Abrupt peaks in perceived risk of occasional drug use after changing the question order in a repeated self-administered survey

Background: Question-order changes in repeated surveys can distort comparisons. We want to describe the evolution of drug risk perceptions among Spanish adolescents and assessing whether the 2006 peaks in perceived risk of occasional drug use can be explained by question-order changes.

Methods: The subjects were secondary students from a biennial national survey during 2000–2012. A one-off intervention was applied in 2006, replacing the two-adjacent items on perceived risk of occasional and regular use of each drug by non-adjacent items. Annual prevalence of high-risk perception were obtained for occasional and regular use of cannabis, heroin, cocaine and ecstasy. Subsequently, the 2006 percent level change (PC) in such were estimated prevalence using segmented Poisson regression, adjusting for various student and parent covariates.

Results: The 2006 PC in prevalence of high-risk perception of occasional drug use ranged from +63% (heroin) to +83% (ecstasy). These PCs were very high in all considered subgroups. However, the 2006 PC in prevalence of high-risk perception of regular drug use ranged from 1% (heroin) to 12% (cannabis). The evolution of preventive interventions does not suggest alternative causal hypotheses for 2006 peaks other than question-order changes.

Conclusion: Within the cognitive heuristics framework, the 2006 spikes in perceived risk of occasional drug use were most likely due to a release of the anchor exerted by perceived risk of regular drug use over that of occasional use triggered by 2006 question-order changes. In repeated surveys it is inexcusable to pre-test the effect of any change in questionnaire format.

Introduction

Repeated surveys are often used to make cross-period or cross-site comparisons of health-related phenomena in both patients and population, in order to improve decision-making on health interventions. These surveys are prone to information bias that can severely limit such comparisons, due to various factors, including differences or changes in measurement instruments, particularly questionnaires. When designing survey questionnaires, most researchers know that they must keep the same question wording in different or successive measurements, but they often forget that changes in the format, ordering or clustering of the questions can strongly affect their interpretation and response (1–4). Changing, deleting or adding a previous question item could greatly alter the answers to the next one, even if its wording is entirely kept (5–12), particularly when questionnaires include multiple question items on similar topics with identical response categories –question grids- (13). In some self-administered question grids, a subsequent question may also affect the preceding one, because the respondent may receive both simultaneously (14, 15). These context or question-order effects can appear when recalling witnessed events, behaviors, tasks performance or people (7, 11, 16). However, they are especially conspicuous for subjective phenomena such as perceptions, beliefs, values, preferences, attitudes or future intentions, since they are usually subject to greater uncertainty in response selection (6, 8–10, 15, 17–19). Susceptibility to order effects depends on numerous factors, including the nature and difficulty of the self-reported subject, specific questionnaire format, and respondent factors such as sex, age, education level, or direct experience on the subject (20–23). It is generally believed that the greater is the respondent’s direct experience or personal involvement in the self-reported issue and their cognitive abilities, the less prone they would be to question-order effects. However, previous evidence on the influence of these factors is often inconsistent (23).

Among the subjective phenomena prone to question order effects are people’s risk perceptions on health-related issues, which are relevant in predicting future decisions and actions of people regarding such issues and implementing strategies to control and reduce the associated health and social harm. Thus, question-order effects have been identified when measuring risk perceptions on diseases/causes of death (cancer, COVID-19, homicide), psychiatric patients, environmental hazards (nuclear power, second-hand smoke, air pollution, food preservatives, electromagnetic radiations, traffic accidents) and health-related behaviors (tobacco, alcohol or drug use) (24–32).

Perceived risk of illicit drug use by adolescents is routinely monitored in many countries (33–35), most of these data employed on secondary analysis on risk prevention (36). ESTUDES is a biennial school survey on drugs aimed at secondary school students conducted in all Spain since 1994, which uses a self-administered paper-and-pencil questionnaire, including questions on subjective judgments on risk of occasional and regular use of different illicit drugs (37). In 2006, the ordering and clustering of the questions on perceived risk of occasional and regular drug use was changed. In statistical analyses, sudden increases in risk perception of occasional use of all considered drugs were found in 2006, which were suspected to be artifacts due to the aforementioned reordering of questionnaire items, so in 2008 and later editions it was decided to restore the initial questionnaire format. The study objective is to describe the evolution over time of the perceived risk of occasional and regular use of illicit drugs among Spanish adolescents stratified according to respondent’s sociodemographic factors, and to assess whether the 2006 peaks in perceived risk of occasional drug use can be explained by question-order changes.

Materials and methods

Study population

Secondary school students aged 14–18 years who participated in the seven editions of ESTUDES biennial survey during 2000–2012. A national representative sample was selected in each edition through a two-stage cluster sampling procedure (school and classroom) (37). The total sample size was 188,921 students, ranging from 20,450 to 32,234 in the different editions.

Study design and variables

The study is conceived as a large-scale unplanned single-group experiment, in which a one-off intervention that had not been applied between 2000 and 2004 was applied in 2006 to the entire study population and ceased to be applied in subsequent years, being analyzed the changes in a target and a control outcome over time. The intervention consisted of changing the ordering of questions on drug risk perception (question-order changes), although in fact it also implies a change in their clustering. Specifically, drug risk perception had been assessed until 2006 by an adjacent item-pair model, in which the pair of questions assessing the risk perception of occasional and regular use of a given drug were consecutively presented one after the other without any intermediate question. However, the questionnaire changed in 2006 towards a non-adjacent item-pair model, in which the pair of questions mentioned for a given drug were not consecutively presented, but in two large different blocks or grids, the first referring to occasional use of all drugs and the second to regular use (Table 1).

TABLE 1

Table 1. Questionnaire models on perceived risk of drug use included in ESTUDES survey (Spain, 2000–2012).

The outcomes were the perceived risk of using occasionally (monthly or less frequently) and regularly (weekly or more frequently) four illicit drugs, specifically cannabis, heroin, cocaine and ecstasy. To assess the risk perception of drug use, the respondents had to indicate the problems (health or otherwise) that entails each of the considered drug use behaviors using an ordinal scale of four responses (no problem, few, quite a few and many) (Table 1). Except for the question-order changes, there were no other changes in questions about outcomes in the analyzed ESTUDES editions, although the questions on cocaine referred simply to “cocaine” until the 2004 edition and “cocaine powder” in the 2006 and later editions (37). For stratification or adjustment the following individual covariates were considered: sex, age, education level, ever had to repeat an annual course, parents’ education level, parents’ employment status, frequency of going out for fun in the evenings as indicator of leisure habits, and use of each considered drug. The education level and course repetition were considered as indicators of academic performance to achieve cognitive skills, and age was used as a proxy of expertise (knowledge on the self-reported subject), since it is assumed that older students have received more information on drug risks (prevention programs, courses, etc.) and have greater capacity to integrate that information. In addition to the ESTUDES variables, data on annual indicators of preventive interventions were obtained from activity reports of the National Plan on Drugs (38), in order to assess if the 2006 changes in outcomes could depend on greater magnitude of such interventions. These indicators include number of schools involved in drug prevention programs, secondary students coverage of such programs, spending index on drug prevention per secondary student and implementation of national preventive campaigns in the media. School drug prevention programs are structured interventions including scheduled sessions (>5) to be developed in the classroom by teachers, or by external prevention experts, often with application manuals, aiming at the development of student’s skills and competencies for life and to avoid drug use. Occasional preventive activities such as talks, distribution of written materials, workshops, awareness days, contests or exhibitions are excluded. The coverage of secondary students by school drug prevention programs is the percentage of total secondary students in Spain (including vocational training) who participated in those programs. The spending index on drug prevention programs per secondary student was calculated by dividing the inflation-adjusted national budget for drug use prevention by the number of secondary students registered in Spain and expressing the result in relation to the year 2000 whose spending per student was assigned the value of 100. The aforementioned budget is the sum of the budgets of the central and regional governments and with it, school and non-school drug prevention interventions are financed. The national preventive campaigns in the media were aimed at informing and sensitizing the Spanish population, especially adolescents aged 12–18 years and their parents or guardians, about the risks of drug use.

Data analysis

After a descriptive analysis, the prevalences of high-risk perception of occasional and regular use of cannabis, heroin, cocaine and ecstasy for each edition of ESTUDES during 2000–2012 were obtained. We considered “high-risk” perception regarding the use of a given drug when the respondent believed that such behavior could cause many problems. Next, two multivariate approaches were used to assess the magnitude of the 2006 change in high-risk perception of drug use compared to previous and subsequent years. The first approach was to estimate the annual Adjusted Prevalence Ratios (aPRs) of high-risk perception and its 95% confidence intervals (95% CIs) from Poisson regression models with robust variance (39) using the year 2000 as a reference. Adjustment covariates were referred to the students (sex, age, indicators of academic performance, leisure habits, and use or not of assessed drugs) or their parents (parents’ education level, parents’ employment status). The second approach was to use an interrupted time series design, which was analyzed with segmented Poisson regression. This design allowed adjustment for the covariates just mentioned plus the underlying time trend (40, 41). As the intervention seems to cause a temporary change (2006) in the outcome level immediately after the intervention, which disappears in the following years, we have adopted the impact model called “temporary level change” (41). This model can be formalized as Y_t = β₀ + β₁T_t + β₂X_t + β_kX_k + ε, where Y_t is the log of annual prevalence, T_t a continuous covariate whose value is the number of years elapsed since 2000, X_t a binary predictor for the intervention (questionnaire change) with a value of 1 in 2006 and 0 other years, X_k a vector for the adjustment covariates other than time, and ε the error term. In this model β₀ represents the intercept or baseline prevalence level, β₁ the change in outcome for each unit increase in sequential calendar year (underlying linear time trend for 2000–2012), β₂ the immediate level change in outcome following the intervention, and β_k the coefficients of different adjustment covariates. To facilitate the interpretation, we transformed the β₂ regression coefficients to relative percent changes (PCs) as 100 (e^β–1). The 95% CI of PCs were estimated as 100 (e^{β ± 1.96SE}–1), where SE is the standard error of β. Analyses were performed using Stata V.14.0 (Stata Corporation, College Station, Texas, USA).

Results

General characteristics of study participants are shown in Table 2.

TABLE 2

Table 2. General characteristics of participants in ESTUDES survey by calendar-year (%) (Spain, 2000–2012).

Evolution of prevalence of high-risk perception of drug use during study period

The evolution of the prevalence of high-risk perception of drug use was very different for occasional and regular use (Figure 1). For occasional use the prevalence followed a relatively stable trend during pre-intervention period (2000–2004), it increased sharply in the intervention year (2006), doubling their figures, and decreased again in post-intervention period (2008–2012), although maintaining a slightly higher level than in 2000–2004. Thus, the prevalence of high-risk perception was 17–21% in 2000–2004, 42% in 2006 and 26–33% in 2008–2012 for cannabis use, 37–40, 75% and 51–58%, respectively, for heroin use, 30–33, 70% and 44–51% for cocaine use, and 26–31, 72% and 48–55% for ecstasy use. Abrupt spikes in prevalence of high-risk perception of regular drug use during 2006 were not observed. Thus, such prevalences in 2000–2012 ranged 51–65% (cannabis), 84–91% (heroin), 83–87% (cocaine), and 77–88% (ecstasy).

FIGURE 1

Figure 1. Evolution of the prevalence of high perceived risk of regular or occasional use of heroin (A), cocaine (B), ecstasy (C), and cannabis (D) by survey year among secondary students aged 14–18 (%). Spain, 2000–2012. High perceived risk: the student thought that a drug use behavior (for example, using heroin regularly) could cause many problems (in health or other aspects). Regular use: using a drug weekly or more frequently. Occasional use: using a drug monthly or less frequently. The lines are shown smoothed.

Impact assessment of 2006 questionnaire change on risk perception of drug use

Like crude prevalences, the aPRs from Poisson regression models showed a very different evolution for occasional and regular drug use (Table 3). Regarding occasional use, the aPR values, using as a reference the prevalence in the year 2000, ranged 0.9–1.2 in pre-intervention years, increased sharply up to 2.0–2.8 in 2006, and decreased again in post-intervention years, staying at a higher level than in 2000–2006 (1.2–2.1). However, the aPRs for regular drug use showed little heterogeneity over time, ranging from 0.9–1.2 during the entire period 2000–2012.

TABLE 3

Table 3. Adjusted-prevalence ratio of high-risk perception¹ of drug use by specific drug use behavior and calendar-year [95% confidence intervals] among secondary students aged 14–18 years (Spain, 2000–2012).

Results from Poisson segmented regression models indicate an upward underlying linear trend in prevalence of high-risk perception of occasional drug use in 2000–2012 with annual PCs of 5, 4, 5, and 7% for cannabis, heroin, cocaine and ecstasy, respectively. Furthermore, there was an immediate relative level change in such prevalence during 2006 of 77, 63, 79, and 83%, respectively (Table 4). However, the 2006 level changes in prevalence of high-risk perception of regular use of cannabis, heroin, cocaine and ecstasy and their corresponding 95% CI were, respectively, 12% (10, 13%), 1% (0, 2%), 3% (2, 4%), and 6% (5, 7%).

TABLE 4

Table 4. 2006 percent level change in prevalence of high-risk perception of occasional drug use¹ [95% confidence interval] by subgroups of secondary students aged 14–18 years (Spain, 2000–2012).

Stratified analysis of impact assessment of 2006 questionnaire change

Focusing on the results of segmented Poisson regression models, a large 2006 immediate percent level increase in prevalence of high-risk perception of occasional drug use was observed in all subgroups of sex, age and academic performance and in both users and non-users of each assessed drug, except in heroin users where only a very slight non-statistically significant increase was observed. These increases were higher in women than men, although the differences were not statistically significant for cannabis. Likewise, the increases were generally greater in students with higher educational level or higher academic performance (as students in the second stage of secondary education or students who had not repeated any complete annual course), except for heroin. Regarding age, no consistent results were observed for the different drugs. Thus, the magnitude of the immediate level increase increased with age for cannabis, decreased with age for heroin, and varied little for cocaine and ecstasy (Table 4).

Evolution of preventive drug use interventions in Spain during study period

The evolution of four indicators of drug prevention interventions aimed at secondary students is shown in Table 5.

TABLE 5

Table 5. Evolution of indicators on drug prevention interventions implemented by the public administration in Spain, 2000–2012.

Three of those indicators (number of schools involved in drug prevention programs, coverage of secondary students by these programs, and spending index on drug prevention per secondary student) followed a similar evolution, with a significant increase between 2000 and 2006, a relative stabilization between that year and 2010 and a subsequent rapid decrease. For example, the coverage of the school drug prevention programs went from 20.7% in 2000 to 55.6% in 2006, 58.7% in 2008, 54.1% in 2010, and 40.5% in 2012. There were annual campaigns in the general media aimed at drug prevention in 2000–2007, but they disappeared in 2008–2012. The maximum relative interannual increases in such indicators were found in 2002 for the coverage of school drug prevention programs (38.0%), in 2003 for the number of schools involved in drug prevention programs (48.8%), and in 2002 for the spending index on drug prevention per secondary student (62.9%). The 2006 relative annual increase of the different interventions ranged from 12.1% for the spending index on drug prevention per secondary student to 31.3% for the coverage of the school drug prevention programs. The absolute maximums of these indicators were in 2008, 2010, and 2006, respectively. Moreover, parents’ and teachers’ involvement in drug prevention increased in 2006–2010 (data not shown in Table 5).

Discussion

In 2006, after a question-order change in ESTUDES survey, there was a sudden and large increase in risk perception of occasional use of cannabis, heroin, cocaine and ecstasy among Spanish adolescents, which largely disappeared in 2008 after restoring the initial questionnaire format. Thus, the relative percent changes in prevalence of high-risk perception immediately after the 2006 intervention ranged from +63% (heroin) to +83% (ecstasy). These increases occurred in all sociodemographic subgroups analyzed. However, the 2006 increases in perceived risk of regular use of these drugs were very small, with relative percentage changes in prevalence of high-risk perception ranging from +1% (heroin) to +12% (cannabis). After assessing the evolution of preventive interventions, no sudden increases in magnitude of these interventions were identified in 2006, which does not suggest alternative causal hypotheses for 2006 peaks in perceived risk of occasional drug use other than 2006 question-order changes.

Question-order changes as the main cause of the 2006 peaks in drug risk perception

It is highly likely that the 2006 peaks in the risk perception of occasional drug use was mainly due to question-order changes for several reasons. First, it was a peak of great magnitude, located only in 2006 and that disappeared after restoring the initial questionnaire format. Second, the Poisson regression model that identified the peak was adjusted for several individual time-varying covariates to avoid confounding. Third, the segmented Poisson regression model to measure the 2006 immediate relative change in prevalence level of high-risk perception was further adjusted for the underlying linear time trend during 2000–2012, which implies some control for unmeasured individual time-varying confounders that change slowly over time (42). Fourth, the peak was identified in all sociodemographic subgroups analyzed, including those of sex, age, education level and academic performance, both in drug and non-drug users (except heroin users). Fifth, the peak was almost non-existent for the perceived risk of regular drug use, considered the control outcome or negative control. Sixth, in 2006 there were no abrupt increases in magnitude of preventive drug use interventions. Thus, although 2006 had the largest budget for drug prevention during the analyzed period, no prevention indicator had in 2006 its highest annual percent increase. Furthermore, the 2006 annual percent increases in prevention indicators ranged 12–31% compared to 63–83% in the outcome. Although local and regional preventive drug interventions are difficult to measure, the occurrence of sudden large-scale synchronized increases in such interventions in 2006 are highly unlikely, mainly due to their partial dependence on the national budget. Finally, among adolescents an inverse relationship between drug risk perception and drug use is usually observed (43–47). However, in 2006–2010 in Spain the apparent huge decreases in perceived risk of cannabis, cocaine and ecstasy were accompanied by decreases in annual and monthly prevalences of use of these drugs (37), suggesting that decreases in perceived risk were not real. Taken together, the evidence suggests a causal relationship between the 2006 questionnaire change (intervention) and the sudden increases in perceived risk of occasional drug use (outcome).

Interpreting the 2006 spike in drug risk perception as an anchor release effect

Although there are several explanations on the mechanism of the question-order effects, perhaps the most accepted rely on cognitive heuristics (13, 23). The mental processes of answering questionnaires usually has various steps (question understanding, retrieving relevant factual data from memory, integrating it into a judgment and answer selection) (1, 48). However, when there is considerable uncertainty, as often happens with subjective phenomena such as health-related risk perceptions, especially under conditions of fatigue or disinterest, respondents often resort to heuristics or mental shortcuts to answer by reducing the cognitive burden. One of the best known is the anchoring-and-adjustment heuristic, which involves initially focusing on an anchor or reference point, which is usually the phenomenon better known, easily recalled from memory, or more salient, and then adjusting the selected answer from the anchor (49–54). This heuristic aligns well with the evidence that human judgements are generally comparative (55), and its effects are extremely ubiquitous across topics, subgroups, and settings. Specifically, in a question grid including several items on a similar topic with a similar response scale, sometimes the response to an adjacent item, which is easier to elaborate, is used by respondents as a self-generated anchor to adjust the response to another item either by reducing the difference between both responses (assimilation) or expanding it (contrast) (51, 56).

The adolescents’ judgments on risk of a given drug are subject to great difficulty and uncertainty because most of them could not base such judgments on personal experience since they are non-users or very sporadic users. Consequently, they would resort to anchoring heuristics, integrating the most easily available external information, which usually refers to regular drug use, since it is a very notorious and quite frequently associated behavior with social or health problems, and subsequently adjusting the response on the risk of occasional use from that reference. As indicated, the 2006 questionnaire change consisted of asking on perceived risk of occasional and regular use of a given drug using two non-adjacent items located in different blocks (non-adjacent item-pair model) instead of two adjacent or consecutive items as in 2000–2004 and 2008–2012 (adjacent item-pairs model). The findings show a 2006 peak in perceived risk for occasional drug use but not for regular use. This suggests that in the adjacent item-pairs model, operative in years other than 2006, the risk perception of regular drug use would act as an anchor item (57), so that adolescents would adjust the response to the immediately preceding item on the risk of occasional use, trying to considerably reduce the risk attributed to regular use (contrast effect) (58). However, in the non-adjacent item-pair model, operative in 2006, the anchor of regular use disappears since this item is far away in another question grid, so the perceived risk of occasional drug use shoots up. In short, the 2006 peak in perceived risk of occasional drug use was most likely due to a release from the anchor exerted by the perceived risk of regular drug use. Such release would have been triggered by the 2006 change in question-order.

As indicated, some respondent factors such as sex, age, education level, or direct experience on the subject (20–23) can modify susceptibility to question-order effects. The higher 2006 peaks in risk perception of occasional drug use among women than men found in our research are consistent with previous findings (20) and with an increased women’s susceptibility to anchoring heuristics (59). Regarding age, our results referring to question-order effects in perceived risk of occasional drug use are mixed, since they show a greater effect among older and younger adolescents, respectively, for cannabis and heroin, and almost no effect for cocaine and ecstasy. However, there is some previous finding indicating that younger adolescents are more strongly influenced by questionnaire format and context than older adolescents (22). Finally, the higher peaks in adolescents with higher than lower educational level or academic performance are difficult to interpret. Thus, although it is usually thought that the greater the cognitive ability of the respondent and their direct involvement or experience in a self-reported issue, the less propensity they will have for question order effects, the truth is that the previous evidence in this regard is usually inconsistent (23). Similarly, it would be expected that the higher the educational level and academic performance of an adolescent, the greater their knowledge on drug effects and greater cognitive ability to develop independent responses without resorting to anchors. However, the evidence on susceptibility to anchoring effect according to education or intelligence level is also inconsistent (4, 21, 60–63). The key to the inconsistencies may be that, although the anchoring heuristic can generate bias, it is also a cognitive resource which in situations of great uncertainty and time constraints helps build responses and judgments (64). Lacking conclusive evidence or social consensus on the risk of occasional use of drugs such as cannabis or ecstasy, the adolescents’ judgments on this involve great uncertainty and cognitive burden. Thus, it may be even more rational and valid to adjust the response from reliable anchors than to elaborate it ex novo relying on one’s own beliefs or those of peers.

Limitations

Given that this study is based on a natural experiment, its main limitation is the potential influence of uncontrolled factors changing between 2006 survey and the others. Notwithstanding, even if it cannot be completely ruled out, the role of potential unknown events (social, political, communicative, etc.) occurring in 2006 is quite unlikely. A detailed search has been made and no one relevant has been found. Other changes in survey methodology (i.e., sampling, fieldwork or database preparation) which could explain the results have not been identified either. In this sense, it is a guarantee that there have not been 2006 abrupt changes in other sociodemographic indicators, drug use, opinion and perceptions from the survey.

On the other hand, dissimilar results observed by drug and sociodemographic subgroup could be argued as a limitation. In fact, these findings do not match with previous knowledge concerning individual susceptibility to cognitive heuristics. However, this background is moderately inconsistent, and this study was not focused on analyzing individual differences. In general terms, the anchoring effect release in 2006 was observed among all drugs and subgroups, and just the magnitude of this fact changed from some groups to others.

Conclusions

The findings suggest that when assessing the magnitude of a subjective health-related phenomenon subject to considerable uncertainty (i.e., the perceived risk of occasional and regular drug use) using two or more consecutive or adjacent items, responses may strongly depend on question ordering. Consequently, in repeated or panel health surveys, some seemingly “cosmetic” changes in ordering or clustering of questions, particularly the insertion of intermediate items, may cause unexpected anomalies or artifacts that severely limit comparisons. The standardization over time, space, groups and individuals of data collection methods or instruments, including questionnaires, is essential to achieve valid estimates of any health measure, particularly subjective ones, and to be able to make valid comparisons based on them. Questionnaire changes should be minimized and if unavoidable they should be carefully pretested through a cognitive interview and piloted before launching (65, 66). Likewise, researchers and decision makers in the health field must be aware that the comparison of findings from two questionnaires with items with the same wording but different ordering or clustering may be affected by question-order biases that invalidate such a comparison. Another important corollary is that the transfer of evidence from self-reported surveys to policies must be very careful and prudent, since the results are highly dependent on the characteristics of the data collection instrument. For this reason, it is also convenient to triangulate the results of these surveys with other sources or methods, such as qualitative techniques.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author contributions

CP-R analyzed the data and wrote the manuscript. JoP and GB originated and designed the study and coordinated the writing of the article. JH, JuP, MD, and MB contributed to the analysis of the data and to the drafting of the manuscript. JG collaborated in the bibliographic search and in the debugging of the database. ER contributed to the interpretation of the results and to the drafting of the manuscript. All authors have contributed to the work, agree to the order in which they are listed, have read and reviwed the final manuscript and approved the final version and its submission.

Funding

This research was carried out in the framework of a project funded by the Government Delegation for the National Drugs Plan (DGPNSD) (Grant 2019I017).

Acknowledgments

The authors thank DGPNSD for the transfer of the analyzed databases.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Abbreviations

aPR, adjusted prevalence ratio; ESTUDES, Survey on Drug Use in Secondary Education in Spain [Encuesta sobre Uso de drogas en Enseñanzas Secundarias en España]; 95% CI, 95% confidence interval; PC, relative percent change.

References

1. Tourangeau, R. Cognitive sciences and survey methods In: Council NR, editor. Cognitive aspects of survey methodology: Building a bridge between disciplines. Washington, DC: The National Academies Press (1984). 73–100.

Google Scholar

2. Tourangeau, RC, Couper, MP, and Conrad, F. Spacing, position, and order: interpretive heuristics for visual features of survey questions. Public Opin Q. (2004) 68:368–93. doi: 10.1093/poq/nfh035

CrossRef Full Text | Google Scholar

3. Schwarz, N. Self-reports: how the questions shape the answers. Am Psychol. (1999) 54:93–105. doi: 10.1037/0003-066X.54.2.93

CrossRef Full Text | Google Scholar

4. Stark, T, Silber, H, Krosnick, JA, Blom, AG, Aoyagi, M, Belchior, AM, et al. Generalization of classic question order effects across cultures. Sociol Methods Res. (2020) 49:567–602. doi: 10.1177/0049124117747304

CrossRef Full Text | Google Scholar

5. Holyk, G. Context effect In: PJ Lavrakas, editor. Encyclopedia of survey research methods. Thousand Oaks, CA: SAGE Publications, Inc. (2008). 141–3.

Google Scholar

6. Bowling, A, and Windsor, J. The effects of question order and response-choice on self-rated health status in the English longitudinal study of ageing (ELSA). J Epidemiol Community Health. (2008) 62:81–5. doi: 10.1136/jech.2006.058214

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Hanley, C, Duncan, MJ, and Mummery, WK. The effect of changes to question order on the prevalence of 'sufficient' physical activity in an Australian population survey. J Phys Act Health. (2013) 10:390–6. doi: 10.1123/jpah.10.3.390

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Garbarski, D, Schaeffer, NC, and Dykema, J. The effects of response option order and question order on self-rated health. Qual Life Res. (2015) 24:1443–53. doi: 10.1007/s11136-014-0861-y

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Lee, S, McClain, C, Webster, N, and Han, S. Question order sensitivity of subjective well-being measures: focus on life satisfaction, self-rated health, and subjective life expectancy in survey instruments. Qual Life Res. (2016) 25:2497–510. doi: 10.1007/s11136-016-1304-8

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Magelssen, M, Supphellen, M, Nortvedt, P, and Materstvedt, LJ. Attitudes towards assisted dying are influenced by question wording and order: a survey experiment. BMC Med Ethics. (2016) 17:24. doi: 10.1186/s12910-016-0107-3

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Jackson, A, and Greene, RL. Question order bias in retrospective evaluations of item and associative recognition. Memory. (2017) 25:481–6. doi: 10.1080/09658211.2016.1188964

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Johnson, AL, Villanti, AC, Glasser, AM, Pearson, JL, and Delnevo, CD. Impact of question type and question order on tobacco prevalence estimates in US young adults: a randomized experiment. Nicotine Tob Res. (2019) 21:1144–6. doi: 10.1093/ntr/nty058

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Siminski, P. Order effects in batteries of questions. Qual Quant. (2008) 42:477–90. doi: 10.1007/s11135-006-9054-2

CrossRef Full Text | Google Scholar

14. Schwarz, NH, and Hippler, HJ. Subsequent questions may influence answers to preceding questions in mail surveys. Public Opin Q. (1995) 59:93–7. doi: 10.1086/269460

CrossRef Full Text | Google Scholar

15. Tourangeau, R, Singer, E, and Presser, S. Context effects in attitude surveys. Effects on remote items and impact on predictive validity. Sociol Methods Res. (2003) 31:486–513. doi: 10.1177/0049124103251950

CrossRef Full Text | Google Scholar

16. Michael, RB, and Garry, M. How do ordered questions bias eyewitnesses? Memory. (2019) 27:904–15. doi: 10.1080/09658211.2019.1607388

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Beebe, TJ, Jenkins, SM, Anderson, KJ, Davern, ME, and Rockwood, TH. The effects of survey mode and asking about future intentions on self-reports of colorectal cancer screening. Cancer Epidemiol Biomark Prev. (2008) 17:785–90. doi: 10.1158/1055-9965.EPI-07-2622

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Thau, M, Mikkelsen, MF, Hjortskov, M, and Pedersen, MJ. Question order bias revisited: a split-ballot experiment on satisfaction with public services among experienced and professional users. Public Admin. (2021) 99:189–204. doi: 10.1111/padm.12688

CrossRef Full Text | Google Scholar

19. Dzielska, A, Kelly, C, Ojala, K, Finne, E, Spinelli, A, Furstova, J, et al. Weight reduction behaviors among European adolescents-changes from 2001/2002 to 2017/2018. J Adolesc Health. (2020) 66:S70–80. doi: 10.1016/j.jadohealth.2020.03.008

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Lee, S, and Grant, D. The effect of question order on self-rated general health status in a multilingual survey context. Am J Epidemiol. (2009) 169:1525–30. doi: 10.1093/aje/kwp070

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Malhotra, N. Order effects in complex and simple tasks. Public Opin Q. (2009) 73:180–98. doi: 10.1093/poq/nfp008

CrossRef Full Text | Google Scholar

22. Diersch, N, and Walther, E. The impact of question format, context, and content on survey answers in early and late adolescence. J Off Stat. (2016) 32:307–28. doi: 10.1515/jos-2016-0018

CrossRef Full Text | Google Scholar

23. Auspurg, K, and Jackle, A. First equals most important? Order effects in vignette-based measurement. Sociol Methods Res. (2017) 46:490–539. doi: 10.1177/0049124115591016

CrossRef Full Text | Google Scholar

24. Taylor, KL, Shelby, RA, Schwartz, MD, Ackerman, J, LaSalle, VH, Gelmann, EP, et al. The impact of item order on ratings of cancer risk perception. Cancer Epidemiol Biomark Prev. (2002) 11:654–9.

Google Scholar

25. Slovic, P, Finucane, ML, Peters, E, and MacGregor, DG. Risk as analysis and risk as feelings: some thoughts about affect, reason, risk, and rationality. Risk Anal. (2004) 24:311–22. doi: 10.1111/j.0272-4332.2004.00433.x

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Kanda, R, Tsuji, S, and Yonehara, H. Perceived risk of nuclear power and other risks during the last 25 years in Japan. Health Phys. (2012) 102:384–90. doi: 10.1097/HP.0b013e31823abef2

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Sarvet, AL, Wall, MM, Keyes, KM, Cerda, M, Schulenberg, JE, O'Malley, PM, et al. Recent rapid decrease in adolescents' perception that marijuana is harmful, but no concurrent increase in use. Drug Alcohol Depend. (2018) 186:68–74. doi: 10.1016/j.drugalcdep.2017.12.041

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Huang, J, Feng, B, Weaver, SR, Pechacek, TF, Slovic, P, and Eriksen, MP. Changing perceptions of harm of e-cigarette vs cigarette use among adults in 2 US National Surveys from 2012 to 2017. JAMA Netw Open. (2019) 2:e191047. doi: 10.1001/jamanetworkopen.2019.1047

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Levy, NS, Mauro, PM, Mauro, CM, Segura, LE, and Martins, SS. Joint perceptions of the risk and availability of cannabis in the United States, 2002-2018. Drug Alcohol Depend. (2021) 226:108873. doi: 10.1016/j.drugalcdep.2021.108873

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Li, Y, Luan, S, Li, Y, Wu, J, Li, W, and Hertwig, R. Does risk perception motivate preventive behavior during a pandemic? A longitudinal study in the United States and China. Am Psychol. (2022) 77:111–23. doi: 10.1037/amp0000885

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Botzen, WJW, Duijndam, SJ, Robinson, PJ, and van Beukering, P. Behavioral biases and heuristics in perceptions of COVID-19 risks and prevention decisions. Risk Anal. (2022) 42:2671–90. doi: 10.1111/risa.13882

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Waddell, JT. Age-varying time trends in cannabis-and alcohol-related risk perceptions 2002-2019. Addict Behav. (2022) 124:107091. doi: 10.1016/j.addbeh.2021.107091

PubMed Abstract | CrossRef Full Text | Google Scholar

33. ESPAD-Group. ESPAD Report 2019: Results from the European school survey project on alcohol and other drugs. Lisbon: European monitoring Centre for Drugs and Drug Addiction (EMCDDA). (2020). Available at: https://www.emcdda.europa.eu/system/files/publications/13398/2020.3878_EN_04.pdf (Accessed April 28, 2022).

Google Scholar

34. UNODC. World drug report 2021. Vienna: United Nations Office on Drugs and Crime (2021). Available at: https://www.unodc.org/unodc/en/data-and-analysis/wdr2021.html (Accessed 28 April 2022).

Google Scholar

35. Johnston, LD, Miech, RA, O’Malley, PM, Bachman, JG, Schulenberg, JE, and Patrick, ME. Monitoring the future national survey results on drug use 1975–2021: Overview, key findings on adolescent drug use. Ann Arbor: Institute for Social Research, University of Michigan (2022).

Google Scholar

36. Helmer, S, Burkhart, G, Matias, J, Buck, C, Engling Cardoso, F, and Vicente, J. “Tell me how much your friends consume”—personal, behavioral, social, and attitudinal factors associated with alcohol and cannabis use among European school students. Int J Environ Res Public Health. (2021) 18:1684. doi: 10.3390/ijerph18041684

PubMed Abstract | CrossRef Full Text | Google Scholar

37. DGPNSD. La Encuesta sobre uso de drogas en Enseñanzas Secundarias en España, ESTUDES Madrid: Delegación del Gobierno para el Plan Nacional sobre Drogas (DGPNSD); (2022). Available at: https://pnsd.sanidad.gob.es/profesionales/sistemasInformacion/sistemaInformacion/encuestas_ESTUDES.htm (Accessed April 28, 2022).

Google Scholar

38. DGPNSD. Memorias del Plan Nacional sobre Drogas. Madrid: Delegación del Gobierno para el Plan Nacional sobre Drogas (DGPNSD); (2022). Available at: https://pnsd.sanidad.gob.es/pnsd/memorias/home.htm (Accessed April 28 2022)

Google Scholar

39. Petersen, MR, and Deddens, JA. A comparison of two methods for estimating prevalence ratios. BMC Med Res Methodol. (2008) 8:9.

Google Scholar

40. Linden, A. Challenges to validity in single-group interrupted time series analysis. J Eval Clin Pract. (2017) 23:413–8. doi: 10.1111/jep.12638

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Lopez-Bernal, J, Cummins, S, and Gasparrini, A. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol. (2017) 46:348–55. doi: 10.1093/ije/dyw098

CrossRef Full Text | Google Scholar

42. Lopez-Bernal, J, Cummins, S, and Gasparrini, A. The use of controls in interrupted time series studies of public health interventions. Int J Epidemiol. (2018) 47:2082–93. doi: 10.1093/ije/dyy135

CrossRef Full Text | Google Scholar

43. Johnston, LD, Miech, RA, O’Malley, PM, Bachman, JG, Schulenberg, JE, and Patrick, ME. Monitoring the future national survey results on drug use 1975–2018: Overview, key findings on adolescent drug use. Ann Arbor: Institute for Social Research, University of Michigan (2019).

Google Scholar

44. Mills, B, Reyna, VF, and Estrada, S. Explaining contradictory relations between risk perception and risk taking. Psychol Sci. (2008) 19:429–33. doi: 10.1111/j.1467-9280.2008.02104.x

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Parker, MA, and Anthony, JC. A prospective study of newly incident cannabis use and cannabis risk perceptions: results from the United States monitoring the future study, 1976-2013. Drug Alcohol Depend. (2018) 187:351–7. doi: 10.1016/j.drugalcdep.2018.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Parker, MA, and Anthony, JC. Population-level predictions from cannabis risk perceptions to active cannabis use prevalence in the United States, 1991-2014. Addict Behav. (2018) 82:101–4. doi: 10.1016/j.addbeh.2018.02.030

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Salloum, NC, Krauss, MJ, Agrawal, A, Bierut, LJ, and Grucza, RA. A reciprocal effects analysis of cannabis use and perceptions of risk. Addiction. (2018) 113:1077–85. doi: 10.1111/add.14174

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Krosnick, JA, and Presser, S. Question and questionnaire design In: JDM Wright and PV Marsden, editors. Handbook of survey research. 2nd ed. San Diego, CA: Elsevier (2009). 263–314.

Google Scholar

49. Tversky, A, and Kahneman, D. Judgment under uncertainty: heuristics and biases. Science. (1974) 185:1124–31. doi: 10.1126/science.185.4157.1124

CrossRef Full Text | Google Scholar

50. Tversky, A. Features of similarity. Psychol Rev. (1977) 84:327–52. doi: 10.1037/0033-295X.84.4.327

CrossRef Full Text | Google Scholar

51. Wegener, DT, Petty, RE, Detweiler-Bedell, BT, and Jarvis, WBG. Implications of attitude change theories for numerical anchoring: anchor plausibility and the limits of anchor effectiveness. J Exp Soc Psychol. (2001) 37:62–9. doi: 10.1006/jesp.2000.1431

CrossRef Full Text | Google Scholar

52. Epley, N, and Gilovich, T. Putting adjustment back in the anchoring and adjustment heuristic: differential processing of self-generated and experimenter-provided anchors. Psychol Sci. (2001) 12:391–6. doi: 10.1111/1467-9280.00372

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Epley, N, and Gilovich, T. The anchoring-and-adjustment heuristic: why the adjustments are insufficient. Psychol Sci. (2006) 17:311–8. doi: 10.1111/j.1467-9280.2006.01704.x

CrossRef Full Text | Google Scholar

54. Gehlbach, H, and Barge, S. Anchoring and adjusting in questionnaire responses. J Appl Soc Psychol. (2012) 34:417–33. doi: 10.1080/01973533.2012.711691

CrossRef Full Text | Google Scholar

55. Kedia, G, Mussweiler, T, and Linden, DE. Brain mechanisms of social comparison and their influence on the reward system. Neuroreport. (2014) 25:1255–65. doi: 10.1097/WNR.0000000000000255

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Sherif, M, Taub, D, and Hovland, CI. Assimilation and contrast effects of anchoring stimuli on judgments. J Exp Psychol. (1958) 55:150–5. doi: 10.1037/h0048784

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Pohl, S, Schulze, D, and Stets, E. Partial measurement invariance: extending and evaluating the cluster approach for identifying anchor items. Appl Psychol Meas. (2021) 45:477–93. doi: 10.1177/01466216211042809

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Wedell, D, Hicklin, H, and Smarandescu, LO. Contrasting models of assimilation and contrast In: DA Stape and J Suls, editors. Assimilation and contrast in social psychology. New York: Psychology Press (2007). 45–74.

Google Scholar

59. Rajdev, AA, and Raninga, AM. Gender and heuristic driven biases: a review of literature. Int J Commer Bus Man. (2016) 5:35–8.

Google Scholar

60. Smith, AR, Windschitl, PD, and Bruchmann, K. Knowledge matters: anchoring effects are moderated by knowledge level. Eur J Soc Psychol. (2013) 43:97–108. doi: 10.1002/ejsp.1921

CrossRef Full Text | Google Scholar

61. Bergman, O, Ellingsen, T, Johannesson, M, and Svensson, C. Anchoring and cognitive ability. Econ Lett. (2010) 107:66–8. doi: 10.1016/j.econlet.2009.12.028

CrossRef Full Text | Google Scholar

62. Teovanovic, P. Individual differences in anchoring effect: evidence for the role of insufficient adjustment. Eur J Psychol. (2019) 15:8–24. doi: 10.5964/ejop.v15i1.1691

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Bruckmaier, G, Krauss, S, Binder, K, Hilbert, S, and Brunner, M. Tversky and Kahneman's cognitive illusions: who can solve them, and why? Front Psychol. (2021) 12:584689. doi: 10.3389/fpsyg.2021.584689

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Lieder, F, Griffiths, TL, Huys, QJ, and Goodman, ND. The anchoring bias reflects rational use of cognitive resources. Psychon Bull Rev. (2018) 25:322–49. doi: 10.3758/s13423-017-1286-8

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Collins, D. Pretesting survey instruments: an overview of cognitive methods. Qual Life Res. (2003) 12:229–38. doi: 10.1023/A:1023254226592

CrossRef Full Text | Google Scholar

66. Grimm, P. Pretesting a questionnaire. Part 2. Marketing research In: JNM Sheth, editor. Wiley International encyclopedia of marketing. New Jersey: John Wiley & Sons, Ltd (2010)NK

Google Scholar