Testing and assessment in psychology. A survey on Italian psychologists at the time of COVID-19 pandemic

Introduction

Italian Universities offer undergraduate and graduate training on tests for students and are also the main providers of Continuous Education credit on test assessment for Italian Psychologists (Alessandri et al., 2021). With both students and professionals, teaching tests requires either to follow the most advanced scientific standards in the choice of which instruments to promote or wide spreading guideline for test use. A further complication in the field of teaching tests is finding a balance between two opposite positions: (1) giving theoretically driven knowledge and leaving that the specific tools will be learned by the psychologists at the beginning of their professional practice; (2) delivering a good knowledge and proficiency of most of the tools used at a given time in a specific community.

With the aim to offer a contribution to these issues, here we present the responses to a survey that involved the psychologists registered to the Italian Board of Psychologists. The survey aimed at providing: (a) the actual picture of psychological testing and assessment in the professional practice; (b) the attitudes toward the use of testing and the evaluation of the quality of the available instruments; (c) the quantity and quality of training received as regards psychological assessment at University, with a particular focus on Norm Referenced Tests (NRT).

We believe that the data presented here will be useful to improve teaching in psychology as well as professional training. Furthermore, the COVID-19 pandemic has accelerated the transition to tools to be administered remotely and the survey also covered this aspect.

Norm-referenced tests

By NRT we refer to psychological assessment techniques which have been standardized in their administration, coding and interpretation, so that test-takers are evaluated in a similar way, irrespectively of where they live or who administers the test. NRT have been evaluated by researchers to assess the extent to which they are effective in measuring a particular psychological feature of respondents (e.g., cognitive functioning, personality traits, or disorders).

When used properly, NRT provide valuable individualized information on respondents on many potentially important aspects of their functioning, and help addressing specific challenges, clarifying diagnoses, and highlighting different strengths and difficulties (e.g., as regards interpersonal, intrapersonal, emotional, behavioral, and cognitive areas). NRT results support providing more effective prevention and intervention by planning tailored treatment for each individual. Also, in perspective, emphasis on NRT would prompt future generations of psychologists to accurately select tests based on their validity and reliability.

It is therefore of particular interest to know what psychologists think about tests, how they used them in general and in specific areas of psychology, and which challenges do they experience in NRT application. This data set includes information on Italian psychologists' attitudes toward NRT, and their online administration with a specific emphasis on the COVID-19 period. Data were collected within the framework of a survey promoted by the EFPA (European Federation of Psychologists Association) Board of Assessment.

EFPA project on assessment and testing

In 2000, the EFPA promoted a survey on psychologists' attitudes toward various aspects of testing in six European countries (i.e., Belgium, Croatia, Slovenia, Spain, The Netherlands, and United Kingdom, Muñiz et al., 2001). The results showed that, in general, European psychologists had a positive attitude toward tests and testing. Their scores also indicated a desire for greater involvement of professional organizations in the regulation of tests along with more control on the qualification in their use. Concern over incorrect use of tests emerged as well. Moreover, results clearly indicated a request for further teaching, since University did not provide adequate training (Muñiz et al., 2001). Italy did not participate to this first investigation.

In 2009, the EFPA Standing Committee on Tests and Testing (EFPA-SCTT) reassessed European psychologists' opinions on tests (Evers et al., 2012). Seventeen European countries participated in the survey, and 11 countries previously not included participated to the study (i.e., Austria, Belgium, Croatia, Czech Republic, Germany, Greece, Lithuania, The Netherlands, Norway, Poland, Romania, Slovakia, Slovenia, Spain, Sweden, Turkey, and United Kingdom). Compared with the previous survey, in 2009 psychologists were more satisfied with information received about test quality, had less concern over quality standards in the use of test materials, but were more preoccupied about illegal copying of test manuals and materials. The new sample showed high appreciation of tests in all countries, but among countries the survey found differences with regards incorrect test use and need for regulations on tests and testing. Moreover, a new dimension was taken into account concerning Internet testing with a low score and high differences between countries (Evers et al., 2012). Italy did not participate also in this second wave investigation.

In 2012 the EFPA Board of Assessment issued a third survey on opinions about tests involving countries worldwide, and comparing results with the point of view of European's psychologists collected in 2009. The following new groups of respondents were included: Brazil, Bulgaria, China, Denmark, Hungary, Indonesia, Israel, Italy, Latvia, Lebanon, New Zealand, and Nigeria. The study was published in 2017 (Evers et al., 2017). Italy participated with a sample of 5,482 participants (80% female).

In 2019 the EFPA Board of Assessment decided to repeat the same survey with the questionnaire used in the 2009 and 2012 administration. In Italy the Assessment Group of the Italian Association of Psychology (AIP) promoted the participation of Italian psychologists belonging to the Italian Board of Psychologists. The EFPA questionnaire was inserted in a wider investigation dealing with (a) the actual picture of psychological testing and assessment in the professional practice; (b) the attitudes toward the use of testing and the evaluation of the quality of the available instruments; (c) the quantity and quality of training received as regards psychological assessment at University. At the we are writing this article, the overall results have not been disseminated yet. However, a comparison between Croatian and Italian psychologists' attitudes is available (Lis et al., 2022).

COVID-19 pandemic

Starting from the beginning of 2020, the world found itself collapsing under the effects of the SARS-CoV-2 pandemic. The subsequent countermeasures to avoid contagious diseases included restrictions mainly aimed at decreasing social contacts and maintaining distance between people. In many countries the peak of the pandemic was managed by a strict lockdown, with little or no access to the hospitals except for acute illness.

All of this induced the need for online assessment. This in turn led to significant changes in clinical practice. Psychologists had to modify their settings to assess patients' mental status and cognitive abilities, moving from in-person to remote assessment by using video calling platforms and other resources. Related to this, our survey included a new part of the online assessment, mandatory during lockdown periods.

Method

Participants

A total of 2,412 Italian psychologists participated in the current study, a percentage of 2% of the 120,601 psychologists registered in the National Order of Psychologists¹ in 2020. However, 922 people (38.2%) did not supply any answer and were deleted from the dataset. The final number of non-empty questionnaires gathered is 1,490.

Males were 229 (15.4% of the whole sample) and females were 1,252 (84%; nine missing values, 6%); mean age was equal to 45.41 years (sd = 11.84). Geographical area of living was recoded in three groups, as follows: Northern Italy, n = 953 (64%), Central Italy, n = 313 (21%) and Southern Italy, n = 224 (15%). As regards the level of education obtained after the Master's degree in Psychology, n = 447 (30%) owned a 2-year post-graduate degree, n = 983 (66%) a 4-year specialization degree enabling the exercise of psychotherapy, and n = 89 (6%) a Ph.D (multiple answers were allowed). To sum up, most respondents were self-employed (n = 1,039, 69.7%), worked in the Clinical psychology field (n = 1,000, 67%), declared more than 15 years of seniority as professional psychologist (n = 591, 39.6%), and more than 15 years of seniority as a member of the CNOP (n = 647, 43%; for details, see Table 1). While the representativeness of the sample was not formally assessed, our data seem to match the features of psychologists working in Italy, with a majority of middle-aged woman working as self-employed professionals and residing in the northern part of Italy.

Table 1

Table 1. Descriptive statistics of items presented in Section 1 of the questionnaire and of selected items belonging to Section 2 and 3.

Measures

The questionnaire included three sections. Section 1. Ten questions asked respondents information on their personal and professional background (V2-V11; e.g., gender, age, geographical area, level of education, job contract, field of specialization, and number of years of registration in the National Board of Psychologists). Section 2. This part included 40 items. Thirty-two items belonged to the EFPA Questionnaire on Test Attitudes of Psychologists (EQTAP), i.e., items Q1_1 to Q1_24, plus the eight options of item Q1_25 (i.e., items Q1_25_1 to Q1_25_8, see Evers et al., 2012, p. 318, Appendix A). Moreover, eight newly developed questions on the same issues (i.e., Q2_1 to Q2_8). All items, were scored on a 5-point Likert-type scale ranging from 1 = completely disagree/very rarely to 5 = completely agree/very frequently. The items assessed several topics, such as the quality of training received in Psychology, legislation and deontology issues, and administration procedures of psychological testing (e.g., “The training received in Psychology Bachelor's Degree courses is sufficient for the correct use of most tests” and “Legislation is needed to control the more serious abuse of testing” from EQTAP; “People to which a test has been administered have the right to receive an explanation of results” from new items). Section 3. This section comprised 47 items, of which and adaptation of n. 26 of the EQTAP (i.e., items Q1_26_1 to Q1_26_3, requiring text format answers, e.g., “Write the name of the three tests you use most frequently in carrying out your profession—Test 1”; see Evers et al., 2012, p. 318, Appendix A). Items from Q3 to Q10 dealt with the respondents' experience with tests (e.g., Q3: “Do you use tests in your work/profession?” answer: yes/no), online testing administration (e.g., Q5_1: “The use of online tests, remotely, is completely similar to the use of in-person tests” answer: yes/no), and professional experience and testing with particular reference to the lockdown due to COVID-19 (e.g., Q5_11: “The current historical moment (COVID-19) can provide the opportunity to learn new digital processes such as online consultation or online testing”). When the response scale was not dichotomous or required a percentage, items were scored on a 5-point Likert-type scale, ranging from 1 = completely disagree/very rarely to 5 = completely agree/very frequently.

Procedure

The survey was administered online through the Qualtrics platform. All members of the CNOP were invited to participate via e-mail on a voluntary basis. The survey was open for answers for 3 months, starting from January 2021 to May 2021. A reminder was sent 4 weeks after the first e-mail.

Overall, of the 1,490 people who provided valid questionnaires, 1,449 answered to at least 75% of items of Section 1 and 2, and 796 to at least 75% of items of all the three Sections. Table 1 summarizes key information on the gathered data along with descriptive statistics concerning selected items. Items displayed in Table 1 were selected based on their salience for readers interested in the demographic of participants and in their general approach to online testing.

Strengths and limitations

The dataset provides a starting point for a better understanding of current views on testing according to a group of Italian professional psychologists. Moreover, it would make it possible to compare them with previous surveys and with those collected in other nations, and to plan strategies to improve the use of testing instruments and their teaching in academia.

As regards limitations, the self-selected convenience sample of respondent was suboptimal if compared with the actual number of psychologists enrolled to the CNOP. Respondents were not entirely representative of the population in terms of geographic area of residence, field of specialization, and number of years of registration in the National Board of Psychologists.

Possible research paths

Based on the presented dataset, scholars can conduct numerous analyses concerning a broad range of research questions on a cross-cultural level, with regard to, for instance, age, gender, level of education, country of origin, academic training for the use of tests, area in which practitioners use psychological tests. In addition, scientists may apply these data to identify predictors of psychological testing use.

Dataset description

The dataset contains the answers of 1,490 Italian psychologists to a questionnaire about the use of assessment tools in their professional practice, which includes a part concerning online testing administration during COVID-19 pandemic.

Data availability statement

The datasets presented in this study can be found in the Open Science Framework (OSF) repository. The names of the repository/repositories and accession number(s) can be found at: https://osf.io/2p3gh/?view_only=b2baaae724874f7e94fe3825fc207fbf. The deposit contains two files: Database.xlsx, containing the raw data; Codebook.xlsx, containing item codes, item descriptions, and response scale/options.

Ethics statement

The studies involving humans were approved by Ethical Committee for Research in Psychology (CERPS) of the Catholic University of the Sacred Heart of Milan (code 62-23). The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

AB: Data curation, Formal analysis, Investigation, Methodology, Writing—original draft, Writing—review & editing. MN: Data curation, Formal analysis, Investigation, Methodology, Writing—original draft, Writing—review & editing. RD: Investigation, Writing—original draft, Writing—review & editing. FA: Conceptualization, Investigation, Methodology, Writing—review & editing. DT: Conceptualization, Investigation, Methodology, Writing—review & editing. FL: Conceptualization, Investigation, Methodology, Writing—review & editing. LP: Investigation, Methodology, Resources, Writing—review & editing. AL: Conceptualization, Investigation, Supervision, Writing—original draft, Writing—review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This research was supported and financed by the Italian Association of Psychology (AIP) for open access fee.

Acknowledgments

The authors wish to acknowledge the Assessment Group of the Associazione Italiana Psicologi (AIP) for the support and the National Council of the Order of Psychologists (CNOP) for the dissemination of the questionnaire.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg.2024.1345995/full#supplementary-material

Footnotes

1. ^The Law n. 56 of the 18th of February 1989, and the Ministerial Decree n. 240 of the 13th of January 1992, altogether established the organization of the profession of psychologist in Italy. The CNOP (National Council of the Order of Psychologists) was founded in 1992, and all professionals who want to practice the profession in Italy need to become registered psychologists after passing a state exam.

References

Alessandri, G., Aschieri, F., Bobbio, A., Daini, R., Laghi, F., Nucci, M., et al. (2021). L'insegnamento degli strumenti di assessment psicologico: una prima esplorazione nella comunità accademica italiana. Giornale Italiano di Psicologia 48, 39–61. doi: 10.1421/101219

Crossref Full Text | Google Scholar

Evers, A., McCormick, C. M., Hawley, L. R., Muñiz, J., Balboni, G., Bartram, D., et al. (2017). Testing practices and attitudes toward tests and testing: an international survey. Int. J. Test. 17, 158–190. doi: 10.1080/15305058.2016.1216434

Crossref Full Text | Google Scholar

Evers, A., Muñiz, J., Bartram, D., Boben, D., Egeland, J., Fernandez-Hermida, J. R., et al. (2012). Testing practices in the 21st century: developments and European psychologists' opinions. Eur. Psychol. 17, 300–319. doi: 10.1027/1016-9040/a000102

Crossref Full Text | Google Scholar

Lis, A., Matešić, K., Antonelli, A., and Aschieri, F. (2022). Similarities and differences in challenges of test development, adaptation, and standardization: a descriptive study of Croatian and Italian psychologists' attitudes. Suvremena psihologija 25, 125–136. doi: 10.21465/2022-SP-252-02

Crossref Full Text | Google Scholar

Muñiz, J., Bartram, D., Evers, A., Boben, D., Matesic, K., Glabeke, K., et al. (2001). Testing practices in European countries. Eur. J. Psychol. Assess. 17, 201–211. doi: 10.1027/1015-5759.17.3.201

Crossref Full Text | Google Scholar