Evaluating the psychometric properties of three WHO instruments to assess knowledge about human rights, attitudes toward persons with mental health conditions and psychosocial disabilities, and practices related to substitute decision-making and coercion in mental health

Background: Instruments to assess the knowledge about the rights of persons with mental health conditions and psychosocial disabilities, the attitudes toward their role as rights holders, and mental health professionals’ practices related to substitute decision-making and coercion are either missing or lack evaluation of their validity and reliability.

Aim: The aim of this study is to evaluate the validity and reliability of three instruments developed to fill this gap in the literature, the World Health Organization’s QualityRights (WHO QR) Knowledge questionnaire, the WHO QR Attitudes questionnaire, and the WHO QR Practices questionnaire.

Methods: A sample of participants was recruited and completed an online survey. Content validity and face validity were assessed for the three questionnaires. Based on the characteristics of the questionnaires, different approaches were used to assess their construct validity (confirmatory factor analysis, known group validity, and convergent and divergent validity). Internal consistency was evaluated using Cronbach’s alpha and test re-test reliability using Pearson’s and Spearman’s r coefficients.

Results: The analyses conducted indicate that the three questionnaires are valid and reliable instruments to evaluate the knowledge about the rights of persons with mental health conditions and psychosocial disabilities, the attitudes toward their role as rights holders, and mental health professionals’ practices related to substitute decision-making and coercion.

Conclusion: This finding lends support to the use of these instruments both within mental health services and in the general population for a better understanding of current knowledge, attitudes, and practices related to a human rights–based approach to mental health in mental health services and the community.

Introduction

People with mental health conditions and psychosocial disabilities are commonly exposed to human rights violations both within the general community and the mental health care system (1–5). The negative consequences of these violations on the health of persons with mental health conditions and psychosocial disabilities have been widely documented but attempts to improve the situation have met with little success. Among the barriers to addressing this problem, three are particularly important: 1) the general public and mental health professionals’ lack of knowledge about human rights (6, 7); 2) the negative attitudes toward people with mental health conditions or psychosocial disabilities’ role as right holders (i.e., their role as persons entitled to exercise all human rights on an equal basis with others) (8–11); and 3) the negative practices implemented in mental health services, such as the use of seclusion, restraint, and substitute decision-making approaches to care that keep people with mental health conditions and psychosocial disabilities disempowered and contribute to the negative image that the general public holds of them (3, 4, 12).

Education on human rights may have a fundamental role in changing negative attitudes and, in turn, practices and behaviors leading to human rights violations. The Knowledge-Attitude-Practice (KAP) framework, also found in literature as Knowledge-Attitude-Behavior framework (13, 14), helps explain this relationship. According to the KAP framework, the accumulation of knowledge about a particular issue leads to changes in the predisposition to respond (the attitude). Over time, this causes a change in practice that is in agreement with the attitude (13). The legitimacy of the KAP approach in the health field, although not entirely free from criticism (15–17), is largely supported by the scientific literature (13, 18–20) and has formed the basis for many successful public health interventions (21–25). Within this article context, according to the KAP framework, improved knowledge of human rights and how to apply rights-based, person-centered approaches among mental health professionals and other stakeholders would lead to positive attitudinal changes toward people with mental health conditions or psychosocial disabilities as rights holders. This would also lead, over time, to reduced human rights violations and improved practices consistent with a human rights–based approach.

The KAP approach has rarely been implemented in its entirety within the mental health and human rights field. There have been numerous initiatives aimed to either increase awareness about human rights (26–31) or challenge the negative attitudes of society toward people with mental health conditions and psychosocial disabilities (32–35). However, only a few initiatives to increase knowledge and change attitudes have also evaluated a change in practices. For instance, among the studies to address mental health professionals’ stigma, only two (36, 37) assessed a general change in practices.

In parallel with these initiatives, many instruments to measure knowledge and attitudes have been developed (31, 38, 39), whereas instruments assessing a change in practices are extremely rare because this aspect is neglected in research. However, even the instruments developed to date present important limitations. The majority of these instruments have not been tested for reliability and validity. This is an issue because the use of instruments that lack evidence of validity and reliability may lead to information bias. Additionally, these instruments often do not follow the human rights approach and language promoted by international human rights treaties such as the United Nations Convention on the Rights of Persons with Disabilities (UN CRPD). The existing instruments were mainly developed by health practitioners and often reflect their position, even when this is in contradiction with the perspective of people with mental health conditions and psychosocial disabilities and their organizations.

As far as we know, there are no validated instruments to specifically evaluate the knowledge regarding persons with mental health conditions and psychosocial disabilities’ human rights included under the UN CRPD. There are several validated instruments to assess negative attitudes toward this group, but none of these instruments focus specifically on attitudes toward their role as rights-holders, although such aspect is fundamental to achieve full participation in society (38). Furthermore, there are no instruments to assess changes in practices related to substitute decision-making and coercion, although such human rights violations are common in mental health services. Instruments able to capture these aspects of practice change are much needed.

The present paper aims to fill this gap by evaluating the validity and reliability of three World Health Organization instruments developed to assess the knowledge about the rights of persons with mental health conditions and psychosocial disabilities, the attitudes toward their role as rights holders, and mental health professionals’ practices related to substitute decision-making and coercion. The study was carried out in Ghana, a country that recently came under scrutiny for human rights violations in mental health and where several interventions are ongoing to tackle these violations (40). The instruments under evaluation could be useful in evaluating some of these initiatives. Indeed, in a further study, we have used them in evaluating change in practices in a Randomized Controlled Trial (RCT) in Ghana, with results forthcoming in a subsequent paper.

Materials and methods

Study setting

The study was conducted in Ghana as part of the project funded by the European Commission (EIDHR 2018–400431) “Empowering persons with psychosocial disabilities to fight for their rights: An implementation of the CRPD and QualityRights principles in Ghana, Lebanon, and Armenia.” Prior to this study, several local and international human rights organizations and scientific publications reported that violations of the rights of persons with mental health conditions and psychosocial disabilities are common in psychiatric facilities and the general community in Ghana (41–45).

Ethics approval and informed consent

The study was approved by the Ghana Health Services Ethics Review Committee (study protocol approval: GSH-ERC 005/01/21. It was conducted according to the Declaration of Helsinki and its revisions. All data were made confidential according to the provisions that protect privacy in Ghana (Data Protection Act, 2012). Participants were required to give informed consent electronically at the recruitment and the second administration of the instruments after 10 days. In this study, as required by the UN CRPD, individuals’ right to legal capacity (including the right to provide informed consent) was not denied on the basis of disability status.

Description of the instruments to evaluate

Three self-report instruments were evaluated in this study:

● The World Health Organization’s QualityRights Knowledge questionnaire (WHO QR Knowledge) measures knowledge about the rights of persons with psychosocial disabilities and mental health conditions included under the UN CRPD. This instrument includes questions around the meaning of different rights (e.g., “Informed consent is when a person’s family members receive information about different possible treatment options in order to make an informed decision”), their application and/or violation (e.g., “Human rights can never be restricted” and “In supported decision-making, support should increase throughout life”) and states’ obligations (e.g., “The Convention on the Rights of Persons with Disabilities is binding on countries which have ratified it”). The items are dichotomous (respondents are required to judge whether the statements are “True” or “False”), with correct responses scoring one point and incorrect responses scoring zero points. Scores on the WHO QR Knowledge questionnaire are calculated by summing up responses across items for each participant as per standard practice. A high score on the questionnaire indicates a high level of knowledge about the UN CRPD principles. More details on the WHO QR Knowledge are provided in Supplementary Table 1 (original version) and Table 1 (final version).

● The World Health Organization’s QualityRights Attitudes questionnaire (WHO QR Attitudes) measures attitudes toward people with psychosocial disabilities or mental health conditions as rights-holders. This instrument comprises items organized in three subscales: Subscale 1 includes questions on the attitudes toward mental health services approach (e.g., “People with dementia should always live in group homes where staff can take care of them” and “The service environment has little to do with people’s mental health and well-being”); Subscale 2 includes questions on the attitudes toward involuntary and coercive practices (e.g., “Controlling people using mental health services is necessary to maintain order” and “The use of seclusion and restraint is needed if people using mental health services become threatening”); and Subscale 3 includes questions on the attitudes toward people with psychosocial disabilities or mental health conditions as decision-makers and full members of society (e.g., “People with psychosocial disabilities/mental health conditions should not be hired in work requiring direct contact with the public” and “Persons with mental health conditions should not be given important responsibilities”). Answers are provided on a five-point Likert scale (“Strongly disagree,” “Disagree,” “Neutral,” “Agree,” and “Strongly agree”) and rated following this scheme: “Strongly disagree” is scored as one point, “Disagree” as two points, “Neutral” as three points, “Agree” as four points, and “Strongly agree” as five points. All items are negatively worded, except for items 7, 13, and 17 (which require reverse scoring). Scores on the questionnaire are calculated by summing responses across each participant’s items. Because the items are ordinal and have at least five categories, they are treated as an ordinal approximation of a continuous variable in the analyses (46, 47). High scores on the questionnaire indicate negative attitudes toward people with psychosocial disabilities’ role as rights-holders. More details on the WHO QR Attitudes are provided in Table 2 and Supplementary Table 2.

● The World Health Organization’s QualityRights Practices questionnaire (WHO QR Practices) measures the practices related to substitute decision-making and coercion in mental health units. This instrument is divided in two subscales. The first subscale measures how frequently the respondent has used practices such as seclusion or restraints in the last 3 months and includes questions such as “I prescribed or administered a treatment although the service user did not want it” or “I yelled or used verbal aggression to get service users to comply with requests.” The answers are provided on a seven-point Likert scale, rated following this scheme: “Never” is scored as one point, “A few times in the last 3 months” as two points, “Once a month or less” as three points, “A few times a month” as four points, “Once a week” as five points, “A few times a week” as six points, and “Every day” as seven points. Items 4 and 5 need to be reverse-scored. A high score indicates practices related to substitute decision-making and coercion are used frequently by the respondent. The second subscale assesses the level of agreement between the respondents and the mental health professionals working in their units in the use of practices related to substitute decision making and coercion. It includes questions such as “Mental health professionals in my unit use restraint to control unsettled situations in the ward” or “Mental health professionals in my unit use seclusion and chemical restraints.” Answers are provided on a five-point Likert scale, rated as follows: “A lot less than me” is scored as one point, “Less than me” as two points, “As much as me” as three points, “More than me” as four points, and “A lot more than me” as five points. Item 11 needs to be reverse-scored. High scores indicate that the mental health professionals working in the units of the respondents are more willing to use practices related to substitute decision-making and coercion than the respondents are. Because the items are ordinal and have at least five categories, they are treated as an ordinal approximation of a continuous variable in the analyses (46, 47). More details on the WHO QR Practices are provided in Supplementary Table 3 (original version) and Table 3 (final version).

Table 1

Table 1. World Health Organization’s QualityRights Knowledge questionnaire (WHO QR Knowledge): Final version with 24 items.

Table 2

Table 2. World Health Organization’s QualityRights Attitudes questionnaire (WHO QR Attitudes).

Table 3

Table 3. World Health Organization’s QualityRights Practices questionnaire (WHO QR Practices): Final version.

Instruments development, testing, and piloting

The WHO QualityRights research team developed the World Health Organization’s QualityRights Knowledge (WHO QR Knowledge) questionnaire and the World Health Organization QualityRights Attitudes questionnaire (WHO QR Attitudes) after a long process of discussion and consultation with different stakeholders (e.g., organizations of persons with mental health conditions and psychosocial disabilities, mental health professionals, human rights advocates, academics, experts in psychometrics, and members of the government) from a variety of countries. The research team of the project “Empowering persons with psychosocial disabilities to fight for their rights: An implementation of the CRPD and QualityRights principles in Ghana, Lebanon, and Armenia” developed the WHO QualityRights Practices questionnaire (WHO QR Practices).

First, the research teams reviewed the literature searching for pre-existing instruments to assess knowledge about disability rights, attitudes toward people with psychosocial disabilities and mental health conditions, and practices related to substitute decision-making and coercion. Then, the research teams developed a first draft of the questionnaires. A group of stakeholders (e.g., organizations of persons with mental health conditions and psychosocial disabilities, mental health professionals, human rights advocates, academics, experts in psychometrics, and members of the government) provided feedback on the drafts, and revisions were incorporated. A second group of stakeholders from different countries then reviewed the revised questionnaires. The final versions of the questionnaires were used in the present study.

Collection of data

Sociodemographic variables

In this study, the questionnaire used to collect data included the following sociodemographic variables: gender; age in years; educational attainment; “main” background (with possibility to select only one option among: person with psychosocial, intellectual, and cognitive disability or mental health condition; person with other disabilities; family member or care partner; mental health or related practitioner; health practitioner; lawyer; human rights advocate; policymaker/analyst; academia; other); region of residence; main language; family member(s) with a psychosocial disability or mental health condition; and identification as “person with a psychosocial disability or mental health condition.”

Additional instruments used

The following instruments were administered with the WHO QR Attitudes questionnaire for evaluating convergent and divergent validity:

● The Community Attitudes Toward the Mentally Ill (CAMI) (48) is a 40-item, self-report instrument to assess the public’s attitudes toward persons with mental health conditions. There are four subscales within the CAMI: Authoritarianism, Benevolence, Social restrictiveness, and Community mental health ideology. High scores on the CAMI subscales indicate endorsement of authoritarianism, benevolence, social restrictiveness, and community mental health ideology. This instrument has been used in a previous study in Ghana (49) and has shown acceptable reliability, with Cronbach’s alpha values ranging between 0.71 and 0.75 for all the subscales, except for the authoritarianism subscale, for which Cronbach’s alpha was 0.31 (likely because the authors did not include two items from the original scale, whereas we included these items in the present study). It was hypothesized that higher scores on the WHO QR Attitudes (showing negative attitudes) would be associated with higher scores on the Authoritarianism and Social Restriction subscales and lower scores on the Benevolence and Community Mental Health Ideology (CAMI) subscales.

● The Attribution questionnaire (AQ) (50) is a 27-item, self-report instrument to measure stigma toward persons with mental health conditions. The AQ consists of a brief vignette about “Harry,” a person with a diagnosis of schizophrenia, followed by questions organized in nine subscales assessing nine constructs: Blame, Anger, Pity, Help, Dangerousness, Fear, Avoidance, Segregation, and Coercion. High scores on the AQ subscales indicate a high endorsement of the corresponding constructs. We hypothesized that higher scores on the WHO QR Attitudes (showing negative attitudes) would be associated with higher scores on the Blame, Anger, Pity, Dangerousness, Fear, Avoidance, Segregation, and Coercion subscales and lower scores on the Help subscale.

Procedure for data collection

All data were collected through an online platform. Participants received a first email with a link to a survey requesting the sociodemographic variables, the WHO QR Knowledge, the WHO QR Attitudes, the WHO QR Practices, the CAMI, and the AQ. After 10 days, participants received a second email with a link to a survey including only the WHO QR Knowledge, the WHO QR Attitudes, and the WHO QR Practices. These data were used for the evaluation of the test–retest reliability.

Sample and recruitment procedure

Opportunistic sampling was used to recruit participants among mental health professionals, members of organizations of persons with disabilities, human rights advocates, academics, and government members, aged 18 years or older and able to speak English. Potential participants were identified using three strategies. First, participants were randomly selected from the register of mental health professionals employed in three psychiatric hospitals (Accra, Ankaful, and Pantang) and invited to be screened for eligibility (via email or phone). Second, trained research assistants, who had professional liaisons with local experts on human rights in mental health, identified potential participants among their contacts (mental health professionals, members of organizations of persons with disabilities, human rights advocates, academics, and government members). This strategy guaranteed that this group of potential participants included only experts in human rights in mental health. Third, the trained research assistants used a recently published list of Ghanaian organizations working in the mental health field to identify and contact potential participants. Potential participants were selected among the organizations’ affiliates who were not experts in human rights in mental health. The potential participants selected through the second and third strategies were then invited to participate in the study by email or phone and provided informed consent electronically. The three strategies allowed us to recruit both experts and non-experts in human rights in mental health, including participants who were mental health professionals and participants who were not. This is important because two of the instruments under evaluation aimed ultimately to target a broader audience. Sample size calculations indicated the need to recruit 280 participants to have enough power to perform the analyses.

Data analytic approach

Description of the sample

Descriptive analyses were conducted for the sociodemographic variables. Univariate analyses were used to examine continuous variables by assessing the mean, the median, and the spread of the data. Tabular analyses were used to examine categorical variables by assessing frequencies. Missing data were reported for each variable.

Validity

In the present study, content validity, face validity, and construct validity were used to assess the validity of the instruments under investigation.

A given instrument is said to have content validity when its development includes a review of existing data and literature, and an independent panel of experts in the subject matter under investigation (usually seven or more) confirms that the instrument items are relevant and reflect the domain of interest (51, 52). In the present study, the instruments were developed following the first criteria, and, then, content validity was assessed by seven experts in human rights in mental health that examined the instruments to ensure that they were consistent with their underlying conceptual frameworks. Experts also evaluated the performance of the items on four dimensions (item consistency with the content area, item wording clarity, items perceived difficulty, and whether the items should be included in instruments) using a dichotomous response scale (yes vs. no) (53, 54). “Yes” answers were scored as 1 and “No” as 0. The maximum overall score for content validity was 476 for the WHO QR Attitudes (each of the 17 items was evaluated on four dimensions by seven experts, and each dimension could have a maximum score of 1: 17 × 4 × 7 = 476), 1,036 for the WHO QR Knowledge, and 336 for the WHO QR Practices. Based on these scores, average content validity indexes were calculated for all the instruments (dividing the actual overall score for content validity by the maximum overall score for content validity) (55). The recommended content validity index cutoff value of 0.75 was considered acceptable (55). Experts had the possibility to provide comments for each item.

A given instrument is said to have face validity when individuals in the target population agree the instrument appears to measure the dimension under investigation (51). In the present study, face validity was assessed by seven lay stakeholders (including persons with mental health conditions or psychosocial disabilities, members of civil society organizations, and mental health professionals). The lay stakeholders examined the instruments using a dichotomous response scale (yes vs. no) to evaluate if the items were clear, easy to understand, and relevant (53, 54). The maximum overall score for face validity was 357 for the WHO QR Attitudes (each of the 17 items was evaluated on three dimensions by seven lay stakeholders, and each dimension could have a maximum score of 1: 17 × 3 × 7 = 357), 756 for the WHO QR Knowledge, and 252 for the WHO QR Practices. Stakeholders had the possibility to provide comments for each item.

Construct validity assessment requires that a conceptual model of the construct of interest is postulated, and its relationships with other relevant constructs in the subject-matter domain have been described (51, 56). If the assessment results are in agreement with the conceptual model and the relationships postulated for the construct of interest, then the questionnaire is considered valid. Based on the characteristics of the questionnaires, different approaches were used to assess their construct validity.

● For the World Health Organization’s QualityRights Knowledge questionnaire (WHO QR Knowledge):

● Known group validity (i.e., the capacity of a questionnaire to differentiate groups of respondents who may be anticipated to have a different score in a predicted direction) (51) was used to compare the total scores for experts and non-experts in human rights using non-parametric Wilcoxon–Mann–Whitney test. We hypothesized that experts, who have greater knowledge on human rights, would have higher scores on the WHO QR Knowledge than non-experts.

● Confirmatory Factor Analysis (CFA) was used to evaluate the one dimension postulated (i.e., knowledge about human rights). A model was fitted including this dimension and using the estimator Weighted Least Square Mean and Variance adjusted (WLSMV) for dichotomous variables. Goodness of fit indices were used to evaluate this model: Tucker–Lewis Index (TLI) ≥ 0.95 and ≥ 0.90 indicating good and sufficient fit, respectively; Root Mean Square Error of Approximation (RMSEA) ≤ 0.05 or ≤ 0.08 indicating good and sufficient fit, respectively (56). Finally, factor indicators with loadings lower than 0.30 were considered for removal, and path diagrams were used for a visual comparison of the factor loadings.

● For the World Health Organization’s QualityRights Attitudes questionnaire (WHO QR Attitudes):

● Confirmatory Factor Analysis was used to evaluate the three dimensions postulated (i.e., 1) Attitudes toward mental health services approach, 2) attitudes toward involuntary and coercive practices, and 3) attitudes toward people with psychosocial disabilities or mental health conditions as decision-makers and full members of society). This factor structure was evaluated using the same approach described for the factor analysis of the WHO QR Knowledge, but using the estimator MLR for continuous variables, that provides better standard error estimation under deviations from normality (56). Factor correlations were estimated using Pearson’s r coefficient (56).

● Convergent and divergent validity (i.e., the capacity of a questionnaire to have high correlations with measures of related constructs and weak correlations with measures of unrelated constructs) (56) were evaluated calculating Pearson’s correlations between the WHO QR Attitudes, the subscales of the Community Attitudes Toward the Mentally Ill (CAMI), and the subscales of the Attribution Questionnaire (AQ). It was hypothesized that higher scores on the WHO QR Attitudes (showing negative attitudes) would be associated with the following: a) higher scores on the Authoritarianism and Social Restriction subscales (CAMI); b) lower scores on the Benevolence and Community Mental Health Ideology subscales (CAMI); c) higher scores on the Blame, Anger, Pity, Dangerousness, Fear, Avoidance, Segregation, and Coercion subscales (AQ); and d) lower scores on the Help subscale (AQ).

● For the World Health Organization’s QualityRights Practices questionnaire (WHO QR Practices):

● Confirmatory Factor Analysis was used to evaluate the two dimensions postulated (i.e., 1) practices related to substitute decision-making and coercion used by the respondent, and 2) level of agreement between the respondents and the mental health professionals working in their units in the use of practices related to substitute decision-making and coercion). This factor structure was evaluated using the same approach described for the factor analysis of the WHO QR Attitudes.

Reliability

In the present study, Cronbach’s alpha was used to assess the internal consistency of the final version of the instruments (after dropping items based on validity assessment) total scales and subscales. Acceptable values of alpha range from 0.70 to 0.95 (57, 58).

Test–retest reliability for the final versions of the instruments was estimated using Pearson’s and Spearman’s r coefficients.

Sample size and power calculation

Given that factor analyses (i.e., CFAs) were performed for all the questionnaires, we referred to this analysis when calculating the sample size. There is no consensus in the literature regarding what sample size is required when performing a CFA to keep power close to 0.80. The majority of researchers consider 5 to 10 subjects per item to provide sufficient power to conduct a CFA (59, 60). Simulation studies have shown that, when variables are normally distributed, an appropriate sample size for a simple CFA model is n = 150 (61).

However, to obtain a more precise estimate of the sample size to be used in the study, sample size calculations were performed using Monte Carlo simulations. The focus was on the model proposed for the CFA of the WHO QR Attitudes because this was the most complex model examined (and thus, its sample size should be sufficiently powered to perform the other, simpler, CFAs).

In the Monte Carlo simulations, 10,000 replications were used for each analysis to ensure that stability had been reached. The sample size calculation was based on the following criteria: 1) parameter and standard error biases do not exceed 10% for any parameter in the model; 2) standard error bias for one of the parameters for which power is being assessed (i.e., subscale at the questionnaire) does not exceed 5%; 3) coverage remains between 0.91 and 0.98; and 4) power is at least to 0.80 (61). Results were estimated from the three-factor proposed for the aforementioned CFA, assuming the lowest conventionally acceptable correlation coefficient of 0.3 to indicate a positive factor loading. Based on these criteria, we calculated that 280 participants were required.

Factor analyses and sample size calculations were performed with MPLUS 8.5 (62). Reliability analyses were performed with R. All other analyses were performed with SAS 9.4 (SAS Institute, Cary NC).

Results

Characteristics of the respondents

Overall, 393 persons agreed to participate of the 788 invited (49.9%). Forty-seven respondents were excluded because, even though they provided informed consent, they did not continue the survey. Twenty-six respondents who completed only the survey questions regarding sociodemographic variables were removed. In total, 320 participants (81.4%) completed at least some of the survey questions from the WHO QR questionnaires and were included in the analyses. There were differences between completers of at least some of the survey questions from the WHO QR questionnaires and “non-completers” (i.e., people who completed only the survey questions on sociodemographic variables) regarding gender (among completers 39.7% were women, 59.7% men, and 0.6%, other gender, whereas, among the 26 non-completers, 57.7% were women and 42.3% men) and median age (among completers, median age was 37, whereas, among the 26 non-completers, median age was 34.5).

More participants completed the WHO QR Attitudes (n = 320) because this was the first questionnaire to be presented in the survey. The WHO QR Knowledge (n = 283) was the second questionnaire. Only mental health professionals were invited to complete the WHO QR Practices (n = 81).

Supplementary Table 4 shows respondents’ (i.e., completers of at least some of the survey questions from the WHO QR questionnaires) characteristics. A total of 39.7% of participants identified as women, 59.7% as men, and 0.6% as other gender. The median age was 37 (mean of 39.3 and standard deviation of 10.2). A total of 10% of participants had a high school degree or an equivalent title, 10% had some college, 28.1% had a college degree, 27.8% a master’s degree and 7.8% a professional degree (the education level in the country is lower as would be expected due to the participants’ selection criteria). Respondents also reported their “main” background (they could select only one option among the ones proposed): 27.5% were mental health practitioners; 17.8% were other health practitioners; 15.9% were persons with a psychosocial disability or mental health condition; 15.0% were persons with other disabilities; 10.9% were human rights advocates; 4.1% were family members or care partners; and the remaining were academics, lawyers, and policymakers. All the Ghanaian regions (except for North-East) were represented, although most participants were from Greater Accra (51.6%). A total of 40.6% of respondents indicated English as their first language. In addition, 35.3% of the participants reported having a person with a psychosocial disability or mental health condition in their close family and 19.1% identified as a person with a psychosocial disability or mental health condition.

One hundred fifty-four participants answered a second test–retest assessment. A total of 113, 124, and 11 participants completed the WHO QR Knowledge, the WHO QR Attitudes, and the WHO QR Practices, respectively.

WHO QualityRights knowledge questionnaire

A total of 283 participants completed the WHO QualityRights knowledge questionnaire. There were no significant differences between the total sample (i.e., the 320 participants that completed at least some of the survey questionnaires from the WHO QR questionnaires) and completers of the WHO QR Knowledge questionnaire regarding gender (in the total sample, 39.7% were women, 59.7% men, and 0.6% other gender, whereas, among the WHO QR Knowledge completers, 38.2% were women 61.2%, men and 0.6% other gender) and median age (median age was 37 both in the total sample and among the WHO QR Knowledge completers).

Validity

Supplementary Table 5 shows the content validity assessment performed by the seven experts in human rights in mental health. The first column reports the evaluation of the items’ consistency, the second of the items’ clarity, the third of the items’ difficulty, and the fourth column describes if the item was deemed deserving to be included in the questionnaire. When all seven experts said, for instance, that one item was clear, that item got a score of 7/7. The maximum score for each item was 28. The questionnaire obtained an overall score of 961/1,036 when evaluated for the performance of its items. An average content validity index 0.92 was calculated from this score, indicating that the WHO QR Knowledge items were overall relevant and clear. Item QR19_K received a low score (18/28) and, after discussion with the experts, was removed from the questionnaire. This item was meant to convey the idea that people cannot be detained (in health facilities) because of their disability (that may lead others to worry about their “social dangerousness”). However, it was interpreted as “people with disabilities can be detained (as it happens for people without disabilities) if they met other criteria such as dangerousness,” and, thus, it was removed.

Supplementary Table 6 shows the face validity assessment performed by seven lay stakeholders. The first column reports the evaluation of the items’ clarity, the second describes if the items were easy to understand, whereas the third reports if the item was deemed relevant. The maximum score for each item was 21. The overall score of the questionnaire was 701/756 and all the items received high scores (ranging from 16 to 21), indicating that the questionnaire was clear, easy to understand, and relevant for the target population.

A Confirmatory Factor Analysis was conducted, fitting a model with one factor as postulated. The items that had factor loadings with values lower than 0.3 (items 1, 3, 12, 13, 16, 20, 23, 30, 32, 34, and 36) or correlated negatively with the underlying factor (items 12 and 18) were excluded from the final model. The final 24-item model showed an acceptable fit (RMSEA = 0.049; CFI = 0.92; TLI = 0.91), and all the factor loadings were supported (see Figure 1), with values greater than 0.3 and significant, indicating that the questionnaire is able to measure the construct of interest.

Figure 1

Figure 1. Confirmatory factor analysis path diagram for WHO QR Knowledge (final version). *Standardized loadings (and standard errors).

With respect to the known group validity, WHO QR Knowledge scores achieved by both the experts and nonexperts group displayed a non-normal distribution and ranged from 8 to 24 (median = 21, mean = 19.87, Confidence Level (CL) = 18.52–21.22, standard deviation = 3.62, lower quartile = 19, and upper quartile = 23) and 5 to 22 (median = 11, mean = 11.98, CL = 11.46–12.49, standard deviation = 4.15, lower quartile = 9, and upper quartile = 15). Participants in the “experts group” were found to be different from “nonexperts” in the hypothesized direction, with experts having higher scores on the WHO QR Knowledge than non-experts (Z = 7.3838, p-value < 0.0001), indicating that the questionnaire is able to differentiate people with knowledge about the rights of persons with psychosocial disabilities and mental health conditions and people without this knowledge.

Reliability

The internal consistency of the final version of the questionnaire, with 24 items, was obtained. The overall Cronbach’s alpha value for the total scale indicated an excellent reliability (0.90). Item analysis indicated that dropping additional items would not have improved reliability. The test–retest reliability suggests stability over time (Spearman’s r = 0.83; 95% CL: 0.77, 0.88).

The final version of the WHO QualityRights Knowledge Questionnaire had 24 items (see Table 1). The minimum score obtained by participants was 5 and the maximum was 24, with a mean of 12.81 ± 4.76 and a median of 12.

WHO QualityRights attitudes questionnaire

A total of 320 participants completed the WHO QualityRights Attitudes questionnaire. The total sample (i.e., the 320 participants that completed at least some of the survey questionnaires from the WHO QR questionnaires) coincides with the sample of completers of the WHO QR Attitudes questionnaire.

Validity

Supplementary Table 7 shows the content validity assessment performed by the seven experts in human rights in mental health on the items’ consistency, clarity, difficulty, and worthiness to be included in the questionnaire. The maximum score for each item was 28. When evaluated for the performance of its items on these four dimensions (item consistency with the content area, item wording clarity, items perceived difficulty, and whether the items should be included in instrument), the questionnaire obtained an overall score of 459/476 and an average content validity index of 0.96, indicating that the WHO QR Attitudes items were relevant and clear.

Supplementary Table 8 shows the face validity assessment performed by seven lay stakeholders. The first column reports the evaluation of the items’ clarity, the second describes if the items were easy to understand, whereas the third reports if the item was deemed relevant. The maximum score for each item was 21. The overall score of the questionnaire was 342/357, and all the items received high scores (ranging from 18 to 21), indicating that they were clear, easy to understand, and relevant for the population of interest.

As shown in Figure 2, we conducted a Confirmatory Factor Analysis, fitting a model with the three factors postulated (attitudes toward mental health services approach, attitudes toward involuntary and coercive practices, and attitudes toward people with psychosocial disabilities or mental health conditions as decision-makers and full members of society). The model showed an acceptable fit (RMSEA = 0.074; CFI = 0.94; TLI = 0.93), and all the factor loadings were supported, with values greater than 0.3 and significant, indicating that our assessment results are in agreement with the conceptual model and the relationships that we postulated for the construct of interest.

Figure 2

Figure 2. Confirmatory factor analysis path diagram for WHO QR Attitudes (final version). *Standardized loadings (and standard errors).

Convergent and divergent validity evidence is presented in Tables 4, 5. The results showed a moderate positive correlation between the WHO QR Attitudes questionnaire total score (whose higher score indicates negative attitudes) and the scores on the CAMI Authoritarianism and Social restrictiveness scales, a moderate negative correlation between the WHO QR Attitudes total score and the score at the CAMI Community mental health ideology scale, and a low negative correlation between the WHO QR Attitudes and the CAMI Benevolence subscale. The results are in the direction we had hypothesized.

Table 4

Table 4. Evaluation of convergent and divergent validity between WHO QualityRights Attitudes and Community Attitudes toward the Mentally III subscales (n = 261).

Table 5

Table 5. Evaluation of convergent and divergent validity between WHO QualityRights Attitudes and Attribution Questionnaire subscales (n = 242).

Similarly, the results showed a moderate positive correlation between the WHO QR Attitudes total score and the scores on the Attribution Questionnaire Coercion and Segregation subscales, and a low positive correlation between the QR questionnaire total score and the scores on the Attribution Questionnaire Dangerousness, Fear, Avoidance, and Anger subscales. A low positive correlation was found between the WHO QR Attitudes total score and the score at the Attribution Questionnaire Pity subscale. Attribution Questionnaire Blame and Help subscales correlation were negligible. The results also are in the direction we had hypothesized.

Reliability

The overall Cronbach’s alpha value for the total scale indicated a good reliability (0.86). The internal consistency of the subscales was moderate or good (Cronbach’s alpha of 0.61 for Subscale 1, 0.77 for Subscale 2, and 0.75 for Subscale 3). Item analysis indicated that dropping additional items would not have improved reliability. The test–retest reliability suggests stability over time (Pearson’s r = 0.84; 95% CL, 0.78, 0.89).

The final version of the WHO QualityRights Attitudes Questionnaire coincides with the original version (see Table 2) and has 17 items. The minimum score obtained by participants was 17 and the maximum was 76, with a mean of 43.51 ± 11.25 and a median of 45.

WHO QualityRights practices questionnaire

Eighty-nine participants reported that they worked in a mental health facility and all of them were asked to complete the WHO QR Practices questionnaire. Among these, 81 participants completed the questionnaire.

There were no meaningful differences between the total sample of participants working in a mental health facility and completers of the WHO QR Practices questionnaire regarding median age (median age was 34 both in the total sample of participants working in a mental health facility and among WHO QR Practices completers). There were slight differences regarding gender (in the total sample of participants working in a mental health facility, women were 51.7%, men were 47.2%, and other gender were 1.1%, whereas, among WHO QR Practices completers, women were 54.3%, men were 44.5%, and other gender were 1.2%).

Validity

Supplementary Table 9 shows that the WHO QR Practices obtained an overall score of 328/336 for the performance of its items on the four content validity dimensions (item consistency with the content area, item wording clarity, items perceived difficulty, and whether the items should be included in instrument). An average content validity index of 0.976 was calculated from the overall score, indicating that the WHO QR Practices items were relevant and clear.

Supplementary Table 10 shows that the overall score of the questionnaire for face validity was 242/252. All the items received high scores (ranging from 18 to 21), indicating that the items were clear, easy to understand, and relevant.

We conducted a Confirmatory Factor Analysis, fitting a model with the two factors postulated (practices related to substitute decision-making and coercion used by the respondent, and level of agreement—between the respondents and the mental health professionals working in their units—in the use of practices related to substitute decision-making and coercion). The only three items depicting positive practices (items 4, 5, and 11) had factor loadings with values lower than 0.3 and were excluded from the final model. The final nine-item model showed an acceptable fit (RMSEA = 0.070; CFI = 0.99; TLI = 0.99), and all the factor loadings were supported (see Figure 3), with values greater than 0.3 and significant, indicating that our assessment results are in agreement with the two factors conceptual model and the relationships we postulated for them.

Figure 3

Figure 3. Confirmatory factor analysis path diagram for WHO QR Practices (final version). *Standardized loadings (and standard errors).

Reliability

The Cronbach’s alpha value for the total scale indicated a good reliability (0.75). The internal consistency was moderate (Cronbach’s alpha of 0.74) for Subscale 1 and good for Subscale 2 (Cronbach’s alpha of 0.89). Item analysis indicated that dropping additional items would not have improved reliability.

Only 11 participants completed the questionnaire after 10 days, so it was not possible to evaluate the test–retest reliability. It is worth mentioning that data for the validation of the questionnaires were collected during a period when mental health workers were on strike and the stressful situation may have had a negative impact on their willingness to continue participating in the study.

The final version of the WHO QualityRights Practices Questionnaire had nine items: subscale A had five items, and subscale B had four items (See Table 3). For subscale A, the minimum score obtained by participants was 5 and the maximum was 26, with a mean of 9.33 ± 4.99 and a median of 7. For subscale B, the minimum score obtained by participants was 4 and the maximum was 20, with a mean of 11.22 ± 4.12 and a median of 12.

Discussion

This study presents three recently designed questionnaires: the World Health Organization’s QualityRights Knowledge questionnaire (WHO QR Knowledge), the World Health Organization’s QualityRights Attitudes questionnaire (WHO QR Attitudes), and the World Health Organization’s QualityRights Practices questionnaire (WHO QR Practices). The analyses conducted indicate that the three questionnaires are valid and reliable instruments to evaluate the knowledge about the rights of persons with mental health conditions and psychosocial disabilities, the attitudes toward their role as rights holders, and mental health professionals’ practices related to substitute decision-making and coercion.

According to the experts engaged, the questionnaires showed excellent content validity. Their items were found to be consistent with the content area, clear, not difficult to understand, and deserving to be included in the instruments. Only one item of the WHO QR Knowledge questionnaire was deemed inadequate and removed. Face validity was also found to be high, with stakeholders from the general population reporting that the items were clear, not difficult, and worthy of being included in the questionnaires. This indicates that the questionnaires contain clear and relevant questions, that capture the domains of interest.

As postulated, the results from the CFAs indicated that an adequate one-factor model should be favored for the WHO QR Knowledge, a three-factor model for the WHO QR Attitudes, and a two-factor model for the WHO QR Practices. For the WHO QR Knowledge, the CFA revealed that 12 items did not contribute adequately to the scale, and, consequently, they were removed. All the other items contributed significantly to the scale, whose construct validity was adequate. The CFA of the WHO QR Attitudes revealed that every item in each of the three subscales (i.e., attitudes toward mental health services approach, attitudes toward involuntary and coercive practices, and attitudes toward people with psychosocial disabilities and mental health conditions as decision-makers and full members of society) contributed significantly to the respective subscales dimension and that the construct validity of each subscale was adequate. For the WHO QR Practices, the CFA revealed that the three items describing positive practices did not contribute adequately to their corresponding subscale; thus, they were removed. However, all other items in each of the two subscales (practices related to substitute decision-making and coercion used by the respondent, and level of agreement—between the respondents and the mental health professionals working in their units—in the use of practices related to substitute decision-making and coercion) contributed significantly to the respective subscales dimension, showing that their construct validity was adequate.

Additionally, the WHO QR Knowledge was able to differentiate between experts in human rights in mental health and non-experts. This supported the known group validity of the questionnaire, indicating that this instrument can be used to measure the construct of interest (i.e., knowledge about the rights of persons with mental health conditions and psychosocial disabilities).

Furthermore, for the WHO QR Attitudes, convergent validity estimates support that this questionnaire does measure what it is supposed to. As expected, high scores on the WHO QR Attitudes, which indicate negative attitudes, were positively correlated with high scores on the Authoritarianism and Social restrictiveness subscales of the CAMI, which reflect a view of persons with mental health conditions and psychosocial disabilities as a threat to society requiring coercive handling. Conversely, high scores on the WHO QR Attitudes were negatively correlated to high scores on the Benevolence and Community mental health ideology subscales of the CAMI, which reflect, respectively, a sympathetic view of persons with mental health conditions and psychosocial disabilities, based on humanistic principles, and the acceptance of their inclusion in the community. Similarly, high scores on the WHO QR Attitudes were positively correlated to high scores on the AQ subscales that indicate endorsement of negative stereotypes and discriminatory practices (i.e., Coercion, Segregation, Dangerousness, Fear, Avoidance, and Anger).

Interestingly, high scores on the WHO QR Attitudes were also positively correlated to the subscale indicating endorsement of Pity, considered in the AQ and other attitudes scales, as well as more generally, a “benevolent feeling” of sympathy toward people facing challenges due to their mental health condition and psychosocial disability. This understanding of disability was the tenet of outdated models of disability, such as the medical and the charity models, according to which people with disabilities have an impairment and, for this reason, they should be objects of pity and receive charitable resources for support. The WHO QR Attitudes was developed in line with the human rights–based approach of the CRPD, which considers people with disabilities as active and valuable members of society and not passive recipients of charity and objects of pity. The positive correlation between high scores on the WHO QR Attitudes and high scores on the AQ Pity subscale further demonstrates that the WHO QR Attitudes is different from previous scales in that its evaluation of attitudes is in line with the human rights–based approach.

The internal consistency for the three questionnaires was good, with overall Cronbach’s alpha values for the total scales of 0.90, 0.86, and 0.75 for the WHO QR Knowledge, the WHO QR Attitudes, and the WHO QR Practices, respectively. The internal consistency of the questionnaires subscales was also moderate or good, with Cronbach’s alpha values ranging from 0.61 to 0.89. The test–retest reliability suggested that both the WHO QR Knowledge and the WHO QR Attitudes are stable over time, whereas it was impossible to evaluate the long-term reliability of the WHO QR Practices because only a few mental health professionals completed this questionnaire during the re-assessment.

This study has several strengths. First, it is the first study to rigorously develop and investigate the psychometric properties of three instruments that follow the human rights approach and language promoted by the United Nations Convention on the Rights of Persons with Disabilities. Content, face, and construct validation methods and reliability testing were conducted to ensure these instruments were valid and reliable in evaluating the dimensions of interest. Second, persons with mental health conditions and psychosocial disabilities were involved in all the instruments’ development and validation phases, in line with the UN CRPD requirements. Third, to our knowledge, the WHO QR Knowledge and the WHO QR Practices are the first validated instruments to specifically evaluate the knowledge regarding persons with mental health conditions and psychosocial disabilities’ human rights as listed in the UN CRPD and the practices related to substitute decision-making and coercion, respectively. Similarly, the WHO QR Attitudes is the first validated instrument to focus specifically on assessing attitudes toward people with mental health conditions and psychosocial disabilities’ role as rights-holders.

Some limitations should also be considered in interpreting the results of this study. First, a convenience sample was used, formed by people from Ghana and it was not possible to collect information on people who refused to participate in the study. This might impose limitations on the generalizability of the findings. A second potential limitation is related to self-reporting and the possibility that participants may have underreported negative attitudes and practices related to substitute decision-making and coercion due to social desirability bias. Third, the test–retest reliability of the WHO QR Practices could not be evaluated due to the limited number of respondents at reassessment. Forth, we used Cronbach’s alpha to assess internal consistency. Although Cronbach’s alpha is a widely used measure for this purpose, it assumes that all items on the scale have equal factor loadings (or are tau-equivalent). When this is not the case, the use of Cronbach’s alpha may lead to an inaccurate estimate of internal consistency. Future studies should be conducted with representative samples and different populations to check if the findings from this research are generalizable. Measures of social desirability such as the Marlowe–Crowne Social Desirability Scale could be used to evaluate if participants may have underreported negative attitudes and practices. The stability over time of the WHO QR Practices should be investigated in a study targeting mental health professionals at recruitment. Furthermore, additional approaches could be used to evaluate internal consistency (e.g., congeneric measurement models).

The World Health Organization’s QualityRights Knowledge questionnaire (WHO QR Knowledge), the World Health Organization’s QualityRights Attitudes questionnaire (WHO QR Attitudes), and the World Health Organization’s QualityRights Practices questionnaire (WHO QR Practices) displayed excellent reliability and validity. This finding lends support to their use both within mental health services and in the general population for a better understanding of current knowledge, attitudes, and practices related to a human rights–based approach to mental health in mental health services and the community.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors upon reasonable request. Requests to access the datasets should be directed to MFM, bWZtb3JvQGdtYWlsLmNvbQ==.

Ethics statement

The studies involving humans were approved by Ghana Health Services Ethics Review Committee (study protocol approval: GSHERC 005/01/21. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided informed consent electronically to participate in the study.

Author contributions

MFM: Conceptualization, Data curation, Formal analysis, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing. LG: Resources, Writing – review & editing. ES: Methodology, Supervision, Writing – review & editing. JA: Resources, Writing – review & editing. JK: Methodology, Supervision, Writing – review & editing. CA: Resources, Writing – review & editing. OG: Methodology, Writing – review & editing. AO: Resources, Writing – review & editing. AN: Formal analysis, Methodology, Writing – review & editing. DT: Project administration, Resources, Writing – review & editing. ND: Resources, Writing – review & editing. HK: Project administration, Resources, Writing – review & editing. FB: Resources, Writing – review & editing. S-aO: Resources, Writing – review & editing. NA: Project administration, Resources, Writing – review & editing. AF: Project administration, Resources, Writing – review & editing. MA: Project administration, Resources, Writing – review & editing. SD: Project administration, Resources, Writing – review & editing. MC: Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing – review & editing. MF: Methodology, Resources, Supervision, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. The work presented was supported by the project “Empowering Persons with Psychosocial Disabilities to Fight for their Rights: An implementation of the CRPD and QualityRights principles in Ghana, Lebanon, and Armenia” (EIDHR/2018/155232-4/97), funded by the European Commission. Additional financial support was received through the project QualityRights, funded by the Sardinia Region (2023-UNCACLE-0291517). The work of Dr. Norcini Pala was supported by NIMH (K01-MH125724 and R01MH131177).

Acknowledgments

The authors would like to acknowledge and thank all mental health professionals, members of organizations of persons with disabilities, human rights advocates and experts, academics, and government members who participated in the study and the WHO QualityRights in Mental health Ghana Team (including members of the Christian Health Association of Ghana, Ghana Health Service, BasicNeeds Ghana, Ta-Excel Foundation, Inclusion Ghana, Special Olympics Ghana, Passion for Total Care, Mental Health Society of Ghana, and MindFreedom Ghana) for its work in implementing the WHO QualityRights initiative in Ghana (including the use of the instruments evaluated in the present study). The first author would also like to acknowledge Sharon Schwartz for her insightful input, the thoughtful guidance, and the warm mentorship during the development of this manuscript and beyond.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer FS declared a shared affiliation with authors MA, SD’O, and MC to the handling editor at the time of review.

The author(s) declared that they were an editorial board member of Frontiers, at the time of submission. This had no impact on the peer review process and the final decision.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2024.1435608/full#supplementary-material

References

1. Kleinman A. Global mental health: a failure of humanity. Lancet. (2009) 374:603–4. doi: 10.1016/S0140-6736(09)61510-5

PubMed Abstract | Crossref Full Text | Google Scholar

2. Patel V, Kleinman A, Saraceno B. Protecting the human rights of people with mental disorders: a call to action for global mental health. In: Michael Dudley DS, Gale F, editors. Mental health and human rights - Vision, praxis, and courage. Oxford University Press (2012). p. 362–75.

Google Scholar

3. Drew N, Funk M, Tang S, Lamichhane J, Chávez E, Katontoka S, et al. Human rights violations of people with mental and psychosocial disabilities: an unresolved global crisis. Lancet. (2011) 378:1664–75. doi: 10.1016/S0140-6736(11)61458-X

PubMed Abstract | Crossref Full Text | Google Scholar

4. Mfoafo-M'Carthy M. Human rights violations and mental illness: implications for engagement and adherence. SAGE Open. (2014) 4:1–18. doi: 10.1177/2158244014526209

Crossref Full Text | Google Scholar

5. Patel V, Saxena S, Lund C, Thornicroft G, Baingana F, Bolton P, et al. The Lancet Commission on global mental health and sustainable development. Lancet. (2018) 392:1553–98. doi: 10.1016/S0140-6736(18)31612-X

PubMed Abstract | Crossref Full Text | Google Scholar

6. Erdman JM. Human rights education in patient care. Public Health Rev. (2017) 38:14. doi: 10.1186/s40985-017-0061-8

PubMed Abstract | Crossref Full Text | Google Scholar

7. Hunt P. The health and human rights movement: progress and obstacles. J Law Med. (2008) 15:714–24.

PubMed Abstract | Google Scholar

8. Nordt C, Rossler W, Lauber C. Attitudes of mental health professionals toward people with schizophrenia and major depression. Schizophr Bull. (2006) 32:709–14. doi: 10.1093/schbul/sbj065

PubMed Abstract | Crossref Full Text | Google Scholar

9. Loch AA, Hengartner MP, Guarniero FB, Lawson FL, Wang YP, Gattaz WF, et al. The more information, the more negative stigma towards schizophrenia: Brazilian general population and psychiatrists compared. Psychiatry Res. (2013) 205:185–91. doi: 10.1016/j.psychres.2012.11.023

PubMed Abstract | Crossref Full Text | Google Scholar

10. Angermeyer MC, Dietrich S. Public beliefs about and attitudes towards people with mental illness: a review of population studies. Acta Psychiatrica Scandinavica. (2006) 113:163–79. doi: 10.1111/j.1600-0447.2005.00699.x

PubMed Abstract | Crossref Full Text | Google Scholar

11. Schomerus G, Schwahn C, Holzinger A, Corrigan PW, Grabe HJ, Carta MG, et al. Evolution of public attitudes about mental illness: a systematic review and meta-analysis. Acta Psychiatrica Scandinavica. (2012) 125:440–52. doi: 10.1111/j.1600-0447.2012.01826.x

PubMed Abstract | Crossref Full Text | Google Scholar

12. Cohen A, Minas H. Global mental health and psychiatric institutions in the 21st century. Epidemiol Psychiatr Sci. (2017) 26:4–9. doi: 10.1017/S2045796016000652

PubMed Abstract | Crossref Full Text | Google Scholar

13. Bettinghaus EP. Health promotion and the knowledge-attitude-behavior continuum. Prev Med. (1986) 15:475–91. doi: 10.1016/0091-7435(86)90025-3

PubMed Abstract | Crossref Full Text | Google Scholar

14. Schwartz NE. Nutrition knowledge, attitudes and practices of Canadian public health nurses. J Nutr Educ. (1976) 8:28–31. doi: 10.1016/S0022-3182(76)80113-6

Crossref Full Text | Google Scholar

15. Baranowski T, et al. The knowledge, attitudes, & behaviors approach how to evaluate performance and learning in complex environments. Performance Improvement. 2004. v43 n9 p8-15

Google Scholar

16. Valente TW, Paredes P, Poppe PR. Matching the message to the process: the relative ordering of knowledge, attitudes, and practices in behavior change research. Hum Commun Res. (1998) 24:366–85. doi: 10.1111/j.1468-2958.1998.tb00421.x

PubMed Abstract | Crossref Full Text | Google Scholar

17. Alexander PA, Dochy PJRC. Conceptions of knowledge and beliefs - a comparison across varying cultural and educational communities. Am Educ Res J. (1995) 32:413–42. doi: 10.3102/00028312032002413

Crossref Full Text | Google Scholar

18. Schrader PG, Lawless KA. The knowledge, attitudes, & behaviors approach how to evaluate performance and learning in complex environments. Perform Improve. (2004) 43:8–15. doi: 10.1002/(ISSN)1930-8272

Crossref Full Text | Google Scholar

19. Ajzen I. Attitude theory and the attitude-behavior relation. In: Krebs D, Schmidt P, editors. New directions in attitude measurement. Walter de Gruyter, New York (1993). p. 41–57.

Google Scholar

20. Tillyard G, DeGennaro V. New methodologies for global health research: improving the knowledge, attitude, and practice survey model through participatory research in Haiti. Qual Health Res. (2019) 29:1277–86. doi: 10.1177/1049732318816675

PubMed Abstract | Crossref Full Text | Google Scholar

21. Tolvanen M, Lahti S, Poutanen R, Seppä L, Pohjola V, Hausen H. Changes in children's oral health-related behavior, knowledge and attitudes during a 3.4-yr randomized clinical trial and oral health-promotion program. Eur J Oral Sci. (2009) 117:390–7. doi: 10.1111/j.1600-0722.2009.00640.x

PubMed Abstract | Crossref Full Text | Google Scholar

22. Mulango ID, Atashili J, Gaynes BN, Njim T. Knowledge, attitudes and practices regarding depression among primary health care providers in Fako division, Cameroon. BMC Psychiatry. (2018) 18:66. doi: 10.1186/s12888-018-1653-7

PubMed Abstract | Crossref Full Text | Google Scholar

23. Bisallah CI, Rampal L, Lye MS, Mohd Sidik S, Ibrahim N, Iliyasu Z, et al. Effectiveness of health education intervention in improving knowledge, attitude, and practices regarding Tuberculosis among HIV patients in General Hospital Minna, Nigeria - A randomized control trial. PloS One. (2018) 13:e0192276. doi: 10.1371/journal.pone.0192276

PubMed Abstract | Crossref Full Text | Google Scholar

24. Chen J, Zhao H, Xia Z, Zhang Y, Lv X, Zhou X, et al. Knowledge, attitude, and practice toward the daily management of PICC in critically ill cancer patients discharged from intensive care units. Cancer Manag Res. (2018) 10:1565–74. doi: 10.2147/CMAR

PubMed Abstract | Crossref Full Text | Google Scholar

25. Shrestha A, Bhattarai D, Thapa B, Basel P, Wagle RR. Health care workers' knowledge, attitudes and practices on tuberculosis infection control, Nepal. BMC Infect Dis. (2017) 17:724. doi: 10.1186/s12879-017-2828-4

PubMed Abstract | Crossref Full Text | Google Scholar

26. Scottish Human Rights Commission. Human rights in a healthcare setting: Making it work for everyone: An evaluation of a human rights-based approach at The State Hospital. Glasgow: Scottish Human Rights Commission (2009) p. 5–6.

Google Scholar

27. Office for Public Management. The impact of a human rights culture on public sector organizations: Lessons from practice. Manchester: Equality and Human Rights Commission (2009) p. 5–99.

Google Scholar

28. Spiers K, Shafii S, Campbell A, Callaghan P. North Lanarkshire dementia demonstrator site evaluation report. North Lanarkshire: National Health Service North Lanarkshire (2013) p. 1–44.

Google Scholar

29. Glowa-Kollisch S, Graves J, Dickey N, MacDonald R, Rosner Z, Waters A, et al. Data-driven human rights: using dual loyalty trainings to promote the care of vulnerable patients in jail. Health Hum Rights. (2015) 17:124–35. doi: 10.2307/healhumarigh.17.1.124

Crossref Full Text | Google Scholar

30. Feldman MA, Owen F, Andrews A, Hamelin J, Barber R, Griffiths D. Health self-advocacy training for persons with intellectual disabilities. J Intell Disabil Res. (2012) 56:1110–21. doi: 10.1111/j.1365-2788.2012.01626.x

Crossref Full Text | Google Scholar

31. Steinert C, Steinert T, Flammer E, Jaeger S. Impact of the UN convention on the rights of persons with disabilities (UN-CRPD) on mental health care research - a systematic review. BMC Psychiatry. (2016) 16:166. doi: 10.1186/s12888-016-0862-1

PubMed Abstract | Crossref Full Text | Google Scholar

32. Henderson C, Noblett J, Parke H, Clement S, Caffrey A, Gale-Grant O, et al. Mental health-related stigma in health care and mental health-care settings. Lancet Psychiatry. (2014) 1:467–82. doi: 10.1016/S2215-0366(14)00023-6

PubMed Abstract | Crossref Full Text | Google Scholar

33. Gronholm PC, Henderson C, Deb T, Thornicroft G. Interventions to reduce discrimination and stigma: the state of the art. Soc Psychiatry Psychiatr Epidemiol. (2017) 52:249–58. doi: 10.1007/s00127-017-1341-9

PubMed Abstract | Crossref Full Text | Google Scholar

34. Thornicroft G, Mehta N, Clement S, Evans-Lacko S, Doherty M, Rose D, et al. Evidence for effective interventions to reduce mental-health-related stigma and discrimination. Lancet. (2016) 387:1123–32. doi: 10.1016/S0140-6736(15)00298-6

PubMed Abstract | Crossref Full Text | Google Scholar

35. Mittal D, Sullivan G, Chekuri L, Allee E, Corrigan PW. Empirical studies of self-stigma reduction strategies: A critical review of the literature. Psychiatr Serv. (2012) 63:974–81. doi: 10.1176/appi.ps.201100459

PubMed Abstract | Crossref Full Text | Google Scholar

36. Yamaguchi S, Wu SI, Biswas M, Yate M, Aoki Y, Barley EA, et al. Effects of short-term interventions to reduce mental health-related stigma in university or college students: a systematic review. J Nerv Ment Dis. (2013) 201:490–503. doi: 10.1097/NMD.0b013e31829480df

PubMed Abstract | Crossref Full Text | Google Scholar

37. Strang J, Hunt C, Gerada C, Marsden J. What difference does training make? A randomized trial with waiting-list control of general practitioners seeking advanced training in drug misuse. Addiction. (2007) 102:1637–47. doi: 10.1111/j.1360-0443.2007.01929.x

PubMed Abstract | Crossref Full Text | Google Scholar

38. Schomerus G. Public attitudes towards protecting the human rights of people with mental illness: a scoping review and data from a population trend study in Germany. Int Rev Psychiatry. (2022) 35:167–79. doi: 10.1080/09540261.2022.2087494

PubMed Abstract | Crossref Full Text | Google Scholar

39. Fox AB, Earnshaw VA, Taverna EC, Vogt D. Conceptualizing and measuring mental illness stigma: the mental illness stigma framework and critical review of measures. Stigma Health. (2018) 3:348–76. doi: 10.1037/sah0000104

PubMed Abstract | Crossref Full Text | Google Scholar

40. HRW. Submission to the Universal Periodic Review of Ghana. (2022) (New York: Human Rights Watch).

Google Scholar

41. HRW. Ghana: People With Mental Disabilities Face Serious Abuse. Take Steps to Improve Plight of Mentally Disabled. New York: Human Rights Watch (2012). Available at: https://www.hrw.org/news/2012/10/02/Ghana-people-mental-disabilities-face-serious-abuse.

Google Scholar

42. HRW. Ghana: Oversight Needed to Enforce Shackling Ban. Monitoring, Community Mental Health Services Needed to End Abuse. New York: Human Rights Watch (2018). Available at: https://www.hrw.org/news/2018/10/09/Ghana-oversight-needed-enforce-shackling-ban.

Google Scholar

43. HRW. Like a Death Sentence. In: Abuses against Persons with Mental Disabilities in Ghana. New York: Human Rights Watch (2012).

Google Scholar

44. Moro MF, Carta MG, Gyimah L, Orrell M, Amissah C, Baingana F, et al. A nationwide evaluation study of the quality of care and respect of human rights in mental health facilities in Ghana: results from the World Health Organization QualityRights initiative. BMC Public Health. (2022) 22:639. doi: 10.1186/s12889-022-13102-2

PubMed Abstract | Crossref Full Text | Google Scholar

45. Moro MF, Kola L, Fadahunsi O, Jah EM, Kofie H, Samba D, et al. Quality of care and respect of human rights in mental health services in four West African countries: collaboration between the mental health leadership and advocacy programme and the World Health Organization QualityRights initiative. BJPsych Open. (2022) 8:e31. doi: 10.1192/bjo.2021.1080

PubMed Abstract | Crossref Full Text | Google Scholar

46. Johnson DR, Creech JC. Ordinal measures in multiple indicator models: A simulation study of categorization error. Am Sociol Rev. (1983) 48:98–407. doi: 10.2307/2095231

Crossref Full Text | Google Scholar

47. Sullivan GM, Artino AR Jr. Analyzing and interpreting data from likert-type scales. J Grad Med Educ. (2013) 5:541–2. doi: 10.4300/JGME-5-4-18

PubMed Abstract | Crossref Full Text | Google Scholar

48. Taylor SM, Dear MJ. Scaling community attitudes toward the mentally ill. Schizophr Bull. (1981) 7:225–40. doi: 10.1093/schbul/7.2.225

PubMed Abstract | Crossref Full Text | Google Scholar

49. Barke A, Nyarko S, Klecha D. The stigma of mental illness in Southern Ghana: attitudes of the urban population and patients' views. Soc Psychiatry Psychiatr Epidemiol. (2011) 46:1191–202. doi: 10.1007/s00127-010-0290-3

PubMed Abstract | Crossref Full Text | Google Scholar

50. Corrigan P, Markowitz FE, Watson A, Rowan D, Kubiak MA. An attribution model of public discrimination towards persons with mental illness. J Health Soc Behav. (2003) 44:162–79. doi: 10.2307/1519806

PubMed Abstract | Crossref Full Text | Google Scholar

51. Fayers PM, Machin D. Quality of Life: The Assessment, Analysis and Interpretation of Patient-Reported Outcomes. 2nd edition. West Sussex, England: John Wiley & Sons Ltd (2007).

Google Scholar

52. DeVon HA, Block ME, Moyle-Wright P, Ernst DM, Hayden SJ, Lazzara DJ, et al. A psychometric toolbox for testing validity and reliability. J Nurs Scholarsh. (2007) 39:155–64. doi: 10.1111/j.1547-5069.2007.00161.x

PubMed Abstract | Crossref Full Text | Google Scholar

53. Downing SM, Haladyna TM. Handbook of Test Development. Mahwah,NJ: Lawrence Erlbaum Associates (2011). doi: 10.4324/9780203874776

Crossref Full Text | Google Scholar

54. Sowtali SN, Yusoff DM, Harith S, Mohamed M. Translation and validation of the Malay version of the Stroke Knowledge Test. J Arrhythm. (2016) New York 32:112–8. doi: 10.1016/j.joa.2015.10.003

PubMed Abstract | Crossref Full Text | Google Scholar

55. Yaghmaei F. Content validity and its estimation. J Med Educ. (2003) 3:25–7. doi: 10.22037/jme.v3i1.870

Crossref Full Text | Google Scholar

56. Raykov T, Marcoulides GA. Introduction to Psychometric Theory. Routledge (2011). doi: 10.4324/9780203841624

Crossref Full Text | Google Scholar

57. Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. (2011) 2:53–5. doi: 10.5116/ijme.4dfb.8dfd

PubMed Abstract | Crossref Full Text | Google Scholar

58. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. (1977) 33:159–74. doi: 10.2307/2529310

PubMed Abstract | Crossref Full Text | Google Scholar

59. Dimitrov DM. Statistical Methods for Validation of Assessment Scale Data inCounseling and Related Fields. Alexandria, VA: A.C. Association (2012).

Google Scholar

60. DeVellis RF. Scale Development: Theory and Applications. 4th ed. ThousandOaks, CA: Sage (2017).

Google Scholar

61. Muthén LK, Muthén BO. How to use a Monte Carlo study to decide on sample size and determine power. Struct Equation Model. (2002) 9:599–620. doi: 10.1207/S15328007SEM0904_8

Crossref Full Text | Google Scholar

62. Muthén LK, Muthén BO. Mplus User's Guide. (1998-2017) (Los Angeles: Muthén & Muthén).

Google Scholar