Abstract
Background: Children in mental health inpatient care require multiple treatments. There is not a comprehensive instrument to assess perceived helpfulness of this combination of interventions.
Aims: To develop and evaluate the psychometric properties of the What was Helpful Questionnaire (WHQ), a tool designed to capture parental perceived helpfulness of the multidimensional management approach used in inpatient children's units.
Methods: A total of 73 inpatients and their families were included in this study. The WHQ consists of six items exploring the perceived helpfulness of different aspects of care. Demographic and clinical variables were collected on admission and discharge. An exploratory factor analysis using polychoric correlations was performed to assess the item structure of the scale and the Cronbach's alpha coefficient was used for internal reliability. Associations were assessed using regressions models.
Results: WHQ is a unidimensional scale with an internal reliability of 0.77. No associations were identified between WHQ total score and age, gender, and Children's Global Assessment Scale scores change. A strong relationship between the WHQ total score and parental Acorn Satisfaction Questionnaire total score was found.
Conclusions: Results add evidence for the validity and the reliability of the WHQ to measure parental perceived helpfulness of interventions offered in inpatient children's units.
Introduction
The prevalence of psychopathology among children and adolescents is high (1) and its impact extends to adulthood affecting academic outcomes (2), increasing the risk of other mental health problems (2, 3) and reducing life satisfaction (4). There is some agreement that 1 in 5 children will suffer from a psychiatric condition before entering adulthood (5).
The clear majority of children and adolescents diagnosed with mental health conditions are treated in outpatient services. However, many young people have severe disorders difficult to manage in community settings and require inpatient input. In the United Kingdom (UK), most hospital mental health beds for under-18s are dedicated to adolescents as their clinical presentation is more frequently, compared to younger children, associated with challenges that need this higher intensity of care (6). Children's inpatient mental health units are a particularly specialized National Health Service provision for young people up to the age of 12 years with complex or/and severe mental health problems. These units offer comprehensive and individualized assessment and multidimensional treatment, including specialist educational input, that can lead to significant positive changes for both children and their families (7, 8). The multidimensional treatment includes behavioral management, psychological and family interventions and, commonly, medication.
There are many reasons behind admission in an inpatient service instead of outpatient treatment, as well as dilemmas regarding inpatient care. Commonly, in the case of conditions associated with significant clinical risk, and particularly retractable or even deteriorating symptoms despite the most intensive outpatient treatment, inpatient care can be offered as an alternative. Admission in an inpatient unit allows detailed assessment of complex presentations in a controlled setting, comprehensive observations, and the development of effective strategies to assist the child and their family manage their difficulties after discharge (9, 10). Importantly, it allows the introduction of a new treatment, such as medication, to be observed (10) and continuous risk management approaches to be implemented (e.g., in cases of self-harm or aggression) (9). Inpatient units can help children improve in their functioning, socialization and academic skills and can have a major impact on young people, who might have a history of difficulties regarding their social adaptation or/and school failure. The individualized assessment and intensive specialist treatment can lead to more effective use of other services in the future (9).
However, there are certain challenges that need to also be taken into consideration. Inpatient admission can have a negative impact on the family or the child due to the dislocation and loss of support from a familiar environment and peer group, especially in cases where units are distant from the children's homes. This situation is not uncommon, considering the aforementioned lack of capacity in inpatient children's mental health services and the uneven geographical distribution of existing units. Other concerns include potential “learning” of new symptoms from other young people as well as institutionalization if the admission is prolonged with subsequent dependency (9, 10).
Today standards of care require the assessment of patient reported outcomes (PROs), that is, outcomes directly reported by patients without further interpretation by clinicians, but also the incorporation of valid and reliable instruments or tools to evaluate patient perception with patient reported outcome measures (PROMs) (11). For example, it is mandatory in United Kingdom (UK), since 2009, to have PROMS for many health conditions (11). In recent years, clinical care has incorporated instruments or tools to measure functional status, health related quality of life, health behaviors (11, 12), patient satisfaction (13) and patient—reported experience measures (PREMs) (14–16).
Measures of patient's satisfaction have acquired significant importance in health care, particularly in treatment outcomes and intervention engagement (17–21) and constitute an indispensable aspect of direct evaluation of health care services and providers (22). Satisfaction is a broad multi-dimensional construct including several aspects of care: access, financial aspects, availability of resources, continuity of care, technical quality, interpersonal manners, and overall satisfaction (22). This evaluation is influenced not only by the actual experience of patients with services (e.g., waiting time in minutes) but also by their expectations (e.g., preferences for long or short office times). Several studies have explored factors related to high satisfaction (23–29) and low satisfaction with psychiatric services (30–33). Moreover, patient's satisfaction measurements have been included in different health systems, for example, United States, Canada, and UK (34), because they help to evaluate the quality of care, quality of services, patient experience, and other patient—based health outcome studies (35).
However, much less attention has received the assessment of the perceived helpfulness of different treatments by patients or their caregivers. Similarly to patient satisfaction, perceived helpfulness is a subjective evaluation; however, helpfulness seems to be a more specific construct. Perceived helpfulness is the perception of the quality of the service or assistance provided by others (people or technological advances) (36). In the medical field, most of the studies have linked helpfulness to treatments. There are few studies exploring psychometric features of questionnaires exploring treatment helpfulness (37, 38), even though the construct has been included in several services and treatment evaluations. For example, perceived helpfulness is likely to facilitate families' involvement, lead to more effective interventions, and improved outcome measures. Lee and colleagues (2010) reported that patients perceived the care and treatment to be helpful after experiencing improvements or changes in their physical (e.g., gastric relief, improved blood circulation, feeling more energetic), cognitive (e.g., reduction of anxiety, sense of calmness, security, self-awareness, hopefulness), or interpersonal (e.g., reduction of interpersonal conflicts) functioning (39). There are few studies exploring the factors influencing perceived helpfulness, but it appears that etiological information (attributed causes of mental health problems), past experiences, nature of treatments and expectations are important (40–42). Moreover, family and patients may have different perceptions of what is helpful. For example, family-based treatment for adolescent eating disorder was perceived as helpful; however, patients seemed to prioritize cognitive in comparation to physical improvements (43).
The majority of the studies exploring perceived helpfulness refers to treatment or services evaluations. For example, a study compared the patients' perceived helpfulness of depression treatment provided by general medical providers (GMPs) and specialty mental health providers (SMHPs) (44). It found that adults who received depression care from an SMHP or a combination of GMPs and SMHPs considered treatment for depression more helpful than adults who received care from a GMP. These findings highlight the importance of type of provider for mental health problems (44). A recent study compared different self-management strategies to deal with depression, finding that not always the most helpful strategies as they were perceived by the patients (completing the treatment, leaving the house regularly) were the most used (45). Rosemblat et al. studied the frequency of use and perceived helpfulness of less structured interventions for bipolar and unipolar depression (46). Wellness strategies, such as listening to music or having adequate sleep, were reported to be helpful and they were associated with perceived treatment effectiveness and greater subjective helpfulness of medication, psychotherapy, and peer support groups (46). In another study on depression among adolescents, counseling and medication were compared in regards to their helpfulness, finding that 32% to 47% of adolescents in the general population perceived depression treatment as extremely or a lot helpful which was lower than the response rate in clinical trials (47). Additionally, adolescents found that medication was more helpful than counseling (47). Regarding treatment adherence, perceived helpfulness may have a role. For example, in a study among 45 participants with schizophrenia spectrum disorders, perceived helpfulness of the previous treating psychiatrist and of previous medication and feeling insufficiently informed about medication significantly predicted medication discontinuation (48). In another study among children and young people with juvenile idiopathic arthritis, the helpfulness of the medication was associated with high adherence (49). Some studies had also examined the parental role of treatment adherence using helpfulness as an explanatory variable. For example, among children and adolescents with psychiatric conditions, medication adherence was greater in children and mothers when mothers felt that the medication helped to reduce the symptoms (50). Similarly, among patients with juvenile idiopathic arthritis factors associated with higher perceived adherence to medications included perceived helpfulness of medications and lower disease severity (51). All the above examples showed how the perceived helpfulness assessment has contributed in the evaluation of service providers, as well as different treatments, and their association to clinical outcomes and treatment adherence. Studying this construct may also help to personalize mental health care and treatments, which is one of the main challenges for the future (52).
There are, already, different tools to assess treatment progress among children and adolescents receiving support in inpatient facilities. For example, symptomatology can be assessed using parent reported symptoms questionnaires such as the Strengths and Difficulties Questionnaire (SDQ) (53), or clinician reported functionality questionnaires such as the Children's Global Assessment Scale (CGAS) (54) or the Health of the Nation Outcome Scales for Children and Adolescents (HoNOSCA) (55). There are also instruments used to assess client satisfaction regarding the care provided by mental health services, such as Acorn Satisfaction Questionnaire (7). So far, there has been no comprehensive tool to assess the caregivers' perceived helpfulness of different treatments offered in an inpatient mental health care service whether alone or in combination.
The aim of the current study was to develop and evaluate the psychometric properties of the What was Helpful Questionnaire (WHQ), a tool designed to capture parental perceived helpfulness of the multidimensional management approach used in inpatient mental health children's units. Given the high level of integration of different treatment approaches in this sample, we hypothesized that WHQ would assess one latent factor measuring parental perceived helpfulness.
Materials and Methods
Setting and Participants
Acorn Lodge provides inpatient assessment and treatment for children aged up to 12 years with severe and complex disorders. These include children with neuropsychiatric disorders such as autism spectrum disorders (ASD) and hyperkinetic disorders (HD), depression, very early onset psychosis and bipolar affective disorder, obsessive compulsive disorder, eating disorders, stress-related disorders, severe encopresis and complicated diagnostic conditions. The service is one of only eight child inpatient units in the UK. It is open seven days a week and offers planned and emergency admissions, including out of hours.
The members of the multidisciplinary team are experienced staff from a variety of professional backgrounds including specialist nurses, a psychologist, a social worker, a family therapist, an occupational therapist and child and adolescent psychiatrists. The ward runs a behavioral model supplemented by other treatment modalities including Cognitive Behavioral Therapy (CBT), family therapy, parenting work, Occupational Therapy (OT) (sometimes working with the child and mother in an action form of therapy, sometimes drama therapy) and pharmacotherapy. The Unit is supported by the Bethlem and Maudsley Hospital School and highly developed educational provision where the children are offered remedial education when appropriate. The school liaises closely with the child's school of origin and with the relevant Educational Authority to help plan appropriate resources for the child's future educational needs. This planning can require funding through complex panel procedures in local authorities. Close liaison with community agencies forms an integral part of the unit's work.
Parents are strongly involved in all aspects of treatment, where their knowledge of their children is valued and their opinions welcome, and participate in family assessments and therapy. Support groups and counseling are also offered for parents and siblings. Parents and family members visit and take children out of the unit as much as possible depending on the children's individual needs. The WHQ was designed in an attempt to capture parental perceived helpfulness of these complex interventions in a valid and reliable way.
Sample
All 73 inpatients discharged from Acorn Lodge from December 2013 to December 2016 were included in this project. All children and their parents are asked to complete a number of outcome and satisfaction measures upon discharge from the unit including the SDQ, CGAS, HONOSCA, ASQ and WHQ as part of their clinical care. The completion of these scales do not take longer than 20 min; however, in few occasions participants will be given as much time as required.
Analysis of the WHQ was part of a wider service evaluation project of anonymized data.
Measures
Socio-Demographic and Clinical Variables
Information on gender, age on admission, ethnicity, family composition, and length of stay (LOS) was collected from the children's clinical notes. We also collected information about receiving medication during the inpatient treatment.
Instruments
What was Helpful Questionnaire (WHQ) at discharge: Two of the authors (SF, MK) created the questionnaire, considering different aspects of care that children normally receive on the ward. The questionnaire was subsequently piloted with a group of parents who did not feel any other aspects of care should be added or any of the included aspects should be removed. The final WHQ (Appendix) consisted of 6 items, rating the perceived helpfulness of each aspect of care on a Likert scale (1 = Strongly Disagree to 5 = Strongly Agree) and was completed by parents at the end of a child's admission. The aspects of treatment included in the questionnaire are briefly described in Table 1. The total score of WHQ ranges from 6 to 30. A higher score means higher perceived helpfulness related to the inpatient care the child received.
Table 1
| Aspect of treatment | Description |
|---|---|
| Family work | Family work/therapy is provided by a Family therapist. This therapist assesses and works with the family using a systemic approach. |
| Work on parenting skills | Parents receive help to build or develop better parenting skills. This work is provided by all members of the multidisciplinary team. |
| Behavioral work for children on the ward | Behavioral work is provided by all members of the multidisciplinary team and constitute a big part of the input a child receives as part of the ward milieu. |
| Individual psychological work for children | Individual psychological work for the child is provided by a trained psychologist or a trainee psychologist under supervision. The main approach is cognitive behavioral therapy. |
| Medication | Medication treatment as part of a comprehensive treatment plan is prescribed by a Consultant Child and Adolescent Psychiatrist and carefully monitored by trained nursing staff. |
| Care Plans | Carefully designed care plans to assist with the child's day-to-day management are agreed with them and their families. Each child has a care coordinator and a nursing care team to ensure that all aspects of care are covered. |
Description of aspects of treatment.
Children's Global Assessment Scale scores (CGAS) on admission and at discharge: (54) The CGAS was originally created as an adaptation of the Global Assessment Scale for adults (56) and designed to reflect the lowest level of functioning for children during a specified time period. It has values from 1, representing the lowest level of functioning, to 100, representing the highest. Scores over 70 represent normal functioning. The CGAS has good interrater reliability (0.84) and test-retest stability (0.85) (54). It has been used extensively in clinical and research settings.
Health of the Nation Outcome Scale for Children and Adolescents (HoNOSCA) on admission and at discharge: (55) The HoNOSCA is a tool used by clinicians to assess clinical change in children and adolescents attending psychiatric clinics. The instrument comprises 15 simple scales measuring behavior, impairment, symptoms, social problems and information problems for those under 18 years of age. The interrater reliability is 0.82 for psychiatric symptoms.
Acorn Satisfaction Questionnaire (ASQ) at discharge: (8) This is a 9-item questionnaire, with seven items responded by parents and two by children, which has been previously used in this population (7). In this study, we only used the report from parents. The total score is the average of the sum of all items. In our sample, the score ranged from 1 to 5, with a mean score of 4.53 (Standard Deviation (SD) = 0.56). The internal reliability in our sample was 0.81.
Data Analysis
We examined the children's sociodemographic variables as well as the items' psychometric characteristics by using descriptive statistics (i.e., means, standard deviations, frequencies, percentages, skewness, and kurtosis).
There are several methodologies and approaches that can be implemented to add evidence of the validity of an instrument (57). All these approaches, from the analyses of the item structure (construct validity) to the discrimination between subjects (discriminant validity) and the agreement between different measures assessing the same construct (concurrent validity) provide different perspective of the performance and validity of an instrument. Generally, after the creation of a new instrument, the first analysis consists of the exploration of the dimensionality and item structure (communalities between items and a latent factor) of the scale using an exploratory factor analysis (EFA). Therefore, an EFA using polychoric correlations was performed. Polychoric correlations are advised for factor analysis when the distributions of ordinal items are asymmetric with excess of kurtosis or high item- total correlation (58). For factor extraction, we used principal axis factoring which is recommended if the assumption of multivariate normality is not completely fulfilled (59), with promax oblique rotation method which allows the factors to correlate (60). The selection of the number of factors was based on eigenvalues ≥1.0. To select the items to be included in the latent factor, we used the criterion of factor loading ≥0.32, cut off point corresponding to ~10% overlapping variance with the other items in the factor (60, 61). The total score of the questionnaire required that all items were answered; therefore, the analysis included completed data.
The Internal consistency was assessed using Cronbach's alpha coefficient. The final score was calculated using the average score for each item with valid response. Acceptable values of alpha range from 0.7 to 0.95 (62). For comparing groups, alpha between 0.7 and 0.8 is considered as satisfactory; however, a minimum of 0.90 is recommended for clinical applications (63).
We explored associations between perceived helpfulness and gender (male as reference), age, CGAS change (change score between baseline and discharge), HoNOSCA change (change score between baseline and discharge) and parental satisfaction using regression models. Due to the small sample size, we only performed univariable analyses to avoid multiple hypothesis testing (64, 65). In addition, and considering that not all patients received medication treatment, we performed a secondary analysis exploring perceived helpfulness using all other 5 items comparing parental responses of children receiving and not receiving medication during inpatient status.
The cut off for statistical significance was established at p < 0.05 and the confidence interval are reported. All analyses were performed in STATA 14.0. All the analyses were conducted using complete data.
Results
Sample Description
A total of 73 patients were included in this study (48%, females). Most of patients were White (72.6%), and the mean age at admission was 123.8 months (SD = 18.2). The average length of admission was 153 days (SD = 93). Most of the patients were receiving medication during their inpatient treatment (83.6%). A 53.4% of children did not live with two biological parents (ICD 10: anomalous parenting).
Regarding the mean CGAS score, it was 30.8 (SD = 11.3) at admission and 57.8 (SD = 14.3) at discharge. The mean HoNOSCA score was 20.8 (SD = 6.7) at admission and 11.3 (SD = 6.5) at discharge.
Regarding the WHQ, 52 (71.2%) carers answer all or some of the questions, while the ASQ was answered by 56 (76.6%) carers.
Psychometrics Properties of the WHQ
Table 2 shows the descriptive statistics of all six items and loading factors. The EFA found one factor with eigenvalue over 1, which explained 91% of the variance (Table 3). All items had a loading factor over the cut off used as reference.
Table 2
| Items | Mn | SD | Var | Skew | Kurt | w |
|---|---|---|---|---|---|---|
| Q1. Family work | 4.36 | 0.78 | 0.60 | −0.98 | 3.22 | 0.75 |
| Q2. Work on parenting skills | 3.77 | 0.91 | 0.84 | −0.73 | 3.58 | 0.77 |
| Q3. Behavioral work | 4.31 | 0.84 | 0.70 | −1.05 | 3.38 | 0.81 |
| Q4. Individual psychological work | 4.29 | 0.81 | 0.65 | −0.81 | 2.71 | 0.70 |
| Q5. Medication | 4.31 | 0.76 | 0.58 | −0.86 | 3.16 | 0.38 |
| Q6. Care plans | 4.31 | 0.84 | 0.70 | −1.67 | 6.82 | 0.59 |
| Total scale | 4.22 | 0.56 | 0.31 | −0.53 | 2.37 |
Descriptives and standardized factor loadings for What was Helpful Questionnaire (WHQ) items.
Md, median; SD, standard deviation; skew, skewness; kurt, kurtosis; w, weights or factor loadings.
Table 3
| Factor | Eigenvalue | Difference | Proportion | Cumulative |
|---|---|---|---|---|
| Factor1 | 2.79345 | 2.37947 | 0.9148 | 0.9148 |
| Factor2 | 0.41397 | 0.13933 | 0.1356 | 1.0504 |
| Factor3 | 0.27464 | 0.30749 | 0.0899 | 1.1403 |
| Factor4 | −0.03285 | 0.10049 | −0.0108 | 1.1295 |
| Factor5 | −0.13334 | 0.1289 | −0.0437 | 1.0859 |
| Factor6 | −0.26225 | −0.0859 | 1 |
Exploratory Factor analyses of the items of the What was Helpful Questionnaire (WHQ).
Table 4 shows the reliability of the scale WHQ. The item-test correlation scores were high for all items, with the exception of the item Medication (0.48). The scale has satisfactory internal consistency (0.77). Calculating the Cronbach's alpha of the scale without the item Medication and using the data of parents of children who did not receive medication, the alpha for the total scale was similar (0.77), however the item-test correlation score of the item Medication improved (0.50).
Table 4
| Items | Item-test correlation | Item-rest correlation | Avarage interitem covariance | Alpha |
|---|---|---|---|---|
| Q1. Family work | 0.75 | 0.61 | 0.23 | 0.71 |
| Q2. Work on parenting skills | 0.75 | 0.59 | 0.22 | 0.72 |
| Q3. Behavioral work | 0.81 | 0.68 | 0.21 | 0.69 |
| Q4. Individual psychological work | 0.71 | 0.54 | 0.24 | 0.73 |
| Q5. Medication | 0.48 | 0.25 | 0.31 | 0.80 |
| Q6. Care plans | 0.61 | 0.41 | 0.26 | 0.76 |
| Total scale | 0.24 | 0.77 |
Reliability of the What was Helpful Questionnaire (WHQ).
Association Analysis
No associations were identified between WHQ total score and age, gender, CGAS change or HoNOSCA change. Considering the mean value of the scale without the medication item (q5) we found that the parents of children receiving medication had a mean score of 4.2 (SD = 0.6), which was not statistically different from the mean score reported from parents of children not receiving medication (4.1, SD = 0.7).
The only strong relationship found was between the WHQ total score and parental ASQ total score was found (β = 0.65, p = 0.000) (See Table 5).
Table 5
| Variable | Beta | P-value | 95% CI | |
|---|---|---|---|---|
| Age | 0.001 | 0.789 | −0.008 | 0.010 |
| Gender (reference Male) | −0.002 | 0.991 | −0.315 | 0.312 |
| CGAS change | 0.0001 | 0.965 | −0.009 | 0.009 |
| HoNOSCA change | −0.003 | 0.836 | −0.027 | 0.022 |
| Parental satisfaction | 0.654 | 0.000 | 0.435 | 0.873 |
Results of the associations analyses.
Discussion
To the authors' knowledge, this is the first study adding evidence for the validity of a scale of perceived helpfulness for children receiving treatment in a mental health inpatient setting. We have found that the WHQ is an easy-to-use tool, with satisfactory psychometric properties, and that the construct of perceived helpfulness measured by this scale is unidimensional, where most items had high factor loadings.
Knowing that the impact of serious mental health conditions in children and their families can be severe and prolonged, it is important to comprehensively assess, not only the specific needs of care (psychological and pharmacological treatment), but also the perceived helpfulness of different aspects of care.
All patients received treatments and services included in the scale, with the exception of medication. A 16% of children did not receive medication, therefore, parents could not respond to this item. We suggest to re-arrange the order of the items in the scale, putting the item “Medication” at the end and after the statement: “If your child had received any medication on the ward, you consider that it was helpful” answering with the same Likert scale. A further evaluation of this amended scale would be warranted.
Each of the treatments provided by the unit is extensively explained to the parents and children, including information about the objective of the intervention, efficacy, potential benefits and risks. Other authors have shown that the beliefs of general public greatly influence the acceptance and perceived helpfulness from different treatments (66). Therefore, providing complete information regarding each intervention implemented by health professionals is important to collaborate in the general care of patients, especially if some the interventions are locally implemented in a unit.
Among the limitations, we have to mention the small sample size, which does not provide sufficient power to conduct a robust exploration of the psychometric features of the questionnaire. This small sample size also had reduced the possibilities to perform other psychometric measures. For example, a confirmatory factor analysis to explore further the relationship between items and to assess theoretical measurement structure. We also cannot consider the association results as definitive because the small sample size only allows us to consider these results as exploratory. Additionally, we could not include other variables, such as the children's diagnosis and explore their associations with parental perceived helpfulness. We also did not ask information regarding demographic data from parents because we wanted to reduce the burden associated to answer a questionnaire too long considering the context of having an inpatient offspring for a long time. However, demographic information such as education, work activities and age may have helped to find differences on understanding and responding the WHQ.
Future research may include a larger sample and young people receiving treatment in the community or in an outpatient setting.
Conclusions
Initiatives to assess parental perceived helpfulness of the treatments received by their children are needed to facilitate the identification of potential gaps between professionals' evaluation of proposed treatment plans and families' opinions about them. The measure designed and assessed in this study is a good candidate to reduce this gap, and it may also be implemented in other settings of care delivery.
Our results added evidence for the validity and reliability of the WHQ to measure parental perceived helpfulness of the multidimensional intervention offered in inpatient children's units. Administration of WHQ in larger independent samples will help to assess the psychometric features of this instrument, for example, testing further the dimensionally of the construct using confirmatory factor analyses; and may also confirm the generalizability of these findings and help clarify the relative contribution of specific aspects of care in perceived parental helpfulness.
Statements
Ethics statement
This project received ethical approval by the South London and Maudsley National Health Service (NHS) Foundation Trust Child and Adolescent Mental Health Services (CAMHS) Clinical Governance/Audit Committee.
Author contributions
MK and SF conceived and designed the study; IM collected the data; MK, SF, and OH supervised the collection of data; IM, JG, and MK analyzed and interpreted the data, and produced the drafting of the manuscripts. All authors provided a critical revision of the manuscript.
Funding
The writing of the manuscript was led by JG while doing a Postdoctoral Research Fellowship in the London School of Hygiene and Tropical Medicine, supported by a scholarship awarded from the National Commission for Scientific and Technological Research (CONICYT) and a scholarship from Universidad de los Andes (Chile).
Acknowledgments
We want to thank all of the children and their families who participated in this study.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Supplementary material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00080/full#supplementary-material
- UK
United Kingdom
- WHQ
What was Helpful Questionnaire
- SDQ
Strengths and Difficulties Questionnaire
- CGAS
Children's Global Assessment Scale
- HoNOSCA
Health of the Nation Outcome Scales for Children and Adolescents
- ASD
Autism spectrum disorder
- HD
Hyperkinetic Disorders
- CBT
Cognitive Behavioral Therapy
- OT
Occupational Therapy
- LOS
Length Of Stay
- ASQ
Acorn Satisfaction Questionnaire
- SD
Standard Deviation
- NHS
National Health Service
- CAMHS
Child and Adolescent Mental Health Services
- EFA
Exploratory Factor Analysis
- ICD
International Classification of Diseases.
Abbreviations
References
1
PolanczykGVSalumGASugayaLSCayeARohdeLA. Annual research review: a meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. J Child Psychol Psychiatry (2015) 56:345–65. 10.1111/jcpp.12381
2
WoodwardLJFergussonDM. Life course outcomes of young people with anxiety disorders in adolescence. J Am Acad Child Adoles Psychiatry (2001) 40:1086–93. 10.1097/00004583-200109000-00018
3
PineDSCohenPGurleyDBrookJMaY. The risk for early-adulthood anxiety and depressive disorders in adolescents with anxiety and depressive disorders. Arch Gen Psychiatry (1998) 55:56–64. 10.1001/archpsyc.55.1.56
4
LayardRClarkAECornagliaFPowdthaveeNVernoitJ. What predicts a successful life? A life-course model of well-being. Econ J. (2014) 124:F720–38. 10.1111/ecoj.12170
5
KielingCBaker-HenninghamHBelferMContiGErtemIOmigbodunOet al. Child and adolescent mental health worldwide: evidence for action. Lancet (2011) 378:1515–25. 10.1016/S0140-6736(11)60827-1
6
FrithE. Inpatient Provition for Children and Young People With Mental Health Problems. London: Education Policy Institute (2017).
7
KyriakopoulosMOugrinDFraserCThomasGMcMahonR. Emergency mental health admissions for children: a naturalistic study. Clin Child Psychol Psychiatry (2015) 20:8–19. 10.1177/1359104513493430
8
GreenJJacobsBBeechamJDunnGKrollLTobiasCet al. Inpatient treatment in child and adolescent psychiatry–a prospective study of health gain and costs. J Child Psychol Psychiatry (2007) 48:1259–67. 10.1111/j.1469-7610.2007.01802.x
9
KurtzZ. The Evidence Base to Guide Development of Tier 4 CAMHS. UK: Department of Health (2009) Available online at: http://webarchive.nationalarchives.gov.uk/20130107105354/http://www.dh.gov.uk/prod_consum_dh/groups/dh_digitalassets/documents/digitalasset/dh_103443.pdf
10
CoghillDBonnarSDukeSGrahamJSethS. Child and Adolescent Psychiatry. New York, NY: Oxford University Press (2009).
11
BlackN. Patient reported outcome measures could help transform healthcare. BMJ (2013) 346:f167. 10.1136/bmj.f167
12
BlackNJenkinsonC. Measuring patients' experiences and outcomes. BMJ (2009) 339:b2495. 10.1136/bmj.b2495
13
WilliamsBCoyleJHealyD. The meaning of patient satisfaction: an explanation of high reported levels. Soc Sci Med. (1998) 47:1351–9.
14
WilliamsAMKristjansonLJ. Emotional care experienced by hospitalised patients: development and testing of a measurement instrument. J Clin Nursing (2009) 18:1069–77. 10.1111/j.1365-2702.2008.02586.x
15
MillerDSteele GrayCKuluskiKCottC. Patient-centered care and patient-reported measures: let's look before we leap. Patient (2015) 8:293–9. 10.1007/s40271-014-0095-7
16
ChristalleEZehSHahlwegPKristonLHärterMSchollI. Assessment of patient centredness through patient-reported experience measures (ASPIRED): protocol of a mixed-methods study. BMJ Open. (2018) 8:e025896. 10.1136/bmjopen-2018-025896
17
HanlonPYeomanLGibsonLEsiovwaRWilliamsonAEMairFSet al. A systematic review of interventions by healthcare professionals to improve management of non-communicable diseases and communicable diseases requiring long-term care in adults who are homeless. BMJ Open. (2018) 8:e020161. 10.1136/bmjopen-2017-020161
18
MetzMElfeddaliIVeerbeekMde BeursEBeekmanAvan der Feltz-CornelisC. Effectiveness of a multi-facetted blended eHealth intervention during intake supporting patients and clinicians in Shared Decision Making: a cluster randomised controlled trial in a specialist mental health outpatient setting. PLoS ONE (2018) 13:e0199795. 10.1371/journal.pone.0199795
19
MetzMJVeerbeekMATwiskJWRvan der Feltz-CornelisCMde BeursEBeekmanATF. Shared decision-making in mental health care using routine outcome monitoring: results of a cluster randomised-controlled trial. Soc Psychiatry Psychiatr Epidemiol. (2018). 10.1007/s00127-018-1589-8. [Epub ahead of print].
20
WykesTCsipkeEWilliamsPKoeserLNashSRoseDet al. Improving patient experiences of mental health inpatient care: a randomised controlled trial. Psychol Med. (2018) 48:488–97. 10.1017/S003329171700188X
21
HolikattiPCKarNMishraAShuklaRSwainSPKarS. A study on patient satisfaction with psychiatric services. Ind J Psychiatry (2012) 54:327–32. 10.4103/0019-5545.104817
22
WareJESnyderMKWrightWRDaviesAR. Defining and measuring patient satisfaction with medical care. Evalu Prog Plan. (1983) 6:247–63. 10.1016/0149-7189(83)90005-8
23
HallJADornanMC. Patient sociodemographic characteristics as predictors of satisfaction with medical care: a meta-analysis. Soc Sci Med. (1990) 30:811–8.
24
HanssonL. The quality of outpatient psychiatric care. A survey of patient satisfaction in a sectorised care organisation. Scand J Caring Sci. (1989) 3:71–82. 10.1111/j.1471-6712.1989.tb00375.x
25
HolcombWRParkerJCLeongGBThieleJHigdonJ. Customer satisfaction and self-reported treatment outcomes among psychiatric inpatients. Psychiatr Serv. (1998) 49:929–34. 10.1176/ps.49.7.929
26
KuosmanenLHatonenHJyrkinenARKatajistoJValimakiM. Patient satisfaction with psychiatric inpatient care. J Adv Nursing (2006) 55:655–63. 10.1111/j.1365-2648.2006.03957.x
27
ItoHShingaiNYamazumiSSawaYIwasakiS. [Patient perceptions and satisfaction of psychiatric services at their discharge]. Seishin shinkeigaku zasshi. (1999) 101:138–47.
28
RohlandBMLangbehnDRRohrerJE. Relationship between service effectiveness and satisfaction among persons receiving Medicaid mental health services. Psychiatr Serv. (2000) 51:248–50. 10.1176/appi.ps.51.2.248
29
HackmanABrownCYangYGoldbergRKreyenbuhlJLuckstedAet al. Consumer satisfaction with inpatient psychiatric treatment among persons with severe mental illness. Commun Mental Health J. (2007) 43:551–64. 10.1007/s10597-007-9098-3
30
BarkerDAShergillSSHigginsonIOrrellMW. Patients' views towards care received from psychiatrists. Br J Psychiatry (1996) 168:641–6. 10.1192/bjp.168.5.641
31
KaplanSBusnerJChibnallJKangG. Consumer satisfaction at a child and adolescent state psychiatric hospital. Psychiatr Serv. (2001) 52:202–6. 10.1176/appi.ps.52.2.202
32
GigantescoAPicardiAChiaiaEBalbiAMorosiniP. Patients' and relatives' satisfaction with psychiatric services in a large catchment area in Rome. Eur Psychiatry (2002) 17:139–47. 10.1016/S0924-9338(02)00643-0
33
BjorngaardJHRuudTFriisS. The impact of mental illness on patient satisfaction with the therapeutic relationship: a multilevel analysis. Soc Psychiatry Psychiatr Epidemiol. (2007) 42:803–9. 10.1007/s00127-007-0229-5
34
BlendonRJLeitmanRMorrisonIDonelanK. Satisfaction with health systems in ten nations. Health Affairs (1990) 9:185–92. 10.1377/hlthaff.9.2.185
35
BowlingARoweGLambertNWaddingtonMMahtaniKRKentenCet al. The measurement of patients' expectations for health care: a review and psychometric testing of a measure of patients' expectations. Health Technol Assess. (2012) 16:1–515. 10.3310/hta16300
36
“helpful”, Merriam-Webster,.com: Merriam-Webster. Available online at: https://www.merriam-webster.com (Accessed January 8, 2019).
37
ChapmanSLJamisonRNSandersSH. Treatment helpfulness questionnaire: a measure of patient satisfaction with treatment modalities provided in chronic pain management programs. PAIN (1996) 68:349–61. 10.1016/S0304-3959(96)03217-4
38
De CivitaMDobkinPLEhrmann-FeldmanDKarpIDuffyCM. Development and preliminary reproducibility and validity of the parent adherence report questionnaire: a measure of adherence in juvenile idiopathic arthritis. J Clin Psychol Med Settings (2005) 12:1–12. 10.1007/s10880-005-0907-y
39
LeeBKirmayerLGroleauD. Therapeutic processes and perceived helpfulness of Dang-Ki (Chinese Shamanism) from the Symbolic Healing Perspective. Cult Med Psychiatry (2010) 34:56–105. 10.1007/s11013-009-9161-3
40
IselinM-GAddisME. Effects of etiology on perceived helpfulness of treatments for depression. Cogn Ther Res. (2003) 27:205–22. 10.1023/A:1023513310243
41
ColmanEMissinneSBrackeP. The role of perceived helpfulness in predicting subjective unmet need and the frequency of health care use. Arch Psychiatr Nurs. (2014) 28:43–9. 10.1016/j.apnu.2013.10.007
42
CarterLReadJPyleMLawHEmsleyRMorrisonA. Causal beliefs in people experiencing psychosis: The relationship to treatment accessed and the perceived helpfulness of treatment. Psychol Psychother. (2018) 91:332–44. 10.1111/papt.12163
43
SinghSAccursoECHailLGoldschmidtABLe GrangeD. Outcome parameters associated with perceived helpfulness of family-based treatment for adolescent eating disorders. Int J Eat Disord. (2018) 51:574–8. 10.1002/eat.22863
44
Kuramoto-CrawfordSJHanBJacobus-KantorLMojtabaiR. Differences in patients' perceived helpfulness of depression treatment provided by general medical providers and specialty mental health providers. Gen Hosp Psychiatry (2015) 37:340–6. 10.1016/j.genhosppsych.2015.04.006
45
van GriekenRAvan TrichtMJKoeterMWJvan den BrinkWScheneAH. The use and helpfulness of self-management strategies for depression: the experiences of patients. PLoS ONE (2018) 13:e0206262. 10.1371/journal.pone.0206262
46
RosenblatJDSimonGESachsGSDeetzIDoederleinADePeraltaDet al. Frequency of use and perceived helpfulness of wellness strategies for bipolar and unipolar depression. Ann Clin Psychiatry (2018) 30:296–304.
47
EdlundMJPettifordAGHamptonJForman-HoffmanVLAultKLColpeLJet al. Adolescents' assessments of the helpfulness of treatment for major depression: results from a national survey. Psychiatr Serv. (2015) 66:1064–73. 10.1176/appi.ps.201400018
48
LincolnTMJungEWiesjahnMWendtHBockTSchlierB. The impact of negative treatment experiences on persistent refusal of antipsychotics. Comprehens Psychiatry (2016) 70:165–73. 10.1016/j.comppsych.2016.07.009
49
MulliganKWedderburnLRNewmanS. The experience of taking methotrexate for juvenile idiopathic arthritis: results of a cross-sectional survey with children and young people. Pediatr Rheumatol Online J. (2015) 13:58. 10.1186/s12969-015-0052-6
50
NagaeMNakaneHHondaSOzawaHHanadaH. Factors affecting medication adherence in children receiving outpatient pharmacotherapy and parental adherence. J Child Adoles Psychiatr Nursing (2015) 28:109–17. 10.1111/jcap.12113
51
FeldmanDEde CivitaMDobkinPLMallesonPMeshefedjianGDuffyCM. Perceived adherence to prescribed treatment in juvenile idiopathic arthritis over a one-year period. Arthritis Rheum. (2007) 57:226–33. 10.1002/art.22534
52
CornettaKBrownCG. Balancing personalized medicine and personalized care. Acad Med. (2013) 88:309–13. 10.1097/ACM.0b013e3182806345
53
GoodmanR. Psychometric properties of the strengths and difficulties questionnaire. J Am Acad Child Adoles Psychiatry (2001) 40:1337–45. 10.1097/00004583-200111000-00015
54
ShafferDGouldMSBrasicJAmbrosiniPFisherPBirdHet al. A children's global assessment scale (CGAS). Arch Gen Psychiatry (1983) 40:1228–31. 10.1001/archpsyc.1983.01790100074010
55
GarraldaMEYatesPHigginsonI. Child and adolescent mental health service use. HoNOSCA as an outcome measure. Br J Psychiatry (2000) 177:52–8. 10.1192/bjp.177.1.52
56
EndicottJSpitzerRLFleissJLCohenJ. The global assessment scale: a procedure for measuring overall severity of psychiatric disturbance. Arch Gen Psychiatry (1976) 33:766–71. 10.1001/archpsyc.1976.01770060086012
57
American Educational Research Association Educational Research Association. Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association (2014).
58
MuthenBKaplanD. A comparison of some methodologies for the factor analysis of non-normal Likert variables: a note on the size of the model. Br J Math Stat Psychol. (1992) 45:19–30. 10.1111/j.2044-8317.1992.tb00975.x
59
FabrigarLRWegenerDTMacCallumRCStrahanEJ. Evaluating the use of exploratory factor analysis in psychological research. Psychol Methods (1999) 4:272–99. 10.1037/1082-989X.4.3.272
60
CostelloABOsborneJW. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Evalu. (2005) 10:1–9.
61
TabachnickBGFidellLS. Using Multivariate Statistics. Boston, MA: Allyn and Bacon (2001).
62
TavakolMDennickR. Making sense of Cronbach's alpha. Int J Med Educ. (2011) 2:53–5. 10.5116/ijme.4dfb.8dfd
63
BlandJMAltmanDG. Statistics notes: Cronbach's alpha. BMJ (1997) 314:572. 10.1136/bmj.314.7080.572
64
RanganathanPPrameshCSBuyseM. Common pitfalls in statistical analysis: the perils of multiple testing. Perspect Clin Res. (2016) 7:106–7. 10.4103/2229-3485.179436
65
GreenlandSSennSJRothmanKJCarlinJBPooleCGoodmanSNet al. Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations. Eur J Epidemiol. (2016) 31:337–50. 10.1007/s10654-016-0149-3
66
JormAFKortenAEJacombPARodgersBPollittPChristensenHet al. Helpfulness of interventions for mental disorders: beliefs of health professionals compared with the general public. Br J Psychiatry. (1997) 171:233–7. 10.1192/bjp.171.3.233
Summary
Keywords
children, inpatient care, helpfulness, mental health, validation
Citation
Mourelatou I, Gaete J, Fewings S, Hickie O and Kyriakopoulos M (2019) What Was Helpful Questionnaire (WHQ): Psychometric Properties of a Novel Tool Designed to Capture Parental Perceived Helpfulness of Interventions in Children Requiring Mental Health Inpatient Care. Front. Psychiatry 10:80. doi: 10.3389/fpsyt.2019.00080
Received
25 September 2018
Accepted
05 February 2019
Published
26 February 2019
Volume
10 - 2019
Edited by
Wai Tong Chien, The Chinese University of Hong Kong, China
Reviewed by
Lawrence T. Lam, University of Technology Sydney, Australia; Raluca Sfetcu, Spiru Haret University, Romania; Steve Lui, University of Huddersfield, United Kingdom
Updates
Copyright
© 2019 Mourelatou, Gaete, Fewings, Hickie and Kyriakopoulos.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Jorge Gaete jgaete@uandes.cl
This article was submitted to Public Mental Health, a section of the journal Frontiers in Psychiatry
Disclaimer
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.