The German EMPATHIC-30 Questionnaire Showed Reliability and Convergent Validity for Use in an Intermediary/General Pediatric Cardiology Unit: A Psychometric Evaluation

Background: Family-Centered Care is a useful framework for improving care for hospitalized children with congenital heart disease. The EMpowerment of PArents in THe Intensive Care-30 (EMPATHIC-30) questionnaire is a widely accepted tool to measure parental satisfaction with Family-Centered Care. Psychometric properties of the EMPATHIC-30 have been evaluated in neonatal and pediatric intensive care units, but not in pediatric cardiac care units. Therefore, our aim was to assess the psychometric properties of the German EMPATHIC-30 in an intermediary/general pediatric cardiology unit.

Methods: We used data from a quality management survey comprising the German EMPATHIC-30, a sociodemographic questionnaire and four general satisfaction items. Data were collected at the intermediary/general pediatric cardiology unit of a specialized heart center in Germany (n = 366). We split the data randomly into two subsets. In the first subset, we assessed internal consistency reliability with McDonald's omega and Cronbach's alpha, and convergent validity using Spearman's rank correlation. Furthermore, we explored the internal structure with Principal Component Analysis (PCA). In the second subset, we validated the resulting structure using Confirmatory Factor Analysis (CFA).

Results: The reliability estimates exceeded 0.70 for all five domain scores and 0.90 for the full-scale score. Convergent validity between EMPATHIC-30 domain scores/ the full-scale score and the four general satisfaction items was adequate (r_s = 0.40–0.74). The PCA suggested three components, accounting for 56.8% of the total variance. Cross-validation via CFA showed poor model fit (χ² = 1545.78, χ²/df = 3.85, CFI = 0.70, TLI = 0.66, RMSEA = 0.13), indicating that the EMPATHIC-30 shows no clear and generalizable factor structure in this sample.

Discussion: The German version of the EMPATHIC-30 exhibited reasonable psychometric properties in an intermediary/general pediatric cardiology unit. Follow-up studies should investigate the factor structure of the EMPATHIC-30 in other pediatric inpatient care settings.

Introduction

Congenital Heart Disease (CHD) is defined as a structural defect of the heart or intrathoracic vessels (1). With a global prevalence of 9.41 per 1,000 births, it represents the most common birth defect worldwide (2, 3). In Europe, ~36 000 children are born with a CHD each year and around 28% of them have moderate to complex heart defects, requiring interventional or surgical treatment (4). During hospitalization, they are exposed to a myriad of stressors, such as separation from their parents, a stressful environment with bright lights and loud noises, restricted mobility, and disrupted sleep. Research shows that children with CHD are at risk for neurodevelopmental impairment, as well as emotional, social, and behavioral difficulties (5–7). Distress during hospitalization may contribute to these challenges (7, 8). Hence, optimizing the hospital environment potentially is an effective strategy to improve neurodevelopmental and psychosocial outcomes of children with CHD, for which Family-Centered Care (FCC) provides a useful framework (9).

Family-Centered Care is an international standard of healthcare provision based on a mutually beneficial partnership among the healthcare providers, patients, and their families (10, 11). In pediatrics, FCC emphasizes the parents as their child's primary source of emotional, social, and developmental support and acknowledges them as integral part of the healthcare team (12). Specific FCC interventions either target the parents (e.g., educational programs, participation of parents in medical rounds), the parent-child dyad (e.g., promoting skin-to-skin contact), or the health-care ecosystem as a whole (e.g., structural implementation of a primary nursing model) (13). Most studies investigating the effects of FCC interventions on child and parent wellbeing have been conducted in Neonatal Intensive Care Units (NICUs), with positive effects reported for physical wellbeing, stress regulation, sleep, and neurodevelopmental outcomes of the child, parent-child attachment, and parental mental wellbeing (14–17). A meta-analysis of randomized controlled trials showed that FCC interventions improve physical health outcomes in premature infants (e.g., weight gain), while their parents experience less anxiety, depression, and stress (18). Despite the positive effects of FCC interventions in neonatology, studies investigating FCC in children with CHD are scarce. However, several authors argue that FCC practices may be similarly beneficial in this population (19–21).

Measuring the subjective experience of provided care is crucial for advances in this area of research, especially when FCC principles are not structurally implemented yet (22). In order to measure parent satisfaction with FCC, the EMpowerment of PArents in THe Intensive Care (EMPATHIC) questionnaire is frequently used (23). Latour et al. (24, 25) originally developed the questionnaire for Pediatric Intensive Care Units (PICUs), based on expert opinions from over 300 PICU nurses and physicians, as well as over 600 parents of children discharged from a PICU. The original scale comprises 65 items, with each item reflecting care aspects from one of the following five domains: Information, Organization, Parental Participation, Care and Cure, and Professional Attitude (23). The domains were identified in qualitative analyses and evaluated quantitatively, by using Confirmatory Factor Analysis (CFA), with separate models for each domain. The authors subsequently developed a shortened version of the questionnaire, the EMPATHIC-30, to improve user friendliness (26). The number of items was reduced by means of multiple regression analysis, resulting in 30 items. In the past years, the EMPATHIC-30 gained international popularity and has been translated from Dutch into various languages, including English, Spanish, Turkish, and German (27–30).

In the original publication of the EMPATHIC-30, Latour et al. (26) found high internal consistency reliability estimates for the five domain scores and the full-scale score. Gill et al. (27) tested the questionnaire's psychometric properties in Australian PICUs, NICUs, and general pediatric wards and reported similar values for the internal consistency reliability (27). Above that, the questionnaire showed adequate convergent validity, as assessed by moderate to strong correlations between each of the domain scores and four general satisfaction items, pointing toward applicability of the questionnaire in these care settings. Orive et al. (28) investigated internal consistency reliability and convergent validity of the questionnaire in Spanish PICUs, with similar results. Only few studies have investigated the construct validity of the questionnaire by using factor analysis. Factor analysis is a statistical method to identify latent variables, which explain covariation amongst a set of measured variables (31). It is therefore an essential approach to generate and evaluate hypotheses about the underlying construct an instrument aims to measure (32). Tiryaki et al. (29) investigated psychometric properties of the EMPATHIC-30 in Turkish NICUs and conducted a CFA in a sample of 238 parents. The authors found a moderate model fit of the final factor solution. However, the factor structure was not reported and thus remains unclear. The German version of the EMPATHIC-30 has not been evaluated psychometrically (30). Furthermore, while the EMPATHIC-30 has been extensively evaluated in different care settings, it has not been psychometrically tested for use in pediatric cardiology units.

Therefore, our aim was to evaluate the psychometric properties, specifically internal consistency reliability, convergent validity, and factor structure of the German EMPATHIC-30 at an intermediary/general pediatric cardiology unit. In order to assess internal consistency reliability, we used McDonald's omega. Although controversially discussed in the literature, we additionally present the classical Cronbach's alpha, to allow for direct comparison to other studies (33, 34). To assess convergent validity, we investigated the relationship between the domain scores and the full-scale score with four general satisfaction items, comparable to the methodology of above-mentioned studies. Furthermore, we investigated the factor structure of the questionnaire, following a two-step procedure. In the first step, we explored the internal structure of the questionnaire on half of the data using Principal Component Analysis (PCA) rather than Exploratory Factor Analysis (EFA). While both PCA and EFA are variable reduction techniques, EFA assumes an underlying construct, which is not measured directly, and PCA reflects a linear combination of variables. We used PCA to explore the internal structure of the questionnaire, because our focus was to explore the structure in total item variance including error, without making assumptions on latent constructs, as these were unknown for the current context (35). In a second step, we used three separate CFA on the other half of the data: The first CFA was conducted to validate the structure resulting from the PCA. The second CFA was conducted to investigate a one-component solution, motivated by potential unidimensionality of the scale. The third CFA was conducted to investigate a five-component solution motivated by the five domains of the EMPATHIC-30.

Materials and Methods

Study Design and Setting

For the psychometric evaluation of the EMPATHIC-30 questionnaire, we used data from a quality management survey comprising the German EMPATHIC-30, a socio-demographic questionnaire, four general satisfaction items and open commentary fields. Data were collected at the intermediary/general pediatric cardiology unit of the German Heart Center Berlin. With its 24 monitored beds and 1,200 yearly admissions, the unit provides specialized care to patients of all ages, ranging from infants to adults, with varying degrees of CHD. This study was approved by the Medical Ethics Committee Charité Virchow (Nr EA2/032/20).

Procedures

All parents of children with CHD hospitalized at the ward were invited to participate in the quality management survey. Participation was voluntary and anonymous. At discharge, doctors handed out a paper and pencil version of the survey together with a return envelope. After completing the survey, parents returned it in a mailbox on the ward. Data collection took place between August 2019 and June 2021.

Materials

The German EMPATHIC-30 questionnaire comprises 30 statements spanning five domains: Information (5 items), Organization (5 items), Parental Participation (6 items), Care and Cure (8 items), and Professional Attitude (6 items). Every statement is rated on a six-point scoring-scale ranging from 1 “certainly no” to 6 “certainly yes,” or rated 0 for the answer alternative “not applicable.”

Sociodemographic information was obtained through a purpose-designed questionnaire. It contains one item to specify the respondent (with options “mother,” “father,” “both mother and father,” and “other relatives” with the option of open-ended specification), as well as items relating to age of the child, place of birth and mother tongue of the parents, length of hospital stay, type of and reason for admission, and undertaken medical procedures.

Four general satisfaction items were included in the survey: Two items are rated on the same six-point scale as the EMPATHIC-30 questionnaire: “We would recommend this unit or ward,” “We would be happy to return to this unit or ward”. Two more items are rated on a ten-point scale, ranging from “very bad” to “excellent”: “Overall performance of doctors” as well as “Overall performance of nurses” (23). Furthermore, commentary fields were included in the survey about general experiences made during admission, hospital stay, and discharge.

Statistical Analyses

Statistical analyses were carried out using SPSS 27 (SPSS Inc, Chicago, Illinois). Non-linear and linear PCA were conducted in SPSS. AMOS, an SPSS extension module, was used for the CFA.

Data Preparation and Preliminary Analyses

Handling of Answer Alternative “Not Applicable”

Non-linear Principal Component Analysis (CATPCA) was performed to determine the best linear replacement values for observed scores in each item individually, for the scores 0 up to 6 (0 corresponding to the answer alternative “not applicable”) (36). Based on transformation plots from nominal optimal scaling, the scores 0 and 6 got assigned a similar quantification; both answer categories had an equivalent interpretation by participants. This was consistent with previous findings by Latour et al. (23). Scores on the answer category “not applicable” were therefore recoded to the highest value of the scale (i.e., 6). In addition, the transformation plots revealed that the answer categories functioned as near-equally spaced linear scale; models with nominal transformation and with numerical transformation after recoding yielded 0.8% difference in explained variance. All subsequent linear analyses were performed using the recoded scores.

Handling of Missing Data

Returned questionnaires with ≥75% of missing items were excluded from analysis. One third of respondents presented at least one missing value and the total percentage of missing data points was 2.3%. Missing data can affect the estimation and interpretation of PCA (37). Little's Missing Completely at Random (MCAR) test was significant, indicating that missings are not missing completely at random, thus indicating a potentially systematic difference between missing and observed values (38). Therefore, multiple imputation, a proven statistical method to estimate missing values, was used on the recoded scores. Missing scores were estimated in 25 sets, applying Markov Chain Monte Carlo sampling and predictive mean matching (39). Results of the statistical analyses were pooled for the imputed data sets whenever possible.

Data Split for Separate Estimation and Validation

The data set was randomly split in half, creating two subsets (A, B) to perform 2-fold cross-validation. All statistical structure and content analyses were performed on set A. Set B was used only as validation set for the confirmatory evaluation of the internal structure via CFA.

Descriptive Statistics

Descriptive statistics of the EMPATHIC-30 scores as well as sociodemographic characteristics of the sample are reported (means and standard deviations for quantitative variables, absolute frequencies and percentages for categorical variables). To check for successful randomization, descriptive statistics for the full set, analysis set A, and validation set B, as well as test statistics for the comparison between set A and B are provided.

Internal Consistency Reliability

The internal consistency reliability of the German EMPATHIC-30 questionnaire on domain and full-scale level was assessed with McDonald's omega. Cronbach's alpha was computed additionally. Values greater than 0.70, 0.80, and 0.90 reflect acceptable, good, and excellent reliability, respectively (40).

Convergent Validity

To examine convergent validity of the questionnaire, we used Spearman's rank correlation test for non-normally distributed data, as assessed visually and through significant Shapiro Wilk tests (p < 0.01). We assessed the relationship between the domain scores/ the full-scale score and the four overall satisfaction statements. Based on findings from other validation studies, we expected moderate to strong correlation coefficients, ranging from 0.40 up to 0.79, indicating adequate convergent validity (41).

Internal Structure

Principal Component Analysis

We conducted a PCA to explore the internal structure of the questionnaire. An oblique rotation should be applied, which reorients the components in order to simplify the mathematical model and interpretation by allowing for intercorrelations between the components. However, this rotation is not implemented for multiply imputed data. Therefore, we conducted a two-step procedure. First, we performed a PCA on the unimputed data set A to determine the number of components. Pairwise deletion was selected to handle missing values. The suitability of the data was assessed with the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy and Bartlett's test of sphericity. In this exploratory stage, the KMO value is interpreted as an approximation of the ratio of potential common variance compared to the total variance in the data and thus provides information if subsequent factor analysis is suitable. Final component extraction was based on the combined Monte Carlo Parallel Analysis and examination of the scree plot (42). Oblique rotation allowing for intercorrelations between the components was applied in this step. For items with cross-loadings, the component on which the item loaded higher was selected. Loadings under 0.30 (<10% shared variance between item and component) were considered as negligible and therefore not considered for inclusion in the component structure. Second, we used the results of this PCA to motivate the number of components in a second PCA on the imputed data set A by using Generalized Procrustes Analyses in the subroutine by Wingerde et al. (43). This subroutine imposed a pre-specified number of components and orthogonal rotation of the component loadings, ignoring intercorrelations between the components.

Confirmatory Factor Analysis

We conducted three separate factor analyses on set B of the sample. First, we conducted a CFA to validate the component structure resulting from the two-step PCA. Second, we conducted a CFA based on a one-component model to investigate potential unidimensionality of the questionnaire. Third, we conducted a CFA based on a five-component model to investigate the validity of the five domains of the EMPATHIC-30 (Information, Organization, Parental Participation, Care and Cure, and Professional Attitude). In the CFA measurement models, correlation between the components is allowed. As combining the results of multiply imputed data is not possible in AMOS, we conducted the analyses on the data with missing values using Full Information Maximum Likelihood Estimation and compared model estimates for robustness. To assess model fit, we used the following fit indices: model-Chi-squared test divided by the degrees of freedom (χ²/df), Comparative Fit Index (CFI), Tucker Lewis Index (TLI), and Root Mean Square Error of Approximation (RMSEA). Cut-off values were: χ²/df <3, CFI of at least 0.90, TLI of at least 0.95, and RMSEA <0.08 (44, 45). A second evaluation of robustness of findings was performed by repeating the same analyses on the other half of the data (set A).

Results

A total of 475 questionnaires were returned between August 2019 and June 2021. The response rate was 68% (percentage of returned questionnaires vs. distributed copies). To ensure homogeneity of the data set, we only included questionnaires filled out by parents. As a result, we excluded 91 questionnaires filled out by adult patients, as well as nine questionnaires filled out by relatives other than parents. Upon first exploration of data, we excluded three more questionnaires with comments in the commentary fields reflecting very high satisfaction, but with lowest possible scores on EMPATHIC-30 items, potentially indicating a mix up between highest and lowest scores. Above that, we excluded six questionnaires with ≥75% missing items. The final number of questionnaires included in the analysis was 366, resulting in 183 questionnaires each for analysis set A and validation set B.

Descriptive Statistics

The child and parent characteristics are presented in Table 1. No significant differences between set A and set B were observed for any of the characteristics, except for the item specifying the respondent, in which a significant shift of mother-only to both parents was seen (X_{2 (2, 356)} = 8.17, p = 0.017). As the proportion of mothers giving their input does not differ in both sets, we view this difference as negligible. Therefore, we consider the reported characteristics of each set representative for the whole group. Below, we present the characteristics of set A, as this set drives the main psychometric analysis. Most children of participating families were either infants (n = 53, 29.6%), toddlers (n = 30, 16.8%) or preschoolers (n = 39, 21.8%) and the mean age was 5.32 years (SD = 6.63). Seventy-six percent of the questionnaires were completed by mothers. The majority of participants were born in Germany (n = 166, 91.2%) and native German speakers (n = 148, 83.1%). Only 7% of hospital admissions were unexpected and the mean length of hospital stay was 6.32 days (SD = 8.86), ranging from 1 to 105 days.

TABLE 1

Table 1. Characteristics of children and parents in set A, B, and the full sample.

Parents gave high ratings on the EMPATHIC-30 and all except four items showed mean scores above 5 (Table 2). On the domain level, mean scores ranged from 5.19 (SD = 0.84) for the domain Organization up to 5.45 (SD = 0.76) for the domain Professional Attitude. The “not applicable” response was given most frequently for the item “The unit could easily be reached by telephone” (n = 42, 23%).

TABLE 2

Table 2. EMPATHIC-30 means and standard deviations for set A, B and the full sample.

Internal Consistency Reliability

McDonald's omega on the domain level ranged from 0.75 (Organization) to 0.87 (Professional Attitude; Care and Cure) and reached 0.95 for the full-scale. Cronbach's alpha on the domain level was only slightly lower and ranged from 0.73 (Organization) to 0.85 (Professional Attitude). The findings are presented in Table 3.

TABLE 3

Table 3. Internal consistency reliability of the EMPATHIC-30 domain scores and full-scale score (n = 178).

Convergent Validity

As shown in Table 4, the correlations between the EMPATHIC-30 domain scores and scores on the four overall satisfaction statements ranged from r_s(183) = 0.40, p < 0.01 between the domain Organization and satisfaction statement “Readmission to ward,” to r_s(183) = 0.68, p < 0.01 between the domain Care and Cure and satisfaction statement “Overall rating doctors.” The lowest correlations were found for the domain Organization, with correlations under 0.50 for all satisfaction statements. Similarly, the correlations between the full-scale score and scores on the four overall satisfaction statements ranged from r_s(183) = 0.62, p < 0.01 for the statement “Readmission to ward” to r_s(183) = 0.74, p < 0.01 for the statement “Overall rating doctors.” All correlations were significant and moderate to high, according to expectation. For an overview of correlations between the domain scores, see Table 5.

TABLE 4

Table 4. Spearman's rank correlation coefficient between domain scores/ full-scale score and scores on the overall satisfaction items (n = 183).

TABLE 5

Table 5. Spearman's rank correlation coefficient between domain scores (n = 183).

Internal Structure

Principal Component Analysis

For the first PCA on unimputed data, sampling adequacy was ascertained by a KMO value of 0.89 and a significant Bartlett's test of sphericity (χ² = 3734.43, p < 0.01). The comparison of empirical data to simulated random data through Monte Carlo Parallel Analysis suggested a three-component solution, explaining 56.8% of the total variance, with each component accounting for the following percentages: 43.7, 7.5, and 5.6%. In the scree plot (Figure 1), we observed the strongest inflection after the first component, which visually supports unidimensionality of the scale. The obliquely rotated component loadings for the three-component solution based on the first PCA are presented in the Supplementary Table. The correlations between the components were ranging from 0.28 to 0.46. The orthogonally rotated combined component loadings from the imputed data in the second PCA showed a similar three-component solution; except for two items (Professional Attitude - Admission, Organization - Efficiency) all of them loaded on the same respective components. Both items showed strong associations with more than one component in either version of the PCA and seemed to contribute mainly to the intercorrelations among the components. The comparison between component loadings for the first, oblique PCA and second, orthogonal PCA are presented in the Supplementary Table, found in the Supplementary Material.

FIGURE 1

Figure 1. Scree plot. Actual data: unimputed complete case observed data from set A. Simulated data: simulated random data from Monte Carlo Parallel Analysis. The strongest inflection in the empirical data is observed after the first component, suggesting unidimensionality. The lines cross between component 3 and 4, indicating that three components based on actual observed data provided more substantive information compared to three components based on purely random data. Four or more components should therefore not be extracted from the observed data.

Confirmatory Factor Analysis

The first CFA was conducted to validate the fit of the three-component solution. As the results of both alternative PCAs showed a comparable three-component solution, we chose to start with a CFA model based on results from the second PCA (on imputed data with orthogonal rotation). However, model fit was poor (χ² = 1545.78, χ²/df = 3.85, CFI = 0.70, TLI = 0.66, RMSEA = 0.13). The model was then further revised using Lagrange Multiplier Tests (LM test), evaluating several alterations aiming at reducing large correlation residuals. The best fitting solution was found after first removing four items and then applying LM test improvements, yet overall model fit remained poor (χ² = 1640.05, χ²/df = 5.05, CFI = 0.73, TLI = 0.70, RMSEA = 0.09).

The alternative model specifying pure unidimensionality showed poor goodness of fit, with χ² = 1642.17, χ²/df = 4.05, CFI = 0.68, TLI = 0.63, RMSEA = 0.13. The five-factor model aiming to evaluate the validity of the five domains also showed poor goodness of fit (χ² = 1561.97, χ²/df = 3.95, CFI = 0.70, TLI = 0.64, RMSEA = 0.13).

Model fit statistics for the respective CFA models are summarized in Table 6. Variance explained by the factors for each CFA model are presented in Table 7. To eliminate lack of power or collateral bias between set A and B as potential cause for finding the current results, we have repeated the same analyses on the other half of the data set. These analyses yielded equivalent results, supporting the robustness of our findings.

TABLE 6

Table 6. Model fit statistic for the respective CFA models.

TABLE 7

Table 7. Variance explained by each factor for the respective CFA models.

Discussion

In this study, we evaluated the psychometric characteristics of the German version of the EMPATHIC-30 for use in intermediary/general pediatric cardiology units. Furthermore, we extended the psychometric assessment in comparison to previous studies by evaluating the internal structure of the questionnaire in this care setting.

On average, parents gave high ratings for their satisfaction with FCC. The McDonald's omega values in our study indicated acceptable to good reliability for the items within the five domains and excellent reliability for the full-scale score. These values are consistent with the findings of other EMPATHIC-30 studies (26–29). We found adequate convergent validity as shown by moderate to strong correlations between the five domains scores/ the full-scale score and the four general satisfaction items. Our results fall in line with previous publications, reporting correlation coefficients in the same order of magnitude (23, 27, 28). Future studies should extend these findings by investigating convergent validity based on methodology that is more elaborate, such as the use of other standardized instruments measuring parent satisfaction with care, as well as by incorporating assessments of discriminant validity.

We used PCA to assess the internal structure of the German version of the EMPATHIC-30. The analyses from the first PCA revealed a three-component structure with an explained variance over 50%. The first component explains beyond 40%, which supports the unidimensionality of the scale and may indicate that the questionnaire adequately measures the construct of interest (satisfaction with FCC) in our population. The three-component structure resulting from the first PCA (conducted on complete case data and allowing for intercorrelations between components) is very similar to the three-component structure resulting from the second PCA (conducted on imputed data, ignoring intercorrelations between components): only two out of 30 items load differently. Considering that the correlations among the components were close to negligible in the first PCA, rotation seems to have a minor impact on the interpretation of the internal structure, which may not be true for missing data (37). Therefore, we are inclined to view the three-component structure resulting from the second PCA as the best approximation of the questionnaire's internal structure in our sample. Although the three-component solution differs from the expected five-component structure, it is plausible and interpretable. Based on the semantic content of the respective items, we label the first component as “Perception and respect of the family's needs,” the second component as “Involvement of and collaboration with the parents,” and the third as “Communication and organization.” However, despite the interpretability of the three components, the cross-validation of the three-component solutions via CFA resulted in poor fit indices. Model revisions did not significantly improve the model fit. A one-component solution to test for unidimensionality also showed a poor fit to the real data. Although the first component captures over 40% of the total variance in PCA, the true score variance seems to be relatively small compared to the random error variance. Additionally, we validated the five-component solution based on the original domains of the EMPATHIC-30, which indicated a poor fit to the real data. According to the poor model fit indices, all tested component models seem to be an oversimplification of the true structure of the questionnaire.

Our findings suggest that the EMPATHIC-30 has no clear and generalizable factor structure in our population. The ambiguous internal structure found in our study needs to be interpreted in light of the construction of the EMPATHIC questionnaires. In the original publication of the EMPATHIC-65, the five domains were defined during expert group sessions and item groupings into the respective domains were performed consensus based (24). While the authors used CFA to evaluate the unidimensionality of each domain (assessing whether the items within every domain measured the same construct), they did not evaluate the underlying factor structure of the questionnaire (23). For the development of the shortened EMPATHIC-30 questionnaire, multiple regression analysis was used to evaluate statistical performance of the items, which might explain the divergence between the conceptual and the data-driven structure of the questionnaire (26). Furthermore, scores on the EMPATHIC-30 were high on average, with relatively small standard deviations. Accordingly, the parents in our sample were highly satisfied and the limited variation may contribute to the unclear factor structure. Still, our data showed sufficient true score variation to find three interpretable dimensions. The non-zero but not very high correlations between domain scores support this claim rather than support a true unidimensional structure. Replication studies may shed light on the question whether the unclear factor structure is sample specific. For instance, individual characteristics may influence interpretation of the items and subsequently, the way items divide into latent factors. Investigating the data-driven internal structure vs. theoretically postulated structure by conducting studies in different cultural settings and (sub-) populations may therefore be an interesting avenue for follow-up research. While we did not find strong support for the five-factor structure, we consider the domains informative, especially as they were thoroughly developed through expert panels. Nevertheless, FCC reflects a multi-faceted construct and we need more conceptual work to explain expert consensus on the one hand, and unclear factorial structure on the other, especially in light of the fact that the questionnaire assesses the subjective experience, as opposed to objective criteria for FCC.

Our study warrants some limitations. This is an analysis of quality management data from a single intermediary/general pediatric cardiology unit. Participation of other pediatric cardiology centers would allow for a more robust interpretation of results and in a prospective study design, additional measurements should be included for psychometric evaluation, specifically allowing for an assessment of discriminant validity. Furthermore, based on our results, differential analyses considering population characteristics like age range, duration of stay, and complexity of disease may be important to further increase our insights into the internal structure of the questionnaire.

To sum up, the German EMPATHIC-30 has no clear and simple factor structure in our population, while showing adequate reliability and convergent validity as assessed with four general satisfaction items. Accordingly, the EMPATHIC-30 is a suitable instrument to measure FCC in intermediary/general pediatric cardiology wards. However, follow-up studies are needed to further investigate the factor structure of the questionnaire. To our knowledge, this is the first study to assess psychometric properties of a standardized assessment of satisfaction with FCC in this population. Identifying care aspects that need to be improved during hospitalization is crucial in order to meet the developmental needs of children with CHD.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Medical Ethics Committee Charité Virchow (Nr EA2/032/20). Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author Contributions

HF, KS, JL, RR, and AG contributed to the conception of the study. HF, RR, and AG developed the data analysis plan. MJ and AG prepared the dataset. RR and AG performed the statistical analysis. AG wrote the first draft of the manuscript. HF and RR reviewed the first version and wrote sections of the manuscript. All the authors contributed to the article and approved the submitted version.

Funding

This study was partially funded by the Stiftung KinderHerz (2511–3–00–022). AG was supported by a doctoral research scholarship of the Deutsche Herzstiftung e.V. Both sponsors were not involved in the study design, data collection, analysis, interpretation, or decision to submit results.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcvm.2022.901260/full#supplementary-material

References

1. Mitchell SC, Korones SB, Berendes HW. Congenital heart disease in 56,109 births. incidence and natural history. Circulation. (1971) 43:323–32. doi: 10.1161/01.CIR.43.3.323

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Liu Y, Chen S, Zühlke L, Black GC, Choy MK, Li N, et al. Global birth prevalence of congenital heart defects 1970-2017: updated systematic review and meta-analysis of 260 studies. Int J Epidemiol. (2019) 48:455–63. doi: 10.1093/ije/dyz009

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Van Der Linde D, Konings EEM, Slager MA, Witsenburg M, Helbing WA, Takkenberg JJM, et al. Birth prevalence of congenital heart disease worldwide: a systematic review and meta-analysis. J Am Coll Cardiol. (2011) 58:2241–7. doi: 10.1016/j.jacc.2011.08.025

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Dolk H, Loane M, Garne E, European European Surveillance of Congenital Anomalies (EUROCAT) Working Group. Congenital heart defects in Europe: prevalence and perinatal mortality, 2000 to 2005. Circulation. (2011) 123:841–9. doi: 10.1161/CIRCULATIONAHA.110.958405

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Marino BS, Lipkin PH, Newburger JW, Peacock G, Gerdes M, Gaynor JW, et al. Neurodevelopmental outcomes in children with congenital heart disease: evaluation and management: a scientific statement from the American heart association. Circulation. (2012) 126:1143–72. doi: 10.1161/CIR.0b013e318265ee8a

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Howell HB, Zaccario M, Kazmi SH, Desai P, Sklamberg FE, Mally P. Neurodevelopmental outcomes of children with congenital heart disease: a review. Curr Probl Pediatr Adolesc Health Care. (2019) 49:100685. doi: 10.1016/j.cppeds.2019.100685

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Clancy T, Jordan B, de Weerth C, Muscara F. Early emotional, behavioural and social development of infants and young children with congenital heart disease: a systematic review. J Clin Psychol Med Settings. (2020) 27:686–703. doi: 10.1007/s10880-019-09651-1

PubMed Abstract | CrossRef Full Text | Google Scholar

8. McGauran M, Jordan B, Beijers R, Janssen I, Franich-Ray C, de Weerth C, et al. Long-term alteration of the hypothalamic-pituitary-adrenal axis in children undergoing cardiac surgery in the first 6 months of life. Stress. (2017) 20:505–12. doi: 10.1080/10253890.2017.1349748

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Ryan KR, Jones MB, Allen KY, Marino BS, Casey F, Wernovsky G, et al. Neurodevelopmental outcomes among children with congenital heart disease: at-risk populations and modifiable risk factors. World J Pediatr Congenit Heart Surg. (2019) 10:750–8. doi: 10.1177/2150135119878702

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Institute for Patient- and Family-Centered Care. Available online at: https://www.ipfcc.org/ (accessed November 2, 2021).

Google Scholar

11. Davidson JE, Aslakson RA, Long AC, Puntillo KA, Kross EK, Hart J, et al. Guidelines for family-centered care in the neonatal, pediatric, and adult ICU. Crit Care Med. (2017) 45:103–28. doi: 10.1097/CCM.0000000000002169

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Committee on Hospital Care and Institute for Patient-and Family-Centered Care. Patient-and family-centered care and the pediatrician's role. Pediatrics. (2012) 129:394–404. doi: 10.1542/peds.2011-3084

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Franck LS, O'Brien K. The evolution of family-centered care: from supporting parent-delivered interventions to a model of family integrated care. Birth Defects Res. (2019) 111:1044–59. doi: 10.1002/bdr2.1521

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Feldman R, Rosenthal Z, Eidelman AI. Maternal-preterm skin-to-skin contact enhances child physiologic organization and cognitive control across the first 10 years of life. Biol Psychiatry. (2014) 75:56–64. doi: 10.1016/j.biopsych.2013.08.012

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Lv B, Gao XR, Sun J, Li TT, Liu ZY, Zhu LH, et al. Family-centered care improves clinical outcomes of very-low-birth-weight infants: a quasi-experimental study. Front Pediatr. (2019) 7:138. doi: 10.3389/fped.2019.00138

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Als H, Duffy FH, McAnulty GB, Rivkin MJ, Vajapeyam S, Mulkern RV, et al. Early experience alters brain function and structure. Pediatrics. (2004) 113:846–57. doi: 10.1542/peds.113.4.846

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Welch MG, Myers MM, Grieve PG, Isler JR, Fifer WP, Sahni R, et al. Electroencephalographic activity of preterm infants is increased by family nurture intervention: a randomized controlled trial in the NICU. Clin Neurophysiol. (2014) 125:675–84. doi: 10.1016/j.clinph.2013.08.021

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Ding X, Zhu L, Zhang R, Wang L, Wang TT, Latour JM. Effects of family-centred care interventions on preterm infants and parents in neonatal intensive care units: a systematic review and meta-analysis of randomised controlled trials. Aust Crit Care. (2019) 32:63–75. doi: 10.1016/j.aucc.2018.10.007

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Torowicz D, Lisanti AJ, Rim JS, Medoff-Cooper B. A developmental care framework for a cardiac intensive care unit: a paradigm shift. Adv Neonatal Care. (2012) 12:S28–32. doi: 10.1097/ANC.0b013e318265aeef

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Lisanti AJ, Vittner D, Medoff-Cooper B, Fogel J, Wernovsky G, Butler S. Individualized family-centered developmental care: an essential model to address the unique needs of infants with congenital heart disease. J Cardiovasc Nurs. (2019) 34:85–93. doi: 10.1097/JCN.0000000000000546

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Peterson JK. Supporting optimal neurodevelopmental outcomes in infants and children with congenital heart disease. Crit Care Nurse. (2018) 38:68–74. doi: 10.4037/ccn2018514

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Dall'Oglio I, Mascolo R, Gawronski O, Tiozzo E, Portanova A, Ragni A, et al. A systematic review of instruments for assessing parent satisfaction with family-centred care in neonatal intensive care units. Acta Paediatr. (2018) 107:391–402. doi: 10.1111/apa.14186

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Latour JM, van Goudoever JB, Duivenvoorden HJ, Albers MJIJ, Van Dam NAM, Dullaart E, et al. Construction and psychometric testing of the EMPATHIC questionnaire measuring parent satisfaction in the pediatric intensive care unit. Intensive Care Med. (2011) 37:310–8. doi: 10.1007/s00134-010-2042-y

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Latour JM, Hazelzet JA, Duivenvoorden HJ, van Goudoever JB. Construction of a parent satisfaction instrument: perceptions of pediatric intensive care nurses and physicians. J Crit Care. (2009) 24:255–66. doi: 10.1016/j.jcrc.2008.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Latour JM, van Goudoever JB, Duivenvoorden HJ, van Dam NA, Dullaart E, Albers MJ, et al. Perceptions of parents on satisfaction with care in the pediatric intensive care unit: the EMPATHIC study. Intensive Care Med. (2009) 35:1082–9. doi: 10.1007/s00134-009-1491-7

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Latour JM, Duivenvoorden HJ, Tibboel D, Hazelzet JA, EMPATHIC Study Group. The shortened EMpowerment of PArents in THe Intensive Care 30 questionnaire adequately measured parent satisfaction in pediatric intensive care units. J Clin Epidemiol. (2013) 66:1045–50. doi: 10.1016/j.jclinepi.2013.02.010

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Gill FJ, Wilson S, Aydon L, Leslie GD, Latour JM. Empowering parents of australian infants and children in hospital: translation, cultural adaptation, and validation of the EMpowerment of PArents in the intensive care-30-AUS questionnaire. Pediatr Crit Care Med. (2017) 18:e506–13. doi: 10.1097/PCC.0000000000001309

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Pilar Orive FJ, Basabe Lozano J, López Zuñiga A, López Fernández YM, Escudero Argaluza J, Latour JM. Spanish translation and validation of the EMPATHIC-30 questionnaire to measure parental satisfaction in intensive care units. An Pediatr. (2018) 89:50–7. doi: 10.1016/j.anpede.2017.08.006

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Tiryaki O, Zengin H, Çinar N, Umaroglu MM, Latour JM. Turkish adaptation and validation of the EMpowerment of PArents in THe Intensive Care (EMPATHIC-30) questionnaire to measure parent satisfaction in neonatal intensive care units. Front Pediatr. (2020) 8:421. doi: 10.3389/fped.2020.00421

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Nagl-Cupal M, Lippoldt J. Übersetzung und kulturelle Adaptierung eines Instruments zur Messung von Family-Centered Care aus der Perspektive von Eltern bzw. nahen Familienmitgliedern Unpublished manuscript Faculty of Nursing Science, University of Vienna (2016).

Google Scholar

31. Bandalos DL, Finney SJ. Factor analysis: exploratory and confirmatory. In: The Reviewer's Guide to Quantitative Methods in the Social Sciences. New York, NY: Routledge (2018). p. 98–122. doi: 10.4324/9781315755649-8

CrossRef Full Text | Google Scholar

32. Tavakol M, Wetzel A. Factor Analysis: a means for theory and instrument development in support of construct validity. Int J Med Educ. (2020) 11:245–7. doi: 10.5116/ijme.5f96.0f4a

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Sijtsma K. On the use, the misuse, and the very limited usefulness of Cronbach's alpha. Psychometrika. (2009) 74:107–20. doi: 10.1007/s11336-008-9101-0

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Dunn TJ, Baguley T, Brunsden V. From alpha to omega: a practical solution to the pervasive problem of internal consistency estimation. Br J Psychol. (2014) 105:399–412. doi: 10.1111/bjop.12046

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Abdi H, Williams LJ. Principal component analysis. Wiley Interdiscip Rev Comput Stat. (2010) 2:433–59. doi: 10.1002/wics.101

CrossRef Full Text | Google Scholar

36. Linting M, van der Kooij A. Nonlinear principal components analysis with CATPCA: a tutorial. J Pers Assess. (2012) 94:12–25. doi: 10.1080/00223891.2011.627965

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Van Ginkel JR, Kroonenberg PM, Kiers HA. Missing data in principal component analysis of questionnaire data: a comparison of methods. J Stat Comput. (2014) 84:2298–315. doi: 10.1080/00949655.2013.788654

CrossRef Full Text | Google Scholar

38. Little RJ. A test of missing completely at random for multivariate data with missing values. J Am Stat Assoc. (1988) 83:1198–202. doi: 10.1080/01621459.1988.10478722

CrossRef Full Text | Google Scholar

39. van Ginkel JR, Linting M, Rippe RC, van der Voort A. Rebutting existing misconceptions about multiple imputation as a method for handling missing data. J Pers Assess. (2020) 102:297–308. doi: 10.1080/00223891.2018.1530680

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Taber KS. The use of Cronbach's alpha when developing and reporting research instruments in science education. Res Sci Educ. (2018) 48:1273–96. doi: 10.1007/s11165-016-9602-2

CrossRef Full Text | Google Scholar

41. Akoglu H. User's guide to correlation coefficients. Turk J Emerg Med. (2018) 18:91–3. doi: 10.1016/j.tjem.2018.08.001

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Watkins M. Monte Carlo PCA for Parallel Analysis (Computer Software). State College, PA: Ed & Psych Associate (2000).

Google Scholar

43. van Wingerde B, van Ginkel J. SPSS syntax for combining results of principal component analysis of multiply imputed data sets using generalized procrustes analysis. Appl Psychol Meas. (2021) 45:231–2. doi: 10.1177/0146621621990757

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model. (1999) 6:1–55. doi: 10.1080/10705519909540118

CrossRef Full Text | Google Scholar

45. Matsunaga M. How to factor-analyze your data right: do's, don'ts, and how-to's. Int J Psychol Res. (2010) 3:97–110. doi: 10.21500/20112084.854

CrossRef Full Text | Google Scholar