Measuring Parent Satisfaction With Care in Neonatal Intensive Care Units: The EMPATHIC-NICU-USA Questionnaire

Background: Neonatal intensive care unit (NICU) patient satisfaction is measured as parent satisfaction. Parents are critical to the family-centered care model and can evaluate care. Several EMpowerment of PArents in THe Intensive Care (EMPATHIC) instruments were developed in the Netherlands to measure parent satisfaction with neonatal and pediatric intensive care. EMPATHIC instruments comprise five domains and a total score: information, care and treatment, organization, parental participation, and professional attitude. To our knowledge, the EMPATHIC has not been adapted for USA use.

Objectives: (1) To select a relevant EMPATHIC instrument for our study. (2) To expand the content reflecting the role of nurses and the cultural heterogeneity of USA NICU infants. (3) To adapt the selected EMPATHIC instrument to USA English. (4) To establish psychometric properties of the linguistically adapted instrument. (5) To evaluate instrument performance with additional items.

Methods: The EMPATHIC-30 was selected based on shortest length, high overlap with neonatal EMPATHIC-N, and availability of a validated Spanish-language version. Six items from the EMPATHIC-N were added, two of which were split into separate items, resulting in the EMPATHIC-38. A neonatal nurse practitioner adapted wording to USA English. Cognitive debriefing was performed with eight NICU parents to evaluate adapted wording. Parent survey data from a study about missed nursing care and NICU parent satisfaction were utilized. Internal consistency of the five domains and overall score was measured by Cronbach's alpha. Spearman's rank correlations were computed for domains and overall score with four validity measures. Differential validity was determined using 13 parent demographic subgroups.

Results: Data were from 282 parents. Parent race was predominantly White (61%) or Black (22%). One fifth were Hispanic. The adapted wording was satisfactory. Four of the five EMPATHIC-30 and EMPATHIC-38 domains had Cronbach alphas at or above 0.70, indicating acceptable reliability. Correlations between the domain, total scores, and validity indicators ranged from 0.30 to 0.57, indicating positive, moderate associations. Results were replicated in demographic subgroups. Reliability and validity of the three domains with additional items were better than or equivalent to values for the original.

Conclusion: The linguistically adapted EMPATHIC-30-NICU-USA and the expanded EMPATHIC-38-NICU-USA exhibit satisfactory psychometric properties and are suitable for use in USA NICUs.

Introduction

The parent/infant dyad is the focus of family-centered care (FCC) in the neonatal intensive care unit (NICU) setting. Parent satisfaction influences parents' fulfillment of their parenting roles, which is instrumental to infant growth and development. The purpose of this study was to evaluate the reliability and validity of the USA adaptation of an EMpowerment of PArents in THe Intensive Care (EMPATHIC) instrument. We report the procedure for adaptation as well as the psychometric evaluation.

Importance of the Family in the Neonatal Intensive Care Unit

A newborn is unique developmentally. Newborns experience rapid growth and development in their first year. During this time, newborns are dependent upon care from the parents. The quality of parenting during this first year is critical to the development of the child. The parents' preparedness to take their child home depends in part on bonding with the infant and the skills they learn from NICU providers to parent an infant who has been critically ill. It is important to understand the family's experiences in the NICU and their satisfaction with it.

Importance of Parent Satisfaction in the Neonatal Intensive Care Unit

In the evaluation of health care services, the measurement of patient satisfaction has become integral. The Hospital Consumer Assessment of Healthcare Providers and Systems is one of several USA national initiatives focused on patient-centered care (1). Patient satisfaction is considered to reflect the achievement of patient-centered care (2–5). Patient-centered care is defined as providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions (6). Because of the unique dyad of the infant and parent in neonatal care, patient-centered care in this setting entails/requires FCC. Family-centered neonatal care is defined as applying comprehensive and holistic care to neonates and their families with respect (7–9).

In the NICU, evaluating satisfaction with care is essential for health care professionals to provide optimal quality of care considering the family's needs (10). Frequent presence of families at the bedside, in some cases 24 h per day 7 days a week, and the integration of developmental and FCC into practice emphasize the importance of evaluating parent satisfaction in the NICU (10, 11). The importance of parent satisfaction and involvement in the NICU is recognized in the literature (12).

A substantial body of literature exists that demonstrates the effect of FCC and other parental measures on infant outcomes. One study indicated that parental perceptions and attitudes toward their infants' disease can determine the success or failure of treatment plans and neonatal readmission (13). Lv et al. (14) demonstrated that very low birth weight infants might experience improved clinical outcomes, including improved breastfeeding practices and nutritional outcomes, when parents are present and caring for them, and allowing parents to be active partners in care may safeguard infants from complications. Studies have further evaluated the effect of FCC care to improve infant weight gain, increased high-frequency exclusive breast milk feeding at discharge, decreased neonatal readmission, decreased parent stress and anxiety, increased parental adherence to interventions, and increased parental satisfaction (12, 15–18).

How Is Parent Satisfaction in the Neonatal Intensive Care Unit Evaluated?

Relatively few neonatal parent satisfaction questionnaires have been reported. A systematic review of instruments for assessing parent satisfaction with FCC in NICUs was conducted by Dall'Oglio et al. (19) who found that, from 2006 to 2016, only two validated instruments could properly assess parent satisfaction with FCC in NICUs and be considered as outcome indicators. The validated instruments found in this 2018 systematic review were the EMPATHIC-N developed by Latour et al. (20) and the Neonatal Satisfaction Survey-NSS-13 developed by Hagen et al. (21) The other nine studies evaluated in the review developed or modified previous questionnaires, including the Parent Satisfaction Survey (22), NICU parental satisfaction tool (23), and modified versions of the nurse–parent support tool (24, 25).

Development of the EMPATHIC Instruments

The 65-item EMPATHIC was developed to measure parent satisfaction in pediatric intensive care units (PICUs) (26). The EMPATHIC was developed in Dutch and published in an English journal (26). The shorter version, the EMPATHIC-30, was derived from the 65-item EMPATHIC (27). There is also a 57-item EMPATHIC-N, which was developed in the Netherlands to measure NICU parent satisfaction (20). The EMPATHIC-N has been translated into Portuguese (28), Greek (29), and Italian (30). The EMPATHIC (26), the EMPATHIC-30 (27), and the EMPATHIC-N (20) all contain five domains of satisfaction: information, care and treatment, organization, parental participation, and professional attitude.

Psychometric Testing of the EMPATHIC Instruments in Other Languages

The EMPATHIC, EMPATHIC-30, and EMPATHIC-N have been translated into and their psychometric properties have been established in seven languages (20, 26, 28–32). The EMPATHIC domains exhibited Cronbach alphas of 0.73–0.93 in the Netherlands (n = 2,046) and from 0.58 to 0.89 in France (n = 72) and Switzerland (n = 100) (26, 32). The EMPATHIC-30 exhibited domain-level Cronbach alphas of 0.56–0.89 across Spain (n = 150) and 0.73–0.81 in the Netherlands (n = 3,354) (27, 31). The Cronbach alphas for the total score for the EMPATHIC-30 was 0.93 from the Netherlands (27) and 0.95 from Spain (31) and 0.87 for the EMPATHIC-N from Greece (29). The EMPATHIC-N exhibited domain-level Cronbach alphas above 0.70 in samples from the Netherlands (n = 441), Italy (n = 162), Greece (n = 256), and Brazil (n = 40) (20, 28–30).

Linguistic Adaptation of the EMPATHIC

The EMPATHIC instruments were developed in the Netherlands and tested in Dutch but published in English-speaking journals for dissemination. A research team translated and culturally adapted the EMPATHIC-30 from Dutch to Australian English, resulting in the EMPATHIC-30-AUS, which they validated in the Australian setting of PICU, NICU, and pediatric inpatient wards (33). A USA English adaptation, however, has not been reported. There was, therefore, a need to make sure that the instrument was appropriate for testing among the English-speaking USA population. Beyond instrument translation from one language to another, even within the same language, different variants are needed to account for linguistic and cultural differences. For this reason, the wording of English surveys developed outside the USA need to be adapted to the USA vernacular and cultural context.

When adapting instruments across languages and countries, Flaherty et al. (34) have found that it is imperative that the survey adaptation undergoes an evaluation that involves content, conceptual, semantic, criterion, and technical equivalence to ensure appropriate use in the new location. When translation of survey instruments occurs, it is necessary to translate not only the literal meaning of the words but also the conceptual relation of the word to the context in order to meet the standards of “conceptual equivalence” (35, 36). Failure to meet adaptation requirements of conceptual and contextual equivalence can lead to unreliable and invalid results from a translated instrument (34, 37).

Materials and Methods

The research team selected and adapted a parent satisfaction instrument in accordance with the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria (38). The COSMIN Study Design checklist provides general recommendations on designing studies to evaluate the properties of patient-reported outcome measures (38). Nine other criteria for evaluating specific properties of a patient-reported outcome measurement include: content validity, structural validity, internal consistency, cross-cultural validity/measurement invariance, reliability, measurement error, criterion validity, hypotheses testing for construct validity, and responsiveness (38).

For this paper, we used the general recommendations for study design, which include: having a clear research aim, a clear description of the patient-reported outcome measure (including its development, origin, existing evidence, and context of use), and a clear description of the target population. We also used the content validity guidelines to support our addition of NICU-specific items to the selected instrument.

Consistent with recommendations from Wild et al. (39) regarding linguistic adaptation, we included cognitive debriefing as part of the process to determine the adequacy of linguistic changes. All 10 steps from Wild et al. (39) were not utilized because the EMPATHIC was published previously in English-speaking journals.

Following linguistic adaptation, we evaluated the resulting instrument's properties. Below, we describe instrument selection, expansion, and linguistic adaptation. Lastly, we discuss the approach to evaluating the instrument's psychometric properties.

Instrument Selection

Of the available EMPATHIC instruments, the EMPATHIC-30 was selected for our NICU study because of its much shorter length than the EMPATHIC-N (30 vs. 57 items), high degree of overlap with the EMPATHIC-N items, and an existing validation for the Spanish language. We were conducting the survey while the infant was in the NICU at the point of discharge and did not want to burden the parents with a much longer instrument at that time. The EMPATHIC-30 contained about half as many items as the EMPATHIC-N, and all but three of the 30 items were identical to EMPATHIC-N items. The three differing items addressed content highly relevant to the NICU parent (being well-prepared for discharge and being asked regularly for our experiences). Furthermore, the significant proportion of Spanish-speaking individuals in the United States calls for inclusion of a Spanish-language version to accommodate these parents. The only validated Spanish version was the EMPATHIC-30.

Instrument Expansion

For the purpose of our study focusing on nursing care and parent satisfaction in the NICU, we desired items that captured the unique nursing role vis-à-vis parents' developmental needs with their higher-risk newborn. Our intention was to increase the comprehensiveness of the instrument for the NICU setting per the COSMIN criteria. With the developer's permission, the research team, i.e., three nursing faculty, including a neonatal nurse practitioner and two economists experienced in NICU outcomes research, added six items from the 57-item EMPATHIC-N, which was developed for the NICU in order to capture the unique aspects of nursing care of NICU infants and their parents (20). The item selection process entailed reviewing all non-EMPATHIC-30 items of the 57 EMPATHIC-N items as candidates to supplement the EMPATHIC-30, keeping in mind the goal of minimizing survey burden. We selected the six items that most directly related to the nurse's unique role vis-à-vis the parents in the NICU. These items were identified as providing a more complete picture of NICU parent satisfaction. One item addressed emotional support, two items addressed the nurse encouraging and training the parent to care for the child, one item addressed the provider taking action immediately if the child's condition worsened, and one item addressed the team paying sufficient attention to the parents and child in spite of the workload. The last item, “our cultural background was taken into account,” was added to explore the heterogeneous cultural backgrounds of parents in USA NICUs. Two of these items had combined nurses and doctors in the EMPATHIC-N version. We split these into separate items to be more precise about doctors' and nurses' roles. The six added items, with two split into individual nurse and doctor items, yielded a total of eight additional items. We refer to the resulting instrument as the EMPATHIC-38.

Linguistic Adaptation of the English EMPATHIC-30-NICU-USA and EMPATHIC-38-NICU-USA

The 38 items were evaluated and modified by a USA neonatal nurse practitioner. The adaptation was evaluated by eight NICU parents. A Ph.D. student conducted cognitive debriefing with the parents individually. Based on the results of the debriefing, item wording was revised. The research team members reviewed the adapted items to confirm their wording. For USA Spanish-speaking parents, the Spanish-language version (31) was adapted to the USA cultural and linguistic context, which will be reported elsewhere.

Evaluating the Reliability and Validity of the English EMPATHIC-30-NICU-USA and EMPATHIC-38-NICU-USA

To evaluate the instrument's psychometric properties, we utilized data from a study of missed nursing care and NICU parent satisfaction (40). These data comprised the EMPATHIC-38-NICU-USA, which has as a subset the EMPATHIC-30-NICU-USA. Thus, both versions were evaluated. By adding the eight derived items from the validated precursor EMPATHIC-N instrument into the same domains of the EMPATHIC-30, the structural validity of each dimension and the scale was upheld. We used Cronbach's alpha to measure the reliability of the five domains and the total score for both the EMPATHIC-38-NICU-USA and the EMPATHIC-30-NICU-USA. We evaluated the correlation of domain score with total scale score. There are four additional items in the EMPATHIC instruments that measure overall parent experience that are not part of calculating the subscales or total score—but are used to assess the construct validity of the domains and total score. These items are described in section Validity Indicators below. To assess construct validity, we calculated Spearman's rank correlations between each of the four validity indicators with each domain and total score of the EMPATHIC-30-NICU-USA and EMPATHIC-38-NICU-USA. We evaluated differential validity in subgroups of respondents.

Recruitment and Data Collection

Secondary data from a study of 30 US NICUs were used. The University of Pennsylvania approved the study protocol from which the secondary data were generated. To serve as a participating site, NICUs obtained institutional review board (IRB) approval through one of three methods: IRB reliance at the University of Pennsylvania, having the project coordinator join the research team as site principal investigator, or obtaining separate IRB approval through their institution. The project coordinator approached eligible parents and explained the study purpose and protocol, including voluntary and confidential participation. The coordinator assured parents that if they chose to not participate, their infant's care and the parents' relationship with the infant's health care providers would not be affected. The coordinator explained to parents that by completing the survey, the parent was giving his or her consent to participate. In each NICU, 10 parents whose infants were scheduled to be discharged in the following 24 h were recruited to answer a survey comprising demographic questions, the EMPATHIC-38, and validity questions in 2018 (40). If an infant died or was transferred to another NICU, the parent was excluded from participating (40). For this psychometric evaluation, only parents who completed the English-language survey version were included. Further information about the NICU sample recruitment approach and inclusion criteria is detailed elsewhere (40).

In participating NICUs, hospital-affiliated project coordinators were identified who were responsible for on-site parent recruitment and overseeing the collection of data (40). At each site, the NICU nurse manager communicated with the project coordinator regarding parents who were eligible to participate (40). Each site's project coordinator recorded the number of eligible parents among those recruited for the purpose of calculating a participation rate (40). When parents were missed or decided not to participate, project coordinators recorded such cases (40). One example of a parent being missed would be if an infant was discharged before the parent could be recruited (40). Prior to data collection, project coordinators attended training webinars about parent recruitment and survey administration guidelines to promote data collection practices that were consistent across NICUs (40). Depending on the hospital's IRB requirements and preferences, a $25 cash or gift card incentive was offered to parent participants (40). IRB approval for this study was obtained from the University of Pennsylvania.

Measures

Parent and Infant Demographics

Demographic questions of the parent include items about race/ethnicity, age, level of completed education, and parent status (i.e., mother, father, both, guardian) of the respondent. Infant characteristic items include whether the infant weighed <1,500 g at birth, in order to capture very low birth weight status, and average length of stay in the NICU in days (Table 1).

TABLE 1

Table 1. Characteristics of infants and parents.

The EMPATHIC-30 and EMPATHIC-38

The EMPATHIC-30 was developed in the Netherlands to measure the satisfaction of parents of hospitalized children (27). It encompasses the care by nurses and physicians as well as organization and cleanliness. The five domains are listed in tabular results. Items were on a 1-to-6 Likert-type scale, with response options ranging from “certainly no” (1) to “certainly yes” (6), with a “not applicable” option. An overall score is calculated as the mean of all items. An expanded 38-item version was developed for our study as described above. The EMPATHIC-N items referred to “your child,” rather than your “infant” (20), and we retained this wording in our adapted version.

Validity Indicators

There were four validity measures: (1) recommend the NICU to others; (2) return to the NICU in the future if needed, both measured on a six-point scale from 0 (certainly no) to 6 (certainly yes); and (3) rate nurses' performance and 4) doctors' performance on a scale from 1 (very bad) to 10 (excellent). The first two validity measures have identical rating scales as the satisfaction items of the measure (six-point scale), while the last two control questions differ in scale (10-point scale) (20).

Data Analysis

Descriptive statistics were calculated and internal consistency, construct validity, and differential validity were evaluated for the 30- and 38-item versions. Descriptive statistics included means and standard deviations for each item, each domain, and the overall score.

Internal consistency reliability of the five domains and the overall score was measured by Cronbach's alpha, where coefficients of ≥0.7 demonstrate acceptable internal consistency (41). Construct validity was evaluated by computing the correlation between each domain and the overall score of each EMPATHIC version with each of the four validity indicators. Differential validity was evaluated by computing these correlations among demographic subgroups of parent respondents. Significance of correlations was based on a p < 0.05. Stata was used for data analyses.

Results

Linguistic Adaptation

Eight parents conducted the cognitive debriefing. The parents' ages ranged from teens to 30 s. Six females and two males were included. The parents were of black (n = 1) or white (n = 5) race and Hispanic ethnicity (n = 2). Five parents commented about the survey broadly. They stated that the items made sense to them. Five parents remarked about eight specific items. All of these parents commented about one item, which was originally (UK version): “The team worked hygienically.” The USA adapted version was: “The team was vigilant about maintaining cleanliness.” Based on parent feedback regarding whether the referent was the unit or the infant, the wording was revised to: “The team was attentive about maintaining hygiene.” The other items that the parents commented on were evaluated by the team. In total, minor linguistic adaptations were made to seven items. For example, “The NICU could be reached by telephone” was changed to “The NICU could be reached easily by telephone.”

Parent and Infant Sample Characteristics

Two-hundred eighty-two parents completed the English-language version. Table 1 displays the sample characteristics. Of the 275 parents who reported their race, 61% were White, 22% were Black or African American, 6% were Other, 5% were Multiracial, 5% were Asian, and 1% were Native American. Twenty percent of parents (n = 56) reported being Hispanic or Latino. The most common answer for highest level of education completed was “high school or a GED equivalent” (n = 118), and 95% of parents had at least a high school education. Three percent of parents indicated grade school completion as their highest level of education. Fifty-three percent of parents were between 25 and 34 years old, 24% were between 18 and 24 years old, 22% were between 35 and 44 years old, and 1% were between 45 and 64 years old. Mothers most often completed this questionnaire alone (73%); however, 14% of fathers completed the questionnaire alone, and 12% of mothers and fathers completed it together. In regard to infant characteristics, 20% of infants for which parents responded about weighed <1,500 g at birth. The average length of stay for infants in the NICU was 19 days.

Across the NICUs, an average of nine parents per NICU responded. In 20 units, all 10 parents completed the English version. From five to 10 parents per unit took the English version. The recruitment approach, which was to recruit the parents of consecutively discharged infants, was designed to achieve a sample representative of the population.

Item Means and Standard Deviations of EMPATHIC-30-NICU-USA and EMPATHIC-38-NICU-USA

The average percent of missing data was 0.66%. For the item “Our cultural background was taken into account,” 100 respondents marked this item not applicable. Means for the 38 items ranged from 4.93 (“During our child's NICU stay, the staff regularly asked about our experience in the NICU”) to 5.92 (“The nurses prepared us well for our child's discharge”); i.e., near the top of the theoretical range, for which a value of 6 corresponded to “certainly yes.” The most variation (SD) among parent experience ratings was found for the item about staff regularly asking about parent experience in the NICU (SD = 1.39). The least variation among parent experience ratings was for the item “The nurses talked to us daily about the care and treatment of our child” (SD = 0.35), and the satisfaction for this item was high at a mean of 5.91 out of 6 (Table 2). The items in the EMPATHIC-30-NICU-USA with the highest and lowest means and SDs were the same ones as noted for the EMPATHIC-38-NICU-USA.

TABLE 2

Table 2. Means and SDs of the statements of EMPATHIC-38-NICU-USA survey.

Overall Parent Experience–Validity Indicators

Table 3 presents results for overall parent experience, performance of doctors, and performance of nurses. Parents highly recommended the NICU to others (mean = 5.85, SD = 0.56) and would return to the NICU in the future if their child required intensive care (mean = 5.84, SD = 0.62). General performance of doctors was highly rated at a mean of 9.41 out of 10 (SD = 1.13). For nurses, general performance was similarly high at a mean of 9.62 (SD = 0.97).

TABLE 3

Table 3. Overall parent experience–validity indicators.

Comparison of the EMPATHIC-30-NICU-USA and the EMPATHIC-38-NICU-USA

Table 4 presents a comparison of the means, standard deviations, Cronbach's alphas, and number of items for the EMPATHIC-38-NICU-USA and EMPATHIC-30-NICU-USA. For the EMPATHIC-38-NICU-USA, there were three domains in which eight items were added: care and treatment (four items), parental participation (two items), and professional attitude (two items). Reliability results are reported below.

TABLE 4

Table 4. Descriptive statistics and alpha coefficients for the EMPATHIC-30-USA and−38-USA domains.

The means and SDs for the care and treatment domain and the professional attitude domain were essentially equivalent (i.e., <0.1 SD different) across the 30- and 38-item versions.

The parental participation domain mean was slightly higher in the EMPATHIC-38-NICU-USA than that in the EMPATHIC-30-NICU-USA (5.64 vs. 5.55). For the parental participation domain, the standard deviation was slightly reduced for the EMPATHIC-38 compared to that of the EMPATHIC-30-NICU-USA (SD 0.52 vs. 0.63).

Reliability of EMPATHIC-30-NICU-USA and EMPATHIC-38-NICU-USA

In our NICU parent sample, both the EMPATHIC-38-NICU-USA and the EMPATHIC-30-NICU-USA demonstrated good overall reliability, with overall Cronbach's alphas of 0.9 and 0.92 (Table 4). The Cronbach's alphas for specific domains were very similar across the 38- and 30-item versions, with the exception of the domains with additional items in the 38-item version, which exhibited higher alpha coefficients. Notably, the additional four items in the care and treatment domain in the EMPATHIC-38 increased the Cronbach's alpha to 0.84 from 0.75 in the EMPATHIC-30. In both EMPATHIC versions, the Cronbach's alphas for four domains exceeded 0.70: information, care and treatment, parental participation, and professional attitude. The Cronbach's alpha for the organization domain (0.68) was slightly below the standard of 0.70 for measuring scale internal consistency (41).

Validity of EMPATHIC-30-NICU-USA and EMPATHIC-38-NICU-USA

For the EMPATHIC-30-NICU-USA, the Spearman's rank correlations between each domain and the four validity indicators were small to moderate, positive, and statistically significant. The correlations ranged from 0.30 to 0.57 (Table 5). The highest correlations were for the two performance indicators with the total score ≥0.50. The weakest correlations were for the domains information and organization (mean r of 0.30 and 0.32, respectively, both come back to NICU). The strongest correlations were for the domains care and treatment and professional attitude (mean r of 0.56 and 0.55, respectively, both doctor's performance).

TABLE 5

Table 5. Spearman's rank correlation of English EMPATHIC-30-NICU-USA and EMPATHIC-38-NICU-USA.

For the EMPATHIC-38-NICU-USA, the Spearman's rank correlations between the information and organization domains and validity items were the same as for the EMPATHIC-30-NICU-USA because items for these domains were not modified. For the care and treatment domain, correlations ranged from 0.34 (come back to NICU) to 0.60 (doctor's performance), which were higher than those in the EMPATHIC-30-NICU-USA. For parent participation and professional attitude, the additional two items in each domain did not change the average correlations with the validity indicators.

For the EMPATHIC-30-NICU-USA, the Pearson's rank correlations between each domain and the overall scale were moderate to large, positive, and statistically significant. The domain correlations to total EMPATHIC-30 score ranged from 0.74 to 0.87 (p < 0.001; Table 6). The highest correlation was for the care and treatment domain. Similarly, for the EMPATHIC-38-NICU-USA, the Pearson's rank correlations between each domain and the overall scale were moderate to large, positive, and all statistically significant. The highest correlation was between the care and treatment domain and the total score (0.91, p < 0.001), while the lowest correlation was found with the organization domain (0.71, p < 0.001). Overall, each domain in both the EMPATHIC instruments was highly correlated with the overall score.

TABLE 6

Table 6. Pearson's correlation of each domain score with English EMPATHIC-30-NICU-USA and EMPATHIC-38-NICU-USA.

Differential validity results are presented in Supplementary Table 1 through M. Differential validity was supported across nine different subgroups by significant correlation coefficients in the hypothesized direction. In five subgroups (i.e., White, non-Hispanic, up to and completion of high school education, above high school education, and mothers as respondents), all correlations were similar in magnitude to the whole sample and were highly significant (p < 0.001). In four subgroups (i.e., non-white race, young adults from 18 to 24, over age 24, and infants weighed <1,500 g at birth), some correlations were weaker, although still significant at the p < 0.01 or p < 0.05 levels.

In four subgroups (i.e., Black, Hispanic, fathers as respondent, and mothers and fathers together as respondent), some correlations between domain scores and differential validity indicators were weaker and non-significant. Notably, the information subscale exhibited the weakest correlations with the validity indicators for the black, non-white, Hispanic, and fathers only as respondents subgroups. The young adults had stronger correlation coefficients than the middle-years adults.

For the black subgroup, there was a weak and statistically non-significant correlation between the information domain and “recommend NICU,” as well as between the information domain and “come back to the NICU.” There was, additionally, a weak and non-statistically significant correlation between the organization domain and “nurse's performance” for the black subgroup.

The Hispanic subgroup exhibited weak and non-significant correlations between the information domain and “recommend to NICU,” “come back to NICU,” and “nurse's performance,” the care and treatment domain and “recommend to NICU” (EMPATHIC-30-NICU-USA only), the organization domain and “come back to NICU,” and the professional attitude domain and “nurse's performance.” The Hispanic subgroup exhibited a weak and non-significant correlation between the professional attitude domain and “come back to NICU” for the EMPATHIC-38-NICU-USA.

The father respondent subgroup exhibited weaker and statistically non-significant correlations between the information domain and “recommend NICU,” the information domain and “come back to NICU,” and the care and treatment domain and “recommend to NICU.” The father respondent subgroup exhibited a weak and significantly non-sufficient correlation between the EMPATHIC-38-NICU-USA professional attitude domain and “recommend NICU.”

Compared to father respondents alone, for mother and father respondents, correlations were greater between the information domain and “recommend NICU” and between the information domain and “come back to NICU” but were statistically insignificant. For mother and father respondents, correlations were weak and statistically non-significant between the care and treatment domain and “recommend NICU,” as well as between the care and treatment domain and “come back to NICU.” The EMPATHIC-38-NICU-USA total score was not significantly correlated with “come back to the NICU.”

Discussion

The parent/infant dyad in the NICU is the focal unit of nursing care due to the infant's unique developmental needs. Furthermore, fostering the parent/infant bond and the parent's role in caregiving following discharge are key expectations of care providers and managers during a NICU stay. NICU parent satisfaction is an important indicator of FCC. Among several instruments for measuring NICU parent satisfaction, the EMPATHIC instrument was specific to the critical care setting and includes several domains that address care provider performance. The EMPATHIC-30 was selected for a national NICU study in the USA. To incorporate unique care features of the NICU as compared to pediatric settings, several items were added to the EMPATHIC-30, yielding the EMPATHIC-38. In preparing to survey US NICU parents, the research team identified a compelling need to linguistically and culturally adapt the instrument to be widely available to the culturally diverse racial and ethnic US NICU parents.

This paper describes the linguistic adaptation and the results. We found that both EMPATHIC-NICU-USA versions performed satisfactorily. Although parents were highly satisfied overall, items exhibited variation in their means from a low of 4.93 for “the staff regularly asked about our experience in the NICU” to a high of 5.92 for “the nurses prepared us well for our child's discharge.” There was greater agreement regarding the item “The nurses talked to us daily about the care and treatment of our child,” indicated by a low SD of 0.35. At the other extreme, the largest SD (1.39) was observed for “the staff regularly asked about our experience in the NICU.”

Both EMPATHIC-NICU-USA versions exhibited high reliability for the total score and several domains and good reliability (>0.70) across most domains. One domain, organization, had lower-than-desired reliability at 0.68. The domains with additional items in the EMPATHIC-38-NICU-USA had higher reliability (Cronbach's alpha) than the EMPATHIC-30-NICU-USA domains.

The validity of the EMPATHIC-NICU-USA versions was supported by significant correlations with the concurrent validity indicators (nurse/physician performance, recommend NICU, return to NICU if needed). The validity values of the EMPATHIC-NICU-USA demonstrated overall Spearman's rank correlations ranging from 0.30 to 0.60, p < 0.001. The domains with additional items and total score in the EMPATHIC-38-NICU-USA had slightly higher (8), equivalent (4), or lower (4) correlation coefficients than the respective EMPATHIC-30-NICU-USA domains. Three of the four weaker correlations were 0.01 lower than the correlation for the comparable domain. One correlation that was 0.05 lower was for the professional attitude domain and the physician performance indicator.

The concurrent validity indicators used were consistent with the general satisfaction items used in the countries Spain (31), the Netherlands (20, 26, 27), France (32), Switzerland (32), and Australia (33). The remaining referenced papers did not report Spearman's correlation to demonstrate validity (28, 29). The range of validity values of the EMPATHIC-NICU-USA was consistent with the EMAPTHIC-30 in the Netherlands (0.37–0.51) (20), Australia (0.38–0.69) (33), and France and Switzerland (0.25–0.63) (32). The range of correlation coefficients was slightly lower than those seen by the EMPATHIC-65 in the Netherlands (0.40–0.59) (26) and was slightly higher than the coefficients demonstrated by the EMPATHIC-N in Italy (0.22–0.57) (30).

The EMPATHIC-38-NICU-USA and EMPATHIC-30-NICU-USA demonstrated overall Cronbach's alphas of 0.94 and 0.92. These values are consistent with high Cronbach alphas in the Netherlands (0.93) (27), Spain (0.95) (31), and Australia (0.91) (33) and higher than in Greece (0.87) (29) and Brazil (>0.70 in all domains) (26). The EMPATHIC-NICU-USA versions had similar domain-level alphas (0.68–0.84) as other linguistic adaptations. In both EMPATHIC-NICU-USA versions, the Cronbach's alphas for four domains exceeded 0.70 except for the organization domain (0.68). Similarly, in other linguistic adaptations, the organization domain occasionally fell below the standard of 0.70. The EMPATHIC-30 exhibited an organization domain-level alpha of 0.56 from Spain (n = 150) (31) and 0.56 from Australia (n = 97) (33), and the EMPATHIC-65 showed an organization domain-level alpha of 0.58 from France (n = 72) and Switzerland (n = 100) (32).

Based on the satisfactory psychometric performance of both versions, we recommend that researchers use the 30-item version for pediatric intensive care settings and the 38-item version for NICU settings.

Our study limitations include substantial “not applicable” responses on one item regarding parents' cultural background being taken into account, which reduced our ability to evaluate its performance. We did not test the EMPATHIC-30 in pediatric intensive care in this study. We encourage USA researchers to test psychometric properties of the linguistically adapted EMPATHIC-30-NICU-USA for PICU settings because the items are identical to the EMPATHIC-30, which was tested originally in the PICU. We did not evaluate whether the structural validity of each domain and the whole scale of the EMPATHIC-30-NICU-USA was upheld in the EMPATHIC-38-NICU-USA after adding items from the EMPATHIC-N, although we added only a few items from the same domains. We note that only three of five domains were altered. The structural validity may show similar factors, which could be tested in future research.

Conclusion

The linguistically and culturally adapted EMPATHIC-30-NICU-USA and the expanded EMPATHIC-38-NICU-USA exhibited satisfactory psychometric properties and are therefore suitable for use in USA NICUs. These satisfactory properties included high internal consistency reliability (>0.90) for the overall score and good internal consistency reliability (>0.80) for four of the five domains. The organization domain was slightly below the acceptable threshold (>0.70). Concurrent validity was supported by significant positive correlations with parent ratings of nurse and physician performance and recommending the NICU to others or willingness to return to the NICU in the future. The 38-item version, which had additional items in three domains (i.e., care and treatment, parent participation, and professional attitude), had higher internal consistency and higher or equivalent correlations for three-quarters of the validity comparisons. The lower correlations were trivially different. We conclude that the EMPATHIC-38-NICU-USA had slightly better performance in measuring parent satisfaction in USA NICUs compared to the EMPATHIC-30-NICU-USA. The validity results were replicated in demographic subgroups. We encourage other researchers to translate this instrument into their languages to increase comparable global evidence and attention to NICU parent satisfaction.

Data Availability Statement

The datasets generated for this study are available on request to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by University of Pennsylvania Institutional Review Board. By completing the survey, participants were giving his or her consent to participate.

Author Contributions

EL, JS, DS, and JR: study concept, design, and data acquisition. EL, JS, DS, KS, and JR: data analysis, interpretation, and manuscript preparation. All authors reviewed and edited the final manuscript.

Funding

This project was supported by Grant No. R01HS024918 (Lake, PI) from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. JS was supported by National Institutes of Health, National Institute of Nursing Research Grant No. T32NR007104 (Aiken, PI).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2020.541573/full#supplementary-material

References

1. Giordano LA, Elliott MN, Goldstein E, Lehrman WG, Spencer PA. Development, implementation, and public reporting of the HCAHPS survey. Med Care Res Rev. (2010) 67:27–37. doi: 10.1177/1077558709341065

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Wolf DM, Lehman L, Quinlin R, Zullo T, Hoffman L. Effect of patient-centered care on patient satisfaction and quality of care. J Nurs Care Qual. (2008) 23:316–21. doi: 10.1097/01.NCQ.0000336672.02725.a5

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Flocke SA, Miller WL, Crabtree BF. Relationships between physician practice style, patient satisfaction, and attributes of primary care. J Fam Pract. (2002) 51:835–41.

PubMed Abstract | Google Scholar

4. Winefield H, Murrell T, Clifford J, Farmer E. The search for reliable and valid measures of patient-centredness. Psychol Health. (1996) 11:811–24. doi: 10.1080/08870449608400277

CrossRef Full Text | Google Scholar

5. Kinnersley P, Stott N, Peters TJ, Harvey I. The patient-centredness of consultations and outcome in primary care. Br J Gen Pract. (1999) 49:711–16.

PubMed Abstract | Google Scholar

6. Institute of Medicine Committee on Quality of Health Care in A. In: Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press (US) (2001)

Google Scholar

7. Harrison H. The principles for family-centered neonatal care. Pediatrics. (1993) 92:643–50.

PubMed Abstract | Google Scholar

8. Ramezani T, Shirazi ZH, Sarvestani RS, Moattari M. Family-centered care in neonatal intensive care unit: a concept analysis. Int J Commun Based Nurs Midwifery. (2014) 2:268–78.

PubMed Abstract | Google Scholar

9. Davidson JE, Aslakson RA, Long AC, et al. Guidelines for family-centered care in the neonatal, pediatric, and adult ICU. Crit Care Med. (2017) 45:103–28. doi: 10.1097/CCM.0000000000002169

CrossRef Full Text | Google Scholar

10. Butt ML, McGrath JM, Samra HA, Gupta R. An integrative review of parent satisfaction with care provided in the neonatal intensive care unit. J Obstet Gynecol Neonatal Nurs. (2013) 42:105–20. doi: 10.1111/1552-6909.12002

PubMed Abstract | CrossRef Full Text | Google Scholar

11. McGrath JM. Strategies for increasing parent participation in the neonatal intensive care unit. J Perinatal Neonatal Nurs. (2011) 25:305–6. doi: 10.1097/JPN.0b013e318235e584

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Bastani F, Abadi TA, Haghani H. Effect of family-centered care on improving parental satisfaction and reducing readmission among premature infants: a randomized controlled trial. J Clin Diagn Res. (2015) 9:SC04–8. doi: 10.7860/JCDR/2015/10356.5444

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Ell K. Social networks, social support and coping with serious illness: the family connection. Soc Sci Med. (1996) 42:173–83. doi: 10.1016/0277-9536(95)00100-X

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Lv B, Sun J, Latour JM. Family-centered care improves clinical outcomes of very-low-birth-weight infants: a quasi-experimental study. Front Pediatr. (2019) 7:138. doi: 10.3389/fped.2019.00138

PubMed Abstract | CrossRef Full Text | Google Scholar

15. O'Brien K, Robson K, Bracht M, Cruz M, Lui K, Alvaro R, et al. Effectiveness of family integrated care in neonatal intensive care units on infant and parent outcomes: a multicentre, multinational, cluster-randomised controlled trial. Lancet Child Adolesc Health. (2018) 2:245–54. doi: 10.1016/S2352-4642(18)30039-7

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Verma A, Maria A, Pandey RM, Hans C, Verma A, Sherwani F. Family-centered care to complement care of sick newborns: a randomized controlled trial. Indian Pediatr. (2017) 54:455–9. doi: 10.1007/s13312-017-1047-9

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Ding X, Zhu L, Zhang R, Wang L, Wang T-T, Latour JM. Effects of family-centred care interventions on preterm infants and parents in neonatal intensive care units: a systematic review and meta-analysis of randomised controlled trials. Austr Crit Care. (2019) 32:63–75. doi: 10.1016/j.aucc.2018.10.007

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Yu Y-T, Hsieh W-S, Hsu C-H, Lin YJ, Lin CH, Hsieh S, et al. Family-centered care improved neonatal medical and neurobehavioral outcomes in preterm infants: randomized controlled trial. Phys Ther. (2017) 97:1158–68. doi: 10.1093/ptj/pzx089

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Dall'Oglio I, Mascolo R, Gawronski O, Tiozzo E, Portanova A, Ragni A, et al. A systematic review of instruments for assessing parent satisfaction with family-centred care in neonatal intensive care units. Acta Paediatr. (2018) 107:391–402. doi: 10.1111/apa.14186

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Latour JM, Duivenvoorden HJ, Hazelzet JA, van Goudoever JB. Development and validation of a neonatal intensive care parent satisfaction instrument. Pediatr Crit Care Med. (2012) 13:554–9. doi: 10.1097/PCC.0b013e318238b80a

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Hagen IH, Vadset TB, Barstad J, Svindseth MF. Development and validation of Neonatal Satisfaction Survey–NSS-13. Scand J Caring Sci. (2015) 29:395–406. doi: 10.1111/scs.12156

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Hurst I. One size does not fit all: parents' evaluations of a support program in a newborn intensive care nursery. J Perinatal Neonatal Nurs. (2006) 20:252–61. doi: 10.1097/00005237-200607000-00014

CrossRef Full Text | Google Scholar

23. Byers JF, Lowman LB, Francis J, Kaigle L, Lutz NH, Waddell T, et al. A quasi-experimental trial on individualized, developmentally supportive family-centered care. J Obstet Gynecol Neonatal Nurs. (2006) 35:105–15. doi: 10.1111/j.1552-6909.2006.00002.x

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Tran C, Medhurst A, OConnell B. Support needs of parents of sick and/or preterm infants admitted to a neonatal unit. In: Paper presented at: PSANZ 2008: The Perinatal Society of Australia and New Zealand 12th Annual Scientific Congress 2008 (Gold Coast QLD).

Google Scholar

25. Domanico R, Davis DK, Coleman F, Davis BO. Documenting the NICU design dilemma: comparative patient progress in open-ward and single family room units. J Perinatol. (2011) 31:281–8. doi: 10.1038/jp.2010.120

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Latour JM, van Goudoever JB, Duivenvoorden HJ, Albers MJ, van Dam NA, Dullaart E, et al. Construction and psychometric testing of the EMPATHIC questionnaire measuring parent satisfaction in the pediatric intensive care unit. Intensive Care Med. (2011) 37:310–18. doi: 10.1007/s00134-010-2042-y

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Latour JM, Duivenvoorden HJ, Tibboel D, Hazelzet JA. The shortened EMpowerment of PArents in THe Intensive Care 30 questionnaire adequately measured parent satisfaction in pediatric intensive care units. J Clin Epidemiol. (2013) 66:1045–50. doi: 10.1016/j.jclinepi.2013.02.010

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Gomez DB, Vidal SA, Lima LC. Brazilian adaptation and validation of the empowerment of parents in the intensive care-neonatology (EMPATHIC-N) questionnaire. J Pediatr. (2017) 93:156–64. doi: 10.1016/j.jped.2016.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Papamichael E, Ioannou M, Talias MA. EMPATHIC-N in a Greek-Cypriot sample: confirming its factorial structure. BMC Health Serv Res. (2018) 18:968. doi: 10.1186/s12913-018-3793-3

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Dall'Oglio I, Fiori M, Tiozzo E, Mascolo R, Portanova A, Gawronski O, et al. Neonatal intensive care parent satisfaction: a multicenter study translating and validating the Italian EMPATHIC-N questionnaire. Ital J Pediatr. (2018) 44:5. doi: 10.1186/s13052-017-0439-8

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Pilar Orive FJ, Basabe Lozano J, Lopez Zuniga A, Lopez Fernandez YM, Escudero Argaluza J, Latour JM. [Spanish translation and validation of the EMPATHIC-30 questionnaire to measure parental satisfaction in intensive care units]. An Pediatr. (2018) 89:50–57. doi: 10.1016/j.anpedi.2017.08.004

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Grandjean C, Latour JM, Cotting J, Fazan MC, Leteurtre S, Ramelet AS. Measurement of parent satisfaction in the paediatric intensive care unit - translation, cultural adaptation and psychometric equivalence for the French-speaking version of the EMPATHIC-65 questionnaire. Intensive Crit Care Nurs. (2017) 38:40–45. doi: 10.1016/j.iccn.2016.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Gill FJ, Wilson S, Aydon L, Leslie GD, Latour JM. Empowering parents of Australian infants and children in hospital: translation, cultural adaptation, and validation of the EMpowerment of parents in the intensive care-30-AUS questionnaire. Pediatr Crit Care Med. (2017) 18:e506–e513. doi: 10.1097/PCC.0000000000001309

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Flaherty JA, Gaviria FM, Pathak D, Mitchell T, Wintrob R, Richman JA, et al. Developing instruments for cross-cultural psychiatric research. J Nervous Ment Dis. (1988) 176:257–63. doi: 10.1097/00005053-198805000-00001

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Squires A. Methodological challenges in cross-language qualitative research: a research review. Int J Nurs Stud. (2009) 46:277–87. doi: 10.1016/j.ijnurstu.2008.08.006

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Jandt FE. An Introduction to Intercultural Communication: Identities in a Global Community. Thousand Oaks, CA: Sage Publications (2017).

Google Scholar

37. Maneesriwongul W, Dixon JK. Instrument translation process: a methods review. J Adv Nurs. (2004) 48:175–86. doi: 10.1111/j.1365-2648.2004.03185.x

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Mokkink LB, Prinsen CAC, Patrick DL, Alonso J, Bouter LM, de Vet HCW, et al. COSMIN Study Design Checklist for Patient-Reported Outcome Measurement Instruments. (2019). Available online at: https://www.cosmin.nl/ (accessed August 13, 2020).

Google Scholar

39. Wild D, Grove A, Martin M, Eremenco S, McElroy S, Verjee-Lorenz A, et al. Principles of good practice for the translation and cultural adaptation process for patient-reported outcomes (PRO) measures: report of the ISPOR task force for translation and cultural adaptation. Value Health. (2005) 8:94–104. doi: 10.1111/j.1524-4733.2005.04054.x

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Lake ET, Smith JG, Staiger D, Hatfield LA, Cramer E, Kalisch BJ, et al. Parent satisfaction with care and treatment relates to missed nursing care in neonatal intensive care units. Front Pediatr. (2020) 8:74. doi: 10.3389/fped.2020.00074

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. New York, NY: McGraw-Hill (1994).