Comparison of Social Validity Ratings With the Effects of a Video-Feedback Intervention for Communication Partners of Individuals With Deafblindness

The parents, teachers, and professional caregivers of individuals with disabilities may benefit from interventions to enhance their educational skills. In previous studies, positive effects were observed of a video-feedback intervention for caregivers (i.e. parents, teachers, and professional caregivers) on their communication with an individual with congenital deafblindness. The intervention they received, was the High-Quality Communication (HQC) intervention. The aim of the current study was to gain insight into the perceived relevance, feasibility, and effectiveness (i.e., social validity) of the HQC-intervention according to these caregivers, and the correspondence between the social validity ratings and the observational effects of the HQC-intervention. Responses on the Social Validity Scale from 25 caregivers who participated in the High-Quality Communication (HQC) intervention revealed that they considered the HQC intervention to be a relevant, feasible, and effective intervention. Comparing the caregiver ratings with observational effects of the HQC intervention at the individual case level, we found no association between the observed effectiveness of this intervention and caregivers’ opinions about its relevance, feasibility and effectiveness. There was however, an association between the rated feasibility and effectiveness, which suggests that the perceived success of the intervention was influenced by caregivers’ experienced competency in supporting the communication of individuals with CDB. The combination of observational and social validity data enabled a critical analysis of the clinical value of the HQC intervention. We recommend that future studies use multiple data source for social validity assessment.

Introduction

In the field of education and care for people with disabilities, there is a growing emphasis on using interventions that improve their participation and quality of life (see Singer et al., 2017). The use of interventions that have proven to be effective in professional practice makes such practice “evidence-based.” According to Cook et al., evidence-based practice in special education means that an operationally defined practice or program is used, for which positive effects have been found on student outcomes in group comparison research or single-subject research, that is methodologically sound (Cook et al., 2014).

Selecting evidence-based interventions for individuals with disabilities requires a collaborative process involving professionals, the individuals with disabilities, and their significant others. To ensure that an intervention is a good “fit,” it is important to determine beforehand which intervention meets the individual’s needs, characteristics, and circumstances (American Psychological Association, 2020). Individuals with congenital deafblindness (CDB; i.e., with combined hearing and visual impairments from birth or before the start of language development; Dammeyer, 2012; Ask Larsen and Damen, 2014) are often unable to participate in decision making about interventions. In these cases, family members or legal guardians, in collaboration with professionals, need to judge whether an intervention is required and, if so, which intervention fits best with the specific needs and characteristics of the individual.

In care and education for individuals with deafblindness, interventions aimed at improving their participation and quality of life commonly focus on the communication skills of their parents, teachers, or professional caregivers (Janssen and Damen, 2018). The individual communication support needs of these caregivers and educators and the characteristics of the communication setting are therefore the topics on which the intervention selection process needs to focus.

The extent to which caregivers and educators agree with intervention goals and procedures provides information about the “social validity” of the intervention. Social validity generally refers to the extent to which “the focus of the intervention and the behavior changes that have been achieved meet the demands of the social community of which the individual is a part” (Kazdin, 1982, p. 18). According to Kazdin (1977), research on the effects of behavioral treatments should not only present the statistical significance of effects but also assess their social validity. Kazdin argues that “small changes in behaviors, even if reliable, may not be sufficient as a criterion for claiming the success of a treatment. The change in behavior must resolve or significantly ameliorate the problem for which the client sought treatment” (Kazdin, 1977, p. 427). To exemplify his argument, Kazdin described a child with autism who, during treatment, exhibited 60 instances of self-destructive behavior per hour instead of the 100 instances he did before treatment. Although the treatment affected the child’s behavior, the intervention did not help the child reach acceptable levels of behavior. In this case, the treatment therefore did not result in a clinically important change (Kazdin, 1977).

Kazdin believes that measurement of the social validity of applied interventions should focus on three criteria: 1) acceptability of the focus of the intervention, 2) acceptability of the procedure, and 3) satisfaction with the results. Strain and Dunlap (2012) describe examples of their use of social validity data in determining the focus and procedures of interventions. They applied a simple interview format to ask family members of children with autism which child-rearing routines caused most stress, and used this information as input for designing an early intervention program. In another example, school-based teams were involved in a field-test of an intervention in which they are guided in developing and implementing behavior support plans. The input of the teams was used to improve the steps and the manual of the intervention. With regard to the acceptability of intervention procedures, information was used from ratings of family members’ comfort level and competence with selected intervention strategies for their child with autism, to determine the length of support these family members received. Strain et al. also explain how they used information on family members’ most valued outcome of an early intervention program for children with autism for the measurement of their satisfaction with this program. The family members expressed that they were most interested in finishing routines with their child in a timely fashion, which made the researchers decide to measure the time to complete a routine in minutes repeatedly, during the baseline and intervention phases (Strain and Dunlap, 2012).

The current study aims to gain insight into the social validity of a video-feedback communication intervention for caregivers of individuals with deafblindness. Video-feedback interventions are pedagogical programs in which parents use guided viewing of interactions on video to evaluate their participation in family interactions (Fukkink, 2008). A meta-analysis of 29 studies revealed that video-feedback interventions have been demonstrated to positively affect parenting behavior and child development (Fukkink, 2008). A randomized controlled trial involving 48 professional caregivers of 23 children who received home-based child care showed, that a video-feedback intervention improved global child care in the intervention group, but not in the control group (Groeneveld et al., 2011). A video-feedback training also appeared to have led to a significant improvement of the quality of interactions between 72 professional caregivers and children and adults with visual and intellectual disabilities in a group home setting, when the last baseline observation of these interactions were compared with the first observation after the start of the video-feedback training (Damen et al., 2011).

In the current study, a video-feedback intervention called the High-Quality Communication (HQC) intervention was used with caregivers of individuals with deafblindness. The caregivers we refer to in this acticle support the daily functioning and development of these individuals with deafblindness at home, at a daycare facility, or at school. Positive effects of the HQC intervention in 11 individual cases have been demonstrated in previous studies (Damen, 2015; Damen et al., 2014; Damen et al., 2015a; Damen et al., 2015b) on communication categories that correspond with three layers of intersubjective development such as described by Trevarthen (See Bråten and Trevarthen, 2007): “dyadic interaction” and “shared emotion” at the first layer, “referential communication,” “meaning negotiation,” and “shared meaning” at the second layer, and “declarative communication” at the third layer. The nonoverlapping of all pairs (NAP) technique (Parker and Vannest, 2009), revealed medium or large effects for all participants with deafblindness for at least one of the communication categories at the first and at least one of the categories at the second layer when comparing the baseline phase with either the first or second intervention phases or the follow-up phase. For four of the 11 cases, a medium or large effect size was also found in declarative communication, a communication category at the third layer of intersubjective development, when the baseline phase was compared with either the first, second or follow-up phase (Damen, 2015).

For six cases (Damen et al., 2014; Damen et al., 2015a), the social validity of the intervention was also assessed. As in other studies that measured the social validity of video-feedback interventions for individuals with CDB (see Janssen et al., 2002; Janssen et al., 2004; Janssen et al., 2006; Janssen et al., 2007; Janssen et al., 2011; Martens et al., 2014a; Martens et al., 2014b; Martens et al., 2017), the satisfaction of the caregivers with the results of the intervention was determined. However, in the analysis, the focus was on the overall satisfaction and not on the satisfaction with the results for single intervention goals. In contrast with the abovementioned studies, it was also not analyzed to what extent caregivers believed the intervention procedure of the HQC intervention was acceptable.

Information about caregivers’ opinions regarding the focus and procedures of the HQC intervention may be relevant when interpreting its effectiveness. Several studies have shown that the extent to which clients believe in the benefits of an intervention, influences its success. This phenomenon is known as the effect of “treatment expectation” and is explained as follows: if a person strongly believes in their treatment, they will be more engaged in the treatment, which will positively influence its effectiveness (see Constantino et al., 2012 for a review). To our knowledge, effects of treatment expectation are predominantly studied in the context of psychotherapy and have so far not been investigated in studies involving people with disabilities. However, we expect that the phenomenon of treatment expectation and the underlying mechanisms are also applicable for interventions for this target group. A study by McClintock et., 2015 and colleagues showed that participants with higher expectations of their psychotherapy had a better alliance with their therapist, were more positive about the therapy sessions, and achieved better results. High expectations of an intervention imply positive opinions about the focus and procedures of that intervention. Such opinions can be measured with social validation instruments. In addition to information about expectations and satisfaction, social validity measurement can also reveal how caregivers experienced the feasibility of the HQC intervention.

The feasibility of an intervention is an important factor in its successful implementation (Daamen, 2013). A study on the use of personal support plans for individuals with severe multiple disabilities found that the proper use of this intervention by support staff – also known as the program integrity – was negatively influenced by the severity of those individuals’ impairments (Zijlstra, 2003). This suggests that there is a relationship between the feasibility of an intervention and its program integrity, which means that feasibility may have a mediating influence on the effectiveness of an intervention.

For this study, we formulated the following Questions: 1) What are the caregivers’ opinions about the relevance of the focus and procedures of the HQC intervention? 2) How do the caregivers evaluate the effectiveness and feasibility of the HQC intervention? 3) To what extent are caregivers’ opinions about the relevance, feasibility and effectiveness of the intervention associated with the observed effectiveness of the intervention?

Materials and Methods

Participants and Setting

11 individuals with congenital deafblindness (CDB) participated in this study. All the individuals exhibited serious cognitive delays, possibly as a result of dual sensory loss since birth or before the start of language development. The participants with CDB were clients of one of two organizations in the Netherlands that provide services to people with deafblindness. The study received approval from the institutional review boards of both organizations. The study adhered to the ethical principles for the involvement of human subjects in medical research that are formulated in the World Medical Association Declaration of Helsinki.

Participant selection was based on the following criteria: 1) visual acuity of less than 0.3 LogMAR, 2) hearing loss of 35 dB or more, 3) sensory disabilities appeared from birth or before the start of language development, 4) a developmental age of nine months or more, 5) a lack of severe epilepsy, and 6) informed consent for their participation by their legal representatives. For each of the 11 participants with CDB, one to four caregivers were involved in the study. In total, 34 caregivers received the intervention, of which 25 caregivers (74% of all caregivers) fully completed the social validity questionnaire. These 25 caregivers were involved in the current study. They included two parents, three teachers, and 20 professional caregivers. Table 1 displays the participants’ characteristics.

TABLE 1

TABLE 1. Characteristics of 11 participants with multiple disabilities.

Intervention

The HQC intervention is set up as a training that is carried out by a communication coach. The training consists of two phases of subsequently 5 and 15 weeks (see Damen et al., 2015a; Damen et al., 2015b). In the first phase, the focus is on improving the attunement of the social partners’ behavior and emotions to those of the individual with deafblindness in order to improve dyadic interactions and shared emotions. Caregivers received education and video feedback from a coach in two individual sessions and one group session. The role of the coach during the video-feedback sessions was to activate the caregivers in reviewing their video-recorded interactions with the individual with CDB, to support them in recognizing their strengths as communication partners, and in setting goals for improving the communication with the individual with DB. In the first phase, the caregivers learned how to recognize and respond to initiatives from the individual with disabilities during interactions.

In the second phase of the intervention, the social partners received education: three sessions of individual video feedback and one group video-feedback session. The aim of the second intervention phase is to enhance caregivers’ ability to recognize communicative attempts from the individual with disabilities, to stimulate the use of communication to refer to objects, people, or situations, and to share thoughts. For example, the caregivers learned how to stimulate the individual with disabilities to give more information about the meaning of their utterances, if this meaning was not clear to the caregiver. The coach had a similar role as in the first phase. However, since the communication abilities in the second phase were more complex than those in the first phase, more video-feedback sessions were carried out by the coach and a longer duration was chosen for this phase.

Measures

Social Validity Evaluation

To measure the caregivers’ opinions about the intervention, we used the Social Validity Scale (SVS; Damen et al., 2011, according to Seys, 1987). The SVS is an existing scale to which a previous study added extra questions about the effectiveness of the intervention on meaning-making aspects (Damen et al., 2014). In the current project, we used the Subjective Effectiveness and Feasibility subscales, both of which showed good internal consistency (Cronbach’s alpha of 0.86 and 0.84).

Relevance

Furthermore, we used three statements to assess caregivers’ opinions about the relevance of the intervention. The first statement was used as an indication of the caregiver’s opinion about the intervention’s focus on supporting caregivers in order to improve communication with their clients with deafblindness. The two other statements concerned aspects of the intervention procedure: the use of a communication coach and the capturing of their communicative interactions on video. Caregivers were asked to evaluate these ideas and respond by choosing one of the following options: 1) very bad idea, 2) bad idea, 3) acceptable idea, 4) good idea, or 5) very good idea.

Feasibility

In the Feasibility subscale, similar categories were addressed in the statements but the caregiver was asked to rate how difficult they found it to use behavior categories in the interaction with the participant with deafblindness. For example, in item 13, the caregiver was asked to choose which answer best completed the sentence “I found sharing emotions….” The options were: 1) very difficult, 2) difficult, 3) feasible, 4) easy, or 5) very easy.

Subjective Effectiveness

In the Subjective Effectiveness subscale, caregivers were asked to rate the amount of positive change in communication categories. For example, item 9 states: “Because of the intervention, my skills in sharing emotions with the participant with deafblindness have been changed in a ... way.” The caregiver was asked to choose the answer that best fits the empty space from the following options: 1) very negative, 2) negative, 3) somewhat positive, 4) positive, or 5) very positive.

Observational Effects

To compare the perceived relevance, feasibility and effectiveness of the intervention with the observational effects, we used observational data from one single-case study (Damen et al., 2014) and two multiple-case studies (Damen, 2015; Damen et al., 2015a) involving a total of 11 participants with CDB. The previous studies tested the effect of the intervention using 286 video observations with a duration between 10 and 20 min. These video observations were recorded in two conditions: 1) before the intervention began, 2) during the intervention, and for the multiple case-studies also after the intervention. The video material was continuously coded for occurrences of six communication categories (See Table 2 for the operational definition of each category): Dyadic Interaction, Shared Emotion, Referential Communication, Meaning Negotiation, Shared Meaning, and Declarative Communication. Codes were assigned to the video material, in accordance with the operational definitions of each of these categories, using a digital coding sheet. One observer coded all the observations and a second observer independently double coded 20% of the material to check for sufficient inter-rater reliability. The inter-rater reliability was determined by calculating the percentage of agreement for each observation category (Brown and Snell, 1993) and ranged between 80 and 100%.

TABLE 2

TABLE 2. Observation categories and corresponding items in two subscales of the Social Validity Scale.

The observation categories were derived from Trevarthen’s theory concerning the development of communication behaviors in three subsequent layers of intersubjective development. Innate intersubjectivity is described as “an awareness specifically receptive to subjective states in other persons” (Trevarthen and Aitken, 2001, p. 4). The Dyadic Interaction and Shared Emotion categories were based on the description of the first layer of intersubjective development, which develops between 0 and 9 months in typical children and is characterized by the child’s awareness of the other. The Referential Communication, Shared Meaning, and Meaning Negotiation categories were based on descriptions of the second layer of intersubjective development, which usually develops between 9 months and 2 years and is characterized by the child’s awareness of mutuality. The Declarative Communication category was defined according to descriptions of the third layer of intersubjective development, which usually develops between 2 and 6 years and is characterized by narrative self- and other awareness.

Data Analysis

For the analysis of the caregiver ratings of the relevance of the HQC intervention, we calculated the mean score and standard deviation for three selected statements about the intervention and calculated if there were significant differences between the scores, using the Wilcoxon signed-rank test. Mean scores and standard deviations were also calculated for the SVS subscales “subjective evaluation” and “feasibility.” Subsequently, we calculated the mean score and standard deviation for each item in those subscales that matched the observed communication categories and the significance of differences between the scores. We selected specific items from the Feasibility and Subjective Effectiveness subscales that corresponded with the operational definitions of the observation categories that were a target in the intervention (see Table 2 for the items used for each observation category). For two observation categories, Referential Communication and Declarative Communication, none of the items fully matched the operational definition and therefore items were selected that concerned communication aspects relevant to these communication categories. However, none of the items in the Subjective Evaluation subscale matched the Declarative Communication category, which implicates that the subjective evaluation of the effectiveness of the intervention for this category was not assessed.

For the analysis of the observational effects, we calculated effect sizes for the difference between the baseline phase and the combined intervention phases for all cases. We only analyzed communication categories that were a target in the intervention for the individual cases. This way of analyzing was different compared to our previous studies, in which we considered for all communication categories if there were medium or large effect sizes for each of the participants with deafblindness, when comparing the baseline with each of the other phases: the first intervention phase, the second intervention phase and, for ten of 11 cases, a follow-up phase.

The effect sizes were the results of a comparison of the observational data points gathered during the two intervention phases with the data points gathered during the baseline phase, using the nonoverlapping of all pairs technique (NAP; Parker and Vannest, 2009). In the NAP technique, each data point of a non-intervention phase is compared to each data point of an intervention phase and the percentage overlap is calculated. An overlap of 0–65% implies no or only a small positive effect which is not significant, an overlap of 66–92% implies a medium positive effect, and an overlap of 93–100% implies a large effect.

To answer our questions on the relation between observational effectiveness and the perceived effectiveness, relevance and feasibility of the intervention, we calculated the Spearman correlation for the observational effects and each of the social validity scores on the level of the single communication categories. For the analysis of the association between the observational effects and the feasibility ratings, the combined mean scores of the three items from the SVS scale that concerned the rated relevance of the interventions was used.

Results

Caregivers’ Opinions About the HQC Intervention

Relevance

The caregivers’ mean scores on the statements about the relevance of the HQC intervention revealed that they had positive opinions about the intervention’s focus, which concerns communication support for caregivers. They also had positive opinions about the two elements of the intervention procedure: receiving communication coaching and using their video recordings, However, there were differences between the statements: while communication support and coaching were generally considered to be ‘a very good idea’ (M = 4.88, SD = 0.33; M = 4.92, SD = 0.28), the use of video recording was only considered to be a “good idea” (M = 3.88; SD = 0.70). The rating of the use of video recording as an idea, was significantly lower compared to the rating of the other statements (Table 3).

TABLE 3

TABLE 3. Scores on the three items of the Social Validity Scale about the relevance of the intervention^a.

Feasibility

The results on the Feasibility subscale in Table 4 show that the caregivers thought the intervention was feasible overall. This was similar for the items on the SVS scale that represented the observed communication categories, except for the Dyadic Interaction category. The caregivers more often thought this category was easy to carry out. The lowest ratings were given to Shared Emotion. There was more variation in the feasibility ratings then in the subjective evaluation ratings. The standard deviations revealed that 68% of the mean feasibility ratings and the ratings for Shared Emotion, Shared Meaning, and Declarative Communication varied between difficult, feasible, and easy. A significant higher feasibility was rated for Dyadic Interaction and Referential communication and a significant lower feasibility for Shared Emotion.

TABLE 4

TABLE 4. Scores on the Feasibility and the Subjective Effectiveness subscales of the Social Validity Scale^a.

Perceived Effectiveness

The ratings on the Subjective Evaluation subscale reveal that caregivers generally rated the HQC intervention as having a positive effect (see Table 4), with an overall mean score of 3.83 on the 5-point Likert Scale. Similar positive ratings were given for the single items that represented the communication categories. The ratings varied between 4.08 for Dyadic Interaction and 3.60 for Shared Emotion. The standard deviations (SD = 0.28–0.70) were all below 1 and thus show that there was little variation in the ratings. A significant higher effectiveness was rated for Dyadic Interaction and a significant lower effectiveness for Shared Emotion.

Observational Effects

Table 5 shows the effect sizes found with the NAP method for the differences between the baseline phase and the combined intervention phases for the communication categories that were a target in the intervention on the level of individual cases. In case Vincent, a large effect was found of the intervention on declarative communication. For Vincent, and five other cases medium effects were found in one or more communication categories. For the other five cases, small effects were found on at least one of the communication categories.

TABLE 5

TABLE 5. Effect sizes of targeted communication categories, based on a comparison between the combined intervention phases and the baseline phase.

Correspondence Between the Perceived Relevance and Observational Effects of the Intervention

No correspondence was found between the caregivers’ opinions about the relevance of the HQC intervention and the observational effects. Caregivers thought it was either a good or a very good idea to carry out the intervention and their positivity toward the intervention was not associated with the effect size. This was confirmed by the very weak correlation between the rated relevance and the effect sizes (r_sp(25) = 0.063, p = 0.515).

Correspondence Between the Perceived Feasibility and Observational Effects of the Intervention

No correspondence was found between the feasibility of the communication categories and the observational effects. Caregivers with clients for whom no effect, or a small, moderate, or large effect size was observed for individual communication categories rated these categories as difficult, feasible, or very easy. A very weak negative correlation was found between the two variables (r_sp(25) = -0.013, p = 0.892).

Correspondence Between the Perceived Effectiveness and Observational Effects of the Intervention

A significant, but weak negative correspondence was found between perceived effects and observational effects (r_sp(25) = −0.219, p = 0.029). The majority of communication partners had rated that the intervention was effective on the targeted communication categories for their clients, or somewhat effective, and this concerned participants for whom no, small or moderate effects in these categories were observed.

Discussion

The analysis of the social validity outcomes in this study revealed that caregivers of individuals with CDB consider the HQC intervention to be a relevant intervention. The data show that caregivers in this study were very positive about receiving a communication intervention, which is in line with studies that report a need for communication support among caregivers of individuals with deafblindness (Janssen et al., 2003; Janssen et al., 2004; Correa-Torres, 2008).

All the participants in our study rated the use of communication support for caregivers of deafblindness and the use of a communication coach as very good ideas on the Social Validity Scale. Although they also considered the recording of their interactions with the participant with deafblindness on video to be “a good idea,” they valued this characteristic of the intervention procedure significantly less than the use of communication support and coaching. This could mean that the caregivers were relatively less motivated for this part of the intervention procedure.

Motivational challenges related to the use of video recordings were also found by Coleman (2000), in a meta-analysis of studies that used video recordings to analyze the interactions between patients and their physician. The use of video recordings did not necessarily alter the behaviors of patients and their physicians who participated in these studies, but it did affect the composition of the study sample. The use of video recordings was a reason for a group of patients to refrain from participating in the research, particularly patients who were embarrassed about their medical problems or were younger (Coleman, 2000).

A recent study by Colliers et al. (2019) about general practitioners’ perceptions about being video recorded during patient interactions found that they had several “mental barriers.” In order for them to agree to participate in such video recordings, conditions needed to be met that mostly relate to “feeling safe, protecting the patient and handling the data carefully” (Colliers et al., 2019, p. 9).

Caregivers rated the HQC intervention on the subscale Subjective effectiveness of the Social Validity Scale as having a positive effect in general. The effectiveness of this intervention was rated highest for the Dyadic Interaction category, which was operationalized in the SVS as the ability of the caregiver to confirm the signals of the individual with deafblindness. The effect was rated lowest for Shared Emotion and this difference was statistically significant when compared with the ratings for Dyadic Interaction and Shared Meaning.

Differences Between Communication Categories

In general, caregivers rated the HQC intervention as feasible on the Feasibility subscale of the Social Validity Scale. However, the data show that there may have been differences between the perceived feasibility of the communication categories. Both Dyadic Interaction and Referential Communication were rated as significantly easier to carry out and Shared Emotion was rated as significantly more difficult to carry out. The lower feasibility rating for Shared Emotion is in line with a study by Martens et al. (2014a) that focused on supporting caregivers of individuals with deafblindness in affective attunement and the sharing of positive and negative emotions. Caregivers in that study reported difficulties with sharing negative emotions and with combining affective attunement and stimulating meaning making in interactions with an individual with CDB (Martens, et al., 2014a). The HQC intervention specifically focuses on meaning making by supporting the caregiver in stimulating the exchange, negotiating and sharing meanings in the second phase of the intervention. The focus on what the individual with deafblindness wanted to express in this phase could have made it more difficult for the caregivers to also attune their behaviors to the participants’ emotions.

Correspondence Between Social Validity Results and Observational Effects

In our study, we found no significant correspondence between the rated relevance, feasibility, of the intervention and the observational effects on the level of the single communication categories for the individual participants, and a significant but weak negative correspondence between perceived and observational effects. Opinions about the intervention’s relevance were quite positive and the overall opinion was that the intervention was feasible and effective, which reveals that there was little variation in the ratings, a phenomenon known as “a restriction-of-range in the data.” We do not know if more variation in the stance toward the intervention would have affected the observed effects.

For the group of caregivers as a whole, we found some similarities between the rated feasibility and rated effects. They rated supporting Dyadic Interaction with the participant with CDB as relatively more feasible (M = 3.52) and they saw the intervention as relatively more effective for this communication category (M = 4.08). However, supporting the participant with CDB in Sharing Emotions was rated as relatively less feasible and the intervention was seen as less effective for this communication category. This suggests a correspondence between the intervention’s perceived feasibility and its perceived effectiveness, which may have been influenced by the caregivers’ experienced competency in supporting the individual with CDB in developing specific communication aspects. The perceived difficulty with Sharing Emotions could be explained by the focus on stimulating the negotiation, exchange and sharing of meanings in the intervention and is in line with findings from Martens et al. (2014a). The low complexity of the work on Dyadic Interaction may be explained by the fact that this communication category develops at the first and most basic layer of intersubjective development, which is characterized by an awareness of the other person. This awareness already manifests itself in typical children at a few weeks old, in the form of attention to the other person, turn taking, and imitation (See Bråten and Trevarthen, 2007).

Implications

An interesting finding is that the caregivers evaluated the feasibility and effectiveness of the intervention differently for individual aspects of communication. A recommendation for practice is therefore to base video-feedback interventions on the individual needs of both the individual with deafblindness and their communication partner, and to monitor those needs during the intervention. According to Janssen and colleagues, communicating with people with CDB is a complex activity (Janssen, Riksen-Walraven, and Van Dijk, 2003). Research by Hartmann (2012) shows that when communication partners experience problems in communicating with a person with CDB, they can experience a lack of self-efficacy. Self-efficacy refers to the extent in which a person feels competent in successfully completing a task (Bandura, 1997). A recommendation for communication coaches is to ask the caregivers to indicate their self-efficacy in specific communication aspects before starting their intervention. The focus in video-feedback interventions is often on the communication skills of the individual with CDB. We recommend that the focus of such interventions is on those communication skills of which the caregivers feels that he is less competent in stimulating them. During the intervention period, the communication coach could then focus on those aspects and could check whether the self-efficacy of the caregiver improves.

The indications of specific difficulties with finding effective ways to share emotions with individuals with deafblindness has implications for the implementation of the HQC intervention. This finding means that the communication coach needs to consider whether communication partners have specific questions about the sharing of emotions and if they need additional support to support them in this (e.g., in the form of modeling or coaching on the job).

Limitations

This study had several limitations. First, the sample was small (N = 25) and there was little variation in the data. Both the perceived relevance and perceived effectiveness were high. Furthermore, the observational data revealed the effectiveness of the intervention for all participants on multiple communication categories, limiting the opportunity for variation in effectiveness to covary with perceptions of social validity.

Also, some of the participating caregivers did not fill out the questionnaire. Although a 74% response rate is usually considered to be sufficient for surveys (see Fincham, 2008) and caregivers for each participant with deafblindness were involved in the study, the caregivers who did not fill out the questionnaire could have had different opinions than those who did. Furthermore, there were caregivers involved in the lives of the participants with deafblindness that did not participate in the intervention but could have been involved in the social validity assessment.

The communication coaches who carried out the intervention were not asked how they valued it. Schwartz and Baer (1991) recommend not only involving the direct consumers of an intervention in the social validity assessment, but also other people who are potentially affected by the intervention.

In addition, based on the use of purposeful sampling in this study and the previously described impact that the use of video recordings can have on the willingness to participate in research (see Coleman, 2000; Colliers et al., 2019), there is a high chance that the selection of the study sample was biased.

The social validity of the study was evaluated in only one way (i.e., by means of a social validity questionnaire) and only after the intervention. According to Hurley (2012), social validity assessment can be done with multiple and various types of instruments including interviews, focus groups, surveys, and video observations. In contrast with the social validity assessment of effects, the assessment of goals and procedures can already be performed at the beginning and during the intervention (Hurley, 2012). Because we only assessed caregivers’ perceptions of the relevance of the HQC intervention after the intervention, their experiences with the intervention could have influenced their ratings of its relevance.

Finally, the operationalization of communication aspects in the coding system and the questionnaire did not always match. This complicates the comparison of the social validity results with the observed effects.

Strength of the Study

The innovative part of this study is that the observational effects of an intervention were compared to the subjective evaluation of that intervention for single participants and on the level of specific communication categories that were selected to intervene on. Such a procedure is rare in effect studies in our field. The comparison of the ratings for the individual communication categories revealed that caregivers differentiated between the communication categories with respect to their feasibility and the extent to which they changed as a result of the intervention. This differentiation suggests that the caregivers’ social validity ratings did not only reflect socially desirable answers, which is a concern some researchers have about social validity assessment (see Garfinkle and Schwartz, 2002). The ratings on the SVS also provided additional information about the clinical value of the intervention that would not have been found if we had only gathered observational data. Although more research is needed to confirm the reliability of our findings, the fact that the caregivers were very positive about the use of a communication coach and less positive about the recording of their interactions on video is an interesting finding, as are their high feasibility and effect ratings for Dyadic Interaction and lower ratings for Shared Emotion.

Future Studies

A recommendation for measuring the social validity of an intervention is to use a more diverse and larger sample. A possible solution is to not only involve the direct consumers, but also other people who are affected by the treatment. In the case of people with multiple disabilities, it is often impossible to ask them to rate the effects of interventions. However, involving as many people as possible who are part of the participant’s life can enhance the validity of social validity assessment. Furthermore, we recommend using a questionnaire together with other social validity instruments, such as interviews, focus groups, or group video analysis. Finally, we recommend creating more than one test moment to provide more insight into how quickly intervention targets are achieved.

Conclusion

This study showed that having a communication coach provide video feedback communication support is highly relevant for caregivers of individuals with deafblindness. The observational effects and the social validity results both revealed that the intervention positively changed caregivers’ communication with the participants. However, there was only a weak negative correlation between the observational effects of the intervention and the perceived effects, when we considered the individual communication categories on the level of single cases. There was also no relation found between the observational effects and the rated relevance and feasibility of the intervention. What we did find, was a relation between the rated feasibility and perceived effectiveness of individual communication categories. Dyadic Interaction was considered easier to support in individuals with CDB and was seen as more positively affected by the intervention. Shared Emotion was considered more difficult to support and was seen as less positively affected by the intervention. This suggests that caregivers based their ratings of the effectiveness of the intervention for individual communication aspects on how competent they felt in supporting individuals with congenital deafblindness in developing these aspects. An interesting finding is also that caregivers were slightly less convinced about the use of video recording in the intervention than about the idea of communication coaching. Their hesitation could be influenced by the fact that watching yourself on video can be uncomfortable. The idea behind video-feedback interventions is that the caregiver is empowered by watching good examples of their interactions on video, which contributes to self-management (see Damen et al., 2020). The coaches in our study were trained and supervised in the use of strategies to activate and empower caregivers. We cannot emphasize enough that future video-feedback interventions should always be carried out by a trained coach.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by the Ethical Board of Bartimeus, the Netherlands. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

SD, CS, and MJ designed the study. SD carried out the study and analyzed the data. CS gave methodological support. SD wrote the first draft. SD, CS, MJ, and WR participated in revising the first, second, and third drafts. The fourth draft was submitted for publication.

Funding

The research undertaken was funded by ZonMW, grant 94308004.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We thank the ZonMW/InZicht foundation for funding our research on the effects of the HQC intervention for individuals with CDB. We thank the caregivers for sharing their evaluation of the intervention by filling in the SVS.

References

American Psychological Association (2020). Guidelines for assessment of and intervention with persons with disabilities. Available at: https://www.apa.org/pi/disability/resources/assessment-disabilities (Accessed February 23, 2020).

Google Scholar

Ask Larsen, F., and Damen, S. (2014). Definitions of deafblindness and congenital deafblindness. Res. Dev. Disabil. 35 (10), 2568–2576. doi:10.1016/j.ridd.2014.05.029

PubMed Abstract | CrossRef Full Text | Google Scholar

Bråten, S., and Trevarthen, C. (2007). “Prologue: from infant intersubjectivity and participant movements to simulation and conversation in cultural common sense,” in In on being moved: from mirror neurons to empathy. 2nd Edn. Editor S. Bråten (Amsterdam, Netherlands: John Benjamins), 21–34.

Google Scholar

Brown, F., and Snell, M. E. (1993). “Measurement, analysis and evaluation,” in Instruction for students with severe disabilities. 2nd Edn. Editor M. E. Snell (New York: Macmillan), 152–183.

Google Scholar

Coleman, T. (2000). Using video-recorded consultations for research in primary care: advantages and limitations. Fam. Pract. 17 (5), 422–427. doi:10.1093/fampra/17.5.422

PubMed Abstract | CrossRef Full Text | Google Scholar

Colliers, A., Coenen, S., Remmen, R., Philips, H., and Anthierens, S. (2019). Looking inside the out-of-hours primary care consultation: general practitioners' and researchers' experiences of using video observations as a method. Int. J. Qual. Methods 18, 1609406919879341. doi:10.1177/1609406919879341

CrossRef Full Text | Google Scholar

Constantino, M. J., Ametrano, R. M., and Greenberg, R. P. (2012). Clinician interventions and participant characteristics that foster adaptive patient expectations for psychotherapy and psychotherapeutic change. Psychotherapy 49 (4), 557–69. doi:10.1037/a0029440

PubMed Abstract | CrossRef Full Text | Google Scholar

Cook, b., Buysse, V., Klinger, J., Landrum, T., and McWilliam, R. (2014). Council for exceptional children: standards for evidence-based practices in special education. Except. Child. 46 (6), 206–212.

Google Scholar

Correa-Torres, S. M. (2008). The nature of the social experiences of students with deaf-blindness who are educated in inclusive settings. J. Vis. Impair. Blind. 102 (5), 272–283. doi:10.1177/0145482X0810200503

CrossRef Full Text | Google Scholar

Daamen, W. (2013). Wat werkt bij het implementeren van jeugdinterventies? [What works when implementing youth interventions? Utrecht, Netherlands: NIZW Jeugd.

Damen, S., Janssen, M. J., Huisman, M., Ruijssenaars, W. A., and Schuengel, C. (2014). Stimulating intersubjective communication in an adult with deafblindness: a single-case experiment. J. Deaf Stud. Deaf Educ. 19 (3), 366–384. doi:10.1093/deafed/enu006

PubMed Abstract | CrossRef Full Text | Google Scholar

Damen, S., Janssen, M. J., Ruijssenaars, W. A., and Schuengel, C. (2015b). Intersubjectivity effects of the high-quality communication intervention in people with deafblindness. J. Deaf Stud. Deaf Educ. 20 (2), 191–201. doi:10.1093/deafed/env001

CrossRef Full Text | Google Scholar

Damen, S., Kef, S., Worm, M., Janssen, M. J., and Schuengel, C. (2011). Effects of video-feedback interaction training for professional caregivers of children and adults with visual and intellectual disabilities. J. Intellect. Disabil. Res. 55 (6), 581–595. doi:10.1111/j.1365-2788.2011.01414.x

PubMed Abstract | CrossRef Full Text | Google Scholar

Damen, S. (2015). A matter of meaning. The effect of partner support on the intersubjective behaviors of individuals with congenital deafblindness. Dissertation. ‘s Hertogenbosch: BOXPress.

Google Scholar

Damen, S., Janssen, M. J., Ruijssenaars, W. A. J. J. M., and Schuengel, C. (2015a). Communication between children with deafness, blindness and deafblindness and their social partners: an intersubjective developmental perspective. Int. J. Disabil. Dev. Educ. 62 (2), 215–243. doi:10.1080/1034912X.2014.998177

CrossRef Full Text | Google Scholar

Damen, S., Prain, M., and Martens, M. (2020). Video feedback interventions for communication partners of individuals with congenital deafblindness: a systematic review. J. Deaf. Stud. Deaf. Educ. 6, 5–44. doi:10.21827/jdbsc.6.36191

CrossRef Full Text | Google Scholar

Dammeyer, J. (2012). Identification of congenital deafblindness. Br. J. Vis. Impair. 30, 101–107. doi:10.1177/0264619612443882

CrossRef Full Text | Google Scholar

Fincham, J. (2008). Response rates and responsiveness for surveys, standards, and the journal. Am. J. Pharmaceut. Educ. 72 (2), 43. doi:10.5688/aj720243

CrossRef Full Text | Google Scholar

Fukkink, R. G. (2008). Video feedback in windscreen: a meta-analysis of family programs. Clin. Psychol. Rev. 28 (6), 904–16. doi:10.1016/j.cpr.2008.01.003

PubMed Abstract | CrossRef Full Text | Google Scholar

Garfinkle, A. N., and Schwartz, I. S. (2002). Peer imitation. Top. Early Child. Spec. Educ. 22 (1), 26–38. doi:10.1177/027112140202200103

CrossRef Full Text | Google Scholar

Groeneveld, M. G., Vermeer, H. J., van IJzendoorn, M. H., and Linting, M. (2011). Enhancing home-based child care quality through video-feedback intervention: a randomized controlled trial. J. Fam. Psychol. 25 (1), 86–96. doi:10.1037/a0022451

PubMed Abstract | CrossRef Full Text | Google Scholar

Hartmann, E. (2012). A scale to measure teachers' self-efficacy in deaf-blindness education. J. Vis. Impair. Blind. (JVIB) 106 (11), 728–738. doi:10.1177/0145482x1210601103

CrossRef Full Text | Google Scholar

Hurley, J. J. (2012). Social validity assessment in social competence interventions for preschool children. Top. Early Child. Spec. Educ. 32 (3), 164–174. doi:10.1177/0271121412440186

CrossRef Full Text | Google Scholar

Janssen, H., and Damen, S. (2018). Communicatiecoaching: een effectieve vorm van professionaliseren in onderwijs en zorg aan personen met doofblindheid [Communication coaching: an effective form of professionalization in education and care for people with deafblindness]. Orthopedagogiek: Onderzoek en Praktijk 57 (3–6), 100–108.

Google Scholar

Janssen, M. J., Riksen-Walraven, J. M., and Van Dijk, J. P. M. (2006). Applying the diagnostic intervention model for fostering harmonious interactions between deaf-blind children and their educators: a case study. J. Vis. Impair. Blind. (JVIB) 100 (2), 91–105. doi:10.1177/0145482X0610000204

CrossRef Full Text | Google Scholar

Janssen, M. J., Riksen-Walraven, J. M., and Van Dijk, J. P. M. (2003). Contact: effects of an intervention program to foster harmonious interactions between deaf-blind children and their educators. J. Vis. Impair. Blind. (JVIB) 97 (4), 215–229. doi:10.1177/0145482X0309700403

CrossRef Full Text | Google Scholar

Janssen, M. J., Riksen-Walraven, J. M., and Van Dijk, J. P. M. (2004). Enhancing the interactive competence of deafblind children: do intervention effects endure? J. Dev. Phys. Disabil. 16 (1), 73–94. doi:10.1023/B:JODD.0000010040.54094.0f

CrossRef Full Text | Google Scholar

Janssen, M. J., Riksen-Walraven, J. M., and van Dijk, J. P. M. (2002). Enhancing the quality of interaction between deafblind children and their educators. J. Dev. Phys. Disabil. 14 (1), 87–109. doi:10.1023/a:1013583312920

CrossRef Full Text | Google Scholar

Janssen, M. J., Riksen-Walraven, J. M., van Dijk, J. P. M., Huisman, M., and Ruijssenaars, W. A. J. J. M. (2011). Fostering harmonious interactions in a boy with congenital deaf-blindness: a single-case study. J. Vis. Impair. Blind. (JVIB) 105 (9), 560–572. doi:10.1177/0145482X110500907

CrossRef Full Text | Google Scholar

Janssen, M. J., Riksen-Walraven, J. M., Van Dijk, J. P. M., Ruijssenaars, W. A. J. J. M., and Vlaskamp, C. (2007). Team interaction coaching with educators of adolescents who are deaf-blind: applying the Diagnostic Intervention Model. J. Vis. Impair. Blind. (JVIB) 101 (11), 677–689. doi:10.1177/0145482X0710101102

CrossRef Full Text | Google Scholar

Kazdin, A. E. (1977). Assessing the clinical or applied importance of behavior change through social validation. Behav. Modif. 1 (4), 427–452. doi:10.1177/014544557714001

CrossRef Full Text | Google Scholar

Kazdin, A. E. (1982). Single-case research designs: methods for clinical and applied settings. New York, NY: Oxford University Press.

Martens, M. A. W., Janssen, M., Ruijssenaars, W. A. J. J. M., Huisman, M., and Riksen-Walraven, J. (2017). Fostering emotion expression and affective involvement with communication partners in people with congenital deafblindness and intellectual disabilities. J. Appl. Res. Intellect. Disabil. 30 (5), 872–884. doi:10.1111/jar.12279

PubMed Abstract | CrossRef Full Text | Google Scholar

Martens, M. A. W., Janssen, M. J., Ruijssenaars, W. A. J. J. M., Huisman, M., and Riksen-Walraven, J. M. (2014b). Applying the intervention model for fostering affective involvement with persons who are congenitally deafblind: an effect study. J. Vis. Impair. Blind. (JVIB) 108 (5), 399–413. doi:10.1177/0145482x1410800505

CrossRef Full Text | Google Scholar

Martens, M. A. W., Janssen, M. J., Ruijssenaars, W. A. J. J. M., Huisman, M., and Riksen-Walraven, J. M. (2014a). Intervening on affective involvement and expression of emotions in an adult with congenital deafblindness. Commun. Disord. Q. 36 (1), 12–20. doi:10.1177/1525740114526926

CrossRef Full Text | Google Scholar

McClintock, A. S., Anderson, T., and Petrarca, A. (2015). Treatment expectations, alliance, session positivity, and outcome: an investigation of a three-path mediation model. J. Clin. Psychol. 71 (1), 41–49. doi:10.1002/jclp.22119

PubMed Abstract | CrossRef Full Text | Google Scholar

Parker, R. I., and Vannest, K. (2009). An improved effect size for single-case research: nonoverlap of all pairs. Behav. Ther. 40 (4), 357–367. doi:10.1016/j.beth.2008.10.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Schwartz, I. S., and Baer, D. M. (1991). Social validity assessments: is current practice state of the art? J. Appl. Behav. Anal. 24 (2), 189–204. doi:10.1901/jaba.1991.24-189

PubMed Abstract | CrossRef Full Text | Google Scholar

Seys, D. M. (1987). Kwaliteit van zorg: zorg voor kwaliteit. Analyse en beïnvloeding van frequentie en kwaliteit van bewoner gerichte zorguitvoering door groepsleid(st)ers in de residentiële zwakzinnigenzorg [Quality of care: care for quality. analysis and modification of frequency and quality of client-guided care by residential staff]. [Dissertation]. Nijmegen, Netherlands: Radboud University.

Google Scholar

Singer, G. H. S., Agran, M., and Spooner, F. (2017). Evidence-based and values-based practices for people with severe disabilities. Res. Pract. Persons Severe Disabil. 42 (1), 62–72. doi:10.1177/1540796916684877

CrossRef Full Text | Google Scholar

Strain, P. S., and Dunlap, G. (2012). Lessons learned about the utility of social validity. Educ. Treat. Child. 35 (2), 183–200. doi:10.1353/etc.2012.0007

CrossRef Full Text | Google Scholar

Trevarthen, C., and Aitken, K. J. (2001). Infant intersubjectivity: research, theory, and clinical applications. J. Child. Psychol. Psychiatry 42 (1), 3–48. doi:10.1111/1469-7610.00701

PubMed Abstract | CrossRef Full Text | Google Scholar

Zijlstra, H. P. (2003). Dansen met olifanten: een onderzoek naar de implementatie van het opvoedingsprogramma in de zorg voor mensen met ernstige meervoudige beperkingen [Dancing with elephants: a study on the implementation of personal support plans in the care of people with severe multiple disabilities. [Dissertation]. Groningen, Netherlands: University of Groningen.

Google Scholar