Operationalizing Behavior Change Theory as Part of Persuasive Technology: A Scoping Review on Social Comparison

Lemke, Mailin; de Vries, Roelof A. J.

doi:10.3389/fcomp.2021.656873

MINI REVIEW article

Front. Comput. Sci., 14 April 2021

Sec. Human-Media Interaction

Volume 3 - 2021 | https://doi.org/10.3389/fcomp.2021.656873

This article is part of the Research Topic 2021 Editor's Pick: Computer Science View all 11 articles

Operationalizing Behavior Change Theory as Part of Persuasive Technology: A Scoping Review on Social Comparison

$\nMailin Lemke&#x;$ Mailin Lemke¹^†

Roelof A. J. de Vries²^*^†

¹Department of Human-Centered Design, Delft University of Technology, Delft, Netherlands
²Biomedical Signals and Systems, University of Twente, Enschede, Netherlands

Theories from psychology or the social sciences are commonly used as a starting point when designing persuasive technologies that aim to evoke a specific behavior change. Ideally, using these theories would (1) help guide the design of the persuasive technology and (2) help evaluate and inform the theory. In this paper, we focused on the first aspect and looked at how papers report on how a theory guided the design of persuasive technology. We performed a scoping review focused on the operationalization of social comparison theory as part of persuasive design. We chose social comparison due to its ubiquitous use in persuasive design as well as its potential positive or negative influence on the user. The former requires careful consideration in a persuasive design prototype. We focused on the proceedings of the Persuasive Technology conference from 2006 to 2020 to gain an understanding of the use of social comparison theory as part of persuasive design. Twelve studies met our inclusion criteria. Explanations of how the theory guides design decisions leading to the final operationalization were sparse. We suggest that conducting manipulation checks and using a systematic approach to reporting design decisions including the potential grounding of design elements in theory could highlight and clarify theoretical insights, and could increase our understanding of how social comparison—and behavior change theory in general—could be efficiently operationalized in persuasive technologies.

1. Introduction

Persuasive technologies often employ theories from other disciplines including psychology, social sciences, or medicine to encourage or support a specific behavior change in the user. The use of such behavior change theories can contribute to an increased effect size of behavior change interventions (Webb et al., 2010). However, interventions often fail due to projects being too ambitious and lacking a detailed understanding of the principles for effective persuasive designs (Lehto, 2012). Critique on the general design research field indicates that the reflection on research findings that can be fed back into the original theory is often missing (Cash, 2018).

Ideally, theory can help guide and ground the process of designing persuasive technology, and in turn, the evaluation of the persuasive designs can help evaluate and inform the theory. With this scoping review we wanted to get a first idea of how theory is operationalized as part of persuasive design prototypes insofar as it is reported in papers. For this purpose, we focused on “Social Comparison Theory,” often simply referred to as “Social Comparison” (SC). We chose to just focus on one theory to get a better understanding of the space of operationalizations. We chose social comparison due to its potential positive or negative effect in evoking the intended behavior change and its implementation in common behavior change strategies such as the “Persuasive System Design Model” (Oinas-Kukkonen and Harjumaa, 2009).

2. The Evolution of Social Comparison

Social comparison theory was initially proposed by Festinger (1954) and focused on the human drive to compare personal abilities and opinions to other people in the absence of objective and non-social norms. The theory outlined different aspects such as the preference of comparing oneself to people who are close to one's own abilities and opinions (Festinger, 1954). Subsequent research has broadened the understanding of the complex nature of comparing oneself to others and expanded the list of so-called “comparison dimensions” including contemporary factors such as accomplishments, possessions, and feelings (Gibbons and Buunk, 1999). The effect of social comparison depends on several contextual factors including the type of “comparison target” (e.g., friend or stranger); the “comparison dimension” (e.g., income or social status); and the “comparison direction” (Arigo et al., 2019). Research furthermore indicates that this comparison direction—including upward (comparing one's situation to someone who is perceived to be superior), lateral (comparing one's situation to someone who is perceived to be on the same level), and downward (comparing one's situation to someone who is perceived to be worse)—has a significant influence on the effect of social comparison (Corcoran et al., 2020; Wayment et al., 2020). Additional contextual factors include the “perceived similarity” with the target which allows to either identify or contrast oneself with the target as well as the “comparison mode” (e.g., online or in person) (Arigo et al., 2019). There are different definitions of social comparison and what it entails. In the context of this study, we refer to Michie et al. (2013, p. 9) definition of “social comparison” as to “draw attention to others' performance to allow comparison with the person's own performance. Note: being in a group setting does not necessarily mean that social comparison is actually taking place.”

3. Social Comparison As Part of Behavior Change Intervention

As discussed in the previous section, social comparison is a dynamic and complex theory which can be understood in various ways. This is also reflected in the way the theory is operationalized and conceptualized in technology. For example, social comparison can have positive effects on individuals' behavior and state of mind (Corcoran et al., 2020), but also evoke negative effects including anxiety, frustration, peer pressure, unnecessary competitiveness, and data safety concerns (Consolvo et al., 2006; Lin et al., 2006; Newman et al., 2011). In addition, the way social comparison has been implemented varies significantly in terms of the type and dimension of comparison as well as design specific features that evoke the comparison effect (Mylonopoulou et al., 2018). Social comparison is embedded in various forms, for example, leaderboards, sharing information with and allowing connection between users, modeling of the correct behavior as well as the provision of normative feedback as part of group settings (Arigo et al., 2020). Moreover, techniques used in behavior change interventions such as “behavioral modeling” and “norm referencing” are often seen as explicit entities next to social comparison (Michie et al., 2013), even though the techniques arguably meet Festinger's description of social comparison (Festinger, 1954). Also, the term “social comparison” is used inconsistently in the context of behavior change technologies and explanations concerning the design features that promote the intended behavior change are often missing. Including these details would allow gaining a better understanding of how social comparison can be effectively used as part of such applications (Arigo and Suls, 2018). A critique which also has been expressed in the general context of behavior change designs (Lehto, 2012).

4. Method

To investigate how social comparison is operationalized as part of persuasive designs, we specifically looked at to what extent the persuasive technologies were “grounded” in theory, and whether an evaluation of the social comparison design included some form of a manipulation check. The term “grounding” is normally mentioned in the context of developing and justifying design theories (Goldkuhl, 2004; Höök and Löwgren, 2012). In the context of this review, we refer to the term grounding as an explanation of how the theory was implemented in the design prototype. We refer to the term manipulation check as one or more questions geared toward understanding each participant's cognizance regarding the condition to which they were exposed (Hoewe, 2017) and the term design prototype as “a representation of a design idea, regardless of medium” (Houde and Hill, 1997, p. 369). We performed a scoping review focusing on the proceedings of the Persuasive Technology conference from 2006 to 2020 and looked at several characteristics relevant to social comparison as well as the operationalization process of the theory. We chose to scope this review on the Persuasive Technology conference proceedings because the conference is specifically focused on designing and implementing persuasive or behavior change strategies into technology, which requires the authors to think about how to operationalize the chosen theory in terms of design features.

4.1. Research Question

The research question for this scoping review was:

How is social comparison operationalized as part of persuasive technology?

With this research question, we wanted to identify how social comparison is operationalized as part of persuasive design prototypes. We were also interested in the process and explanation of using the theory to develop a design prototype.

4.2. Scoping Review

4.2.1. Identification and Screening

This review followed the PRISMA Extension guidelines for Scoping Reviews (Tricco et al., 2018). We used the Persuasive Technology proceedings database¹ for the purpose of the review which consists of 363 papers published between 2006 and 2020. In September 2020 we conducted a computerized search using the search term “social comparison.” Studies had to include the search term in the title, abstract, or main manuscript. We used three inclusion criteria: (1) The use of social comparison as part of the study intervention; (2) report on the development of a design prototype; and (3) involvement of participants as part of the design evaluation process. All papers were assessed for eligibility and coded by two reviewers (both authors). In a first step, abstracts and titles were assessed and in a consequent step, eligible papers were read in full. Discrepancies were resolved through discussion.

4.2.2. Data Charting Process and Data Items

For the data charting process, we captured relevant information electronically. The included studies were coded for the following aspects relevant to the operationalization process: [ref] reference to foundational social comparison definition included; [grounding] explanation how theory is used to justify design implementations; [check] use of manipulation check; [direction] upwards, lateral or downwards social comparison direction; [com.] social comparison combined with other strategies; [target] who “you” are compared against; [general evaluation] main evaluation outcome of the paper; and [SC eval.] post-hoc evaluation of social comparison specifically, see Table 1. These chosen categories were partially based on another scoping review (Arigo et al., 2019) and additional dimensions we deemed relevant for answering our research question. We also captured design specific information relevant to the context of operationalization including: [design] the design facilitating social comparison; [study context] type of behavior targeted; [medium] implementation form of social comparison; and [type] nature of social comparison, see Table 2.

TABLE 1

Table 1. Summary of dimensions of social comparison information extracted from papers included in the scoping review relevant for operationalization.

TABLE 2

Table 2. Summary of dimensions of social comparison information extracted from papers included in the scoping review relevant for design implementation.

5. Results and Discussion

The search term led to 47 unique articles. Twenty-one articles were rejected after reading the title and abstract because they did not include participants as part of the study or did not report on the development of a design prototype. Twenty-six articles were consequently reviewed in full. Fourteen more articles were excluded because they did not meet our inclusion criteria. In the end, 12 articles were included in the qualitative synthesis. The coding results of these articles are reported in Tables 1, 2.

As stated at the beginning of this paper, the use of a theory as part of a design is twofold: It should help guide the design of the persuasive technology and help evaluate and inform the theory. The focus of this scoping review was on identifying how social comparison is operationalized as part of persuasive design applications, as well as what the process and explanation is of using the theory to develop a design prototype.

Ten of the 12 reviewed studies reported a reference they used to support the use of social comparison (Table 1). This is a necessary step to understand what the results mean and to interpret the results in the context of the referential social comparison theory. However, there was a lack of description concerning the grounding process of the design elements for all but one study. This is not necessarily surprising, considering that there is no clear template available for reporting on behavior change theories operationalized as practical designs. Especially in the context of conference papers that we reviewed that have a limited word count. The observed lack of details might therefore be due to a lack of reporting rather than a lack of performing these steps. However, this lack of reporting does limit our understanding of the operationalization process. We also observed a similar pattern in the context of manipulation checks, where only some papers included comprehension questions to assess if participants could identify the depicted strategy or some pre-user test to assess whether users could recall the message content that included the social comparison aspect used as part of the experiment. This lack of manipulation checks limits our understanding of the validity of the implementation.

Details were also mostly lacking for intended social comparison direction. Just four studies described using upward comparison and one study described using all three comparison directions, even though literature has shown that the comparison direction can have a drastic influence on the effect (Corcoran et al., 2020; Wayment et al., 2020) and should therefore be considered carefully in the design. Seven of the 12 reviewed studies combined multiple behavior change strategies. The inclusion of multiple behavior change techniques has been claimed to be a rational approach to increase the effect size of the interventions (Webb et al., 2010). However, it could also pose a barrier if study designs cannot be explained in detail and if design elements are intertwined and assessed simultaneously. An example of why it is important to be clear about what strategies you think your design represents can be seen in Busch et al. (2016). “Statistics” are presented as part of the study as a representation of the two strategies “social comparison” as well as “self-monitoring.” The statistics displayed are a line-chart with “you” vs. the “average” over time. While in other papers a line chart is used as a representation of “system credibility” (Ruer et al., 2016). This makes one wonder whether it is something specific in this design that makes it encompass social comparison and self-monitoring. Arguably, the “over time” dimension facilitates self-monitoring the most, so is it that specific design element that makes this display of statistics a combination of social comparison and self-monitoring? Using designs this way obfuscates the effect of the specific strategy. Research in the context of apps promoting physical activities has raised similar questions around app features that arguably facilitate social comparison even though they are not used as such (Arigo et al., 2020). It seems that a specific design can represent a multitude of strategies, and also a specific strategy can be represented by a multitude of designs. The target of the social comparison strategy included a group of people as well as specific and identifiable users, non-identifiable users, and users with a personal connection to the participant. The target used for the social comparison effort seemed to be mostly determined by the use case and there is little explanation how the target was chosen in relation to the potential social comparison impact, even though related work (Arigo et al., 2019) highlights that the comparison target is an important feature.

The level of general evaluation was quite different between the papers, from the actual effect that the design had in terms of motivating specific behaviors, to whether people perceived themselves to be persuaded, to whether the design encouraged something more metaphorical like saving fictional money. All these different forms of evaluation make it hard to compare across study results and to draw higher level conclusions. Furthermore, only five of the papers evaluated in some form or another how social comparison played a role in their study, which is crucial in understanding what worked or did not work in the implementations or designs. This is connected to the differing levels of interaction, contexts, or ways of comparing that the prototypes allowed. For example, design prototypes were described in the form of interactive prototypes that users could engage with but also as storyboards that provided a third-person description of social comparison (Table 2). And also, the study context ranged from contributions to online movie ratings platform to sustainable transportation habits. The ways of comparing had several media and types, such as text-based or visualization-based comparisons or comparing oneself to group averages or specific individuals. These elements can be essential in the development of a design prototype. For example, a scenario-based representation of a leaderboard showing different users can foster an upward as well as downward comparison direction, depending on user's score and position on the leaderboard. It remains unclear at this stage if these design elements influence the effect that social comparison can have on the user of persuasive technology. Depending on the level of refinement of the design prototype, level of interaction, or type of comparison different results can be expected.

All in all, in answer to our research question, we can say that the explanation of the operationalization of social comparison as part of persuasive technology is very limited, at least in terms of how it is described in the papers. Therefore, we argue for using a more systematic approach to reporting design decisions, such as the grounding of design elements in theory and using manipulation checks. It is important to understand the fidelity of the designs and interventions in relation to their foundational theory for the interpretation of results. This helps to clarify theoretical insights, and to develop an understanding of how theory, in this case social comparison, could be efficiently implemented in persuasive technologies. For many of the discussed operationalization dimensions details were missing but without these details it remains unclear how these designs relate—conceptually and methodologically—to social comparison theory (Lambert et al., 2017). Potentially, this does not only apply to operationalizations of social comparison, but to other behavior change theories as well. In that sense, this paper is a “call to arms” to encourage researchers to explicate more on the grounding of their designs to make the potential inferences based on their results more tangible and to expedite reproducibility and replicability.

An important limitation to the current scoping review is the scope. For this review, we focused on just one database for the review process. Although we would argue that the Persuasive Technology conference proceedings are an important data source to use to investigate how theory is operationalized as part of persuasive designs, it is by no means the only place where papers on theory-based persuasive or behavior change technologies are published. Future work would benefit from seeing whether our findings are also representative of the broader field of theory-based persuasive or behavior change technologies.

6. Conclusion

This paper presents a scoping review of the use of social comparison in the Persuasive Technology conference proceedings. With this paper, we intended to survey these proceedings to highlight the current trends and potential gaps. We found gaps in how persuasive technology research reports on the operationalization process. We think filling these gaps is required to propel the field forward in terms of getting replicable results and systematizing the operationalization process. To this end, we suggest two methodological improvements for the field: The consistent use of grounding and manipulation checks.

We think it is important that the process of operationalizing a theory into a practical (persuasive) design prototype is explained in as much detail as possible. This will facilitate future research by replicating or reproducing relevant elements of the operationalization process and open up the design space for the formation of designerly knowledge.

Author Contributions

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

Funding

This research was funded by the Pride and Prejudice project by the 4TU federation (www.4tu.nl) under Grant No. 4TU-UIT-346.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

1. ^https://link.springer.com/conference/persuasive

References

Anagnostopoulou, E., Magoutas, B., Bothos, E., Schrammel, J., Orji, R., and Mentzas, G. (2017). “Exploring the links between persuasion, personality and mobility types in personalized mobility applications,” in International Conference on Persuasive Technology (Amsterdam: Springer), 107–118.