The emotions effect on a virtual characters design–A student perspective analysis

Interaction between people and virtual characters through digital and electronic devices is a reality. In this context, the design of virtual characters must incorporate emotional expression at a nonverbal level looking for effective communication with the user. This exploratory study investigates the design features of an avatar functioning as a virtual assistant in educational contexts. From a multidisciplinary approach, the user's research was elaborated by a semi-open questionnaire of self-perception of emotional characteristics: likeability, attractiveness, and applicability of a set of six 2D and 3D characters. The results extracted from a sample of 69 university students provide a relevant information on design features and open new lines for future research. Aspects such as Ekman's basic emotion discrimination and the design of facial expression are analyzed. The incorporation of other body parts, their spatial orientation and contextual elements, seems to contribute to effective emotional communication. The results also highlight how the design of a virtual character should take into consideration the complexity involved in facial gestures and changes in relation to the vertical axis and planes of movement. Finally, this article discusses the complexity involved in expressing a given emotion in a virtual character.

Introduction

Nowadays, the interaction between electronic devices and people is a common routine. We interact with software, virtual characters, and other people through mobile devices and computers. The importance of virtual agents being able to show emotions is increasing in popularity and research effort (Lin et al., 2010; Liebold and Ohler, 2013). Research in Affective Computing is not only about designing software or algorithms to recognize and interpret emotions but also being able to show empathy and emotions to the users (Picard, 1997). Emotion and empathy are two issues that are becoming relevant elements when designing virtual characters; these are two essential resources to enhance communication through verbal and nonverbal expressions (Becker-Asano and Wachsmuth, 2009; Paiva et al., 2017).

The relationship between emotions and the body has been widely studied. Researchers like Damasio (1999) analyzed the physiological processes that explain how emotions emerge from physical signals that the body expresses when reacting to external stimuli; feelings emerge once the brain interprets the emotions.

In recent years, numerous research projects focused on using physical agents such as humanoid robots as a vehicle of affective communication (Tapus and Mataric, 2007; Asada, 2015; Tsiourti et al., 2017). Nevertheless, this fact has generated certain incongruence between the robot's unnatural movements and the socioemotional context of the interaction, which sometimes causes certain rejections or a lack of empathy, often unconsciously, on the part of the users (Tsiourti et al., 2019). Despite the efforts that various studies made to make a robot as human-like as possible, there is always a part that our brain detects and categorizes as unreal: for example, the speed of movement or the dissociation of voice and body language (Ma et al., 2016) and the problem of transferring the complexity of facial and body movements to a physical entity (Mirnig et al., 2015). In fact, they go so far as to point out the users' success in recognizing more emotions in the virtual entity than in the physical entity (Saldien et al., 2010). Recent studies conclude that humans respond socially to virtual humans if they show human-like characteristics (Park and Catrambone, 2021).

Virtual characters are an alternative for digitally modeling movements and emotions to interact with users. Several studies used digital avatars in areas such as education (Chao et al., 2012), psychology (Tanaka et al., 2017), and health (Peña et al., 2020). In a study by Johnson et al. (2016), the authors presented design guidelines for avatar-based interactive applications to evaluate and assist people with social communication disorder (SCD). The research defined a taxonomy of human–avatar interactions to support cognitive processes while considering emotional states and empathic behaviors; they considered the emotion categories proposed by Paul Ekman, excepting disgust (Ekman and Friesen, 1971).

The iAnimate Live project (Kellems et al., 2020) has shown the effect of the interaction between students with special needs in clinics and schools, for example, improving the engagement of individuals when speaking with a virtual character. Virtual characters can be more cost-effective in teaching social skills than traditional methods and helping them learn social skills.

Recently, studies pointed out that using virtual characters, with facial expressions, gestures, or human emotions, in areas such as coaching and education increases attention in students. Furthermore, virtual environments could be useful for increasing the understanding of specific contents in both high schools (Ketelhut et al., 2013) and higher education (Alvarez and Olivera-Smith, 2013). In a study by Chen et al. (2012), researchers analyzed the importance of including empathy as a design element in a virtual avatar, where it was found that, if avatars show positive emotions to the users, they improve in reading by completing the proposed educational exercises. It has been proven that including recognition of emotions by the affective system is essential. Nevertheless, it is also important that the avatar should communicate those emotions. It is fundamental for feedback and empathy with the users. In a study by Chao et al. (2012), the recognition of emotions by the user is included but the communication of emotions by the avatar remains unattended. A comparative analysis by Wang et al. (2021) found a greater user receptiveness when using an avatar with emotional responses unlike using an avatar with a flat facial expression.

Other studies reflected the importance of the creation of avatars for people of different ages and why the users want to identify with them (Paleczna and Szmigielska, 2020) or the importance of adapting the design of virtual agents (e.g., Pedagogic Conversational Agents) to the needs of learners (Pérez-Marín, 2021). Therefore, the avatar should be properly perceived by the users, showing clear emotions, showing correct gestures, and generating empathy to improve the interaction and interest in users.

On this basis, some research questions will guide the work: What visual characteristics make a student choose one or the other avatar in relation to the expression of a specific emotion? In the students' perception, what elements can clarify or facilitate the expression of a particular emotion? Can we increase our understanding of the design characteristics that a virtual agent or character should have to be perceived in the desired way?

Our overall objective is to gather insights about design guidelines of avatars as assistive virtual agents in educational interfaces through an exploratory study. In this study, the end-users, or target users, will be young university students, as this research is the first phase of a virtual coach project in higher education.

Specifically, this study aimed to cover several objectives: first, to explore the emotional perception of the students interacting with a variety of character images and to evaluate the consensus between the emotion perceived by the student and the designer's intention; second, to explore which characters are perceived as more realistic, alive, attractive, trustworthy, and intelligent; and third, to extract information about the applications these characters may have and how to improve emotional representation, according to the students' opinion.

The structure of the paper is as follows: Section Materials and methods introduces the research methodology, the selection criteria of the characters and emotions, and the design of the evaluation questionnaire for the users to be studied; Section Results shows the quantitative and qualitative results of our research. Finally, Section Discussion and future work provides the discussion and conclusions on the effect of emotions on the design of virtual characters.

Materials and methods

The design approach used in this research is determined by the information we acquired from the potential users, in this case, young university students.

According to the standard ISO 9241-210:2019 (Ergonomics of Human-System Interaction - Part 210: Human-centered Design for Interactive Systems (ISO 9241-210:2019), 2019), human-centered design is characterized by a clear understanding of user needs, involvement and giving importance to the user's feedback, among other things. Due to this approach and the nature of the topic, our research tactic is multidisciplinary, with representatives from fields such as animation and digital design, user experience, applied psychology, non-verbal communication, and software engineering.

The user research method is based on questionnaires to gather insights into what specific aspects of virtual characters are preferred by students for this type of assistive software agent, in line with the objectives.

Given the exploratory nature of this study, a mixed approach was used by incorporating both quantitative and qualitative information in the questionnaire. The approach is thus more complex but provides complete information by collecting the opinions and contributions of the participants. All participants signed a consent form, agreeing to the anonymized use of a self-report questionnaire in the current study.

Emotions in virtual character design

Given numerous human–robot interaction (HRI)–related studies (Jack et al., 2014; Wittig et al., 2015; Mcginn, 2020; Wang et al., 2021), where emotions are managed using Ekman's Facial Action Coding System (FACS) (Ekman and Friesen, 1978), our research focuses on 6 basic emotions: happiness, surprise, disgust, sadness, anger, and fear (Ekman et al., 1969; Piórkowska and Wrobel, 2017).

In the process of designing characters, the artist explores a range of the characters' facial expressions (McCloud, 1993; Preston and Radcliff-Mendoza, 1995) through the character expression sheet (Figure 1).

FIGURE 1

Figure 1. Basic character expression sheet designed by Michelle Kham.

Character designs for user research

In the recent research with avatars (Kellems et al., 2020), despite using a character showing natural facial expressions in the experiment, some participants mentioned feeling uncomfortable because of the Uncanny Valley Effect. This may be caused by the fact that the avatar is represented as very human-like and very realistic (MacDorman and Ishiguro, 2006) according to Mori's original Uncanny Valley theory, which advises to escape the valley and avoid designs of very realistic artificial beings (Mori et al., 2012). It is important to note that digital characters are considered to be artificial with X degree of realism (Andreotti et al., 2021). Mori's original theory is also being reinterpreted by Kätsyri et al. (2015), who divided Mori's artificial beings into four groups of human-likeness: clearly artificial, somewhat human-like, almost human-like, and fully human-like (Kätsyri et al., 2015). The CG characters' degree of realism also depends on the texture and color treatment applied to them, hence, the use of light and volume illusion (Thomas and Johnston, 1997). Furthermore, regarding realism degrees, Kätsyri et al. proved that semi-realistic characters tended to fall into the valley (Kätsyri et al., 2016), depending on how their degree of realism is being represented (realistic or cartoon), CG characters could lay in the valley or not (Araujo et al., 2022). Therefore, we avoid using character designs with any realistic degree.

Other research projects (Loewen et al., 2021) determined that the participants, sampled between ages 17 and 24 (same target as our research), preferred idealized to realistic avatars. This is what Frank Biocca called The Cyborg's Dilemma (Biocca, 1997): “a paradoxical situation in which the development of increasingly “natural” and embodied interfaces leads to “unnatural” adaptations or changes in the user.”

In our research, given the literature about the Uncanny Valley (UV) effect and Cyborg's Dilemma, cartoon avatar designs were chosen to avoid both effects and make the avatars more approachable to the students. The fact that all our virtual agent designs chosen have anthropomorphic characteristics is linked to the anthropomorphism phenomenon in HRI, widely studied by Zlotowski et al. (2015).

To establish the guidelines to design this virtual avatar, a variety of designs were chosen from different artists around the world. The primary source was ArtStation.com, which is a webpage where artists share and publish their work. More than 300 concept designs were considered, but only six were finally used in the research, meeting the criteria summarized in Table 1 and having permission to use them in the study. The artists were contacted through email and they accepted to be part of the study. The author names and the characteristics of the character designs are shown in Table 1. The main criteria concerning the cartoon representation technique were to choose characters from the two main groups: 2D and 3D (Figure 2). Two designs for each group (both genders) were selected, and two more drawings were added because we also wanted to see whether it was important for the participant's perception that the art style was distorted or vaguely sketched. All designs were changed into black and white colors to avoid influencing the participants using them.

TABLE 1

Table 1. Character designs used in the questionnaire and their authors.

FIGURE 2

Figure 2. 2D and 3D character designs (see Table 1 for author references and character detail information).

It is important to note that a character design could be drawn directly in two dimensions (2D) or created and modeled in three dimensions (3D) (Figure 2) (Preston and Radcliff-Mendoza, 1995). The main difference between 2D and 3D designs is a deeper perception in the 3D models that could lead to an unconsciously more understanding of the character anatomy structure. Regarding the 3D structure and its rig, some studies using 3D designs investigated the importance of using enough amount of facial blendshapes to represent emotions digitally according to Ekman's Action Units (AU; Carrigan et al., 2020). They found some FACS more noticeable than others and established their order of importance. Although they focused on realistic virtual human faces, those important Action Units that they remarked correspond with the Main AU, mentioned in the previous section. Regarding 2D designs, as animation experts recommend, designers should work with forms and volumes, not just lines (Thomas and Johnston, 1997). This research intended to discern whether using a 2D or 3D design is important for representing emotions.

The characteristics taken into consideration to choose the characters were color-filled, sketch, and distorted shapes inside the 2D group. Regarding the 3D group, the final image could just be represented in color filled as given the characteristics of the 3D techniques. A 3D image could not be represented or perceived as a sketch because it is modeled not a drawing (Figure 2).

The color-filled 2D drawings (characters number 1, 2, and 4 from Figure 2) could give the perception of finished drawings. The sketch drawing (character number 3), for instance, could give the perception that it is just a concept, is not finished, and therefore, is not a character yet that the audience could empathize with (Araujo et al., 2022).

Character number 4 (Lorenzo) was chosen for its general shape to give participants a cartoon human-like character with the most distorted and exaggerated shape. Even though all the characters have human proportions developed using a rounded form method, the final exaggerated and sharp shape can suggest different personalities or even other kinds of living characters. The use of sharp shapes affects the character appeal and the perception of its personality (Thomas and Johnston, 1997). The use of atypical features on characters could also help viewers to experience the UV effect (Kätsyri et al., 2015).

In some studies with virtual agents, only the Agent's head is shown (Chao et al., 2012), so there is a lack of more information from their body language to understand the situation. The effect of body language has been proven in other studies, in which the probability of acceptance increases when the user obtains a complete experience if the robot accompanies its performance with gestures (Salem et al., 2013). Therefore, we will show at least our avatar in a medium shot, simulating a video call communication to better approach participants. In this medium shot, the characters will not show their arms, just their shoulders, to better understand the correlation between animacy and likeability. Some of the characters will also present different spatial orientations with respect to the observer, which can provide information about the dynamism and relational capacity of the character.

Evaluation method: Questionnaire design

Several references were analyzed as input for the questionnaire design. Krägeloh et al. (2019) analyzed six different questionnaires for measuring social acceptability factors related to social robots. Based on semantic differential scales, Bartneck et al. (2009) studied different scales and questionnaires of fundamental design elements and proposed a design to capture anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety in human-to-robot interactions for each aspect. These authors proposed what is known as the Godspeed Questionnaire Series. According to the review of Weiss and Bartneck (2015), the Godspeed questionnaire has been used frequently to measure the direct impression of users in their interaction between humans and robots.

The survey design includes the usage of different types of questions: open-ended to capture qualitative data about students' perceptions, context-specific multi-choice (Likert scale) to capture a specific option, and a semantic differential scale as the one proposed by Bartneck et al. (2009). A simplified and reviewed version of Goodspeed semantic differential scales was used as suggested by Weiss and Bartneck (2015), to avoid overlapping of items in each scale, while maintaining all five scales: the perception of anthropomorphism (is the character more human-like or more machine-like), animacy (what is the perception of life), likeability (how attractive is the character), perceived intelligence of the character, and perceived safety (what is the level of trust induced by the character). Character names, given by the character designers, were shown in the questionnaire as an important part of the animacy and character personality perception. This could also help to create an emotional bond between user-avatar, even though this could influence the character gender perception. The avatar cannot be considered as an object, it is a character (Czerwonka et al., 2021).

Initially, students were asked to access the survey through a welcome page and to choose one character to respond to the whole questionnaire for the selected character (Figure 3). Limiting the questionnaire to one selected character helped to know which character was the most interesting at first glance, and this also limited the total amount of time for survey response to assure student participation. Instructions and time limit were also given to ensure that the time needed to complete the questionnaire was a maximum of 20 min. Each student received the welcome web page link by email.

FIGURE 3

Figure 3. Welcome page for the questionnaire showing the characters (elaborated based on images from authors cited in Table 1).

The questionnaire was structured in several sections to capture the relevant data according to the research goals: (0) student context; (1) student's perception of emotional characteristics of the character; (2) student's perception of the attractiveness of the character based on physical aspects; (3) student's overall impression of the character; and (4) student's opinion about the applicability of each character and their preferred character.

In section (0), the first two questions gather the context for each respondent (student's email as a simple control mechanism of duplicated responses, and the student's degree).

Questions of section (1) were repeated for all six emotions of the character chosen by the participant, using an individual emotion image each time, following the sequence: (1) happiness, (2) surprise, (3) disgust, (4) sadness, (5) anger, and (6) fear. Students were asked to identify the emotion shown in each picture and the level of intensity of the perceived emotion (questions 3 and 4). This section also captures how the students describe the emotion with their own words and what would they add to the image to intensify the perceived emotion (questions 5 and 6).

Section (2) captures the perception of muscular tension, the visual focal point of the character, degree of dynamism, and gender (questions 7 to 10). Emotional expressions are multimodal behavioral patterns. Numerous studies analyzed the physiological and neurological levels of the relationship between muscle tension, eye contact, and perception of movement with the emotions expressed through them (Shafir et al., 2016; Keltner et al., 2019; Scheer et al., 2021). Movement analysis has made it possible to expand knowledge about the relationship between certain motor patterns and expressed emotions, and this is used assiduously in fields as diverse as therapy, education, and the performing arts (Shafir et al., 2016). The human mirror neuron system supports the relationship between the observed facial configuration of another individual (in this case, a character), including their gaze, and the possible inference of emotions through neural activation. The perception of movement (or of a possible movement) generates through the activation of the specular system emotions similar to those that are observed (Schmidt et al., 2020). Furthermore, the chosen elements, muscular tension, visual focus, and perception of movement bring the character the so-called intentional state, i.e., to attribute to an inanimate element the existence of a human mind with beliefs, desires, and emotions (Dennett, 1987). A unique image grid is used to show all six emotions at once for a given character (e.g., the example shown for character Ann in Figure 1) (Isbister, 2006).

Section (3) contains questions related to the mentioned semantic differential scales. The number of options in each scale was limited to three, e.g., in the anthropomorphism scale, the options were human – not human, natural – artificial, conscious – not conscious (questions 11–15). In this section, a unique character image grid was used for the selected character, as the example shown for character Ann in Figure 1.

In section (4), students were asked to specify whether they prefer a virtual character with direct visual contact while it interacts with the user (question 16); what type of virtual assistant they prefer (question 17); student's opinion on the application for the character (question 18); what general aspect the student prefer (question 19); and what characteristics they liked most (question 20). No images were used in this section.

Final question 21 captures the ranking in terms of preference among all the characters, asking the student to sort the characters by preference. For this purpose, the third type of image shows all characters with the same emotion, in this case, happiness (Figure 2).

The questionnaire was written in Spanish and developed using Microsoft Sway for the welcome page and MS Forms for the questionnaire itself.

The detailed list of questions can be found in Appendix A.

Data analysis

Both descriptive and inferential analyses were performed. A chi-squared test was used to assess the existence of significant differences between the characters and the perception of the emotional characteristics. A Kruskal test was used to perform the quantitative analysis of the impression of the students of the six characters, and a Mann–Whitney U test to analyze the pairwise differences.

Finally, an exact one-sample proportion test was used to analyze the main characteristics of the characters required by the students. A p < 0.05 was considered statistically significant, and IBM SPSS v.24 software was used to carry out the statistical analysis.

The qualitative information extracted from the questionnaire was transcript and codified. Successive readings were done until saturation of information was reached. Evidence about students' perceptions, degree of coherence with the designer's intention, and suggestions for intensifying the emotions were extracted following an inductive approach for the six emotions studied. Nvivo 12.0 was used to analyze the information.

Results

General context results

There are a total of 69 completed questionnaires. Of the total number of participants, 44% were women and 56% men. The small sample size for some of the characters analyzed means that, as a whole, the results presented should be considered preliminary.

Student perception of emotional characteristics of the character (quantitative results)

The level of accuracy in identifying each emotion was measured by analyzing what emotions were correctly identified by the students (extracted from responses to the multiple-choice question 3). Sadness, anger, and happiness were the emotions best identified by students. The p-values in Table 2 measure whether there are differences in identifying each emotion in the chosen character. Sadness, fear, and surprise are the three emotions whose identification significantly depends on the character. Results also showed that it is more difficult to identify sadness in Yuriy and fear and surprise in Paul. Concerning anger, happiness, and disgust, these emotions are equally identified in the six characters.

TABLE 2

Table 2. Accuracy in identifying each emotion and the overall accuracy by character.

Question 3 also allows insights into the global misclassification rate for each emotion (Table 3). Surprise is partly misclassified with disgust (18.8%) and fear (17.4%) as well as disgust with fear (23.2%) and fear with surprise (37.7%).

TABLE 3

Table 3. Confusion matrix in emotion identification.

Responses to question 4 capture the intensity of each emotion perceived by the students, over a three-level scale (low, medium, high). Surprise (56.4%), sadness (48.7%), and fear (44.3%) are the most intense perceived emotions. Table 4 contains the global summary for each emotion.

TABLE 4

Table 4. Intensity perceived for each emotion (n = number of responses for each level and emotion, % over the total number of answers for each emotion).

Student perception of attractiveness based on physical aspects (quantitative results)

Lorenzo, Hodga, and Smaragd are the characters with the highest perceived muscular tension, with being Paul and Ann the characters with the lowest perceived muscular tension (question 7). Responses to question 8 show that Ann and Smaragd are perceived as having the most specific visual focus, and the character with the least specific visual focus is Paul. Regarding the degree of mobility of the characters and how active they are perceived by the students (question 9), Hodga is the top character in this aspect, followed by Smaragd and Lorenzo. Yuriy, Ann, and Paul show a discrepancy in perceptions, indicating less consensus among students about the level of movement of these characters. The gender of each character (question 10) is perceived unanimously by all students for Yuriy, Ann, Hodga, and Smaragd. There are some different perceptions of the gender of the characters Paul and Lorenzo. Table 5 contains the overall perception of these aspects.

TABLE 5

Table 5. Perceptions related to various character features (n = number of responses for each character, % over responses for each level and character).

Even if there are no significant differences between the character and muscular tension (p = 0.620), perception of visual focus (p = 0.648), and perception of movement (p = 0.207), we did find significant differences in the character in terms of the perception of the gender (p < 0.001).

Students' overall impression of characters (quantitative results)

Using the Godspeed approach for the perception of artificial characters, the results were analyzed to get insights into the following aspects:

• Which character has been tagged as the most anthropomorphic

• Which character has been tagged as the most alive?

• Which character has been tagged as the most attractive?

• Which character has been tagged as the most intelligent?

• Which character has been tagged as the one who inspired the most confidence?

Table 6 summarizes the findings for questions 11 to 15, using a global score index based on the normalized sum of responses for each aspect and character. A maximum score of 5 points could be reached by any character. The overall best scores were achieved by Ann (4, 61), Smaragd (4, 33), and Paul (4, 30). Lorenzo has the lowest global score (2, 5) because it has very low attractiveness and is the least inspiring confidence even though this character scores highest in “Most alive character.” Table 6 also shows the p-values of the Kruskal test that analyzes whether there are significant differences in the values of each aspect for the six characters. There are significant differences for all the aspects, as well as for the global index, except for alive. Performing pairwise comparisons among the six characters by means of the Mann–Whitney U-test, we conclude that Lorenzo is significantly less anthropomorphic and less attractive than the other characters and that Yuriy and Lorenzo look significantly less intelligent than the other characters. Finally, the global index is significantly smaller for Lorenzo, followed by Yuriy and Hodga, and finally, the global index is considerably higher for Ann, Paul, and Smaragd (see Table 6).

TABLE 6

Table 6. Global perception of each character (normalized global score index over all responses for each character).

Student opinion on applications for each character (quantitative results)

The multiple-choice question 16 (Do you like a virtual character to look at you when interacting with you?) shows that more than 60% of the students do not care about this aspect (Table 7). Only Hodga and Ann are below average for this answer, with 33.3% and 50%, respectively. The multiple-choice question 17 (In a virtual assistant, would you prefer...?) indicates that nearly 30% prefer a real person and that more than 40% prefer a virtual character like the one shown in the images. Question 18 is multiple-answer (Do you think the character seen above would be suitable to be...?). It shows a leveled opinion of the students among all possible options, except for “I don't see any use for it” and “Others.” Question 19 is a multiple-choice question (Do you prefer the character to have an appearance of…?). It indicates that more than 70% of the students (without differences between genders) prefer a human-like appearance. This question showed some differences between genders. Male students' preferred abstract (9% men vs. 2% women) and robotic (6% men vs. 0% women). Female students' preferred animal (6% female vs. 2% male) and object (4% female vs. 0% female). Question 20 is a multiple-answer question (What did you like about your character?). It indicates that gender is the least preferred characteristic (5%), with a leveled response among the rest of the options. Table 7 contains these details.

TABLE 7

Table 7. Student opinion on several application aspects for each character.

The results in Table 7 allow us to conclude that more than 50% of the individuals prefer a virtual assistant with visual interaction (p = 0.045), with human appearance (p = 0.001) and consider the relevant aesthetic (p = 0.001) and gesture and expression (p = 0.003).

Two inputs are used to analyzed the preference expressed by students. One is the character selected by students when they started the survey (from the welcome page – Preference P1). The second input is the results from final question 21, which allows the students to rank the characters by preference, from first to sixth place (explicit preference P2). A comparison with the score for the first place from question 21 shows that Yuriy improved (P1 < P2), Paul and Smaragd did not change preferences (P1 = P2), and Ann, Lorenzo, and Hodga got lower scoring at the end of the questionnaire (P1 > P2). Comparing P1 with the score for sixth place shows that Lorenzo got a significant difference. This character was chosen by 12 students on the welcome page and was voted 42 times as the least preferred character overall. Table 8 shows a summary of results for P1 and P2. No female students chose the characters Hodga and Yuriy. Ann and Smaragd were chosen indistinctly according to the gender of the student. Paul character was chosen by 22% of female students vs. 13% of male students. Lorenzo was chosen by 15% of male students vs. 5% of female students.

TABLE 8

Table 8. Student preferences before and at the end of the questionnaire.

Using the information in Table 8, we aggregated the partial rankings given by the students to obtain a total ranking. For this aim, even if many mathematical tools are available, we consider the classic approach of de Borda (1781) (see Table 9).

TABLE 9

Table 9. Global ranking given by the Borda count.

This global ranking is in accordance with our previous results. Previously, we observed that Lorenzo was considered to be significantly less anthropomorphic, less attractive, and inspiring less confidence than the other avatars, so he is ranked in the last position. Similarly, Lorenzo and Yuriy have a significantly smaller global index than the other characters, which is why they are ranked in the last two positions.

Student's perceptions on characters' emotional expression (qualitative results)

The content analysis of questions 3 and 4 provides information on the students' perception of the set of emotions shown by the characters, the degree of coherence with the designer's intention, and suggestions for intensifying them. Some pieces of evidence are presented in quotation marks and coded by student and character number. The translation of the quotes from Spanish to English was done by one of the research team members.

Happiness

Most students adequately identify this emotion for some of the characters, such as Paul and Smaragd. In many cases, students described the emotion as synonymous such as friendliness, optimism, enthusiasm, and security, “it gives me the feeling of being in front of a kind, self-confident and honest person” [Std33-Smaragd] or “he transmits me companionship and enthusiasm” [Std55-Paul]. For some characters, this emotion relates to feelings of calm and confidence, “more than happiness it generates me relief and calm” [Std64-Paul] and “it produces more comfort than happiness” [Std4-Ann], and even serenity, “a feeling of not very intense happiness but the atmosphere that appears when you talk to a friend and listen to him attentively” [Std12-Yuriy]. In some cases, there is even a connection with memories of a happy childhood: “It makes me feel happy, because it reminds me of the drawings I used to see as a child” [Std67-Paul]. This image also generates the ability to awaken a closeness on the part of the observer: “I feel empathy and it transmits tenderness to me” [Std16-Paul].

A variety of resources appears when the students are asked about what they would add to improve the representation of emotion. They suggest adding elements such as “having arms” [Std1-Hodga], “hands that accompany the expression in a more gestural way” [Std66-Paul], “someone at his side” [Std7-Ann], “appropriate clothing” [Std16-Yuriy], or “a context - like carrying a backpack, a computer, a pencil - to be able to situate what the character is doing and be able to empathize more with him” [Std68-Paul]. Other students suggest improving details, especially in the eyes and mouth, as well as a shift in the axis of the position, for example, “some freckles and eyes looking toward me.” [Std61-Paul], “a bigger smile, more like a laugh” [Std-41-Smaragad], and “smile more with the eyes” [Std3-Smaragd].

Surprise

Surprise is sometimes either not completely identified or appears tinged with a negative quality, “it provokes surprise but in a negative way as if you have just received bad news” [Std2-Hodga]. It also appears confused with a wide variety of emotions such as sadness, disappointment, indifference or uncertainty, “not pleasant, surprise, disillusionment perhaps” [STd9-Ann], and even emotions such as disgust, “it is a face of unpleasant disgust, as if toward another person” [STd55-Paul], or fear “surprised perhaps with fear” [Std26-Lorenzo]. The association with emotions and disgust appears mostly in the case of the character Paul, where only two of the 23 students identify the gesture with surprise. This also occurs in the case of Ann, only two of the 6 students identify surprise, but the others indicate fear or sadness.

This confusion in determining the emotion shown is also reflected in the proposals to improve the characteristics of the character. When there is a correct identification with the designer's intention, the students suggest modifications either in details of the face, “the mouth should be more closed, it seems too exaggerated” [Std33-Smaragad] or “more expression in the eyes” [Std41-Smaragd], or in external elements that can provide contextual information. So, “a train coming in or out of the mouth” [Std20-Lorenzo] or “putting hands on the face” [Std31-Smaragd] is suggested. One of the students who associate disgust rather than surprise to the image puts the focus on the importance of the rest of the body, “bodily expressions mainly. The same facial expression can mean different things depending on the body expression - what the hands do, if it is shock and he jumps, if it's disgust and he walks away from the scene, etc.” [Std68-Paul].

Disgust

Students do not identify this emotion with the designer's intention. Many of them associate other emotions such as fear, anger, and sadness, “it can provoke fear and disgust” [STd56-Paul]. This makes them contradict themselves. One student who chooses the emotion sadness says “I feel sorry for someone so angry” [Std22-Lorenzo]; another uses fear and disgust as synonyms [Std15-Yuriy]. Some note inconsistencies in the facial gesture itself “it is a bit confusing, on the one hand, it seems to reflect elation, but the wrinkles between the eyebrows seem to show disgust, despite the smile” [Std45-Smaragd]. Those who agree with the designer' proposal state that the emotion is in relation to something external, for instance, “he is seeing something he does not like” [Std24-Lorenzo], “the drawing is rejecting something, it seems that he is refusing to do something because it disgusts him - he closes his eyes as if he does not want to see it-” [Std36-Smaragd]. They also use other terms such as displeasure, pity, or shame.

In relation to suggestions to intensify the emotion generated, specific details of the face appear, for example, “stick out your tongue” [Std4-Ann], “frown and nose more” [Std41-Smaragad], or “lower the corners of your lips” [Std3-Hodga], elements mentioned by several participants. Other students consider “a 3D design could better define the emotion” [Std5-Ann] or the incorporation of some external element to “understand the source of the emotion” [Std47-Paul].

Sadness

Sadness is correctly identified by 65 students for the different characters, except for the character of Yuriy where three students indicate other emotions such as disgust or anger. An extensive vocabulary appears to characterize this emotion: disappointment, longing, loneliness, nostalgia, hopelessness, worry, decay, and resignation. Some students indicate what the character makes them feel and they empathize with him: “it makes me sad and also makes me want to ask him how he is” [Std67-Paul] or “it makes me sad, very sad” [Std17-Lorenzo]. This concern for the character repeatedly appears in Paul's case. In Smaragd's character, it seems that some credibility is missing in the face “he seems to be pouting, his sadness is not very convincing” [Std33-Smaragd].

There is unanimity on the use of tears to accentuate the expression together with modifications in the eyes and mouth: “maybe adding tears and lowering the chin could increase the intensity of the emotion” [Std12-Yuriy], “some kind of wrinkle in the face” [Std36-Smaragd], or “sadder eyes” [Std59-Paul]. Again, the importance of the rest of the body to intensify the emotion appears, such as “position of the shoulders” [Std63-Paul] or the context “maybe a gray sky, rain, cold colors” [Std67-Paul].

Anger

There is unanimity in the recognition of anger in the characters of Hodga, Lorenzo, and Smaragd. The participants mention “you can see that he is angry or upset about something” [Std1-Hodga] and terms such as being upset, disapproval, frustration, and anger appear. Several of them mention that there is more anger than annoyance, that is, they do not appreciate the high intensity in the image, for example, “it gives a feeling of being angry but without reaching an extreme anger” [Std27-Lorenzo] or “annoyance or repressed anger, it seems that someone is being unfair with her or with someone around her” [Std33-Smaragd]. Even terms such as superficiality or indifference appear in relation to this limited intensity of emotion. Some students discriminate between the emotion expressed by the character and the one they feel when visualizing it. There is no identification with it, for example, they assign the emotion of surprise to the image but mention “I am surprised to see him angry, he does not look like the classic character who is often angry” [Std60-Paul] or “I am afraid because I feel that he is scolding me for something and he is very angry” [Std48-Paul].

Regarding the suggestions to intensify the emotion again, we can distinguish between those that are characteristics referring to the character's own features, e.g., “wrinkle the nose more, furrow the eyebrows more and tense the mouth more” [Std33-Smaragd] or “pupils narrower” [Std47-Paul], or from the context “a little background fire would do him good” [Std20-Lorenzo]. Some contributions refer to body position “be staring straight at me” [Std61-Paul] and coherence in the message that can be enhanced when incorporating the rest of the body and changes in relation to the vertical axis: “Body expression, it is rare in real life to shout at someone leaning forward and with your arms behind you. The more we express ourselves verbally and facially, the more we use our hands and the rest of the body. It is logical to be leaning forward in an “attack” position, but the hands should accompany - as a natural instinct of intimidation trying to appear larger and more dangerous –” [Std68-Paul].

Fear

This emotion is identified in almost all the characters except for Paul. All students (22) who chose Paul associated with the surprise emotion instead of fear. In the case of Yuriy, half of the students associated the character with surprise instead of fear. In the rest of the answers there is more unanimity, despite some descriptions related to the concept of surprise, for example, “unpleasant surprise” [Std29-Smaragd], or disgust also appears “between scared and surprised, between fear and disgust like when you see a bug or a cockroach” [Std33-Smaragd]. It is interesting how some participants identify themselves as feeling the same emotion, “it transmits fear, therefore it is what I feel” [Std35-Smaragd], while others distinguish the expression of the character from their own emotion when viewing it. So, a student who identifies fear says “it causes me distress” [Std31-Smaragd]. It is very evident the confusion with the character Paul; students are surprised and use words such as singing, enjoyment, and illusion. In the case of Yuriy, some students describe the observed contradiction, choose fear, and explain “it can be confused with surprise because of the open eyes, but the slightly open mouth and the teeth can indicate a possible jaw tremor, expressing fear” [Std12-Yuriy].

Among the suggestions provided to intensify the emotion “more pronounced neck muscles may show a greater degree of intensity” [Std12-Yuriy] or “I would wrinkle the brow down more - as it was under tension-and open his mouth wider” [Std33-Smaragd]. Again, the context and the complement of other body parts appear as improvements “sounds and expression with other body parts and gestures accompanying the facial gestures” [Std41-Smaragd] or “bodily expressions - hands beside the face or on top of the head grabbing the hair” [Std68-Paul].

Discussion and future work

The fundamental question that drives this research is to explore the design aspects of digital characters that fit best for a software avatar applied to education support systems. In this sense, an ideal virtual character could be designed based on the questionnaire result analysis and students' perceptions.

The decision about using a human-like character for a virtual agent could be settled, because 72% of participants preferred human-like agents, regarding the results shown in Table 7. Therefore, the character should have a human aspect, with anthropomorphic elements such as Hodga, Yuriy, and/or Smaragd.

For an attractive avatar, design elements like the ones shown in Lorenzo appear not to be appropriate. Given the results about the lowest global score (Table 6), the idea of using rounded shapes instead of sharp shapes is reinforced to give appeal to the character. This seems to be also connected to confidence scores. The design of Ann, Lorenzo, and Hodga (characters that were perceived more alive by users) allows observing muscle tension lines as well as a rotation of the torso with respect to the vertical axis, thus increasing the sensation of movement and approach to the observer. Therefore, design elements used in characters like Ann, Lorenzo, and Hodga could be considered for an alive avatar.

This perception leads to the relationship between the personality of virtual agents and the empathy or rejection that they can generate. This idea is linked to the stereotypical expectations of personality that people hold when thinking about particular jobs or roles, as in some studies shown in the introduction (Joosse et al., 2013). The role of this ideal character as a virtual assistant needs to consider the results of its design. For example, Paul, Ann, and Lorenzo are chosen as the best characters to ask academic and university questions, and Hodga is the best character to be a virtual assistant to study. This result could also reinforce the idea of using stereotypical elements to the character according to the task, such as glasses like the ones our character Paul wears. None of the students commented about it directly, just one student who chose Paul, mentioned elements regarding educational context “like carrying a backpack, a computer, a pencil - to be able to situate what the character is doing and be able to empathize more with him.” Nevertheless, Paul was not perceived as the most intelligent character. For the design of an intelligent avatar, using design elements such as formal dress, haircut, and gender representation (e.g., the ones used in Ann) could be considered. On the other side, elements such as scruffy looks and goofy expressions (e.g., the ones used in Yuriy, Lorenzo, or Hodga) are not recommended to be used.

Therefore, one important thing for further research is to highlight what qualities our virtual character should have to implement in the personality of our entity and thus, determine what movements will be congruent with that personality.

Regarding the results about virtual character gender, it is interesting to note that the female characters were scored as the best fit to share personal concerns and the characters who inspire more confidence by a big margin (Table 6). Also, students perceived them as the most confident ones. This result could be unconsciously influenced by the stereotypical association between women and education roles and should be considered for further work (Eagly and Koenig, 2021). There is also a remarkable result about Paul's and Lorenzo's gender (Table 5). It can be seen that doubts were generated about those character's gender. Regarding Paul, the reason could be the sketch technique itself because it has fewer details in its design anatomy structure, and factions. Regarding Lorenzo's design, the exaggerated and less human-like shape could suggest some other kind of character with non-binary gender. Also, the high percentage of male perception could be influenced by its male names. Some considerations regarding the gender of the students and their initial choice of character should also be made. No female students chose the characters of Hodga and Yuriy, and some further differences were observed in the selection according to the gender of the students.

The results about a global expression of emotions indicate that the more detailed a character's design is (e.g., Smaragd and Hodga), the more accurate seems the emotion perception by the participants. Additionally, on numerous occasions, the emotion that the designer wants to represent is not adequately captured by the students, particularly in the case of surprise, fear, and disgust, as shown in the confusion matrix (Table 3). Normally, these emotions are difficult to observe and recognize by people who are not experienced in emotional recognition (Ekman, 2007). Also, they are difficult to represent in two dimensions, given the natural synthesis result from drawing. This could be extracted from the participant's results because Paul, a 2D sketch, has been designed with fewer details in comparison with others like Smaragd and Hodga, which have the best results related to surprise recognition (Table 2). It is important to note that both Smaragd and Hodga are 3D representations, while others, like Lorenzo or Ann, are very detailed 2D designs. They also have a good score in emotional accuracy (Table 2).

This could lead to the conclusion that the 3D style is not mandatory to represent emotions in better way. A 2D design could be used if it is done with enough details and anatomy representation.

That is also settled for a specific representation of emotions. The characteristics of the following pairs of character-emotion should be considered: Ann-Happiness, Hodga-Surprise, Smaragd-Surprise, Hodga-Fear, Ann-Sadness, Hodga-Sadness, Smaragd-Sadness, Lorenzo-Sadness, Lorenzo-Anger, and Hodga-Anger (Figure 4).

FIGURE 4

Figure 4. Relevant character-emotion pairs.

Regarding character design, especially when difficult emotions need to be expressed, it seems essential to incorporate a contextual element that helps to understand the emotion as well as to incorporate other parts of the body, changes in relation to the vertical axis and planes, and sensation of movement on it. This corroborates previous studies (Salem et al., 2013) and reinforces the idea for future research not only to use a 2D static image but to use an animated character that interacts with the student. Furthermore, contextual information and cultural dependence could influence the perception and consideration of certain visual elements. The focus of this paper has been directed toward the analysis of different face gesture designs. In this sense, evidence suggests facial expressions and other emotional movements are cross-culturally universal (Izard, 1994; Scherer et al., 2001; Dael et al., 2012). Different objectives aimed for example at the analysis and facilitation of body-mind integration for clinical proposals would have required an understanding of social context (Ono et al., 2019; Chang, 2021) since movement is influenced by the psychophysical orientation of culture (Chang, 2021).

It is also important to note that some students show difficulties both in discriminating emotions and in the use of the vocabulary that could represent them, which opens a line of future research regarding one of the elements of emotional intelligence (the recognition and expression of emotions) (Gartmeier and Tettegaah, 2015). Other students differentiate and indicate them verbally, between the emotion that the character seems to represent and the one they feel. There is not always absolute identification.

In any case, there are some important limitations identified during this study. First, there was no pre-post questionnaire design, so we could not measure the improvement regarding a starting level. Second, it would be important to analyze students' ability to recognize and understand emotions through standardized tools. This would make it possible to refine the results. Third, the small sample size for some of the characters (as shown in Table 10). This fact does not assure the convergence of the statistic to the chi-square distribution; hence, these results can only be taken as preliminary. The small sample size has also prevented a segmented analysis by gender. The choice of one or the other character could have an added bias due to the gender of the students and it remains for future work to expand the sample by incorporating these aspects into the study. Finally, it is worth mentioning that the designs made by the artist are deeply connected to the understanding of virtual character's emotions, which could bias the participant's perceptions of them.

TABLE 10

Table 10. General results about student context.

Several future research opportunities can be identified based on the results of this study and the mentioned limitations:

1. Design and implement a virtual coach prototype for university students, based on the insights gathered in this research, to study the impact of applying this type of character as a education support system.

2. Define virtual coach design guidelines with a focus on emotion representation.

3. Incorporate movement analysis into the virtual design guidelines. Study embodiment impacts this type of education supports systems and deepens non-verbal communication by incorporating movements, understanding the avatar as a character with a unique personality and appearance that can generate empathy and credibility in the user.

4. Study the application of emotions for the design of a user-centric education support system.

5. Deepen the study of the relationship between gender and roles in the design of digital characters.

6. Study the use of color and its impact on character acceptance and empathy.

Data availability statement

The original data of this contribution can be requested to the corresponding author.

Ethics statement

The studies involving human participants were reviewed and approved by Universidad Europea de Madrid. The patients/participants provided their written informed consent to participate in this study.

Author contributions

AC-P, MV-C, and CM: conceptualization, software, and visualization. AC-P, CM, and R-MR-J: methodology. CM and R-MR-J: validation. AC-P, MV-C, CM, and R-MR-J: formal analysis, investigation, writing–original draft preparation, and writing–review and editing. MV-C: resources. CM: data curation. AC-P: supervision and project administration. All authors have read and agreed to the published version of the manuscript.

Acknowledgments

This research was supported by Universidad Europea de Madrid through the project: CIPI/21/083 “Embodiment en plataformas virtuales y robóticas. Análisis comparativo.” The authors thank the Artists: Lisa Henke, Michelle Kham, Jean Paul Medellin, Adrián de la Cruz, and Jimmy Levinsky, for allowing us to use their designs for this study.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fcomp.2022.892597/full#supplementary-material

References

Alvarez, I. M., and Olivera-Smith, M. (2013). Learning in social networks: Rationale and ideas for its implementation in higher education. Educ. Sci. 3, 314–325. doi: 10.3390/educsci3030314

CrossRef Full Text | Google Scholar

Andreotti, L., Weber, M. L., da Silva, T. L., de Andrade Araujo, V. F., and Musse, S. R. (2021). Perception of Charisma, Comfort, Micro and Macro Expressions in Computer Graphics Characters. 2021 20th Brazilian Symposium on Computer Games and Digital Entertainment (SBGames) (Brazil), 107–116.

Google Scholar

Araujo, V., Melgare, J., Dalmoro, B., and Musse, S. (2022). Is the Perceived Comfort With CG Characters Increasing With Their Novelty? IEEE Comput. Graph. Appl. 42, 32–46. doi: 10.1109/MCG.2021.3090198

PubMed Abstract | CrossRef Full Text | Google Scholar

Asada, M. (2015). Towards Artificial Empathy. Int. J. Soc. Robot. 7, 19–33. doi: 10.1007/s12369-014-0253-z

CrossRef Full Text | Google Scholar

Bartneck, C., Kuli,ć, D., Croft, E., and Zoghbi, S. (2009). Measurement instruments for the anthropomorphism, animacy, likeability, perceived intelligence, and perceived safety of robots. Int. J. Soc. Robot. 1, 71–81. doi: 10.1007/s12369-008-0001-3

CrossRef Full Text | Google Scholar

Becker-Asano, C., and Wachsmuth, I. (2009). Affective computing with primary and secondary emotions in a virtual human. Auton Agents Multi Agent Syst. 20, 32. doi: 10.1007/s10458-009-9094-9

CrossRef Full Text | Google Scholar

Biocca, F. (1997). The cyborg's dilemma: progressive embodiment in virtual environments [1]. J. Comput. Mediat. Commun. 3, 12–26. doi: 10.1111/j.1083-6101.1997.tb00070.x

CrossRef Full Text | Google Scholar

Carrigan, E., Zibrek, K., Dahyot, R., and McDonnell, R. (2020). “Investigating perceptually based models to predict importance of facial blendshapes,” in Motion, Interaction and Games (MIG '20). (New York, NY: Association for Computing Machinery), 1–6. doi: 10.1145/3424636.3426904

CrossRef Full Text | Google Scholar

Chang, M. H. (2021). “Cultural Consciousness and the Global Context of Dance/ Movement Therapy,” in The Art and Science of Dance/Movement Therapy (New York, NY: Routledge) 349–366.

Google Scholar

Chao, C., Lin, H. K., Lin, J., and Tseng, Y. (2012). “An affective learning interface with an interactive animated agent,” in 2012 IEEE Fourth International Conference On Digital Game And Intelligent Toy Enhanced Learning (Takamatsu), 221–225.

Google Scholar

Chen, G. D., Lee, J. H., Wang, C. Y., Chao, P. Y., Li, L. Y., and Lee, T. Y. (2012). An empathic avatar in a computer-aided learning program to encourage and persuade learners. Educ. Technol. Soc. 15, 62–72.

Google Scholar

Czerwonka, S., Alvarez, A., and McArthur, V. (2021). One ring fit to rule them all? An Analysis of avatar bodies and customization in exergames. Front. Psychol. 12, 4941. doi: 10.3389/fpsyg.2021.695258

PubMed Abstract | CrossRef Full Text | Google Scholar

Dael, N., Mortillaro, M., and Scherer, K. R. (2012). Emotion expression in body action and posture. Emotion 12, 1085–1101. doi: 10.1037/a0025737

PubMed Abstract | CrossRef Full Text | Google Scholar

Damasio, A. (1999). The Feeling of What Happens Body and Emotion in the Making of Consciousness. Pennsylvania: Harcourt Brace and Company.

PubMed Abstract | Google Scholar

de Borda, J.-C. (1781). Mémoire sur les élections au scrutin. Histoire de l'Académie Royale des Sciences.

Google Scholar

Dennett, D. (1987). Then Intentional Stance. Durham, NC: The MIT Press.

Google Scholar

Eagly, A. H., and Koenig, A. M. (2021). The vicious cycle linking stereotypes and social roles. Curr. Direct. Psychol. Sci. 30, 343–350. doi: 10.1177/09637214211013775

CrossRef Full Text | Google Scholar

Ekman, P. (2007). Emotions Revealed: Recognizing Faces and Feelings to Improve Communication and Emotional Life. New York, NY: Henry Holt.

Google Scholar

Ekman, P., and Friesen, W. (1971). Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17, 124–129. doi: 10.1037/h0030377

PubMed Abstract | CrossRef Full Text | Google Scholar

Ekman, P., and Friesen, W. V. (1978). Facial Action Coding System: Investigator's Guide. Salt Lake City, UT: Consulting Psychologists Press.

Google Scholar

Ekman, P., Sorenson, E. R., and Friesen, W. V. (1969). Pan-cultural elements in facial displays of emotion. Science 164, 86 LP–88. doi: 10.1126/science.164.3875.86 Ergonomics of Human-System Interaction- Part 210: Human-centred design for interactive systems (ISO 9241-210:2019). (2019). Pub. L. No. ISO 9241-210:2019, 33. Available online at: https://www.iso.org/standard/77520.html (accessed December 19, 2021).

PubMed Abstract | CrossRef Full Text | Google Scholar

Gartmeier, M., and Tettegaah, S. Y. (2015). Emotions, Technology, Design, and Learning. Cambridge, MA: Academic Press.

Google Scholar

Isbister, K. (2006). “Culture: research findings and characters,” in Better Game Characters by Design. San Francisco, CA: Morgan Kaufmann.

Google Scholar

Izard, C. E. (1994). Innate and universal facial expressions: Evidence from developmental and cross-cultural research. Psychol. Bull. 115, 288–299. doi: 10.1037/0033-2909.115.2.288

PubMed Abstract | CrossRef Full Text | Google Scholar

Jack, R. E., Garrod, O. G. B., and Schyns, P. G. (2014). Dynamic facial expressions of emotion transmit an evolving hierarchy of signals over time. Curr. Biol. 24, 187–192. doi: 10.1016/j.cub.2013.11.064

PubMed Abstract | CrossRef Full Text | Google Scholar

Johnson, E., Hervás, R., Franca, C., Mondejar, T., Ochoa, S., and Favela, J. (2016). Assessing empathy and managing emotions through interactions with an affective avatar. Health Inf. J. 24, 182–93. doi: 10.1177/1460458216661864

PubMed Abstract | CrossRef Full Text | Google Scholar

Joosse, M., Lohse, M., Pérez, J. G., and Evers, V. (2013). “What you do is who you are: The role of task context in perceived social robot personality,” in 2013 IEEE International Conference on Robotics and Automation (Karlsruhe), 2134–2139.

Google Scholar

Kätsyri, J., Förger, K., Mäkäräinen, M., and Takala, T. (2015). A review of empirical evidence on different uncanny valley hypotheses: support for perceptual mismatch as one road to the valley of eeriness. Front. Psychol. 6, 390. doi: 10.3389/fpsyg.2015.00390

PubMed Abstract | CrossRef Full Text | Google Scholar

Kätsyri, J., Mäkäräinen, M., and Takala, T. (2016). Testing the ‘uncanny valley' hypothesis in semirealistic computer-animated film characters: an empirical evaluation of natural film stimuli. Int. J. Hum. Comput. Stud. 97, 149–61. doi: 10.1016/j.ijhcs.2016.09.010

CrossRef Full Text | Google Scholar

Kellems, R. O., Charlton, C., Kversøy, K. S., and Gyori, M. (2020). Exploring the use of virtual characters (Avatars), live animation, and augmented reality to teach social skills to individuals with Autism. Multimodal Technol. Interact. 4, 48–59. doi: 10.3390/mti4030048

CrossRef Full Text | Google Scholar

Keltner, D., Sauter, D., Tracy, J., and Cowen, A. (2019). Emotional expression: advances in basic emotion theory. J. Nonverb. Behav. 43, 133–160. doi: 10.1007/s10919-019-00293-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Ketelhut, D. J., Nelson, B., Schifter, C., and Kim, Y. (2013). Improving science assessments by situating them in a virtual environment. Educ. Sci. 3, 172–192. doi: 10.3390/educsci3020172

CrossRef Full Text | Google Scholar

Krägeloh, C. U., Bharatharaj, J., Sasthan Kutty, S. K., Nirmala, P. R., and Huang, L. (2019). Questionnaires to measure acceptability of social robots: a critical review. Robotics 8, 88. doi: 10.3390/robotics8040088

CrossRef Full Text | Google Scholar

Liebold, B., and Ohler, P. (2013). “Multimodal emotion expressions of virtual agents, mimic and vocal emotion expressions and their effects on emotion recognition,” in 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction (Geneva), 405–410.

Google Scholar

Lin, S., Zhigang, L., and Aihua, D. (2010). Research of emotion modeling for intelligent virtual agent,” in 2010 The 2nd International Conference on Computer and Automation Engineering (ICCAE) (Singapore), 286–289.

PubMed Abstract | Google Scholar

Loewen, M. G. H., Burris, C. T., and Nacke, L. E. (2021). Me, myself, and not-I: self-discrepancy type predicts avatar creation style. Front. Psychol. 11, 1–7. doi: 10.3389/fpsyg.2020.01902

PubMed Abstract | CrossRef Full Text | Google Scholar

Ma, G., Gao, J., Yu, Z., Chen, X., Huang, Q., and Liu, Y. (2016). Development of a Socially Interactive System with Whole-Body Movements for BHR-4. Int. J. Soc. Robot. 8, 183–192. doi: 10.1007/s12369-015-0330-y

CrossRef Full Text | Google Scholar

MacDorman, K. F., and Ishiguro, H. (2006). The uncanny advantage of using androids in cognitive and social science research. Interact. Stud. 7, 297–337. doi: 10.1075/is.7.3.03mac

CrossRef Full Text | Google Scholar

McCloud, S. (1993). Understanding Comics: The Invisible Art. New York, NY: Tundra Publishing Ltd.

Google Scholar

Mcginn, C. (2020). Why do robots need a head? The role of social interfaces on service robots. Int. J. Soc. Robot. 12, 281–295. doi: 10.1007/s12369-019-00564-5

CrossRef Full Text | Google Scholar

Mirnig, N., Strasser, E., Weiss, A., Kühnlenz, B., Wollherr, D., and Tscheligi, M. (2015). Can you read my face?: A methodological variation for assessing facial expressions of robotic heads. Int. J. Soc. Robot. 7, 63–76. doi: 10.1007/s12369-014-0261-z

CrossRef Full Text | Google Scholar

Mori, M., MacDorman, K. F., and Kageki, N. (2012). The uncanny valley [From the Field]. IEEE Robot. Autom. Mag. 19, 98–100. doi: 10.1109/MRA.2012.2192811

CrossRef Full Text | Google Scholar

Ono, Y., Aoyagi, S., Sugimoto, M., Yamazaki, Y., Yamamoto, M., and Nagata, N. (2019). “Application of classification method of emotional expression type based on laban movement analysis to design creation,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Springer International Publishing).

Google Scholar

Paiva, A., Leite, I., Boukricha, H., and Wachsmuth, I. (2017). Empathy in virtual agents and robots: a survey. ACM Trans. Interact. Intell. Syst. 7, 1–40. doi: 10.1145/2912150

CrossRef Full Text | Google Scholar

Paleczna, M., and Szmigielska, B. (2020). Avatars in education : age differences in avatar customization. Psychol. Educ. 57, 203–205. Available online at: https://ruj.uj.edu.pl/xmlui/bitstream/handle/item/267531/paleczna_szmigielska_avatars_in_education_age_differences_in_avatar_customization_2020.pdf?sequence=1&isAllowed=y

Google Scholar

Park, S., and Catrambone, R. (2021). Social responses to virtual humans: the effect of human-like characteristics. Appl. Sci. 11, 7214–7238. doi: 10.3390/app11167214

CrossRef Full Text | Google Scholar

Peña, J., Li, B. J., and Ratan, R. (2020). “Digital embodiment and improving health outcomes,” in Technology and Health (Elsevier), 27–47.

Google Scholar

Pérez-Marín, D. (2021). A review of the practical applications of pedagogic conversational agents to be used in school and university classrooms. Digital 1, 18–33. doi: 10.3390/digital1010002

CrossRef Full Text | Google Scholar

Picard, R. W. (1997). Affective Computing. Cambridge, MA: MIT Press.

Google Scholar

Piórkowska, M., and Wrobel, M. (2017). “Basic emotions,” in Encyclopedia of Personality and Individual Differences (Rochester, NY: Springer).

Google Scholar

Preston, B., and Radcliff-Mendoza, C. (1995). Cartoon Animation with Preston Blair, Revised Edition!: Learn Techniques for Drawing and Animating Cartoon Characters. London: Walter Foster Publishing.

Google Scholar

Saldien, J., Goris, K., Vanderborght, B., Vanderfaeillie, J., and Lefeber, D. (2010). Expressing emotions with the social robot probo. I. J. Soc. Robot. 2, 377–389. doi: 10.1007/s12369-010-0067-6

CrossRef Full Text | Google Scholar

Salem, M., Eyssel, F., Rohlfing, K., Kopp, S., and Joublin, F. (2013). To Err is Human(-like): Effects of Robot Gesture on Perceived Anthropomorphism and Likability. Int. J. Soc. Robot. 5, 313–323. doi: 10.1007/s12369-013-0196-9

CrossRef Full Text | Google Scholar

Scheer, C., Kubowitsch, S., Dendorfer, S., and Jansen, P. (2021). Happy enough to relax? How positive and negative emotions activate different muscular regions in the back - an explorative study. Front. Psychol. 12, 511746. doi: 10.3389/fpsyg.2021.511746

PubMed Abstract | CrossRef Full Text | Google Scholar

Scherer, K. R., Banse, R., and Wallbott, H. G. (2001). Emotion inferences from vocal expression correlate across languages and cultures. J. Cross Cult. Psychol. 32, 76–92. doi: 10.1177/0022022101032001009

CrossRef Full Text | Google Scholar

Schmidt, S. N. L., Sojer, C. A., Hass, J., Kirsch, P., and Mier, D. (2020). fMRI adaptation reveals: the human mirror neuron system discriminates emotional valence. Cortex 128, 270–280. doi: 10.1016/j.cortex.2020.03.026

PubMed Abstract | CrossRef Full Text | Google Scholar

Shafir, T., Tsachor, R. P., and Welch, K. B. (2016). Emotion regulation through movement: unique sets of movement characteristics are associated with and enhance basic emotions. Front. Psychol. 6, 2030. doi: 10.3389/fpsyg.2015.02030

PubMed Abstract | CrossRef Full Text | Google Scholar

Tanaka, H., Adachi, H., Ukita, N., Ikeda, M., Kazui, H., Kudo, T., et al. (2017). Detecting dementia through interactive computer avatars. IEEE J. Transl. Eng. Health Med. 5, 1–11. doi: 10.1109/JTEHM.2017.2752152

PubMed Abstract | CrossRef Full Text | Google Scholar

Tapus, A., and Mataric, M. J. (2007). “Emulating empathy in socially assistive robotics.” in AAAI Spring Symposium: Multidisciplinary Collaboration for Socially Assistive Robotics (California), 93–96.

Google Scholar

Thomas, F., and Johnston, O. (1997). The Illusion of Life: Disney Animation. New York, NY: Hyperion.

Google Scholar

Tsiourti, C., Weiss, A., Wac, K., and Vincze, M. (2017). Designing emotionally expressive robots: a comparative study on the perception of communication modalities,” in Proceedings of the 5th International Conference on Human Agent Interaction (Bielefeld).

Google Scholar

Tsiourti, C., Weiss, A., Wac, K., and Vincze, M. (2019). Multimodal Integration of Emotional Signals from Voice, Body, and Context: Effects of (In)Congruence on Emotion Recognition and Attitudes Towards Robots. Int. J. Soc. Robot. 11, 555–573. doi: 10.1007/s12369-019-00524-z

CrossRef Full Text | Google Scholar

Wang, H., Gaddy, V., Beveridge, J. R., and Ortega, F. R. (2021). Building an Emotionally Responsive Avatar with Dynamic Facial Expressions in Human—Computer Interactions. Multimodal Technol. Interact. 5, 13–38. doi: 10.3390/mti5030013

CrossRef Full Text | Google Scholar

Weiss, A., and Bartneck, C. (2015). “Meta analysis of the usage of the Godspeed Questionnaire Series,” in 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (Kobe: IEEE Press), 381–388. doi: 10.1109/ROMAN.2015.7333568

CrossRef Full Text | Google Scholar

Wittig, S., Rätsch, M., and Kloos, U. (2015). “Parameterized facial animation for socially interactive robots,” in Mensch und Computer 2015 – Proceedings, eds S. Diefenbach, N. Henze, and M. Pielot (Berlin: De Gruyter Oldenbourg), 355–358.

Google Scholar

Zlotowski, J., Proudfoot, D., Yogeeswaran, K., and Bartneck, C. (2015). Anthropomorphism: opportunities and challenges in human–robot interaction. Int. J. Soc. Robot. 7, 347–360. doi: 10.1007/s12369-014-0267-6

CrossRef Full Text | Google Scholar