Feasibility of an Autism-Focused Augmented Reality Smartglasses System for Social Communication and Behavioral Coaching

Background: Autism spectrum disorder (ASD) is a childhood-onset neurodevelopmental disorder with a rapidly rising prevalence, currently affecting 1 in 68 children, and over 3.5 million people in the United States. Current ASD interventions are primarily based on in-person behavioral therapies that are both costly and difficult to access. These interventions aim to address some of the fundamental deficits that clinically characterize ASD, including deficits in social communication, and the presence of stereotypies, and other autism-related behaviors. Current diagnostic and therapeutic approaches seldom rely on quantitative data measures of symptomatology, severity, or condition trajectory.

Methods: Given the current situation, we report on the Brain Power System (BPS), a digital behavioral aid with quantitative data gathering and reporting features. The BPS includes customized smartglasses, providing targeted personalized coaching experiences through a family of gamified augmented-reality applications utilizing artificial intelligence. These applications provide children and adults with coaching for emotion recognition, face directed gaze, eye contact, and behavioral self-regulation. This preliminary case report, part of a larger set of upcoming research reports, explores the feasibility of the BPS to provide coaching in two boys with clinically diagnosed ASD, aged 8 and 9 years.

Results: The coaching intervention was found to be well tolerated and rated as being both engaging and fun. Both males could easily use the system, and no technical problems were noted. During the intervention, caregivers reported improved non-verbal communication, eye contact, and social engagement during the intervention. Both boys demonstrated decreased symptoms of ASD, as measured by the aberrant behavior checklist at 24-h post-intervention. Specifically, both cases demonstrated improvements in irritability, lethargy, stereotypy, hyperactivity/non-compliance, and inappropriate speech.

Conclusion: Smartglasses using augmented reality may have an important future role in helping address the therapeutic needs of children with ASD. Quantitative data gathering from such sensor-rich systems may allow for digital phenotyping and the refinement of social communication constructs of the research domain criteria. This report provides evidence for the feasibility, usability, and tolerability of one such specialized smartglasses system.

Background

Autism spectrum disorder (ASD) is a childhood-onset neurodevelopmental disorder with a rapidly increasing prevalence. ASD is now thought to affect as many as 1 in 68 children, and over 3.5 million people, in the United States (1). The economic impact of ASD is substantial with an aggregated total cost in the United States of approximately $236 billion (2). This figure includes direct and indirect costs from a variety of sources, including medical care, special education, and lost parental productivity (2, 3). Many children with ASD benefit from intensive behavioral interventions, although the costs are often prohibitive, ranging between an additional $40,000 and $60,000 per child per year in the United States (4).

The rapid rise in the number of individuals diagnosed with ASD has strained the limited behavioral therapy resources, and many children face considerable delays in obtaining these much-needed interventions (5). These interventions are crucial given that there are no medical treatments to improve the deficits seen in ASD. ASD is principally characterized by impairments in social communication, and the presence of repetitive and restrictive behaviors (6). Evidence-based behavioral therapies have sought to address these deficits by focusing on improving communication, social interactions, and reducing challenging behaviors (7). Vital socially salient information is transmitted through facial emotions, expressions, eye gaze, and other nuanced social cues. Unfortunately, many people with ASD have been found to demonstrate deficits in facial processing, potentially accounting for some of their difficulties in social communication (8).

The high demand for therapeutic autism services combined with recent technological advances have led to a rapid increase in attempts to deliver such interventions through digital modalities (9). Compared to traditional behavioral therapies, evidence for these digital interventions are limited, but there are some research studies to suggest that digital interventions may improve specific skills and behaviors in people with ASD (Table 1).

TABLE 1

Table 1. Evidence for digital interventions in people with autism spectrum disorder.

Furthermore, children with ASD may be especially receptive to such digital interventions (17, 18). Children with ASD have been shown to have a preference for electronic media (19), game-like elements (20), computer-generated speech (21), and those with particular visual strengths may be especially adept at engaging with digital modalities (22). Current diagnostic and therapeutic approaches seldom rely on quantitative data measures of symptomatology, severity, or condition trajectory, and it is hoped that digital tools may help to provide such data. Emerging technology and mobile sensor-rich devices may help to deepen and refine new paradigms for understanding social communication, such as those outlined within the social processes construct of the National Institute of Mental Health’s Research Domain Criteria (RDoC) (23) and to undertake digital phenotyping and subtyping of behavioral conditions (24).

Some of the most promising technologies include augmented reality, and some reports have highlighted the early use of augmented reality interventions in children with ASD (25–27). The Brain Power System (BPS) is a smartglasses-based augmented reality system for children and adults with ASD to teach themselves life skills that may facilitate or enhance self-sufficiency. The BPS consists of a number of gamified applications; the social communication module includes applications designed to teach users to recognize facial emotional expressions and to attend to the faces of others. The BPS collects quantitative data about the user’s environment and interactions through the use of an array of inbuilt sensors and analyzes these data through the use of artificial intelligence (AI), including the use of Affdex emotion AI (Affectiva, Boston, MA, USA) (28). This report will explore the feasibility and usability of the BPS through caregiver interview, staff observation, and structured validation with autism rating scales.

The BPS was developed in response to several areas of need highlighted by people with ASD and their families. One of the most frequently encountered concerns by parents was a feeling of “disconnection” from their children during conversations. This sense of disconnection is likely related to several previously outlined social communication deficits in ASD, including deficits in social-emotional reciprocity, non-verbal communicative behaviors used for social interaction, and deficits in developing, maintaining, and understanding relationships (6).

Methods

In this report, the feasibility of the BPS was assessed when it was used to provide a single session of a behavioral coaching intervention to two males with ASD. The tolerability, usability, and behavioral changes associated with the intervention were assessed via a validated autism inventory, and subjective caregiver and user report. The aberrant behavior checklist (ABC) was the validated autism inventory used in this study (29).

Users

The BPS system was trialed on two male users aged 8 and 9 years of age who had a specialist-derived clinical diagnosis of ASD according to the DSM-5 criteria (6). Both users screened positive for ASD when assessed with the Social Communication Questionnaire (30), a validated caregiver questionnaire of ASD symptoms that is frequently used as a screener for entrance to research studies (31). User demographics are laid out in Table 2. Each participant was accompanied by their caregiver to the intervention session.

TABLE 2

Table 2. Demographics of users.

Brain Power System

The BPS is a smartglasses-based behavioral aid designed to help children and adults with ASD with emotional understanding, face directed-gaze, eye contact, and self-control. The BPS is a combination of hardware and software add-ons that may be integrated onto a variety of smartglasses platforms. The physical attributes of the BPS in this report are shown in Figure 1. The BPS includes a family of gamified applications and has onboard sensors that it uses to capture real-time data across several different categories, such as movement, physiology, in-app performance, video, and audio. Software analysis of data from the smartglasses’ inbuilt gyroscope and accelerometer allows for calculation and real-time tracking of a user’s head direction and movements. The BPS can also give real-time visual and auditory feedback to users via a small computer screen above the right eye and a bone conduction speaker behind the right ear.

FIGURE 1

Figure 1. Brain Power System (partial side view).

Brain Power System applications have been designed with the goal of enhancing social and cognitive skills development in children and adults with ASD. Face Game and Emotion Game are two gamified BPS applications and are designed to help children improve their ability to engage in face and eye directed gaze and to recognize facial emotion, respectively.

Face Game and Emotion Game Apps

Faces are a rich source of socially salient information and humans are drawn to attend to faces, and to gaze at facial features in a stereotypical manner (32). People with ASD have been found to have reduced attention to the faces of others, and compared to their typically developing peers, may demonstrate altered facial feature gaze patterns (33, 34). Two key theories have been put forward to help explain the diminished eye contact in autism, gaze aversion, and gaze indifference. Gaze aversion suggests that the individual with ASD actively avoids eye contact as it is threatening, anxiety-inducing, or linked to physiologic overstimulation (35–37). Gaze indifference, on the other hand, supports the notion that reduced eye contact in ASD may be a passive phenomenon, where the eyes of others are not seen as being engaging or a relevant stimulus (38). Certainly, given the heterogeneity seen in ASD, different theories may hold true for different individuals.

Face Game aims to coach users to attend visually to another person’s face while attempting to overcome the suspected challenges identified by both gaze indifference and gaze aversion. Face Game uses gamified augmented reality to increase user engagement and interest in human faces, addressing an underlying challenge in gaze indifference. Face Game is also designed to help address some of the proposed challenges described in gaze aversion theory. Face Game is designed to allow users to have a comfortable starting point, using videogame engagement principles, but proceeding to gradually help users to engage and interact with human faces in the real-world. Face Game provides for multiple different levels, and as the user progresses, the digital elements gradually become more subtle, while real-world interactions are emphasized.

Face Game applies computer vision algorithms to a real-time camera feed and detects human faces anywhere in the user’s field of view. These detected human faces are then overlaid with an augmented reality cartoon face in a manner that attempts to engage the user and attract his or her attention. When the user turns to look directly at the augmented reality cartoon, it gradually fades, revealing the underlying human face. At this time, Face Game rewards the user with “points.” The points rapidly tally for a period while the user’s gaze is directed at their partner’s face. Point accumulation eventually slows and then stops increasing to prevent rewarding and, therefore, promoting staring. The user can then repeat the process of looking away and looking back at the face in order to accumulate more points. The central region of the face—for instance, the eye regions—contains the most socially salient information. Face Game provides a higher point reward when users look closer to the more socially salient central region of the face. Closed loop feedback is possible through the collection of in-app metrics and quantitative sensor-data, allowing for customization of game difficulty and rewards.

During typical early childhood development, infants rapidly become able to recognize expressed facial emotions. Facial emotions carry with them important social information about a person’s affective state, intentions, and surroundings (39). New technology may be particularly useful in assessing and measuring facial expressions in a manner that may improve the social communication constructs of RDoCs (23).

Facial emotion recognition appears to be impaired in ASD (40) and reduced emotional understanding is linked to reduced real-world social behavior and adaptive functioning (41). Emotion recognition training may be effective in ASD, however, generalizability to real-world situations is unclear (40). Emotion Game was developed as a means of helping users assess and identify different human facial emotions. The aim of Emotion Game is to help augment emotion recognition training provided by therapists. Like Face Game, Emotion Game also detects human faces, and through software assessment, can accurately identify a range of human facial emotional expressions. Emotion Game uses a visual display to prompt the user to correctly identify the facial emotion in the detected human face by presenting them with two emoticon choices. Users are instructed to choose the correct emoticon with a slight tilt of their head. The BPS detects the head movement and awards points depending on the result of the selection. The experience can also feature deeper gamified elements, such as customized on-screen rewards, and adapts to individual performance based on data-driven closed-loop feedback.

Experimental Design

Two boys with ASD had a single BPS coaching session with two BPS applications, Face Game and Emotion Game. This report is part of a larger ongoing IRB-approved clinical trial program.

The coaching session consisted of an initial orientation period where the users and caregivers became familiar with the system had the opportunity to wear the device. Once they were comfortable with the device, the user and caregiver had discrete periods of time to learn to play Face Game and Emotion Game. The participants, caregivers, and study staff had the opportunity to pause or stop the coaching session at any time for any reason including bathroom breaks, food or water breaks, changes in behavior, and lack of tolerability of device. During the intervention, there was additional external audio and video monitoring in place.

Outcome measures consisted of the ABC and a caregiver and user post-intervention interview. The ABC is an empirically developed scale designed to measure behavioral symptoms in individuals with developmental disorders across five domains: irritability/agitation; lethargy/social withdrawal; stereotypic behavior; hyperactivity/non-compliance; and inappropriate speech (29). The ABC was conducted pre-intervention to obtain a baseline measure, and also at 24 h post-intervention. The post-intervention ABC assessed the entire 24 h immediately following intervention. The total ABC score and subscale scores were calculated. Immediately following the intervention, every caregiver had a post-intervention interview. The caregiver interview assessed how the user had interacted with the BPS, exploring feasibility, tolerability, and usability. The caregivers were also asked to identify changes in the user’s social communication and behavior, including questions about perceived changes in emotional connection, behavioral issues, and verbal and non-verbal communication.

Ethical Considerations

This report was conducted as part of the “Brain Empowerment—Youth Oriented Unconstrained Reality for Social Engagement and Lifeskills Formation Study” approved by Asentral, Inc., Institutional Review Board (Newburyport, MA, USA). The children’s participation in this study was discussed with their legal guardian and informed consent was obtained. The guardians were informed that they could withdraw consent at any time and for any reason.

Results

Both of the users and their respective caregivers completed the intervention session without any adverse effects noted by the study staff or reported by the participant and caregiver.

Caregiver Report

Caregivers had a post-intervention structured interview to assess the feasibility and functionality of the BPS. The BPS was felt to have both high tolerability and engagement. Caregivers also felt that both males found the system enjoyable and fun to use (Table 3). The caregivers were also interviewed about their perception of changes in the user’s communication, interaction, and behavioral control during the intervention (Table 4). Additionally, caregivers were asked to rate their own stress level. Both caregivers reported users had improved non-verbal communication, eye contact, and social engagement. Both caregivers felt that verbal communication was unaffected. One caregiver reported decreased emotional connection and behavioral control, while the other noted improvements in both of these areas. One caregiver felt their own stress levels were greatly improved.

TABLE 3

Table 3. Caregiver report on user interaction with Brain Power System.

TABLE 4

Table 4. Caregiver perceptions of user and caregiver emotional and behavioral change.

Aberrant Behavior Checklist

The pre-intervention ABCs for both User A and User B demonstrate considerable symptom burden across all subscales, with a total baseline symptom score of 60 and 53, respectively (maximum score 174) (Table 5). The post-intervention ABC, which covered the 24 h post-intervention, demonstrated improved symptoms across all subscales in both users. Additionally, both users had a zero post-intervention ABC score in the lethargy/social withdrawal subscale, and User A had a zero score in the inappropriate speech and stereotypic behavior subscales. Both users were noted to have had a substantial decrease in the hyperactivity/non-compliance subscale, the largest subscale contributor to their baseline ABC total score. The pre-intervention and post-intervention ABC-subscale scores for User A and User B are visualized in Figures 2 and 3, respectively.

TABLE 5

Table 5. Aberrant behavior checklist (ABC)-subscale score pre- and post-intervention.

FIGURE 2

Figure 2. User A: pre- and post-intervention aberrant behavior checklist (ABC) subscale scores.

FIGURE 3

Figure 3. User B: pre- and post-intervention aberrant behavior checklist (ABC) subscale scores.

Discussion

Augmented reality interventions provide an opportunity to enhance social communication in ASD, and our report demonstrates preliminary evidence of usability and feasibility of one combination of smartglasses and augmented reality technology. Both users managed to use the device without any observable adverse effects, and the BPS was found to be well tolerated, easy to use, and highly engaging according to the caregivers. Caregivers also noted that the users had improved social interactions, through improvements in non-verbal communication, social engagement, and eye contact while using the BPS. It was reassuring to see that ABC scores 24 h post-intervention did not get worse, when compared to the pre-intervention baseline. In fact, it was noted that both children had considerable reductions in symptoms across all five ABC subscales, including irritability/agitation, lethargy/social withdrawal, stereotypic behavior, hyperactivity/non-compliance, and inappropriate speech. While these findings highlight promising feasibility, and usability, and may also suggest some improvement to subjective and objective behavior, there are a number of notable limitations. First, the intervention was tested in two young males, and our findings cannot be expected to generalize to the broader ASD population, which is highly demographically and clinically heterogeneous. Additionally, in this report, the primary aim was to assess feasibility through the use of a single intervention session. To further understand the improvements in ASD symptoms as measured by the ABC, larger studies will be required. Our ongoing research efforts are investigating the outcomes of repeated coaching sessions, in addition to longitudinal monitoring of the ABC to see if there are sustained changes to the ABC based on the use of these interventions. We will also need explore how females with ASD may respond to the BPS, as sex and gender may influence symptoms presentations, and we should be mindful of the range of sexually dimorphic differences in social communication, memory, and cognitive flexibility (42).

When using technologies like the BPS, a vast amount of user data is collected through the inbuilt smartglasses sensors, and analyzed through the applications. These quantitative datasets, encompassing visual, audio, physiologic, and movement data around social interactions, may allow for deeper insights into baseline social communication deficits, and improvements in interventions. Through the use of machine learning we may be able to adopt a data-driven approach to identifying optimal therapeutic and educational approaches for each individual user.

Smartglasses may offer a number of distinct advantages compared to applications delivered via smartphones. With smartglasses, users are heads-up as opposed to immersed in a screen, and they remain hands-free, thus able to use their hands to engage in both non-verbal social communication and undertake educational/occupational tasks. This type of mobile and lightweight technology allows users to coach themselves in the privacy of their own home, and whenever is most convenient. Finally, such technology can be rapidly scaled to meet demands.

Conclusion

The ASD community has considerable difficulty accessing effective and timely therapeutic interventions. This situation may potentially be addressed through the use of digital interventions, such as augmented reality. Augmented reality may be especially effective as it may deliver visual and auditory cues while the user is simultaneously engaged in natural or structured social interactions. Additionally, wearable technologies contain sensors that can be used to record and quantitatively monitor a user’s interaction. This report highlights the need for further research into the use of augmented reality technologies as a therapeutic tool for people with ASD. To the authors’ knowledge, this is the first published report of the use of augmented reality smartglasses, such as Google Glass, as a behavioral aid in a pediatric population. In the two reported cases, augmented reality smartglasses demonstrated high feasibility, usability, and tolerability. Overall, the results are encouraging but should be considered in the context of the limitations outlined.

Ethics Statement

This report includes data from human participants who used the Brain Power System. The use of the Brain Power System by children and adults with autism is under AIRB 2015-405A; “Brain Empowerment – Youth Oriented Unconstrained Reality for Social Engagement and Lifeskills Formation,” IRB-approved by Asentral INC IRB, who is affiliated with the Commonwealth of Massachusetts’ Department of Public Health.

Author Contributions

NS is the inventor of the Brain Power System. JS, AV, and NS helped to design and create the intervention. AV was the lead on the writing of this technology report, with additional review and contributions by JS, RL, and NS.

Conflict of Interest Statement

This report was supported by Brain Power, a neurotechnology company developing a range of augmented reality technologies for smartglasses.

References

1. Baio J; Developmental Disabilities Monitoring Network Surveillance Year 2010 Principal Investigators; Centers for Disease Control and Prevention (CDC). Prevalence of autism spectrum disorder among children aged 8 years: Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2010. MMWR Surveill Summ (2014) 63(2):1–21.

PubMed Abstract | Google Scholar

2. Buescher AV, Cidav Z, Knapp M, Mandell DS. Costs of autism spectrum disorders in the United Kingdom and the United States. JAMA Pediatr (2014) 168(8):721–8. doi:10.1001/jamapediatrics.2014.210

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Lavelle TA, Weinstein MC, Newhouse JP, Munir K, Kuhlthau KA, Prosser LA. Economic burden of childhood autism spectrum disorders. Pediatrics (2014) 133(3):e520–9. doi:10.1542/peds.2013-0763

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Amaral D, Geschwind D, Dawson G, editors. Autism Spectrum Disorders. Oxford: University Press (2011).

Google Scholar

5. Zablotsky B, Pringle BA, Colpe LJ, Kogan MD, Rice C, Blumberg SJ. Service and treatment use among children diagnosed with autism spectrum disorders. J Dev Behav Pediatr (2015) 36(2):98–105. doi:10.1097/DBP.0000000000000127

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Association AP. Diagnostic and Statistical Manual of Mental Disorders (DSM-5^®). Arlington: American Psychiatric Association Publishing (2013).

Google Scholar

7. Wong C, Odom SL, Hume KA, Cox AW, Fettig A, Kucharczyk S, et al. Evidence-based practices for children, youth, and young adults with autism spectrum disorder: a comprehensive review. J Autism Dev Disord (2015) 45(7):1951–66. doi:10.1007/s10803-014-2351-z

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Tanaka JW, Sung A. The “eye avoidance” hypothesis of autism face processing. J Autism Dev Disord (2016) 46(5):1538–52. doi:10.1007/s10803-013-1976-7

CrossRef Full Text | Google Scholar

9. Odom SL, Thompson JL, Hedges S, Boyd BA, Dykstra JR, Duda MA, et al. Technology-aided interventions and instruction for adolescents with autism spectrum disorder. J Autism Dev Disord (2015) 45(12):3805–19. doi:10.1007/s10803-014-2320-6

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Ramdoss S, Lang R, Fragale C, Britt C, O’Reilly M, Sigafoos J, et al. Use of computer-based interventions to promote daily living skills in individuals with intellectual disabilities: a systematic review. J Dev Phys Disabil (2012) 24(2):197–215. doi:10.1007/s10882-011-9259-8

CrossRef Full Text | Google Scholar

11. Ramdoss S, Lang R, Mulloy A, Franco J, O’Reilly M, Didden R, et al. Use of computer-based interventions to teach communication skills to children with autism spectrum disorders: a systematic review. J Behav Educ (2011) 20(1):55–76. doi:10.1007/s10864-010-9112-7

CrossRef Full Text | Google Scholar

12. Ramdoss S, Machalicek W, Rispoli M, Mulloy A, Lang R, O’Reilly M. Computer-based interventions to improve social and emotional skills in individuals with autism spectrum disorders: a systematic review. Dev Neurorehabil (2012) 15(2):119–35. doi:10.3109/17518423.2011.651655

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Wainer AL, Ingersoll BR. The use of innovative computer technology for teaching social communication to individuals with autism spectrum disorders. Res Autism Spectr Disord (2011) 5(1):96–107. doi:10.1016/j.rasd.2010.08.002

CrossRef Full Text | Google Scholar

14. Tanaka JW, Wolf JM, Klaiman C, Koenig K, Cockburn J, Herlihy L, et al. Using computerized games to teach face recognition skills to children with autism spectrum disorder: the let’s face it! program. J Child Psychol Psychiatry (2010) 51(8):944–52. doi:10.1111/j.1469-7610.2010.02258.x

CrossRef Full Text | Google Scholar

15. Ayres KM, Maguire A, McClimon D. Acquisition and generalization of chained tasks taught with computer based video instruction to children with autism. Educ Train Dev Disabil (2009) 44(4):493–508.

Google Scholar

16. Burke RV, Andersen MN, Bowen SL, Howard MR, Allen KD. Evaluation of two instruction methods to increase employment options for young adults with autism spectrum disorders. Res Dev Disabil (2010) 31(6):1223–33. doi:10.1016/j.ridd.2010.07.023

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Pennington RC. Computer-assisted instruction for teaching academic skills to students with autism spectrum disorders: a review of literature. Focus Autism Other Dev Disabl (2010) 25(4):239–48. doi:10.1177/1088357610378291

CrossRef Full Text | Google Scholar

18. King AM, Thomeczek M, Voreis G, Scott V. iPad^® use in children and young adults with autism spectrum disorder: an observational study. Child Lang Teach Ther (2014) 30(2):159–73. doi:10.1177/0265659013510922

CrossRef Full Text | Google Scholar

19. Shane HC, Albert PD. Electronic screen media for persons with autism spectrum disorders: results of a survey. J Autism Dev Disord (2008) 38(8):1499–508. doi:10.1007/s10803-007-0527-5

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Mayes SD, Calhoun SL. Analysis of WISC-III, Stanford-Binet: IV, and academic achievement test scores in children with autism. J Autism Dev Disord (2003) 33(3):329–41. doi:10.1023/A:1024462719081

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Schlosser RW, Blischak DM. Is there a role for speech output in interventions for persons with autism? A review. Focus Autism Other Dev Disabl (2001) 16(3):170–8. doi:10.1177/108835760101600305

CrossRef Full Text | Google Scholar

22. Tincani M, Boutot E. Technology and autism: current practices and future directions. In: Edyburn D, Higgins K, Boon R, editors. Handbook of Special Education Technology Research and Practice. Whitefish Bay, WI: Knowledge by Design (2005). p. 413–21.

Google Scholar

23. Health NIoM. Behavioral Assessment Methods for RDoC Constructs. Bethesda, MD: National Institute of Mental Health Services DoHaH (2016).

Google Scholar

24. Insel T, Cuthbert B, Garvey M, Heinssen R, Pine DS, Quinn K, et al. Research domain criteria (RDoC): toward a new classification framework for research on mental disorders. Am J Psychiatry (2010) 167:748–51. doi:10.1176/appi.ajp.2010.09091379

CrossRef Full Text | Google Scholar

25. Escobedo L, Tentori M, Quintana E, Favela J, Garcia-Rosas D. Using augmented reality to help children with autism stay focused. IEEE Pervasive Comput (2014) 13(1):38–46. doi:10.1109/MPRV.2014.19

CrossRef Full Text | Google Scholar

26. da Silva CA, Fernandes AR, Grohmann AP. STAR: speech therapy with augmented reality for children with autism spectrum disorders. In: Cordeiro J, Hammoudi S, Maciaszek L, Camp O, Filipe J, editors. Enterprise Information Systems. ICEIS 2014. Lecture Notes in Business Information Processing, Vol. 227. Cham: Springer (2015).

Google Scholar

27. Escobedo L, Nguyen DH, Boyd L, Hirano S, Rangel A, Garcia-Rosas D, et al., editors. MOSOCO: a mobile assistive tool to support children with autism practicing social skills in real-life situations. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM (2012).

Google Scholar

28. McDuff D, Mahmoud A, Mavadati M, Amr M, Turcot J, Kaliouby Re, editors. AFFDEX SDK: a cross-platform real-time multi-face expression recognition toolkit. Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM (2016).

Google Scholar

29. Aman MG, Singh NN, Stewart AW, Field CJ. The aberrant behavior checklist: a behavior rating scale for the assessment of treatment effects. Am J Ment Defic (1985) 89(5):485–91.

PubMed Abstract | Google Scholar

30. Rutter M, Bailey A, Lord C. The Social Communication Questionnaire: Manual. Torrance: Western Psychological Services (2003).

Google Scholar

31. Chandler S, Charman T, Baird G, Simonoff E, Loucas T, Meldrum D, et al. Validation of the social communication questionnaire in a population cohort of children with autism spectrum disorders. J Am Acad Child Adolesc Psychiatry (2007) 46(10):1324–32. doi:10.1097/chi.0b013e31812f7d8d

CrossRef Full Text | Google Scholar

32. de Haas B, Schwarzkopf DS, Alvarez I, Lawson RP, Henriksson L, Kriegeskorte N, et al. Perception and processing of faces in the human brain is tuned to typical feature locations. J Neurosci (2016) 36(36):9289–302. doi:10.1523/JNEUROSCI.4131-14.2016

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Riby DM, Hancock PJ. Viewing it differently: social scene perception in Williams syndrome and autism. Neuropsychologia (2008) 46(11):2855–60. doi:10.1016/j.neuropsychologia.2008.05.003

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Klin A, Jones W, Schultz R, Volkmar F, Cohen D. Visual fixation patterns during viewing of naturalistic social situations as predictors of social competence in individuals with autism. Arch Gen Psychiatry (2002) 59(9):809–16. doi:10.1001/archpsyc.59.9.809

CrossRef Full Text | Google Scholar

35. Corden B, Chilvers R, Skuse D. Avoidance of emotionally arousing stimuli predicts social-perceptual impairment in Asperger’s syndrome. Neuropsychologia (2008) 46(1):137–47. doi:10.1016/j.neuropsychologia.2007.08.005

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Tottenham N, Hertzig ME, Gillespie-Lynch K, Gilhooly T, Millner AJ, Casey BJ. Elevated amygdala response to faces and gaze aversion in autism spectrum disorder. Soc Cogn Affect Neurosci (2014) 9(1):106–17. doi:10.1093/scan/nst050

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Kaartinen M, Puura K, Makela T, Rannisto M, Lemponen R, Helminen M, et al. Autonomic arousal to direct gaze correlates with social impairments among children with ASD. J Autism Dev Disord (2012) 42(9):1917–27. doi:10.1007/s10803-011-1435-2

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Moriuchi JM, Klin A, Jones W. Mechanisms of diminished attention to eyes in autism. Am J Psychiatry (2017) 174(1):26–35. doi:10.1176/appi.ajp.2016.15091222

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Ekman P, Rolls E, Perrett D, Ellis H. Facial expressions of emotion: an old controversy and new findings [and discussion]. Philos Trans R Soc Lond B Biol Sci (1992) 335(1273):63–9. doi:10.1098/rstb.1992.0008

CrossRef Full Text | Google Scholar

40. Berggren S, Fletcher-Watson S, Milenkovic N, Marschik PB, Bolte S, Jonsson U. Emotion recognition training in autism spectrum disorder: a systematic review of challenges related to generalizability. Dev Neurorehabil (2017) 10:1–14. doi:10.1080/17518423.2017.1305004

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Trevisan DA, Birmingham E. Are emotion recognition abilities related to everyday social functioning in ASD? A meta-analysis. Res Autism Spectr Disord (2016) 32:24–42. doi:10.1016/j.rasd.2016.08.004

CrossRef Full Text | Google Scholar

42. Halladay AK, Bishop S, Constantino JN, Daniels AM, Koenig K, Palmer K, et al. Sex and gender differences in autism spectrum disorder: summarizing evidence gaps and identifying emerging areas of priority. Mol Autism (2015) 6(1):36. doi:10.1186/s13229-015-0019-y

PubMed Abstract | CrossRef Full Text | Google Scholar