Amy Jane Griffiths
John Brady
Nicholas Riley
James Alsip- Attallah College of Educational Studies, Chapman University, Orange, CA, United States
Some students with autism spectrum disorder and other learning differences may have superior visual acuity, increased attentional focus, and logical thinking abilities, lending to an affinity for science, technology, engineering, and mathematics (STEM) fields. At the same time, economists report that, the United States will experience a 28.2% increase in STEM-related jobs between 2014 and 2024. Although students with disabilities (SWD) can help to fill those positions, 85% of SWD graduates are either underemployed or unemployed as they enter young adulthood. Thus, there is a need to develop, evaluate, and report outcomes of STEM preparation programs specifically tailored to SWD. This mixed-methods study was designed to develop an evaluation procedure to measure a STEM school’s program for SWD and to analyze the first two years of data to help shape the evaluation process. A comprehensive evaluation model of STEM education for children with learning differences was developed and tested. Implications for practice and future research are discussed.
Introduction
In the United States, more than 156 million jobs are available, with projections of 0.7% annual increases in available jobs over the next 10 years (U.S. Department of Labor, Bureau of Labor Statistics, 2017a). Further, 15% of the U.S. workforce is in computer, engineering, and science careers; these and other STEM-related fields comprise the top 30 occupations expected to grow the fastest by 2026 (U.S. Department of Labor, Bureau of Labor Statistics, 2017b; U.S. Department of Labor, Bureau of Labor Statistics, 2018). This progressive increase in job opportunities, coupled with the evolution of technology, is creating higher demands for diversity of thought, experience, perspective, and background in the labor market.
When considering diversity of thought, researchers have attempted to identify potential skill sets or common characteristics of individuals with autism spectrum disorder (ASD) and related learning differences such as specific learning disabilities, mental health disorders impacting learning (i.e., emotional disturbance), and other health impairment (e.g., attention deficit hyperactivity disorder). To better understand how a STEM-based educational model may influence and support the employment of youth with identified disabilities in STEM fields, investigators sought to develop an evaluation procedure to measure the impact of such a model on STEM-related outcomes.
Disability and Employment
The Individuals with Disabilities Education Act (IDEA) (P. L. 101–476) outlines specific categories of disabilities under which children may be eligible for special education and related services in the school setting. As defined by IDEA, a child with a disability may qualify under the following categories: intellectual disability, hearing impairments, speech or language impairments, visual impairments, emotional disturbance, orthopedic impairments, autism, traumatic brain injury, other health impairments, or specific learning disabilities; and due to this disability the student requires special education and related services to access their education.
In this study, particular attention is paid to ASD (as defined by IDEA) as the school program was specifically designed for the skills and deficits associated with the diagnosis, there is an increased need for attention regarding this population, and the majority of the student population had an ASD or related diagnosis identified in the school setting. Autism is formally categorized as a neurodevelopmental disability under Autism Spectrum Disorders (ASD) in the most recent edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM) (American Psychiatric Association, 2013). The challenges that individuals with autism experience are well documented. They include challenges with social skills, both verbal and nonverbal communication struggles, restrictive interests, repetitive behaviors, the need for sameness and routine, and difficulty understanding others’ intentions or emotions. People with autism may demonstrate rigid thinking and process information in exact, less flexible terms. The degree to which these characteristics are present varies by individual and can manifest in a range of ways (Masi et al., 2017).
With recent studies demonstrating an increase in the classification of autism, it is estimated that 50,000 young adults with autism will soon be transitioning out of K-12 school services and into adult life (Baio et al., 2018). Over the next 10 years, close to half-million people with autism will reach adulthood; this generational cohort is referred to as Generation A (Hurley-Hanson et al., 2020). The estimated cost of supporting and servicing this population has exceeded $268 billion in the US and is expected to reach $461 billion by 2025 (Leigh and Du, 2015). Parents of children with autism report high expenses and costs of care that only increase when students reach adulthood and are faced with unemployment. Despite individuals with autism possessing the capacity and desire to pursue employment, approximately 80% are unemployed (Hendricks, 2010; National Autistic Society, 2019). As a result, adults with autism often report lower levels of life quality, financial outcomes, and daytime activities (Taylor and Seltzer, 2011). Employment is recognized as a crucial milestone in adulthood that is critical to enhancing one’s self-concept, persistence, and adaptability (Zikic and Hall, 2011); additionally, it provides access to life essentials and socialization.
Education and Employment in STEM
Projected growth for STEM related jobs between 2014 and 2024 is 28.2%, compared to 6.5% for all occupations (Fayer et al., 2017). Although students with disabilities (SWD) can help to fill these positions, 85% of SWD graduates are either underemployed or unemployed as they enter young adulthood. This disconnection between ability and success results in lost opportunities for these individuals, who, notably, are an untapped resource for the nation. We need a process to connect the skills of these individuals with future job opportunities. A clear path forward is through STEM education with a particular focus on enhancing and measuring the learning of students with disabilities.
According to the President’s Council of Advisors on Science and Technology (PCAST, 2010), STEM education is a major factor in solving global challenges in the areas of energy, health, environmental protection, and national security. STEM education allows for the development of a more capable and flexible workforce that is able to compete in a global market. In its 2010 executive report, PCAST called for the creation of at least a thousand STEM-focused high schools, middle schools, and elementary schools over the next decade, particularly in historically underserved and minority communities.
Although only 5% of jobs required specialized knowledge at the beginning of the 20th century, over 70% did so by 2009 (Mansilla and Jackson, 2011). Further, due to the rapidly developing global economy and the resulting changes in the workplace, the competitiveness of the United States depends on the country’s cultivation of citizens who enter the labor force with well-developed STEM abilities and skills (Alper, 2016). In the development of a highly skilled and adaptable workforce, STEM academies or high schools provide extended time for students to further their expertize. More than ever, there is a need to develop students’ “21st-Century skill or competencies” such as communication, collaboration, innovative thinking, and creativity (Larson and Miller, 2011), and the educational experience needs to be adapted accordingly.
21st Century Competencies in STEM Education and Employment
The term 21st Century skills or competencies has various meanings to different professionals, and descriptions of these skills are not consistent across the literature. There are also significant challenges in assessing outcomes related to these skills, as they are not easily measured using traditional academic assessments. Soland et al. (2013) identify and categorize a broad range of competencies and related assessment approaches that illustrate vital skills required now and in the future. Soland et al. (2013) outline broad categories defined by the National Research Council and related organizations with expertize in these competencies. The broad categories include: (a) Cognitive competencies, (b) Interpersonal competencies, and (c) Intrapersonal competencies. The cognitive category typically includes mastery of critical academic content, across various disciplines and may also include critical thinking and creativity. The interpersonal category includes the skills that students need in order to connect to and relate to other people. These competencies require a foundation of basic communication and collaboration. The intrapersonal category includes the competencies that involve the individual’s attitudes and behaviors that influences their ability to problem solve in everyday life across settings (e.g., school, work, community).
Individuals With Autism and Related Disorders Pursuing STEM
Little documentation exists regarding the unique abilities and perspectives that employees with ASD can offer a marketplace in need of skilled workers. However, traits commonly seen as strengths in individuals with autism are related to visual acuity, more deliberative decision making, increased attentional focus, logical thinking, an affinity for technology, and professional and occupational interests in STEM fields (Crespi, 2016). STEM fields, in particular, present a growing need for a skilled workforce in these areas, potentially giving individuals with autism an ideal fit for this sector of the labor market (U.S. Department of Labor, 2020a). However, despite these unique skills, effectively transitioning into the workforce can be challenging and often results in unfavorable outcomes.
Individuals with disabilities are marginalized in the workforce and only 21% report being gainfully employed (U.S. Bureau of Labor Statistics, 2018). Individuals with disabilities have comprised as little as 7% of the workforce in select STEM disciplines and face unprecedented barriers, along with women and minority ethnic and racial groups (National Science Foundation, 2017). Additionally, teachers of STEM topics often struggle to disseminate and facilitate instruction in an inclusive manner for individuals with disabilities (Lee, 2011). Despite emerging practices encouraging the development of STEM skills for individuals with disabilities, their journey remains long and arduous (Bellman et al., 2018; Schreffler et al., 2019).
No exception is provided to individuals with autism in their career pathways as their distinct needs and skills are often viewed as shortcomings. This is evident in a national sample representing individuals with autism in special education that found only 32% of individuals with autism were enrolled in 2- or 4-year colleges (Wei et al., 2013). Conclusions from this study found that among 11 disability categories, young adults with autism were least likely to enroll in college, the only exception being individuals with intellectual disabilities. Despite clear disproportionality in college settings, it is recognized that a relatively higher percentage of individuals with autism elect to pursue select STEM majors compared to other disability classifications and the general population (Wei et al., 2017).
Unfortunately, even once a job is obtained, maintaining employment is challenging for individuals with autism. Difficulties with interpersonal skills and interpreting workplace dynamics may lead to increased stress or conflicts (Richards, 2012; Hayward et al., 2018). Research has shown that individuals, a majority being males, who had fewer traits of autism, less maladaptive behaviors, and were not diagnosed with comorbid intellectual or cognitive deficits, were more prone to positive vocational outcomes (Eaves and Ho, 2008; Taylor and Seltzer, 2011). Due to these concerns, we are compelled to begin understanding school-based STEM further encourage positive outcomes for individuals with disabilities, particularly ASD and related disorders, pursuing STEM.
A Theoretical Framework for STEM Education and Students with Disabilities
A large body of research has indicated that teachers struggle to facilitate inclusive STEM classrooms and may require additional training and skills to increase access and learning for students with disabilities (Bargerhuff et al., 2010; Lee, 2011; Rule et al., 2009). This population of students is underrepresented in traditional designs for instruction.
As students with disabilities would significantly benefit from entering the STEM fields, have unique skills to offer, and tend to struggle more than their peers in STEM disciplines various interventions and instructional approaches have been developed and evaluated to improve outcomes in each of the STEM disciplines (Hwang and Taylor, 2016).
In recent years, researchers and educators have focused on understanding and designing STEM instruction around the needs of students with disabilities. Basham and Marino (2013) identify Universal Design for Learning (UDL) as an appropriate framework to implement STEM education for students with disabilities. Originally, universal design was developed in the field of architecture to encourage product designs to support an environment more accessible to all people. Since, UD has grown to be present in a variety of disciplines in education, including instruction and learning (Schreffler et al., 2019). Universal design promotes consideration being given to an individual’s ability or skill level, learning style or preference, age, gender, sexual orientation, culture, and disabilities (Burgstahler, 2017). Universal design for learning includes 3 critical factors include providing multiple means of (1) engagement, (2) representation, and (3) action and expression (Center for Applied Special Technology, 2018). These principals represent the why, what, and how of learning, respectively, and are intended to support learners that are purposeful and motivated; resourceful and knowledgeable; as well as strategic and goal-directed (CAST, n.d.b). General guidelines for integrating UDL into teaching and learning have been developed by the Center for Applied Special Technology (CAST). These aspects of learning are essential assets for increasing access of STEM curriculum for students with disabilities (Schreffler et al., 2019).
STEM education can be viewed as an interdisciplinary approach which requires active collaboration across disciplines inside and outside of STEM subject areas. For example, researchers have suggested embedding content literacy and reading into STEM instruction (Israel et al. (2013); Kennedy and Wexler (2013). Other suggest that teaching mathematics and sciences are complementary and a co-teaching method across subjects improves STEM instruction outcomes (Moorehead and Grillo, 2013). Hwang and Taylor (2016) suggest that by integrating the arts in STEM education, students with disabilities are more likely to be successful. Their STEM education framework focuses on teaching problem-solving skills within science contexts; generalizing problem-solving skills when engaging in hands-on classroom activities, including art and music components into a lesson to increase student motivation, promoting varied use of technology when necessary, and maintaining a focus on real world connections.
STEM Programs for Students with Disabilities
There are limited studies in regard to SWD and STEM-related programs. One of the few is a qualitative case study at a single STEM high school, whereby teachers reported that significant modification and accommodation of the academic content, cooperative class group work, and overall social expectations and environment of the school presented significant challenges for SWD (Bargerhuff, 2013). Although this study is a step in the right direction, more research is needed. Due to the limited information on STEM programs that are tailored to individuals with disabilities, the focus of this study is on developing an evaluation process of student outcomes (e.g., skill acquisition, participant experiences, post-graduation success) related to tailored STEM programming. In addition, early outcome data will be used to adjust the originally designed evaluation procedure.
The purpose of the STEM academy that is the focus of this study is to cultivate the particular learning differences of SWD in STEM and to prepare students to succeed in an evolving job market. The curriculum infuses social skills with 21st-century skills (e.g., collaboration, communication, critical thinking, creativity, innovation, transfer of knowledge, global awareness, peer support) into a project-based STEM program. The inclusion of these factors are integrated throughout the day across disciplines, including those outside of what is considered to be STEM fields. Students schedules are developed based on core academic requirements for graduation and college entrance, however, the courses are taught in a manner that is conducive for interdisciplinary STEM learning. In addition, elective classes and extracurricular activities related to STEM fields are encouraged and made available. In both formal and informal learning environments, skills training and “in-the-moment feedback on the core competencies are provided throughout the day. A key tenant of the common core method, project-based learning is an approach to instruction and student work that focuses on student-led investigation into complex problems. The result is publicly shared, student-created projects (Edmunds et al., 2017). Project-based learning evaluations have been used to measure whether students are successfully learning and implementing 21st-century skills and have been found to provide a more holistic picture of student skills than do standardized evaluations alone (Meyer and Wurdinger, 2016).Similar to the strategies posed by Hwang and Taylor (2016), the problem-solving process used within the project-based learning approaches provides a practical foundation for teachers to teach STEM in an integrative manner. The ultimate goal is for students with disabilities to have both the content knowledge and the practical skills to solve complex problems in the real world.
In this study, we describe and analyze an evaluation model for a school dedicated to STEM education for students with varied learning needs. With a place for SWD in the workforce, we need to draw on the 25% of college SWD who have enrolled in undergraduate STEM fields (U.S. Department of Education, 2012). To better understand how well the application of a new educational model is working, we designed this mixed-methods study to with two core research goals (a) to develop an evaluation procedure to measure the key elements of a STEM school’s current program, including 21st Century skill acquisition; post-graduation outcomes, and student/parent/teacher experiences, and to (b) use the first two years of outcome data to enhance the evaluation process.
Materials and Methods
Setting
The academy described in this study is located on the west coast of the United States and was created from a pilot project that focused on teaching STEM skills to students with disabilities by a nonprofit organization in 2013. The school is considered a non-public school and is generally funded by local school districts who send their students who have not made academic progress in the district schools. Students may attend if their parents privately pay for the services, however, this is relatively uncommon (<1% of those enrolled). The program combines 21st-century skills and a project-based learning approach within a STEM curriculum for students with high-functioning ASD, ADHD, and other social and learning differences. “High Functioning” Autism Spectrum Disorder (HFASD). is sometimes used to provide a brief descriptor of a particular group of individuals with ASD (i.e., those with average to above-average cognitive skills). However, the term HFASD can be problematic, as the term suggests those with average or above cognitive skills perform well in other functional areas, while the evidence indicates this is a poor predictor of functional skills (Alvares, et al., 2020) To ameliorate some of these concerns, while still maintaining a “short descriptor”, school personnel use example behaviors based on the Diagnostic and Statistical Manual of mental disorders, 5th edition to determine inclusion. Specifically, for the purposes of this study, a person with HFASD is defined as someone who has identified themselves as having Autism, Asperger’s, or HFASD has approximately average intellectual ability (when compared to peers) but may have marked difficulties in social interactions, including communication. The student may have difficulty initiating or responding to social interactions, or may not seem interested in interacting with peers or teachers. The person may be able to have a conversation when necessary, but may have difficulty keeping the conversation going or knowing how and when to end the conversation. The person may require a structured environment and/or schedule and may not deal well with change. The person may have difficulty with organization and planning.
The academy accepts students from kindergarten through 12th grade (K-12) who have social and learning differences and have achieved a grade of B or better in STEM-related classes or have demonstrated significant skills in a STEM related area. Students accepted must have a positive record review and show potential to benefit from an experiential, hands-on, problem-solving approach to learning. The stated mission of the academy is to connect the particular strengths of students accepted into the program with an innovative and rigorous STEM curriculum that prepares them for future success. As the high school was the most developed of the schools in terms of curriculum and enrollment, this project will focus on the high school.
To create a learning environment that supports students STEM skill development the academy emphasizes several 21st-century skills, including critical thinking, creativity, innovation, transfer of knowledge, global awareness, collaboration, and communication (Larson and Miller, 2011). These skills are taught through project-based instruction in classes such as Algebra, Geometry, Art, U.S. History, English, Biology, and Spanish. Each academic subject is structured around one large project per academic quarter, for which each student is evaluated on his or her demonstration of a given 21st-century skill.
Further, the academy utilizes measures of social-emotional functioning and well-being to evaluate student development in the areas of optimism, persistence, self-control, empathy, and creativity (Furlong et al., 2014). The faculty incorporate these skills in the classroom through project-based learning, feedback, and modeling in addition to academic instruction.
Procedures
This study used a mixed-methods design to allow researchers to consider the complex interactions of variables that exist in program development and to answer comprehensive questions in regard to the initial outcomes of the program. Program evaluation case studies can provide valuable insight regarding the specific program as well as provide an understanding of similar programs (Mills et al., 2010). The present study involved both qualitative and quantitative data to help create an evaluation process that will ultimately lead to an understanding of the successes and drawbacks of the initial implementation of the academy’s unique curriculum.
The program evaluation consisted of four main components: (a) a comprehensive rubric to evaluate the demonstration of 21st-century skill areas; (b) interviews, focus groups, and survey data to assess teacher and student/parent experiences pre- and post-graduation; (c) standardized assessments (Social-Emotional Health Survey [SEHS] and Torrance Tests of Creative Thinking [TTCT]) to determine levels of social-emotional functioning and creativity; and (d) record reviews to further investigate student progress and engagement in the curriculum.
Participants
For the duration of the study, student enrollment fluctuated as students entered and exited throughout the school year. A total of 132 students were included in the present study with the majority (92%) as receiving special education services through an Individualized Education Plan (IEP). The remaining 8% of students were enrolled privately, although they had disabilities their local school district did not agree to fund the school, as they felt they were able to provide adequate services in their schools. Their age ranged from 11- 19 years old, with a mean age of 15. The sample consisted of 73% students who identified as male, 13% who identified as female, and the remaining students identified as “other.” Of the overall student population, 53% qualified for special education under the category of Autism, while 22% had an eligibility of Emotional Disturbance. Other eligibilities included Other Health Impairment (15%) and Specific Learning Disability (2%). The demographic makeup of the student body included students who identified as White (77%), Asian (8%), Hispanic (6%), Black (3%), and two or more races (5%), and each student’s primary language was English.
During the first year of implementation, the academy had eight teachers who taught courses such as Algebra, Geometry, Art, US History, English, Biology, and Spanish. Of the eight teachers, two had previous experience teaching a STEM curriculum, and six had previous experience in working with SWD.
The academy had 11 teachers during the second year of implementation, six of whom had previous experience teaching a STEM curriculum and seven who had previous experience with working with SWD. The demographic composition of teachers included White (46%), Asian (36%), and Hispanic (18%), with the number of years of experience as ranging from 1 year to 25 years, with a mean of 4.7 years.
Overall Framework for Developing Evaluation Procedure
Although there is not one set of agreed-upon competencies that are considered “21st-century skills,” there are many that appear across the extant research. After a comprehensive analysis of the literature, the team developed a list of skills, and then used a consensus process with the staff to decide on which ones the program would be a focus. Using information gathered from scholarly reports, a framework was developed that contained the various competency areas. Competencies were framed within three large categories: cognitive, interpersonal, and intrapersonal, while additional areas were included as subcategories within one of the three main competency areas. Table 1 contains each 21st-century skill area and the tool that was used to measure that specific skill. Due to the inherent overlap of various competency areas and a lack of measurement tools for some areas, a comprehensive rubric was developed, using various 21st-century skills. Additionally, in order to gather a broad understanding of the program impact, we built out an evaluation process that looked at post-graduate outcomes, and student, parent, and teacher experiences.
TABLE 1. 21st-century skill areas.
Cognitive Competencies
Cognitive competencies included critical thinking, creativity, innovation, and transfer of knowledge.
Critical thinking Critical thinking is perhaps the most widely recognized 21st-century skill. Critical thinking includes the ability to use inductive and deductive reasoning, inferencing, and analysis as well knowing the right questions to ask (Soland et al., 2013). Critical thinking is important in understanding academic content and is correlated with career success.
Creativity Although definitions of creativity tend to be culturally specific, it is usually defined by descriptors such as unusualness, adaptability, and helpfulness. Creativity results in products that are both novel and useful (Plucker et al., 2004) and is typically valued in any field that places significance on entrepreneurship.
Innovation Although creativity and innovation are closely related, innovation refers to acting on creative ideas to create something new (Road Map Project, 2014). It is not just a novel idea or new way of approaching a problem but also the execution of a new way of doing things.
Transfer of knowledge Transfer of knowledge refers to the ability to apply previously learned knowledge in a novel setting. The selection of this skill was based on its inclusion in The Road Map Project’s (2014) publication on skill-building strategies that support school success and the skill’s clear application in the academic setting.
Interpersonal Competencies
Interpersonal competencies included global awareness, communication (oral and written), peer support, and teamwork/collaboration.
Global awareness Many researchers maintain that empathy is the key feature of global awareness, in that a student who feels empathy toward individuals of other cultures is considered globally aware (Bachen et al., 2012). Global awareness implies an ability to recognize how events in one part of the world affect those in other parts of the world (Soland et al., 2013) and is considered an important skill in the changing job market.
Oral/written communication Communication is a broad category that overlaps with several other 21st-century skills. Although communication is difficult to separate from skills such as collaboration and leadership, it is typically included as a 21st-century skill due to its importance in the workplace. Communication, along with collaboration and leadership, is vital to facilitate teamwork and conflict resolution (Pellegrino and Hilton, 2012).
Peer support Much of the extant research indicates that peer support may have a large impact on overall student success. According to Brown and Braun (2013), peer culture is one of the most powerful forces that affects the lives of children and was thus included as a skill in the present study.
Teamwork/collaboration Similar to communication, collaboration is often defined as a group of skills, rather than a single skill, that can be difficult to separate. Collaboration is frequently considered a form of communication that includes competencies related to conflict resolution and negotiation (Pellegrino and Hilton, 2012; Lai et al., 2017).
Intrapersonal Competencies
Intrapersonal competencies include empathy, optimism, persistence, and self-control.
Empathy Although empathy is often considered a subskill of global awareness, it also can be included as an individual skill. Empathy involves perspective-taking of individuals in circumstances different than one’s own (Soland et al., 2013). In the classroom, empathy includes caring and compassion for others, a skill that can greatly affect one’s ability to collaborate and problem solve (Brown and Braun, 2013).
Optimism Optimism is broadly defined as a generalized expectancy that one will experience good outcomes in life (Scheier and Carver, 1985). Optimism tends to correlate with overall social-emotional well-being, school connectedness and success, and even physical health (Boman et al., 2009; Kirschman et al., 2009; You et al., 2014).
Persistence Level of persistence is strongly related to a belief in one’s self (Shechtman et al., 2013) and is represented as such in the SEHS. Persistence, for the purposes of this study, concerns academic persistence (e.g., one’s likelihood to persist through a difficult task/ask for teacher clarification).
Self-control Self-control refers to one’s ability to control his or her behavior and regulate his or her own emotions. Effective self-control has been shown to play a role in reducing alcohol and drug dependency, dropout and nonattendance, and other conduct problems (Zins et al., 2007).
Measures
Various measures were used in the current study to assess student progress in the academy, student development of 21st-century skills, student creative potential and social-emotional functioning abilities, and student, parent, and teacher experiences in the program. Measures included (a) a project-based rubric; (b) standardized assessments; (c) record review; and (d) interviews, focus groups, and survey data.
Project-Based Rubric for 21st-Century Skill Areas
To aid in the development of the academy, a project-based rubric was developed for the academy to help teachers to evaluate student demonstrations of 21st-century skills as well as to establish accurate measures of 21st-century skills in students with disabilities at STEM schools. The rubric was developed by investigators and uses a Likert-type scale for which 1 is the lowest possible score and 4 is the highest. Corresponding to each numbered score is a word whose first letter is found in the STEM acronym. Scores descriptors are presented in Table 2. The rubrics were completed by the classroom teacher after each core classroom project. The rubric with skills descriptions and scoring instructions can be accessed by contacting the first author.
TABLE 2. Project based rubric for 21st century skills: score descriptors.
Standardized Assessment
Standardized assessments were used to evaluate students in creativity and social emotional health.
Torrance Tests of Creative Thinking Students were evaluated using the TTCT, which measures the ability to develop and elaborate upon ideas, the number of statistically infrequent ideas, the degree of psychological openness, and humor. An administrator on the leadership team of the school, who was trained to administer this assessment, administered it to the students in small groups. Assessments were sent to the test development company for scoring. The TTCT includes two forms of the test, TTCT-Verbal and TTCT-Figural, that together form a complete battery for measuring creative potential (Kim, 2011).
Although the predictive validity of any measure of creativity or divergent thinking is typically low due to the wide array of domains, the TTCT has one of the largest norming samples and has demonstrated valuable longitudinal data and high predictive validity (Kim, 2006). According to a meta-analysis conducted by Kim (2008), creativity test scores, such as those determined by the TTCT, predict creative achievement better than do IQ test scores. Further, research indicates that the TTCT predicts creative achievement more accurately than do divergent thinking or creative potential measures.
Social Emotional Health Survey The SEHS, a self-report survey based on the co-vitality model (Furlong et al., 2014), was used to assess student social-emotional functioning. Students completed this survey during class time, and was administered by classroom teachers who were provided with an administration script. Assessment data were processed by the survey developer and sent back to the research team. The SEHS has demonstrated strong predictive evidence of student-reported emotional and behavioral problems (You et al., 2014) and their effects on achievement. For the purposes of this evaluation, the survey sought to elicit answers from students regarding their perceptions of peer support, empathy, optimism, persistence, and self-control. The response format for the majority of the questions was based on a Likert-type scale.
Record Review
Attendance records and grade point average (GPA) were used as a measure of student engagement and progress in the program. Deidentified student record data were sent to the research team by a school administrator for analysis and review.
Interviews, Focus Group, and Surveys
Student and teacher interviews, parent surveys, and teacher focus groups were used to assess student and parent overall experiences both pre- and post-graduation. Surveys were sent by school administrators to the parents. The research team, including the principal investigator and two trained graduate research assistants, conducted the interviews and focus group.
Student interviews Brief, semi-structured interviews with randomly selected students were conducted after Years 1 and 2 of implementation of the evaluation process. Questions were designed to elicit insight into student experiences with the academy and included items related to feelings, personal change, and areas for improvement.
Teacher interviews Brief, semi-structured interviews were conducted with randomly selected teachers. Questions were designed to elicit insight into teacher experiences in the STEM program and included items in regard to the culture of the academy among students, teachers, and administrators; the values of STEM education; the drawbacks or issues of STEM education; and the impact of project-based learning for students.
Parent surveys A parent survey was created and administered to parents as a means to gather additional information regarding their child’s background. The survey items related to the reason for enrollment at the academy, previous social skills training programs, previous attendance, extra-curricular involvement, job experience, and goals that the parents have for their child both at the academy and in his or her post-graduate life.
Post-graduation interviews Current and past graduates and, in some cases, their parent or guardian were interviewed by the research team about post-graduation plans and activities and the graduates’ preparedness to take on those plans.
Teacher focus group A focus group that comprised all of the academy teachers was conducted to assess experiences with the current evaluation process and to gather feedback regarding further development of the procedures. Questions included the utility of the rubrics, personal alignment with evaluation procedures, and helpfulness in planning and instruction in 21st-century skill areas for individual students and entire classes. In regard to social-emotional instruction, the focus group included questions regarding the ease of instruction. Finally, teachers were asked to reflect on whether the assessment was accurately measuring what was taught and student outcomes.
Results
The evaluation procedure was develop with the team of researchers and school administrators. It focused on skill development, post-graduation outcomes, as well as the overall experiences of all participants (i.e., parents, students, and teachers). As the present study includes only the first two years of evaluation data, much of the initial data was used to enhance the evaluation procedure, but was not used to make determinations about the outcomes of the academy’s programming.
As discussed, methods of evaluation consisted of (a) a comprehensive project-based rubric; (b) standardized assessments; (c) record reviews; and to measure experiences (outside of skill acquisition) we used (d) interviews, focus groups, and survey data.
Project-Based Rubric
Students were evaluated on their demonstration of 21st-century skills on one to four projects, using the project-based rubric. During Year 1, an overall mean score was calculated for each student after Quarter 3 and again after Quarter 4. Data from the first two quarters of the first year were not collected, as the system of evaluation was not yet developed. During Year 2, an overall mean score was calculated for students after each quarter.
A total mean score was used to calculate progress in the domains of critical thinking, innovation, transfer of knowledge, global awareness, collaboration, and communication during Year 1, while a paired-samples t-test was conducted to evaluate the change in total mean scores from the first and second quarters. The data indicate a statistically significant increase in the project-based rubric scores from Time 1 (M = 13.26, SD = 3.44) to Time 2 (M = 14.50, SD = 3.47), t(50) = -2.51, p < .01 (two-tailed). The mean increase in scores was 1.24 with a 95% confidence interval that ranged from 2.2 to 0.24. These preliminary data indicate an increase in overall demonstration of 21st-century skills from Quarter 3 to Quarter 4.
Total mean score was again used for Year 2 to calculate progress in the domains of critical thinking, innovation, transfer of knowledge, global awareness, collaboration, and communication. During Year 2 of the study, however, there were not enough students with scores in each quarter to conduct a paired-samples t-test or to evaluate changes in mean scores, with just eight students with scores in Quarters 1 and 4, and 19 students with scores in Quarters 3 and 4. The results of rubric indicated that after the end of the first year students saw an increase but at end of second year we didn’t have enough data to evaluate.
Standardized Assessments
Torrance Tests of Creative Thinking
Students’ scores for the Creativity Index of the TTCT were recorded, as the measure’s creators noted that this value serves as an overall indicator of creative potential. This standard score value is developed through pooling the creative strength ratings and the average standard score from the profile.
In Year 1, of the 64 students who completed the TTCT, 57.8% had scores that fell within the average range (85–115) on a normal distribution. Of the students, 14% had scores that fell within the below-average range, while 28.1% had scores that fell within the above-average range. The mean score for the group was 102.73 (average).
In Year 2, of the 69 students who completed the TTCT, 49.3% had scores that fell within the average range (85–115) on a normal distribution. An additional 14.5% of students had scores that fell within the below-average range, while 36.2% had scores that fell within the above-average range. The mean score for the group was 107.26 (average).
A paired samples t-test was conducted to evaluate the impact of the academy’s programming on student scores from Years 1–2. There were 36 students who were enrolled and took the assessment in both years. There was no significant difference in scores from Time 1 (M = 105.19, SD = 21.625) to Time 2 (M = 109.69, SD = 22.068), t(35) = -1.242, p < .223 (two-tailed).
Social-Emotional Health Survey
During Year 1, 51 students completed the SEHS. The results related to specific skills, including peer support, empathy, optimism, persistence, and self-control, are presented in Table 3. The percentage of students who positively responded to the items is listed in the table. For example, in Year 2, when assessing peer support, approximately one-third of the students indicated some positive peer support, specifically in response to the following items: To, “I have a friend my age who really cares about me,” 44.1% of students indicated that this was “like me” or “very much like me”; to, “I have a friend my age who will talk with me about my problems,” 32.3% of students indicated this was “like me” or “very much like me”; and to, “I have a friend my age who helps me when having a hard time,” 29.6% of students indicated this was “like me” or “very much like me.” In addition, the SEHS provides a total “covitality” score that encompasses belief in self, belief in others, emotional competence, and engaged living.
TABLE 3. Percentage of students who endorsed positive responses to SEHS items.
The mean score for all students who took the survey at the academy was 118, which falls within the high-average range. During the second year, 125 students took the SEHS, with 99 students’ responding to most or all questions. The results related to each individual domain are included in Table 3. The mean covitality score for all students who took the survey at the school in Year 2 was 122.26 (SD = 64.59), falling within the high-average range. When thinking about evaluation process, because there was a large variation in responses, individual data should be used to address individual student needs and may be useful in tracking progress over time. This total mean score, however, indicates a general positive trend in social-emotional health at the academy.
Record Review
Attendance records and grades were used to get a sense of student engagement and progress in the program. GPA was calculated on a 4-point scale (with students in AP classes as having the opportunity to score above a 4.0) and reported across three time points (Spring 2017, Fall 2017, and Spring 2018). Data from Fall 2016 were not collected, as the team was still developing the evaluation plan for this study. GPA ranged from 1.5 to 4.5, with the mean GPA for each time point as follows: Spring 2017 (M = 3.38, SD= .67), Fall 2017 (M = 3.39, SD= . 96), and Spring 2018 (M = 3.35, SD= .98). The results of a Friedman test indicated that there was no statistically significant difference in GPA across the three time points, X2 (2, N = 28) = 5.46, p < .065.
Attendance was calculated throughout the two years across four time points (Fall 2016, Spring 2017, Fall 2017, and Spring 2018), and students had relatively high and stable attendance rates. They were calculated as a percentage that included days attended over days possible. Attendance ranged from 5% to 100% attendance, with the mean attendance rate for each time point as follows: Fall 2016 (M = 94.1%, SD= 5.81), Spring 2017 (M = 92.1%, SD= 6.23), Fall 2017 (M = 92%, SD= 5.98), and Spring 2018 (M = 87%, SD= 4.71). Although we were not able to obtain actual attendance rates and GPA scores for these students at their previous schools, many students or parents reported at intake that they were at this particular school due to poor attendance and poor grades in their previous school setting. Given a history of poor attendance and low GPAs, the average attendance rates and GPAs for these students is relatively high.
In addition to the evaluation of the specific skills, data was also collected on the overall experience of students and teachers in this newly developed program, including their experience with the evaluation process.
Interviews, Focus Group, and Surveys.
Qualitative Analysis Approach
Data from interviews, surveys and focus groups were reviewed by the research team, entered in a review document, and analyzed for common constructs. Data were analyzed using a narrative approach. The reviewers independently identified themes from the review documents using a standardized table with detailed instructions. Data were coded, summarized, and categorized. Reviewers then compared tables and notes, resolved disagreement through discussion or, if required, adjudication by a third reviewer. Finally, a table was created with the common themes and sample statements were used to illustrate each theme.
Student interviews Student interviews were analyzed for common themes in regard to experiences at the school. Themes found through analysis of interview data included fitting in with a peer group, poor previous schooling experiences, applicability of the STEM program, change/growth while attending a STEM school, experience of project-based assignment format, benefits of the STEM curriculum, and behavior of peers. Themes and the corresponding statements from individual student surveys during Years 1 and 2 are represented in Table 4.
TABLE 4. Themes from parent surveys and student and teacher interviews on STEM school programming.
Teacher interviews Interviews with teachers also were analyzed for common themes and experiences. Although the academy has a limited number of teachers, 60% were interviewed during Year 1 and 50%, during Year 2. In Year 1, themes included school culture, importance of the arts, definition of student success, value of a STEM education, and project-based instruction. In Year 2, themes included school culture, project-based instruction, desire for consistency, innovation lab importance, teacher support needs, and real-life applications. Relevant themes and the corresponding statements from individual teachers in Years 1 and 2 are presented in Table 4.
Teacher focus group At the completion of the evaluation period, a total of 10 teachers participated in the focus group related to the process of the evaluation. Themes from the teacher focus group include rubrics, standardized assessment (social-emotional and creativity measures), additional areas to measure, support needs that affect evaluation, and overall evaluation concerns. Table 5 presents the themes and representative teacher statements.
TABLE 5. Themes from teacher focus group on STEM school programming.
Post-graduation interviews In Year 1, a total of five graduates, or their parents, were interviewed about post-graduation outcomes. This represents all of the recent graduates at the time. Two of the respondents were parents of the graduates, and three of the respondents were past students. In Year 2, a total of 16 graduates or their parents were interviewed about post-graduation outcomes. Four of the respondents were parents of the graduates, 11 respondents were past students, and one did not indicate whether the respondent was a past student or parent. The post-graduation survey contained items in regard to the student’s current employment, post-secondary education, enrollment, type of degree pursuing and in what field, sources of support for pursuing post-secondary education, and level of preparedness. Table 6 includes the primary outcomes identified by Year 1 and Year 2 students.
TABLE 6. Post-graduation survey results.
Parent surveys In Year 1, we did not obtain enough parent surveys to conduct any meaningful analysis. In Year 2, however, 35 parent surveys were obtained. At the time of the survey, parents reported that their child had been attending the academy from 1 to 39 months, with a mean enrollment of 14.9 months. In the survey, parents were asked whether their child had ever been employed. The results indicated that only 11.43% of students had ever been employed, 48.57% had never been employed, and 40% were too young to be employed. Parents were then asked how well the academy is preparing their child for future employment on a 5-point Likert-type (1 = not at all prepared, 2 = slightly prepared, 3 = moderately prepared, 4 = very prepared, 5 = extremely prepared). Of the 35 parent responses, 5.7% responded extremely prepared, 25.71% responded very prepared, and 34.29% responded moderately prepared for future employment. The remainder of parents indicated that their child was slightly prepared for future employment, or they did not respond due to their child’s young age or having just recently enrolled in the academy. Using the same scale, parents were asked how well the academy is preparing their child for college. Of the 35 parent responses, 2.85% responded extremely prepared, 22.86% responded very prepared, and 34.29% responded moderately prepared for college. The remaining parents responded slightly prepared for college or did not respond due to their child’s young age or having recently enrolled.
In addition, content was analyzed for themes for parents’ enrolling their child in the academy. The themes included academic challenge while addressing their child’s individual needs, fewer behavior disruptions, more STEM opportunities, challenges due to their student’s disability, and the safe and supportive environment of school. Parents also reported a desire for post-secondary education and employment, independence, happiness, and social skill development for their child. See Table 4 for statements related to common themes.
Discussion
The purpose of this study was to develop an evaluation procedure to measure the outcomes of a STEM school for individuals with special needs and to analyze the first two years of data to better shape the evaluation process. Each of these areas is discussed below.
Evaluation Procedure Development
This study included the conception of a model that organized and evaluated key aspects of what a STEM school may hope to address. Following the development of the model, the evaluation team collected data, using a variety of methods.
This in-depth evaluation process of a STEM program designed for students with disabilities has implications for researchers, program developers, and education stakeholders. The research provides a comprehensive process for evaluating a STEM school for SWD and offers specific measures that other research and educators may utilize when examining similar constructs or piloting comparable programs.
When the evaluation method was executed at the school level, the evaluation team learned which aspects were useful, those that needed adjustment, and areas that were not as effective and required further study. The aspects of the evaluation that were useful included collecting data through a mixed-methods approach on multiple areas of the student’s school experience. A variety of information was gathered through each method, and the coupling with the more qualitative approaches allowed us to understand what aspects of the program were most salient for students.
The use of surveys and focus groups allowed us to develop recommendations for the school that would be applicable for individuals who develop such programs and evaluation procedures. For example, feedback on the evaluation procedure was obtained to ensure that the appropriate items were being measured. As can be seen in the teacher focus group, much of what was measured was helpful in providing a snapshot of the students’ current perceptions and needs. Nevertheless, there is room for improvement in terms of measuring some additional constructs (e.g., motivation, parent-teacher interactions) and providing teachers and students with the context and training to understand the purpose of the measures and to use them more effectively (e.g., to administer correctly and frequently, to use the data to monitor progress and design interventions).
Although having multiple measures and approaches was useful, it was also time consuming and somewhat burdensome to collect, organize, and develop meaning from the various sources. Many data points were not returned from teachers, parents, or students. As a considerable amount of data was collected, and multiple demands were placed on the school site, school personnel were not always vigilant in gathering the necessary data from all students; thus, at times, it was difficult to evaluate individual student progress in the program and then tie these data back to the relevant components in the model. Rather, we were able only to evaluate broad program outcomes based on the data collected. As schools begin to develop STEM programming for individuals with more specialized needs, a method for effectively gathering individual student progress data will be particularly important. Specifically, to achieve a better response rate, we suggest that streamlining of the data collection process should be considered, particularly when developing evaluation protocols, and only the most relevant measures should be used.
It also may be prudent to consider additional ways of engaging the parents/caregivers of these students so that they are more involved and responsive when feedback and support are needed from the school community. Given the nature of the school setting, and that many students travel from far distances, non-public schools may need to develop creative practices for parent involvement (e.g., virtual gatherings, opportunities for frequent communication).
Despite some setbacks, the current evaluation model and procedure for evaluating a STEM program for SWD is a promising start. Now that this project has been established, and some measures can be adjusted based on feedback received, a more thorough picture of progress can be developed.
Initial Outcomes to Inform the Evaluation Process
Early outcome data on 21st Century Skill acquisition were collected with the following related outcomes: Students showed a statistically significant increase in critical thinking, innovation, transfer of knowledge, global awareness, collaboration, and communication on the comprehensive rubric of 21st-century skills from the beginning of Year 1 (M = 13.26, SD = 3.44) to the end of Year 1 (M = 14.50, SD = 3.47), t(50) = −2.51, p < .01 (two-tailed). On the Social-Emotional Health Survey (SEHS), the mean score for all students was 118 during Year 1 and 122.23 for Year 2, both falling within the high average range. No statistically significant difference was found in student Grade Point Average (GPA) or attendance across either year. With regard to post-graduation data, after Year 1, 80% of students pursued post-secondary education with 100% pursuing a bachelor’s degree in a STEM-related field. After Year 2, 100% of students pursued a post-secondary education with 75% pursuing a bachelor’s degree in a STEM-related field.
The data related to student, teacher, and parent experiences indicated many areas of strength (e.g., project-based learning; focus on STEM fields; smaller, safer environment for students who struggled in previous environment) and some areas for potential improvement (e.g., supporting teachers in creating learning communities to enhance their level of knowledge and comfort with teaching, engaging, and managing behaviors of students at various skills levels).
Although teachers were motivated and excited to work in an innovative program, many struggled with understanding how to effectively teach by using the non-traditional approach of project-based learning while still addressing state standards. School professionals who are developing such programs should consider establishing learning communities for teachers to collaborate, learn, and receive support from one another. They may want to provide additional training, particularly related to curriculum development and implementation in the STEM fields, using a project-based approach.
Many of the areas of identified need are commonly seen during the development of an innovative educational program, particularly those that serve youth with unique challenges and strengths. The mixed methods evaluation approach allowed the academy to use progress-monitoring data to inform the development of the program in “real time” rather than the academy’s waiting until the end of the data collection process to make adjustments. As data are collected and needs are clarified, adjustments can be made to the program and results shared throughout the evaluation process. We suggest that schools who are developing comparable programs or processes utilize a similar approach in order to maximize the use of data-based decision making throughout implementation.
Limitations
As this research was a single-site study of a STEM school for SWD, when considering the findings of this study, the limitations of case study research must be taken into account. A limitation of the case study design is that it is difficult to generalize results. Specifically, the study did not include a control group to compare findings; thus, we are unable to draw conclusions about the effectiveness of STEM programs for SWD. In addition, the research encompassed only one school with a relatively small student body. Further, the data in regard to the program are limited, as they were from the first two years of the school, and some of the research measures were specifically created for this study, as no existing measures met our evaluation needs. In addition, these measures do not currently have published validity and reliability data for their use with individuals with disabilities. Further, different measures contained different numbers of participants, ranging from 5 postgraduate student interviews to 99 student surveys for the social-emotional measure.
Another limitation of this study, related to the first limitation, is the difficulty with generalizing this research to a wider population due to the sample. More than half of the students (53%) had an IEP eligibility of ASD. Thus, the findings may not be indicative of students with diverse disabilities. In addition, all student participants’ primary language was English; therefore, the results may not generalize to students with different primary languages. In addition, due to the difficulty with collecting data from the school site, we are not able to provide an analysis of certain information, such as that on parent survey forms, including information about students’ educational history and the reasons for enrolling in the STEM academy. Despite the aforementioned limitations, we have taken the first step in creating a comprehensive developmental evaluation to measure student outcomes in STEM school for SWD. In this article, we begin to address the gap in the research regarding the promotion of STEM skills for SWD, there is more work to be done.
Implications for Future Research
A subsequent study of the academy should utilize a longitudinal design to investigate student outcomes over time. The current focus on creating an environment of acceptance and of developing related social skills seems to be an asset of the program. It would be great for the team to try and articulate how this is done in the school setting so that it can be replicated by related programs. In general, future research should explore the impact that STEM-specific programs have on students’ educational and employment outcomes. For example, subsequent research regarding this academy could utilize a longitudinal approach in the following years to look at the program graduates’ educational and employment outcomes to investigate whether graduates are more likely to pursue post-secondary STEM education and careers. In addition, research could examine the barriers that SWD may face when participating in a STEM program as well as factors that may promote these students’ success in a STEM-specific program and STEM careers.
Conclusion
Using existing research and models, we have systematically developed and evaluated a comprehensive evaluation process for understanding outcomes of STEM schools on the learning of SWD. However, there is a significant need for further research on the evaluation, preparation, and participation of SWD in STEM education and careers. We must understand the impact of these programs on long-term outcomes of individuals as they enter and continue along their career paths.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by Chapman University Institutional Review Board. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.
Author Contributions
AJG designed and conceptualized the work, collected data, supervised the data entry and analysis, interpreted the data and drafted a substantial portion of the article. JB assisted with designing and conceptualizing the work, interpreting the data, and revised the article. NR Assisted with data collection, data entry and analysis, assisted with interpretation of the data and drafted a substantial portion of the article. JA Assisted with initial literature review, interpretation of the data, and revised a substantial portion of the article. VT Added to initial literature review, and assisted with data collection, entry, and analysis. LG Completed data entry, assisted with analysis, and substantially revised the final draft. All authors read and approved the final manuscript.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Alper, J. (2016). Developing a national STEM workforce strategy: a workshop summary. Washington, DC: National Academies Press.
Alvares, G. A., Bebbington, K., Cleary, D., Evans, K., Glasson, E. J., Maybery, M. T., et al. (2020). The misnomer of ‘high functioning autism': intelligence is an imprecise predictor of functional abilities at diagnosis. Autism 24, 221–232. doi:10.1177/1362361319852831
American Psychiatric Association (2013). Diagnostic and statistical manual of mental disorder. 5th edn. Washington, DC, USA: American Psychiatric Publishing.
Bachen, C. M., Hernández-Ramos, P. F., and Raphael, C. (2012). Global empathy scale. PsychTests 11, 33–39. doi:10.1037/t16972-000
Baio, J., Wiggins, L., Christensen, D. L., Maenner, M. J., Daniels, J., Warren, Z., et al. (2018). Prevalence and characteristics of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2012. MMWR Surveill Summ 65 (SS-6), 1–23. doi:10.15585/mmwr.ss6503a1
Bargerhuff, M. E. (2013). Meeting the needs of students with disabilities in a STEM school. Am. Second. Educ. 41 (3), 3–20. doi:10.1177/1053451214546401
Basham, J. D., and Marino, M. T. (2010). Introduction to the topical issue: shaping STEM education for all students. J. Spec. Educ. Technol. 25 (3), 1-2. doi:10.1177/016264341002500301
Bellman, S., Burgstahler, S., and Chudler, E. H. (2018). Broadening participation by including more individuals with disabilities in STEM: Promising practices from an engineering research center. Am. Behav. Sci. 62, 645–656. doi:10.1177/0002764218768864
Boman, P., Furlong, M. J., Shochet, I. M., Lilles, E., and Jones, C. N. (2009). “Optimism and the school context,” in Handbook of positive psychology in schools. Editor F. M. Furlong (New York, NY, USA: Routledge), 197–212.
Brown, B. B., and Braun, M. T. (2013). “Peer relations,” in Research, applications, and interventions for children and adolescents: a positive psychology perspective. Editors C. Proctor, and P. A. Linley (New York, NY, USA: Springer Science & Business Media), 149–164. doi:10.1007/978-94-007-6398-2_9
Center for Applied Special Technology. (2018). UDL on campus: home. Retrieved February 23, 2018, from http://udloncampus.cast.org/home#.WIYVqLYrKRs
Crespi, B. J. (2016). Autism as a disorder of high intelligence. Front. Neurosci. 10, 300. doi:10.3389/fnins.2016.00300
Eaves, L. C., and Ho, H. H. (2008). Young adult outcome of autism spectrum disorders. J. Autism Dev. Disord. 38 (4), 739–747. doi:10.1007/s10803-007-0441-x
Edmunds, J., Arshavsky, N., Glennie, E., Charles, K., and Rice, O. (2017). The relationship between project-based learning and rigor in STEM-focused high schools. Interdiscipl. J. Probl.-Based Learn. 11 (1). doi:10.7771/1541-5015.1618
Fayer, S., Lacey, A., and Watson, A. (2017). STEM occupations: past, present, and future. Retrieved from https://www.bls.gov
Furlong, M. J., You, S., Renshaw, T. L., Smith, D. C., and O’Malley, M. D. (2014). Preliminary development and validation of the Social and Emotional Health Survey for secondary school students. Soc. Indicat. Res. 117 (3), 1011–1032. doi:10.1007/s11205-013-0373-0
Hayward, S. M., McVilly, K. R., and Stokes, M. A. (2018). “Always a glass ceiling.” Gender or autism; the barrier to occupational inclusion. Res. Autism Spect. Disord. 56, 50–60. 10.1016/j.rasd.2018.09.001.
Hendricks, D. (2010). Employment and adults with autism spectrum disorders: challenges and strategies for success. J. Vocat. Rehabil. 32, 125–134. doi:10.3233/jvr-2010-0502
Hurley-Hanson, A. E., Giannantonio, C. M., and Griffiths, A. J. (2020). Autism in the workplace: creating positive employment and career outcomes for generation A. London, UK; Palgrave Macmillan.
Hwang, J., and Taylor, J. C. (2016). Stemming on STEM: a STEM education framework for students with disabilities. J. Sci. Educ. Stud. Disab. 19 (1), 39–49. doi:10.14448/jsesd.09.0003
Israel, M., Maynard, K., and Williamson, P. (2013). Promoting literacy-embedded, authentic STEM instruction for students with disabilities and other struggling learners. Teach. Except. Child. 45 (4), 18–25. doi:10.1177/004005991304500402
Kennedy, M. J., and Wexler, J. (2013). Helping students succeed within secondary-level STEM content using the “T” in STEM to improve literacy skills. Teach. Except. Child. 45 (4), 26–33. doi:10.1177/004005991304500403
Kim, K. H. (2006). Can we trust creativity tests? A review of the Torrance Tests of Creative Thinking (TTCT). Creativ. Res. J. 18, 3–14. doi:10.1207/s15326934crj1801_2
Kim, K. H. (2008). Meta-analyses of the relationship of creative achievement to both IQ and divergent thinking test scores. J. Creativ. Behav. 42, 106–130. doi:10.1002/j.2162-6057.2008.tb01290.x
Kim, K. H. (2011). The APA 2009 division 10 debate: are the torrance tests of creative thinking still relevant in the 21st century?. Psychology of Aesthetics, Creativity, and the Arts 5 (4), 302–308. doi:10.1037/a0021917
Kirschman, K. J. B., Johnson, R. J., and Roberts, M. C. (2009). “Positive psychology for children: development, prevention, and promotion,” in Oxford handbook of positive psychology. Editor F. M. Lopez (New York, NY, USA: Oxford University Press), 133–148.
Lai, E. R., DiCerbo, K. E., and Foltz, P. (2017). Skills for today: what we know about teaching and assessing collaboration. London, UK: Pearson.
Larson, L. C., and Miller, T. N. (2011). 21st Century skills: prepare students for the future, Kappa Delta Pi Rec., 47:3, 121–123. doi:10.1080/00228958.2011.10516575
Lee, A. (2011). A comparison of postsecondary science, technology, engineering, and mathematics (STEM) enrollment for students with and without disabilities. Career Dev. Except. Individ. 34 (2), 72–82. 10.1177/0885728810386591
Leigh, J., and Du, J. (2015). Brief report: forecasting the economic burden of autism in 2015 and 2025 in the United States. J. Autism Dev. Disord. 45 (12), 4135–4139. doi:10.1007/s10803-015-2521-7
Mansilla, V. B., and Jackson, A. (2011). Educating for global competence: preparing our youth to engage the world. New York, NY, USA: Asia Society.
Masi, A., DeMayo, M. M., Glozier, N., and Guastella, A. J. (2017). An overview of autism spectrum disorder, heterogeneity and treatment options. Neurosci. Bull. 33 (2), 183–193. doi:10.1007/s12264-017-0100-y
Meyer, K., and Wurdinger, S. (2016). Students’ perceptions of life skill development in project-based learning schools. J. Educ. Iss. 2 (1), 86–91. doi:10.5296/jei.v2i1.8933
Moorehead, T., and Grillo, K. (2013). Celebrating the reality of inclusive STEM education Co-teaching in science and mathematics. Teach. Except. Child. 45 (4), 50–57. doi:10.1177/004005991304500406
A. J. Mills, E. Durepos, and E. Wiebe Editors. (2010). Encyclopedia of case study research. Thousand Oaks, CA: Sage.
Pellegrino, J. W., and Hilton, M. L. (2012). Education for life and work: developing transferable knowledge and skills in the 21st century. Washington, DC, USA: The National Academies Press.
Plucker, J. A., Beghetto, R. A., and Dow, G. T. (2004). Why isn’t creativity more important to educational psychologists? Potentials, pitfalls, and future directions in creativity research. Educ. Psychol. 39 (2), 83–96. doi:10.1207/s15326985ep3902_1
President’s Council of Advisors on Science and Technology (2010). Report to the president: prepare and inspire: K-12 education in science, technology, engineering and math (STEM) for America’s future. Washington, DC, USA: White House Retrieved from June 20, 2017https://nsf.gov/attachments/117803/public/2a--Prepare_and_Inspire--PCAST.pdf.
Richards, J. (2012). Examining the exclusion of employees with Asperger syndrome from the workplace. Person. Rev. 41 (5), 630–646. 10.1108/00483481211249148
Road Map Project (2014). Skills and dispositions that support youth success in school part 2: strategies for building motivation, engagement and 21st-century skills (Scholarly project). Youth Development Executives of King County and The Road Map Project Retrieved from http://www.roadmapproject.org/wp-content/uploads/2012/07/Skills-Dispositions-Part-Two-Strategies-May-2014-Final.pdf.
Rule, A. C., Stefanich, G. P., Haselhuhn, C. W., and Peiffer, P. (2009). A Working Conference on Students with Disabilities in STEM Coursework and Careers. ERIC Document Reproduction Service No. ED 505 568.
Scheier, M. F., and Carver, C. S. (1985). Optimism, coping, and health: assessment and implications of generalized outcome expectancies. Health Psychol. 4, 219–247. doi:10.1037//0278-6133.4.3.219
Schreffler, J., Vasquez, E., Chini, J., and James, W. (2019). Universal design for learning in postsecondary STEM education for students with disabilities: a systematic literature review. Int. J. STEM Edu. 6 (8), 1–10. doi:10.1186/s40594-019-0161-8
Shechtman, N., DeBarger, A., Dornsife, C., Rosier, S., and Yarnall, L. (2013). Promoting grit, tenacity, and perseverance: critical factors for success in the 21st century (Draft). Washington, DC, USA: Department of Education Office of Educational Technology.
Soland, J., Hamilton, L., and Stecher, B. (2013). Measuring 21st-century competencies: guidance for educators (Scholarly project). Asia Society Retrieved from https://asiasociety.org/files/gcen-measuring21cskills.pdf.
Taylor, J. L., and Seltzer, M. M. (2011). Employment and post-secondary educational activities for young adults with autism spectrum disorders during the transition to adulthood. J. Autism. Dev. Disord. 41 (5), 566–574. doi:10.1007/s10803-010-1070-3
U.S. Department of Labor, Bureau of Labor Statistics (2017a). Employment projections. Retrieved from https://www.bls.gov/news.release/ecopro.nr0.htm.
U.S. Department of Labor, Bureau of Labor Statistics (2017b). May 2017 National occupational employment and wage estimates United States. Retrieved from https://www.bls.gov/oes/current/oes_nat.htm.
U.S. Department of Labor, Bureau of Labor Statistics (2018). June 21). Persons with a disability: labor force characteristics news release https://www.bls. gov/news.release/disabl.htm.[Press release]
Wei, X., Yu, J., Shattuck, P., McCracken, M., and Blackorby, J. (2013). Science, technology, engineering, and mathematics (STEM) participation among college students with an autism spectrum disorder. J. Autism Dev. Disord. 43 (7), 1539–1546. doi:10.1007/s10803-012-1700-z
Wei, X., Yu, J. W., Shattuck, P., and Blackorby, J. (2017). High school math and science preparation and postsecondary STEM participation for students with an autism spectrum disorder. Focus Autism Other Dev. Disab. 32 (2), 83–92. doi:10.1177/1088357615588489
You, S., Furlong, M. J., Dowdy, E., Renshaw, T. L., Smith, D. C., and O’Malley, M. D. (2014). Further validation of the social and emotional health survey for high school students. Appl. Res. Qual. Life 9 (4), 997–1015. doi:10.1007/s11482-013-9282-2
Zikic, J., and Hall, D. T. (2011). Toward a more complex view of career exploration. Career Dev. Q. 58 (2), 181–191. doi:10.1002/j.2161-0045.2009.tb00055.x
Keywords: stem education, students with disabilities, evaluation of STEM programs, assessment, autism
Citation: Griffiths AJ, Brady J, Riley N, Alsip J, Trine V and Gomez L (2021) STEM for Everyone: A Mixed Methods Approach to the Conception and Implementation of an Evaluation Process for STEM Education Programs for Students With Disabilities. Front. Educ. 5:545701. doi: 10.3389/feduc.2020.545701
Received: 25 March 2020; Accepted: 28 December 2020;
Published: 02 February 2021.
Edited by:
Ida Ah Chee Mok, The University of Hong Kong, Hong KongReviewed by:
Di Liu, East China Normal University, ChinaChina Jonte Taylor, Pennsylvania State University, United States
Copyright © 2021 Griffiths, Brady, Riley, Alsip, Trine and Gomez. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Amy Jane Griffiths, YWdyaWZmaXRAY2hhcG1hbi5lZHU=