Correlates of Academic Performance in Online Higher Education: A Systematic Review

Chung, Jennifer; McKenzie, Stephen; Schweinsberg, Ashleigh; Mundy, Matthew Edward

doi:10.3389/feduc.2022.820567

SYSTEMATIC REVIEW article

Front. Educ., 25 February 2022

Sec. Digital Education

Volume 7 - 2022 | https://doi.org/10.3389/feduc.2022.820567

Correlates of Academic Performance in Online Higher Education: A Systematic Review

Jennifer Chung¹*

Stephen McKenzie¹

Ashleigh Schweinsberg¹

Matthew Edward Mundy²

¹School of Psychological Sciences, Monash University, Melbourne, VIC, Australia
²Monash Centre for Professional Development and Monash Online Education, Monash University, Melbourne, VIC, Australia

The existing steady and continual rise of online learning in higher education has been accelerated by COVID-19 and resulted in a move away from solely on-campus teaching. Prior to the pandemic, online education was providing higher education to students who were returning to study to up-skill, are employed full-time, caring for family members, living rurally or remotely and/or for whom otherwise face-to-face campus learning was not a preference or option. To understand how we can better support online students in their unique circumstances and create an optimal learning environment, we must understand the factors associated with academic achievement within an online setting. This systematic review involved a search of relevant databases published between January 2009 and May 2021 examining factors and constructs related to academic performance in online higher education settings. Across 34 papers, 23 (67.6%) explored factors and constructs related to student characteristics including cognitive and psychological, demographic, university enrolment, and prior academic performance. Twenty-one (61.8%) papers explored learning environment factors including engagement, student experience, course design, and instructor. Our overall synthesis of findings indicates that academic performance in online learning is most strongly associated with motivation (including self-efficacy), and self-regulation. We propose three main implications of our review for online learning stakeholders such as educators and designers. Firstly, we argue that the wellbeing of online learners is important to understand, and future research should explore its impact on students’ experience and success in online higher education. Secondly, we emphasise the importance of developing and designing online courses utilising relevant frameworks and evidence-based principles. Finally, we propose an approach to promoting improved student cognitive and psychosocial experiences (such as self-efficacy, self-regulation, and perceived support) could be achieved by creating and incorporating an online learning orientation module at the commencement of enrolment.

Systematic Review Registration: (website), identifier (registration number).

Introduction

In recent decades there has been a rapid increase in the number and nature of online or web-based subjects, courses, programs, and degrees. In both the United States (U.S) and Australia, tertiary online learning and distance education enrolment is increasing steadily by approximately 5% per year (Seaman et al., 2018; Australian Government Department of Education Skills and Employment, 2020). Approximately 15% of total higher education students, in both the U.S and Australia, are enrolled exclusively in an external or distance mode (Seaman et al., 2018; Australian Government Department of Education Skills and Employment, 2020). Furthermore, Allen and Seaman (2013) report that in the U.S, 32% of all tertiary students take at least one online course.

Moore et al. (2011) conducted a literature review on terms used in research and academia to describe distance learning, e-learning and online learning and subsequently reported the results of surveying 43 conference attendees at an educational technology conference in 2009. Moore et al. (2011) asserted that terminology, definitions as well as spelling of terms differed, and furthermore, differed between countries. Similarly and more recently, Singh and Thurman (2019) systematic review provides insight into the definitions of online learning used during 1988–2018. Online learning was the most commonly used term, and its defining elements include technology to deliver education, time including both synchronous and asynchronous elements, interactivity through a variety of contexts, physical distance, and in addition, confusion over related terms to describe online learning was also apparent (Singh and Thurman, 2019).

The current paper will not present an updated literature review on these terms and their terminology, and hereafter we will refer to online learning and on-campus learning. Where online learning refers to fully or wholly online learning. In fully online learning, the learning environment is generally asynchronous, requires greater self-directed learning, and often involves less live contact with instructors and peers. There is no face-to-face attendance required, however in some cases students may be asked to participate in final exams at a campus location, which is therefore not strictly fully online learning (Allen and Seaman, 2013). This differs from hybrid or blended learning, which involves a combination of face-to-face and technology enhanced instruction (Rasheed et al., 2020). More recently, on-campus learning has shifted towards this model of blended learning.

Online learning has also typically attracted a unique type of student with characteristics such as their being older in age, living rurally or juggling multiple responsibilities such as full-time employment and/or caring for families (Colorado and Eberle, 2010; Kahu et al., 2013; Johnson, 2015). Advantages of online education include its allowing an expansion of education opportunities including to individuals in remote or rural geographic locations. Although there has been an increase in the number of adult learners and online leaners recently, there are fewer studies on them in comparison to studies on traditional university students (Kara et al., 2019). From an institutional viewpoint, according to Allen et al. (2016), 77% of institutions that provide online or distance learning believe that it is critical to their long term strategy.

As a consequence of the COVID-19 pandemic, universities worldwide are now needing to offer online or blended learning as alternatives to fully face-to-face learning. Initially, the race to convert face-to-face teaching to fully digital and remote was a reaction to lockdowns, however there is now a shift to the new normal consisting of long-term changes, almost 2 years into the pandemic. This changing landscape in online education has meant that students who previously studied in the face-to-face mode are now also studying online, in addition to students who were typically already attracted to online study.

In addition to differences in the students who may typically elect to study online, there are also inherent differences in how the learning is undertaken (e.g., asynchronous). Based on this phenomenon, are there more specific skills or traits that help students to learn in fully online environments? Secondly, are there structures within the online learning environment that enhance online students’ learning and outcomes? Whilst there are most likely similarities for all students, whether their study is undertaken face-to-face or online, we contend that asynchronous online study is likely to require additional unique skills.

How can we help students succeed in online learning? To this end, what is success in online learning? Success in this context may be defined as academic, personal growth and learning, and employability. Whilst other and broader forms of success are important, in this study we focus on academic performance as a measure of success or competencies in relation to learning objectives, and as a measure of performance outcome. In this study we define academic success based on performance and attainment in online courses or degrees. Academic achievement is typically defined as an academic result in an assignment, quiz, exam, single subject, or whole degree.

Therefore, to answer our initial question of how we can help students succeed in online learning we need to better understand the factors and variables that contribute to successful online learning. This will allow evidenced based decision to be implemented when creating learning environments and better support students to achieve their best possible outcome. Despite the growing interest in online higher education, and particularly with COVID-19 rapidly increasing the online uptake, there is a lack of consolidated review on factors associated with academic performance in online learning in higher education.

Therefore, the aim of this systematic review was to explore the factors associated with academic performance within online learning environments. This was achieved by evaluating empirical studies that have examined a variable, construct, or factor associated with academic achievement in fully online learning environments. This systematic review was performed in line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement guidelines (Liberati et al., 2009). The overarching research question for this study is “What factors have been examined in relation to academic performance in online higher education?” To focus the study, this question is divided into the follow sub-questions:

RQ1 What constructs that relate to student factors or individual characteristics have been examined in relation to academic performance in online higher education?

RQ2 What constructs that relate to the learning environment, course design and student experience have been examined in relation to academic performance in online higher education?

Methods

Eligibility Criteria

Papers were restricted to peer-reviewed journal papers published within the last decade in English language journals, between the years January 2009 and May 2021.

Search Strategy and Data Sources

The search strategy included reviewing peer-review papers within the databases PsycINFO, Medline, CINAHL Plus, SCOPUS, ERIC, A + Education, and ProQuest Ed Journal. This search was undertaken for papers that matched derivatives of undergraduate student, academic performance, university, online education and predictor. Boolean operators, subject and MeSH headings were used where appropriate. Search terms are provided in Table 1. This search was performed in May 2021.

TABLE 1

TABLE 1. Search terms.

Types of Studies

All studies were required to examine a predictor(s) in relation to academic performance or attainment. Studies were required to involve either a single subject or degree which was delivered fully online. Fully online study was defined as the absence of the requirement of any face-to-face attendance at a physical on campus university, or where authors stated that at least 80% of the study was completed online (Allen and Seaman, 2013). Studies that examined traditional classroom, blended/hybrid, or distance education where more than 20% face-to-face attendance was required, or where the mode and requirement of face-to-face attendance was not clear in the article, were not included in this review.

Types of Participants

Studies with students enrolled in an undergraduate or equivalent degree within a university, college or equivalent to higher education, as participants were included in this review. No additional participant characteristics were excluded.

Types of Factors or Constructs

Factors, constructs or variables including student and course related, academic or non-academic, were included in this review. Studies that included the variable of interest as the online mode of delivery itself were excluded, as well as studies that solely examined non-generalisable online learning strategies, systems or tools that are web-based. Where variables were non-generalisable and specific to the population of interest (e.g., studies examining military status within a population of military students), the studies were excluded.

Types of Outcome Measures

Studies that examined the relationship or influence of a variable (meeting the above criteria) on participants’ actual online academic performance were included. Academic performance was defined as the achievement of a numerical grade in an online assignment, quiz, exam, subject(s), or degree (grade point average; GPA). Self-report or self-perception measures of academic performance were not included. Studies that did not include an outcome measure of academic performance were not included (e.g., student satisfaction, teacher evaluation etc.). In this review, when referring to academic performance we use the term synonymously with achievement and grades.

A summary of the inclusion and exclusion criteria is presented in Table 2.

TABLE 2

TABLE 2. Inclusion and exclusion criteria.

Selection Process

We originally employed a search criterion that captured a broader definition of online learning to include blended/hybrid and distance education, and the addition of studies that were published between 2005 and 2009. During the title and abstract screening, we further refined the eligibility criteria to exclude learning environments that were blended/hybrid and/or distance modes. It was evident that there was a distinction in the terminology used where blended, hybrid and distance modes were greater than 20% face-to-face, and online learning was used to facilitate on-campus teaching. Secondly, we also restricted inclusion to articles published after 2009, as the number of articles returned were adequate and therefore allowed us to conduct a more recent review. Studies where data was only analysed by the combination of study modes were excluded (e.g., where blended, online, and on-campus data were collated). However, studies that analysed the data by instruction mode (fully online), as well as combined modes, were eligible, and only the findings relating to the fully online instruction mode were extracted. The same rationale was applied with regards to undergraduate and other types of students (e.g., postgraduate).

JC and AS independently screened the titles and abstracts of the papers for eligibility. Subsequently, the full texts of potential papers were examined for inclusion in the systematic review. At each stage of the screening process, discrepancies were discussed by the reviewers until agreement was reached. Finally, pearl growing was conducted by examining the reference lists of papers deemed relevant during full-text review.

Data Extraction and Bias

Data from the included papers were extracted and summarised, and authors were contacted where clarification was necessary. Where findings remain unclear, this is indicated in the results of this review.

The included papers were assessed using the critical appraisal tool from the JBI Checklist for Analytical Cross Sectional Studies and the JBI Checklist for Quasi-Experimental Studies (Moola et al., 2020; Tufanaru et al., 2020). Following guidelines, the papers were classed as: (a) low risk of bias; all criteria met, (b) moderate risk of bias; one criterion unclear or not met, (c) high risk of bias; two or more criteria unclear or not met. Quality assessments are presented in the results.

Results

After removing duplicates, the search identified 2,579 unique articles. In the initial title and abstract screening phase, 2,450 articles did not meet the criteria for full review and were excluded. The reviewers searched the reference lists of the remaining papers and found 20 relevant papers, totalling 129 articles for full-text review. A further 95 papers were excluded that did not meet the inclusion criteria. The exclusion reasons at the stage of full-text review are provided in Figure 1. In total, 34 papers were included in this systematic literature review (Figure 1).

FIGURE 1

FIGURE 1. PRISMA flow diagram of papers included in review.

A summary of study details of the 34 papers is presented in Table 3. A total of 24 papers described the subject study mode as a synonym of fully online (i.e., the absence of any in-person component), and 10 papers met the criteria of 80% of the study mode online. In all 10 papers the on-campus component included attending the campus or another location to complete final exams; they did not include any face-to-face attendance for teaching and learning components.

TABLE 3

TABLE 3. Systematic review table (alphabetical order according to first author).

After data extraction, unique factors were clustered into themes using a thematic synthesis. Across the 34 studies, 46 unique factors/constructs that were examined in association with academic student performance were found. Given the heterogeneity of the methods and factors examined across these studies, it was not tenable to conduct a meta-analysis.

Academic Outcome Measures

Of the papers reviewed, a total of 26 defined academic performance by the final grade achieved in the subject. Eighteen papers relied on this outcome solely (Dotterweich and Rochelle, 2012; Ryabov, 2012; Barbeau et al., 2013; Cho and Shen, 2013; Guidry, 2013a, 2013b; Joo et al., 2013; Liu et al., 2013; Castillo-Merino and Serradell-López, 2014; Helms, 2014; Attardi and Rogers, 2015; Carceller et al., 2015; Nemetz et al., 2017; Alkış and Temizel, 2018; Wakeling et al., 2018; Soffer et al., 2019; Kuo et al., 2020; Goad et al., 2021). Whereas eight papers utilised both final grade in the subject as well as grade achieved in an assignment, quiz, or exam (Hegeman, 2015; You, 2016; De Vlieger et al., 2017; Hurlbut, 2018; Baker et al., 2019; Weigel, 2019; Abe, 2020; Raza and Reddy, 2021). Four papers measured performance on a single assignment, quiz or exam (Altmyer and Yang, 2010; Hodges and Kim, 2010; Zhan and Mei, 2013; Daffin and Jones, 2018), three used GPA (Gibson et al., 2010; Bravo-Agapito et al., 2021; Malkawi et al., 2021); and one paper used grades on multiple assignments, quiz or exams (Rajabalee et al., 2020).

Thematic Synthesis of the Results

The unique factors (variables, or constructs) were grouped into domains that were either related to the student (i.e., at the individual level), or the learning environment (i.e., a variable related to the experience of the online subject). The identified domains, themes, sub-themes, and findings are described in the Discussion.

Discussion

Student Factors

This section of the discussion answers RQ1: What constructs that relate to student factors or individual characteristics have been examined in relation to academic performance in online higher education? These factors were defined as variables that are specific and unique to each student/individual, and that are not explicitly related to the student experience in any particular subject or programme.

Two themes emerged from the student domain: non-academic factors, and academic factors, of which a further two sub-themes emerged. In the non-academic theme we identified variables that were related to cognitive or psychological functioning, or that were related to the demographics of the individual. In the academic theme, we further grouped the variables into prior academic performance, or that were related to students’ enrolment at university. Of the 34 papers reviewed, 23 (67.6%) explored a factor or construct associated with the student theme. The themes, sub-themes and factors in the student domain are presented in Table 4.

TABLE 4

TABLE 4. Thematic synthesis of factors investigated—student domain.

Non-Academic Factors

Cognitive and Psychological Factors

In this sub-theme, constructs were grouped based on the themes developed by Richardson et al. (2012). In Richardson et al. (2012) systematic review and meta-analysis, “non-intellectual psychological correlates” with GPA were grouped into five distinct research domains. We applied this grouping to the current sub-theme and therefore also identified papers that related to the five constructs/distinct research domains. In total, 12 papers (35.3%) investigated the five constructs; motivation factors (n = 8), self-regulation learning strategies (n = 6), personality traits (n = 2), students’ approach to learning (n = 2), psychosocial contextual (n = 1).

Within the motivation construct, factors explored included motivation (general), self-efficacy, local of control, task value, learner persistence, and academic self-concept. The relationship between self-efficacy and achievement in learning has been well-documented, more recently research has been conducted in the online learning setting. Self-efficacy, as defined by Bandura (1997), is “beliefs in one’s capabilities to organize and execute the courses of action required to produce given attainments” (p.3). Notably, of the seven papers that explored self-efficacy, five found significant relationships where higher self-efficacy was associated with greater academic achievement (Hodges and Kim, 2010; Joo et al., 2013; Alkış and Temizel, 2018; Kuo et al., 2020; Goad et al., 2021). Kuo and colleagues investigated internet self-efficacy and found that greater belief and confidence in one’s computer and internet skills to achieve their task, were significantly more likely to achieve higher academic results. These findings align with previous research in online learning (Wang et al., 2013). The general construct of motivation (Castillo-Merino and Serradell-López, 2014; Alkış and Temizel, 2018) was also associated with student performance.

Self-regulation in academic learning has been well-documented for over 2 decades and refers to self-directive processes and beliefs in one-self that enable learners to apply abilities into academic performance skills (Zimmerman, 2008). Two of the three papers that investigated self-regulation (general) found that increased self-regulation (Cho and Shen, 2013; Kuo et al., 2020) were associated with greater achievement. These findings are consistent with the multitude of research that has demonstrated support for this strong relationship (Wang et al., 2013; Broadbent and Poon, 2015). Other self-regulation learning strategies explored were also found to be positively associated with student success in online learning include self-discipline (Nemetz et al., 2017), time management (Baker et al., 2019), and organisation (Goad et al., 2021).

Finally, in both papers that examined personality using the Five Factor Model, conscientiousness (Alkış and Temizel, 2018; Abe, 2020), and openness to experience (Abe, 2020) were significantly associated with performance, with small to moderate, and moderate positive correlations, respectively. As conscientiousness is associated with persistent, goal directed behaviour, and openness to experience describes individuals who seek intellectually challenging and novel experiences, this is consistent with prior research in the higher education setting (Richardson et al., 2012; Vedel, 2014; Abe, 2020). Non-significant results were reported for the remaining factors explored in relation to approach to learning (learning preferences, Altmyer and Yang, 2010; initiative, Goad et al., 2021) and psychosocial contextual factors (social presence, Zhan and Mei, 2013).

Demographics and Employment Factors

Nine factors and variables relating to these factors were identified across 10 papers (29.4%). Gender and age were explored in nine papers each. One of the nine papers investigating gender found a significant difference regarding academic achievement. Utilising forward entry regression, Gibson et al. (2010) found that although the model was significant, the combined 14 predictors (including gender) only accounted for 5.8% of variance on the achievement measure. Furthermore, gender was one of 11 variables that accounted for a combined total of 1.9% of variance. Therefore, the authors concluded that these findings were inconclusive. In 3 of the 9 papers investigating age (Altmyer and Yang, 2010; Gibson et al., 2010; Bravo-Agapito et al., 2021), older students demonstrated better grade performance than younger students. Finally, students who worked more than 20 h per week (Goad et al., 2021), and who were Black, Non-Hispanic (Gibson et al., 2010) were associated with lower academic performance. In summary, whilst demographic and student characteristics were explored in many of the papers in this review, very inconclusive findings have been found in relation to their relationship with online student performance.

Academic Factors

Prior Academic Performance

Three variables across 12 papers (35.3%) were explored that were objective measures of students’ prior academic performance. The three variables include GPA (n = 9), secondary school score (n = 4), and prior grades at university, not GPA (n = 2). In 8 out of 9 papers reviewed, students’ higher GPA scores were associated with increased academic performance in the current subject (Altmyer and Yang, 2010; Dotterweich and Rochelle, 2012; Ryabov, 2012; Guidry, 2013a, 2013b; Liu et al., 2013; Helms, 2014; Goad et al., 2021). Moderate (r = 0.55) to strong (r = 0.67), and strong (r = 0.63), significant positive correlations were found in Barbeau et al. (2013) and Attardi and Rogers (2015), respectively, when grades obtained in 2nd year subjects were measured against the current subject undertaken by 3rd and 4th year students. In both papers, the incoming grades were gathered from the core subjects in the Bachelor’s degree. Whereas academic performance prior to university (i.e., at secondary level), was not found to be significantly predictive of current performance at university, in comparison to other studies utilising data mining, such as Saa et al. (2020). In summary, the reviewed findings suggest that students’ grades within university are predictive of their future performance in university subjects, however achievement at secondary level does not seem to be an accurate predictor of this.

Enrolment

We identified 10 factors associated with students’ enrolment at university, either past or current that were examined across 8 papers (23.5%). The main factors include enrolment load (n = 4), number of subject attempts (n = 3), online learning experience (n = 3), number of credit hours earned (n = 3) and year level at university (n = 2).

There were 21 investigations of the 10 unique enrolment factors, examined across 8 distinct papers. Of these 21 investigations (analyses) only five significant associations were found (Dotterweich and Rochelle, 2012; Ryabov, 2012; Goad et al., 2021). Examined in four papers, students’ enrolment load (in either traditional campus or online study) was not associated with academic achievement. This indicates that online study success is not limited to or based on the amount of study students decide to undertake. Rather, students may match their study load with their self-perceived study capacity.

Previous experience in online learning was explored by Dotterweich and Rochelle (2012), Goad et al. (2021), and Liu et al. (2013). Dotterweich and Rochelle (2012) found that students undertaking a business statistics subject, who had previously undertaken an online subject (in another discipline), performed 4.6% lower than those who had no previous online learning experience. This may seem counterintuitive, and the authors hypothesised that this may be due to the unique nature of learning a quantitative subject. Previous experience in a qualitative driven, descriptive subject may have resulted in a “false sense of confidence” (Dotterweich and Rochelle, 2012, p. 135), and potentially a learning style interference. However, Goad et al. (2021) found students who had taken at least one other online unit prior to their online physical education subject, were more likely to complete the subject (i.e., passing grade or higher) than those with no prior experience.

Regarding year level, Goad et al. (2021) found that in an online physical education subject, students who were in their 2nd, 3rd, or 4th year were twice as likely to achieve a passing grade than students in 1st year. However, this finding is more likely to be associated with student level and experience at university (i.e., comparing performance of a 4th year student to a 1st year subject), rather than the ability of older students’ to achieve greater success in online learning. Finally, the number of subject attempts was not associated with current achievement (Dotterweich and Rochelle, 2012; Guidry, 2013a, 2013b).

Learning Environment Factors

This section of the discussion answers RQ2: What constructs that relate to the learning environment, course design and student experience have been examined in relation to academic performance in online higher education? This second domain in the review describes the learning environment in which students undertake their online learning. Four themes emerged from the learning environment grouping: engagement, student experience, course design, and instructor. Of the 34 papers reviewed, 21 (61.8%) explored a factor or construct associated with the learning environment theme. The themes and factors in the learning environment domain are presented in Table 5.

TABLE 5

TABLE 5. Thematic synthesis of factors investigated—Learning environment domain.

Engagement

These variables were associated with the students’ level or nature of engagement either with the Learning Management System (LMS) directly or their engagement with learning in the course. Eight engagement factors were identified and investigated across 10 papers (29.4%). The most commonly investigated factor was the amount of time or the pattern of LMS use/login (n = 9). In 6 of the 9 papers, more frequent time spent on the LMS was associated with increased student performance (Dotterweich and Rochelle, 2012; Ryabov, 2012; Cho and Shen, 2013; You, 2016; Alkış and Temizel, 2018; Soffer et al., 2019; Bravo-Agapito et al., 2021). Alkış and Temizel (2018), Cho and Shen (2013) both found moderate positive correlations between time spent on the LMS and academic performance. Ryabov (2012) multinomial logistic regression demonstrated that increased time online would result in moving from low grades (F) to middle (C or D), and from middle to higher (A or B). Furthermore, an increase of LMS use of an hour per week resulted in the odds of improving a F grade to D increasing by 0.53, and from F to A, an odds ratio of 1.23 (Ryabov, 2012). Soffer et al. (2019) explored the patterns of use across the semester and students who evenly spread their activity across the semester, rather than focused on one dominant quarter of the semester scored higher on average in final course grades. Additionally, greater LMS activity towards the final exam period was also associated with higher grades, as too did re-accessing course content (Soffer et al., 2019).

Related to LMS use, activity on discussion forums was examined in three papers. Bravo-Agapito et al. (2021) concluded that frequent LMS use as measured by LMS login and participation in forums may be due to reviewing content when instructors post new materials and forum replies, which may explain greater performance. Carceller et al. (2015) investigated the construct of social capital as measured by students’ activity on discussion forums. In this large study which included 1,458 students across 20 Arts subjects, it was found that increased forum participation was significantly associated with greater final marks than marks for those students who didn’t participate during the semester. More specifically, they argued that high-scoring students are more likely to be connected with more students, and that those connections are likely to occur with other high-scoring students. Carceller et al. (2015) posited that this compounds the experience for low-scoring students with less social capital. Lower social capital (position of lower advantage, fewer ways to satisfy needs) will also result in access to reduced and poorer quality resources (Carceller et al., 2015). Future research would benefit from further examining the nature of this relationship, given the potential implications for students are large and the potential compounding nature of this phenomenon.

In regards to engagement with live classes or tutorials, Hurlbut (2018) found that when students attended at least one live class they achieved higher final marks, whereas simply reviewing the recorded class was not significantly associated with performance. However, the authors report a limitation in the study and in the LMS as they were unable to determine whether students watched the recordings in full or part and were only able to identify students that had clicked on the link. This may explain the differences in findings between viewing and reviewing the class (Hurlbut, 2018). In Nemetz et al. (2017), both attending class or reviewing class recordings were not significantly associated with performance.

Student Experience

This sub-theme was categorised and measured by students’ perception and appraisal of factors associated with the online course. Nine student experience factors were examined across four papers (11.8%). It can be concluded that students who achieve higher grades rated having higher perceptions of their study environment (Goad et al., 2021), were more comfortable in an online education environment (Hurlbut, 2018) and perceived the instructions for learning requirements to be clear (Nemetz et al., 2017). Two variables that were investigated but that were not significantly associated with academic achievement during the COVID-19 pandemic were satisfaction and attitude. Across two semesters in the first year of COVID-19 (2020) at a university in the United Arab Emirates, Malkawi et al. (2021) did not find any differences in student performance when examining satisfaction and motivation towards e-learning as measured by the 15 items developed for their study. Malkawi et al. (2021) findings are very likely to be confounded by the very unique student experiences observed during a world-wide pandemic, in which many individuals across the world had never experienced before. In regards to learning satisfaction, Joo et al. (2013), found small to medium correlations with academic achievement.

Course Design

This sub-theme includes factors associated with how the online course or subject was designed (e.g., presence of exam invigilation). Four factors were investigated across four papers (11.8%). Significant findings were found in Daffin and Jones’ (2018) large (n = 1,694) quasi-experimental study with psychology students. Performance on proctored (invigilated) exams was significantly lower compared to exams that were not proctored in the following semester. The authors suggest that this may be due to students feeling more anxious when they are being watched, or that in a proctored exam they cannot use notes or get help from other sources. The authors also found that the average time it took to complete an exam almost doubled when it was non-proctored. The remaining three factors (instructor versus student regulated versus course format; publisher versus instructor generated materials; and audio and video versus text interaction) that were investigated by one paper each were not significantly associated with performance.

Instructor

This sub-theme consists of factors that were related to the instructor in the course. In total, three papers (8.8%) explored two instructor variables. This may be surprising given the nature of online study where the main social and live interaction that students may have in their course is with their instructor. Two of the three papers examining instructor effectiveness (n = 2) and communication with instructors via social media (n = 1) did not reveal any significance on academic achievement in online learning. In a cohort of students taking an online mathematics course, De Vlieger et al. (2017) reported that an increase of 1 standard deviation (SD) of the quality of instructor effectiveness, resulted in 0.25 SD increase in course grades in the first mathematics unit, and in the following unit would show an increase of 0.04 SD. The authors reported that an increase of 1 SD in instructor effectiveness equates to an instructor in the 87th percentile of effectiveness. The model used in this study is based on the sort of value-added model, often used in K-12 (Ehlert et al., 2014; De Vlieger et al., 2017). Finally, Nemetz et al. (2017) explored the concept of students wanting to connect to their lecturer via Facebook and grade performance, however no significant relationship was found.

Implications and Critical Reflections

To our knowledge, this systematic review is the first of its kind undertaken to review empirical studies that examine factors which relate to academic achievement in undergraduate online higher education. Our review, that included 34 papers, revealed that of the student and learning environment factors which have been examined and explored, only a select few variables consistently reported significant associations with academic performance in online learning in this category.

Sixty-seven percent of the papers reviewed explored student-centric or factors and constructs that relate to the individual. Twenty-nine percent of papers explored demographic characteristics, and overall findings were inconclusive in relation to academic performance. Overall, our review does not indicate that there are particular student demographic characteristics which are associated with risk of low academic performance.

In total, almost one third of papers (29.4%) reviewed explored cognitive and psychological constructs, and overall, many reported significant associations with academic performance. Self-regulation strategies, motivation factors (including self-efficacy) and personality, were each examined by two or more studies and reported positive associations with greater student performance in online programmes. According to Shen et al. (2013), online instructors can support online students’ self-efficacy and learning satisfaction by regularly monitoring students’ activities and completion of tasks and therefore provide timely support and guidance where needed. Furthermore, Shen et al. (2013) findings revealed that online self-efficacy was related to social interactions both with peers and instructors. Students would benefit from instructors fostering a sense of community and social presence including direct interactions via discussion boards and demonstrating social interactions in an online learning community (Artino and Stephens, 2009; Shen et al., 2013).

The heavy focus of the constructs explored in the cognitive and psychosocial factor sub-theme, such as self-regulation strategies and motivation factors was not surprising. However, constructs associated with psychological states and mental wellbeing (within psychosocial contextual factors) were not explored in the papers that met our methodology criteria. Richardson et al. (2012) systematic and meta-analytic review that examined non-intellective correlates of performance (as measured by GPA), of which thematic synthesis in this review was based on, lists a domain of “psychosocial contextual influences.” In this domain, authors include mental health and wellbeing including general stress, academic stress, and depression (Richardson et al., 2012).

Understanding and addressing student wellbeing is currently a very topical factor due to the recent and on-going effects of COVID-19. After the worldwide outbreak of COVID-19, there have been reports and concerns about how it has impacted university students. Some impacts include shifting from face-to-face to fully online classes resulting in changes in assessment and evaluation, international students, travel restrictions and border closures, and more generally impacts on mental health (Eaton and Turner, 2020; Sahu, 2020; Ye et al., 2020).

However even prior to COVID-19, it was widely accepted that psychological distress, reduced wellbeing and mental health is a growing problem and concern in university students (Stallman, 2010; Larcombe et al., 2016; Thorley, 2017; Sharp and Theiler, 2018). Greater psychological distress and decreased wellbeing in general populations of university students has been associated with reduced academic performance (Stallman, 2010), and negatively impacted on academic self-efficacy (Grøtan et al., 2019), motivation and other academic outcomes including lower engagement and increased drop-out rates (Biasi et al., 2018). Whilst we suspect that lower mental wellbeing may similarly impact online learners, this phenomenon needs to be explored in future research. Research questions could include: Are the anxieties or stressors experienced by online students similar to those experienced by students studying face-to-face (e.g., assessment, learning content)? Are there additional stressors experienced due to or related to the fully online learning environment (e.g., using the LMS)? Are the stressors related to individual factors of which online learners are more likely than face-to-face learners to experience (e.g., juggling employment and/or caregiving responsibilities)? Improving understanding in these areas would have potentially large implications for how online learning should best be designed to support student wellbeing and promote academic success (Larcombe et al., 2021).

In regards to the learning environment, which 61.8% (n = 21) of the papers investigated, unsurprisingly, greater activity and engagement with the LMS was found to be associated with more positive academic outcomes. Measures of LMS activity varied across papers, including total time spent in them, number of logins, pattern of logins, and participation and contribution to discussion forums. Whilst this finding may have been expected, this supports the contention that for online learners, the LMS platform and engagement with it is highly important for success. We contend that this demonstrates the importance of placing considerable time, expertise, and funding into ensuring that the LMS is engaging, interactive, of high quality, and importantly–that the technology used is innovative and effectively facilitates learning in a fully remote and asynchronous mode. Pre-existing course materials cannot always be directly translated from existing modes into fully online delivery. Online activities and materials need to be re-designed, if not completely transformed, with the online learner and environment in mind, rather than making pre-existing materials fit into the online landscape (Broadbent and Lodge, 2020). It is important that evidence-based principles for design and delivery are utilised, and that frameworks relevant to online learning are consulted (see, e.g., Cranney et al., 2020). Furthermore, evaluation of online learning materials needs to be on-going and iterative to achieve best practice (Broadbent and Lodge, 2020). However, relying purely on LMS activity as a predictor of success may be superficial and more specialised measures may be more appropriate and provide a more in-depth understanding of students’ online experiences.

Satisfaction was explored in only one paper in this review and the findings supported its positive relationship with learner achievement (Joo et al., 2013). Other factors that satisfaction has been associated with include self-efficacy (Joo et al., 2013; Shen et al., 2013), perceptions of support (Lee, 2010; Lee et al., 2011), and preparedness for learning, in which learning orientation sites can be used as a proactive strategy (Abdous, 2019). In Shen et al. (2013), self-efficacy to complete an online course predicted 12% of the variance in satisfaction, with the remaining dimensions of self-efficacy (interacting socially, interacting with classmates, interacting with instructors) predicting between 1–4% of variance each. The authors suggest that instructors play a large role in supporting online students self-efficacy, particularly those new to online study including by monitoring and responding in a timely way and promoting social interactions to ease potential anxieties (Shen et al., 2013).

In regards to perceptions of support, this may include instructional, peer and technical (Lee, 2010; Lee et al., 2011). When students feel more supported by their online course, they are more likely to be satisfied. In Kurucay and Inan (2017) quasi-experimental study, students who worked in groups and therefore experienced greater learner-learner interactions, achieved significantly higher grades than students who worked individually. The authors concluded that this highlights the importance of student interactions in online courses, helping students to feel part of a community (Kurucay and Inan, 2017).

Lastly, a proactive approach and strategy to provide support (both academic and non-academic) and increasing students’ preparedness and readiness for online learning is through an online learning orientation (Cho, 2012; Horvath et al., 2019; Lee and Choi, 2011; Taylor et al., 2015). Online learning orientations are programs, curricula, modules or resources that provide training or support to students in preparation for online learning (Liu, 2019). The offerings, length, content and nature of these orientations vary significantly within and between institutions. In Abdous (2019), after completing their online orientation, there was a strong influence of students’ sense of preparedness on their feelings of anxiety. Similarly, Liu (2019) evaluation of a self-paced online learning orientation improved students’ readiness in social competencies, technical and communication domains. In summary, well designed orientation programs for online learners can address relevant factors associated with success such as facilitating the development of organisational and effective time management skills, increasing confidence in using technology and the LMS, improving retention, and fostering or establishing a sense of community (Garivaldis et al., 2022; Horvath et al., 2019; Liu, 2019).

Recommendations for Educators, Designers and Other Key Stakeholders

Based on our findings in this systematic review, as well as the points we have raised in discussing the implications of our review, we summarise here our recommendations for educators, instructional designers and other key online learning stakeholders. These include:

• Actively seek to understand student wellbeing and stressors that students are experiencing in relation to their course (such as the structure, LMS usability) or their characteristics as online learners. Consider how and where further resources or support would best be provided to your students.

• Value the importance of evidence-based principles when designing online learning curriculum and delivery. Refer to relevant frameworks to guide best teaching and delivery practice, that are specifically relevant to the online environment and to online learner characteristics. Seek support (such as expertise and financial) and endeavor to take an iterative approach to redevelopment.

• Create and/or incorporate an online learning orientation module or program. Focus on incorporating content that will support students’ preparedness for online learning such as self-regulation, self-efficacy, perceptions of support, and social interactions through fostering a sense of community. Reflect on the general needs of online learners, as well as specific needs of your cohort of students. Determine if any existing resources within your course, department, or institution may be adapted for inclusion in your course or course supporting resource. Consider your available time and resources and determine how a small-scale module (or prototype module) can be developed, potentially by collaborating with another online course educator. If your program already includes an online learning orientation, take an iterative approach; for example,—reevaluate the topics included, how or if they benefit your students, and how or if your orientation might be redesigned to further support your online students. Finally, consider how your orientation materials fit into the “bigger picture” of your online course and your institutional context, with a goal to complement your existing resources.

Limitations

Our systematic review included papers published over a 12-years timeframe. Whilst this is a strength as it allows a broad range of publications to be included, we also acknowledge that the teaching, practice and research conducted in online learning has changed. Given that the online learning space has seen an almost exponential growth in this decade, findings in papers published in 2009 may not be representative of what is occurring now, particularly given the shift to online learning during COVID-19. However, cognitive and psychosocial factors are related to student experiences, not only to online students, therefore the importance of these factors are likely to remain stable across time. Also, in conducting a review of the online learning literature, it is clearly evident that there are large differences in terminology used to describe online learning (Moore et al., 2011). Our systematic review attempted to include and exclude papers based on the description of the online course meeting our definition/cut off criterion of content and delivery that is, greater than 80% online (Allen and Seaman, 2013). However, we acknowledge that where descriptions were unclear to the reviewers (and thus were excluded based on the inclusion criteria), it is possible that the courses did in fact meet the definition of 80% online and should have been included. Whilst we are unable to confirm this, it does suggest that additional studies could be included in future reviews if the inclusion criteria were adjusted. Additionally, whilst we provide our rationale for choosing to examine fully online learning, given the shift to blended or hybrid learning due to COVID-19, a similar review with this learning environment would be very timely and provide insight to many educators and institutions worldwide.

Though not specifically a limitation in how this review was conducted, we acknowledge that using solely academic performance as the dependent variable in a review may be putting too much emphasis on academic performance. Whilst measuring academic performance allows educators to evaluate the level of learning and competencies in a given subject area, and can be used for further selection processes (e.g., continued study), there are also inherent limitations with relying on grades as a metric of success (Madigan and Curran, 2021). Certain types of assessments may promote surface-level learning, rather than deeper learning (Flynn et al., 2020; Madigan and Curran, 2021), may not necessarily reflect actual learning (Arum and Roksa, 2011), and grading approaches can differ significantly across institutions (York et al., 2015).

By evaluating student success based on academic performance, we have not had the opportunity to discuss other factors that relate to student success in higher education. Other highly-relevant non-academic factors impacting upon success may include wellbeing, fulfilment, enjoyment, satisfaction, employability, general and transferable skills and knowledge, valuing deep learning, the building of professional and non-professional relationships, connections and sense of community within the university. These non-academic success factors may also act as a support and protective factor to the achievement of academic success (Vesely et al., 2007; Sadera et al., 2009; Stallman, 2010). Evaluating only one type of academic success construct contributes to the narrowing focusing in education research (York et al., 2015). Therefore, whilst the importance of academic performance will never, and nor should it ever be overlooked–it is also time to shift focus toward ensuring and understanding how to best support students to also attain non-academic successes (Chung and McKenzie, 2020).

Conclusion

This study presents a systematic literature review revealing the factors that are associated with academic performance in undergraduate online higher education. We firstly discussed the need for such a review, given the large increase in fully online learning, and secondly the unique nature of studying solely online, mostly asynchronously, and often appealing to a different cohort of students consisting of more students who are mature-age, returning to study, and who are employed and/or caregiving. We then examined 2,599 studies published from 2009 to 2021 by applying PRISMA guidelines for systematic reviews. Thirty-four papers were included in the review and factors that were examined in relation to academic performance were extracted. We conducted a thematic synthesis of the correlates and found factors and constructs that related to student factors or individual characteristics (67.6%; RQ1) and the learning environment, course design and student experience (61.8%; RQ2).

We showed in this review that whilst research has examined a range of cognitive psychosocial constructs (35.3%) that relate to motivation factors, self-regulated learning, personality traits, approach to learning and psychosocial contextual factors; the primary focus was on motivation constructs and self-regulation strategies, and these variables demonstrated a high correlation with academic performance. As expected, many papers reviewed the impact of demographic variables and employment (29.4%) on academic achievement, as well as academic history including prior academic performance (35.3%) and other enrolment related variables (23.5%). These findings were mainly inconclusive. In regards to the learning environment domain, the majority of the papers reviewed examined engagement with the LMS (which reported to be highly predictive of academic success; 35.3%), and the remaining factors were examined by one to two papers only, and included variables relating to student experience (14.7%), course design (11.8%), and instructor (8.8%). Whilst we believe examining student experience and course design factors is important, the papers that explored these constructs in our review were mainly singular and often very specific and not necessarily easily applicable in other contexts (e.g., perception of more homework required; Nemetz et al., 2017). Overall, the focus of the topics and research themes examined by the papers in our review are consistent with Martin et al. (2020) systematic review that examined the broader research themes in online learning research.

Lastly, we turned our attention to factors and constructs that were either not examined, or examined by only a few papers in our review. Whilst this could have included a myriad of variables, given the current global circumstances, we focused on wellbeing and mental health, and the relationship between satisfaction and other cognitive and psychosocial constructs, specifically self-efficacy, perceptions of support and preparedness for online learning. We contend that the mental health and wellbeing of online learners needs to be more fully examined and the nuances of online learning and characteristics of online leaners needs to be better understood, including their impacts on student wellbeing and mental health. We argue that the learning environment and materials need to be designed specifically for online learners, rather than translating existing materials, and should be achieved by consulting relevant frameworks and evidence-based principles for best practice online learning design and delivery. Finally, we suggested an approach to promoting greater cognitive and psychosocial experiences (such as satisfaction, preparedness, self-efficacy, perceived support), which could be achieved by the implementation of an online learning orientation at the commencement of student online learning enrolment.

Data Availability Statement

The raw data supporting the conclusion of this article will be made available by the authors, without undue reservation.

Author Contributions

JC, MM, and SM conceptualised the research question, study methodology and review protocol. JC conducted the literature review and literature search. JC and AS conducted the screening and reviewed papers for eligibility. JC extracted the data, analysed the results and prepared the manuscript for publication. MM and SM provided expert guidance throughout. All authors reviewed and contributed to the final manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Abdous, M. h. (2019). Influence of Satisfaction and Preparedness on Online Students' Feelings of Anxiety. Internet Higher Educ., 41, 34–44. doi:10.1016/j.iheduc.2019.01.001