Tongguang Li1†
Yizhou Fan2,3*†
Yuanru Tan4†Yeyu Wang4
Shaveen Singh1
Xinyu Li1
Mladen Raković1
Joep van der Graaf5
Lyn Lim6Binrui Yang4
Inge Molenaar5
Maria Bannert6Johanna Moore3Zachari Swiecki1
Yi-Shan Tsai1
David Williamson Shaffer4
Dragan Gašević1- 1Faculty of Information Technology, Monash University, Clayton, VIC, Australia
- 2Graduate School of Education, Peking University, Beijing, China
- 3School of Informatics, University of Edinburgh, Edinburgh, United Kingdom
- 4Department of Educational Psychology, University of Wisconsin-Madison, Madison, WI, United States
- 5Behavioural Science Institute, Radboud University, Nijmegen, Gelderland, Netherlands
- 6School of Social Sciences and Technology, Technical University of Munich, Munich, Bavaria, Germany
Self-regulated learning (SRL) is the ability to regulate cognitive, metacognitive, motivational, and emotional states while learning and is posited to be a strong predictor of academic success. It is therefore important to provide learners with effective instructions to promote more meaningful and effective SRL processes. One way to implement SRL instructions is through providing real-time SRL scaffolding while learners engage with a task. However, previous studies have tended to focus on fixed scaffolding rather than adaptive scaffolding that is tailored to student actions. Studies that have investigated adaptive scaffolding have not adequately distinguished between the effects of adaptive and fixed scaffolding compared to a control condition. Moreover, previous studies have tended to investigate the effects of scaffolding at the task level rather than shorter time segments—obscuring the impact of individual scaffolds on SRL processes. To address these gaps, we (a) collected trace data about student activities while working on a multi-source writing task and (b) analyzed these data using a cutting-edge learning analytic technique— ordered network analysis (ONA)—to model, visualize, and explain how learners' SRL processes changed in relation to the scaffolds. At the task level, our results suggest that learners who received adaptive scaffolding have significantly different patterns of SRL processes compared to the fixed scaffolding and control conditions. While not significantly different, our results at the task segment level suggest that adaptive scaffolding is associated with earlier engagement in SRL processes. At both the task level and task segment level, those who received adaptive scaffolding, compared to the other conditions, exhibited more task-guided learning processes such as referring to task instructions and rubrics in relation to their reading and writing. This study not only deepens our understanding of the effects of scaffolding at different levels of analysis but also demonstrates the use of a contemporary learning analytic technique for evaluating the effects of different kinds of scaffolding on learners' SRL processes.
1. Introduction
1.1. Self-regulated learning and scaffolding
Self-regulated learning (SRL) encompasses multiple cognitive, motivational, and emotional aspects of learning and has been thoroughly researched and integrated into education (Panadero, 2017). Contrary to views of value achievement, SRL emphasizes the mechanisms by which learners actively control and adjust their learning in response to varying educational contexts (Zimmerman, 1986). Various SRL models have been proposed throughout the years. One such model is the COPES model by Winne and Hadwin (1998), which describes four learning phases—task definition, goal setting and planning, enactment, and adaptation—coupled with five learning facets – condition, operation, product, evaluation, and standard (the COPES model). In this model, learners' strategies are influenced by internal and external conditions to manage learning information and, in turn, produce a learning product. This product is then evaluated against internal and external standards to facilitate learning adaptation. Similarly, Bannert (2007) proposed a comprehensive SRL framework for hypermedia learning, further subdividing SRL into cognition, metacognition, and motivation. This framework has proven particularly useful in guiding the analysis of trace data gathered in hypermedia environments to understand learners' SRL processes (Bannert et al., 2014; Fan et al., 2022a; Srivastava et al., 2022; Lim et al., 2023). Consequently, cognitive and metacognitive processes of SRL can be operationalized as patterns and sequences of learning actions, encompassing activities such as orientation—collecting information about the learning task to implement SRL strategies—and evaluation—monitoring learning progress throughout the learning process (Siadaty et al., 2016b; Saint et al., 2020; Fan et al., 2022a; Srivastava et al., 2022).
The beneficial impact of SRL on academic performance has been frequently highlighted in the literature (Greene and Azevedo, 2007, 2009; Broadbent and Poon, 2015; Broadbent, 2017; Maldonado-Mahauad et al., 2018). Moreover, compared to other intrinsic factors such as self-efficacy and motivation, the ability to use effective SRL processes in learning are considered to be more predictive of successful academic performance (Pintrich and De Groot, 1990). However, literature has shown that many students find it difficult to use SRL processes without guidance (Greene and Land, 2000; Bannert, 2009; Jovanović et al., 2017; Guo, 2022). Thus, there is a need to aid learners in their development of SRL.
As suggested by Bannert and Reimann (2012), effective SRL instruction should encompass several facets: (1) integration—contextually integrating with the specific learning domain; (2) explanation – elucidating how the suggested SRL processes can be effectively applied; and 3) training—providing ample training such that learners' can effectively use SRL processes. Scaffolding, defined as structured guidance for acquiring skills within a specific learning context until they can perform independently (Pea, 2004), may offer a viable solution to these instructional necessities.
1.2. Adaptive scaffolding
Research suggests that SRL scaffolding is positively associated with improvements in academic performance and learning processes. Specifically, SRL scaffolding has been examined from various perspectives, such as the persistence of scaffolding effects (Bannert et al., 2015; Sonnenberg and Bannert, 2019), effectiveness of technological scaffolds (Milikić et al., 2018; Lahza et al., 2022), the utility of scaffold training (Bannert, 2009), impact on group activities and group performance (Molenaar et al., 2011), influence of demographic factors (Pieger and Bannert, 2018), and association with different goal orientations (Duffy and Azevedo, 2015). Moreover, the effects of scaffolding have been examined in diverse contexts, including educational settings (Azevedo et al., 2004; Bannert, 2009; Sonnenberg and Bannert, 2016) and workplaces (Siadaty et al., 2016a,b). However, a critical characteristic of the previous studies is that the scaffolding was primarily fixed—i.e., the content of the scaffolding was the same for each student, and the design of the content was largely informed by the findings from the existing literature (Bannert, 2009; Bannert et al., 2015; Pieger and Bannert, 2018; Guo, 2022).
Wong et al. (2021), for example, divided learners into three groups — fixed question prompt, fixed recommendation prompt, and no prompt—and investigated the effect of each compared to a control condition. Their results suggested that neither type of fixed scaffolding significantly affected SRL processes, such as time-management, self-reflection, planning, and self-monitoring. The authors concluded that scaffolding designed to flexibly target specific SRL processes could be more effective. Similarly, a recent systematic literature review by Guo (2022) found that the adaptivity of scaffolding is a strong moderator of the relationship between SRL processes and learning. Thus, adaptive scaffolding, which responds to actions of individual learners and targets their specific deficiencies, may be more effective in supporting learners' SRL processes (Guo, 2022).
Although some previous studies have examined the effects of adaptive scaffolding on SRL processes, this study has several limitations. First, early studies implemented adaptive scaffolding that was only partially automated, limited their use in larger-scale learning contexts, confounding the results. For example, Azevedo et al. (2004) investigated differences in learners' SRL processes among three scaffolding conditions (adaptive scaffolding, fixed scaffolding, and no scaffolding) and found that learners who received adaptive scaffolding from human tutors more frequently regulated their learning by activating prior knowledge, utilized more diverse learning strategies, and engaged in more help-seeking behaviors. However, the researchers caveated their findings by suggesting that their positive findings could have been due to the presence of the human tutor.
More recently, the adoption of adaptive scaffolding has gathered momentum through the implementation of automated rule-based algorithms. For example, Duffy and Azevedo (2015) implemented rule-based adaptive scaffolding and found that learners who received this kind of scaffolding used more SRL strategies and spent more time viewing the learning material. Munshi et al. (2023) also designed an automated rule-based adaptive scaffolding system that tailored SRL suggestions to learners. Specifically, the adaptivity is based on learners' real-time learning behaviors, and learners were offered a procedure or a piece of knowledge that they struggled to properly apply. For instance, when a learner employed an ineffective strategy during a task, such as adding erroneous elements during knowledge construction, immediate scaffolding guided them toward self-assessing their understanding.
While these studies are a step forward in investigating the effectiveness of automated adaptive scaffolding, they were limited because they were only compared to control conditions in which no scaffolding was provided. To more completely understand the utility of automated adaptive scaffolding, it needs to be compared to both a control condition with no scaffolding and a condition in which fixed scaffolding is provided.
1.3. Segmentation analysis
In addition to the limitations described above, prior studies have focused on the effects of scaffolding using data aggregated over the entire learning task (Sitzmann et al., 2009; Azevedo et al., 2011; Molenaar, 2014; Duffy and Azevedo, 2015; Sonnenberg and Bannert, 2016; Srivastava et al., 2022). For instance, Sitzmann et al. (2009) assessed the effectiveness of adaptive scaffolding by measuring the activation of SRL processes through a questionnaire provided at the end of a learning task. In another study, Duffy and Azevedo (2015) extracted log file data and measured learners' SRL processes based on the overall frequency of several learning behaviors. In contexts where multiple scaffolds are provided throughout the learning task, this practice limits our understanding of how each specific scaffold is associated with SRL processes. In other words, this approach results in an “averaging out” of the detailed effects of scaffolding.
To address the above limitation, some have suggested segmenting the learning task for analysis to provide better insights into the effects of the individual scaffolds (Molenaar, 2014; Knight et al., 2017; Saint et al., 2022). In the context of implementing SRL scaffolds, segmentation involves dividing the overall learning task into multiple segments based on appropriately defined time windows (e.g., between scaffolding events) and then examining the SRL processes within each segment (Knight et al., 2017; Fincham et al., 2018; Saint et al., 2022). Once the overall learning task is divided into segments, researchers can identify the immediate or lagged changes in SRL processes after each scaffold to evaluate the association between specific scaffolds and SRL processes. This approach can increase our understanding of the effect of each individual scaffold (Saint et al., 2022).
Despite these potential benefits, few studies have implemented such a segmentation approach. Among them, two major limitations still remain. First, previous studies have largely focused on changes in general cognitive behaviors, overlooking changes in metacognitive or SRL processes. For example, Munshi et al. (2023) implemented six scaffolds throughout the learning task and evaluated the behavioral changes immediately after each scaffold. However, this study only examined changes in learning behaviors prompted by each scaffold (e.g., if the scaffold suggests that learners take a quiz to assess their understanding, how many learners took the quiz?) and did not address changes in SRL processes (e.g., if the scaffold suggests that learners should monitor their learning processes, to what extent do they adopt a more monitoring-oriented learning process). Second, segmentation analysis has mostly been conducted over longer learning periods – either segmenting a whole learning semester into different weeks (Mahzoon et al., 2018) or a whole week into different days (Dorodchi et al., 2018), while few studies have implemented a lower-level segmentation analysis of a single learning task. Conducting such studies at the segment-level should provide a more detailed information about the effect of specific scaffolds on SRL behaviors in addition to the general effects of scaffolding.
1.4. Research questions
In this study, we sought to address the limitations of a prior study that investigated the effect of adaptive scaffolding on SRL processes. To do so, we collected learner interactions (i.e., trace data) with an online environment and compared the SRL processes of learners in three conditions—an adaptive scaffolding condition (AS), a fixed scaffolding condition (FS), and a control condition (CN) in which no scaffolding was provided. This analysis was conducted at two levels: (i) the overall task level and (ii) the task segment level, where segments were defined according to the timing of the scaffolding. Our study was guided by the following research questions:
1. How is adaptive scaffolding, compared to fixed scaffolding and no scaffolding, associated with SRL processes when analyzed at the task level?
2. How is adaptive scaffolding, compared to fixed scaffolding and no scaffolding, associated with SRL processes when analyzed at the task segment level?
At the task level, we hypothesize that learners who receive adaptive scaffolding will be more likely to engage in high-cognitive and metacognitive SRL processes [as defined in Bannert's SRL model (Bannert, 2007)] compared to those receiving fixed or no scaffolding. Our hypothesis is grounded in existing literature (Sonnenberg and Bannert, 2015; Siadaty et al., 2016a; Wong et al., 2021). For example, Siadaty et al. (2016a) identified that those who received technological scaffolding exhibited more micro-level SRL processes within the forethought or preparatory phase of SRL. At the task segment level, we hypothesize that learners who receive adaptive scaffolding will tend to comply with the scaffolding recommendations more so than those in the other conditions. In other words, when examining differences at the task segment levels, which are defined by time periods between scaffolds, we expect the SRL processes of students in the adaptive condition to align more with the most recent scaffold they received because it was tailored to their prior behaviors. These hypotheses are grounded in the adaptive nature of scaffolding that tailors its assistance to address the unique needs and learning gaps of each individual learner (Duffy and Azevedo, 2015; Guo, 2022; Lim et al., 2023; Munshi et al., 2023).
Previous research has highlighted the necessity of recognizing the relationships between the SRL process that learners use instead of viewing these processes in isolation (Saint et al., 2022). This conceptual shift is vital because a learner might appear to be engaged in, for example, re-reading the text while also checking task instructions to understand its key aspects. Therefore, taking the co-occurrence of multiple SRL processes into consideration is critical. Moreover, previous studies have acknowledged that SRL processes are context-sensitive and sequence-specific—that is, any given SRL process can precede or follow other SRL processes, and different orders imply different meanings (Fan et al., 2023). To account for the connected and sequential nature of SRL processes, we used the network analytic technique, ordered network analysis (ONA) (Tan et al., 2023). This technique, as well as the details of our experimental design and data, is described in the sections below.
2. Materials and methods
2.1. Research context and design
2.1.1. Participants
We conducted this study with participants from a graduate level academic writing course at a large university in China. Participants were non-native English speakers. The expected learning outcome of the course was to improve the academic writing skills of first-year graduate students, for whom English was not their first language. As part of the course, participants were tasked with completing a writing assignment on a Moodle-based learning platform that integrated instrumentation tools and learning analytics-based scaffolding (a detailed description of the learning platform is summarized in Section 2.1.2). This study received approval from the ethics committee prior to the commencement of data collection.
Participants for this study were recruited from two separate offerings of the same course (the first round in November 2021 and the second round in April 2022). Consequently, the course design, task design, and learning context covered in both rounds of data collection were identical. The participants hailed from various disciplinary backgrounds and did not receive monetary incentives for their participation in the study. A total of 437 students (137 from round 1 and 300 from round 2) participated in two rounds of data collection, which resulted in a total of 161 valid participants whose data were complete and usable. The data of 276 participants were excluded because they i) did not consent to their data to be analyzed; ii) technical errors (e.g., incomplete data records or scaffolding not successful triggered); or iii) did not submit a complete writing product. A summary of participant numbers is presented in Table 1. Overall, the participants for each group were randomly assigned and similarly distributed across the study conditions: 53 learners in the control (CN) group (32 from round 1 and 21 from round 2), 57 learners in the fixed scaffolding (FS) group (28 from round 1 and 29 from round 2), and 51 learners in the adaptive scaffolding (AS) group (22 from round 1 and 29 from round 2). The sample consisted of 55 percent female and 45 percent male university students, with minority ethnic groups comprising 12 percent of the population. Their academic majors were diverse, spanning physics, engineering, ecology, and computer science, among others.
Table 1. Summary of participant information across two rounds of data collection.
2.1.2. Learning platform and task design
The learning platform used in this study was an extended version of the Moodle learning environment, where participants were asked to complete their writing assignment. As depicted in Figure 1, the platform interface consisted of several main functional zones, including the catalog and navigation zone for learners to navigate and access reading materials, the reading zone for displaying content and enabling learners to use annotation tools for note-taking or highlighting, and the essay writing zone for learners to compose their essays. The platform also incorporated various instrumentation tools, such as a search tool, timer, planner, and scaffolding tool. Such instrumentation tools have been shown useful for capturing trace data and measuring learners' SRL processes (van der Graaf et al., 2021). Lastly, scaffolding was provided to learners via a pop-up window, prompting them to regulate their SRL processes. Detailed explanations regarding the types of scaffolding deployed on the platform are presented in the Section 2.1.3.
Figure 1. Snapshot of the learning environment.
Participants in this study were asked to complete four activities related the topic of AI and education: (i) a pre-task activity that consisted of a pre-survey, a pre-knowledge test including 10 multiple choice questions about AI and education and a consent form for participation; (ii) a training activity where participants were instructed on how to use and interact with those embedded instrumentation tools (e.g., how to create annotations and tags); (iii) a two-hour main task activity that involved reading and writing, i.e., a 300–400 word essay on AI and education; and (iv) a post-task activity consisting of a post-task knowledge test including 10 multiple choice questions about AI and education, a transfer test (10 multiple choice questions about the application of AI in medicine), and a post-task survey. The training on how to interact with scaffolds in the training activity was deemed crucial as previous studies have shown that the effectiveness of scaffolds on academic outcomes improves when learners receive prior training (Bannert, 2009). For the main writing task, participants were provided with reading materials covering three topics—AI in education, differentiation in education, and scaffolding in education. Based on these materials, participants were asked to compose the essay. The main task was set with a time limit of 120 minute, and the average time spent on the main task was 113 minute. Given that the participants were non-native English speakers, and the task was conducted in English, this imposed an inherent time pressure. This pressure was further amplified by the considerable volume of text contained within the reading materials. We purposefully designed the task this way to encourage participants to adopt a selective reading approach, guided by the task instruction and/or rubric.
2.1.3. Scaffolding design
Participants in this study were allocated to one of three study conditions: the CN, FS, and AS groups. The CN participants received no scaffolding. For the FS group, scaffolds were not differentiated among different participants—everyone received the same scaffolds that were designed according to the participants' general learning needs as referenced from the relevant literature and lab studies (van der Graaf et al., 2022). Lastly, the AS group participants received personally tailored scaffolding, the adaptivity of which was determined by an algorithm implementing a rule-based approach. This algorithm included relevant suggestions in the scaffolds based on real-time analysis of SRL processes derived from the three types of trace data (see Section 2.2). The scaffolding was delivered via pop-up windows, and the scaffolding content in the AS group was adaptively adjusted according to observed SRL processes in trace data. Figure 2 illustrates an example of the differences between fixed and adaptive scaffolding pop-up windows. Fixed scaffolds were presented with all learning suggestions (e.g., check and revise your writing according to the marking rubric) that were posited to be useful to learners regardless of their SRL processes. In contrast, if the real-time analysis based on trace data revealed that learners had performed certain SRL processes (e.g.,check and revise your writing according to the marking rubric), the adaptive scaffolds would hide relevant prompts and would only suggest SRL processes that had not been observed in trace data. In cases where learners performed all three suggested SRL processes before the triggering time of the scaffolds, the scaffold windows were hidden.
Figure 2. Segmentation and scaffolding design example.
For FS and AS, five scaffolds were embedded within the main task activity (Table 2). The timing for each scaffold was fixed at 5th, 17th, 40th, 52nd, and 88th minute, which also guided the timing of our segmentation for data analysis (Figure 2). The timing and content design of the scaffolds were informed both theoretically, by aligning with the cycle of SRL (Winne, 1997), and empirically, by incorporating findings from previous studies (Lim et al., 2021; Srivastava et al., 2022). For instance, high-performing learners were shown to employ more metacognitive strategies, such as monitoring their rereading processes and selectively reading, compared to less successful learners (Lim et al., 2021). In response, the third scaffold was designed to prompt learners to monitor their reading process. Similarly, a prior lab study identified a positive association between essay revision and academic performance, as measured by the essay score (Raković et al., 2022). Accordingly, the last scaffold was created to prompt learners to review their written work before submission. Each scaffold had a distinct theme (i.e., purpose), including understanding the task (1st), starting to read (2nd), monitoring the reading process (3rd), starting to write (4th), and monitoring the writing process (5th). Within each scaffold, based on its main theme, three suggestions were included (Table 2).
Table 2. Content of the scaffolding.
2.2. Data collection and analysis
2.2.1. Data collection
Data were collected and processed according to the trace-based SRL measurement protocol (see the Electronic Appendix at this link, containing an Action Library and a Process Library, which make up the trace parser) to generate SRL process data for each participant (Siadaty et al., 2016b; Saint et al., 2020; Fan et al., 2021b, 2022a,b). This study collected three types of trace data: (1) time-stamped navigational logs (i.e., clickstreams), (2) mouse traces incorporating mouse movements and scrolls, and 3) keyboard strokes. The selection of these data types was based on Winne and Hadwin's model of SRL (Winne and Hadwin, 1998), which posits that learning conditions, learners' operations on information, and the standards they employ for self-evaluation are adjustable and typically vary over time (Winne, 2017). Therefore, considering SRL as a dynamic process, it is crucial to collect time-stamped learning trace data to monitor alterations in learning behavior. Once collected, the data were processed following the trace parser, grounded theoretically in Bannert's self-regulated hypermedia learning framework (Bannert, 2007) and adopted in preceding studies (Fan et al., 2022a; Srivastava et al., 2022). The trace parser facilitated the processing of learning trace data through the action and process libraries, mapping trace data onto SRL processes. The validity of collecting trace data and processing via the trace parser has been affirmed in previous studies using think aloud data (Fan et al., 2022b). Because not all AS learners were presented with every scaffold, for each segment, the data from learners who had not received the corresponding scaffold were excluded. For instance, if a learner has shown all the anticipated SRL processes for the upcoming fifth scaffold, the scaffolding window would be concealed from that learner. Consequently, that learner's trace data in the fifth segment would not be included in our analysis. Table 3 summarized the number of scaffolds that have been triggered for AS learners in each segment.
Table 3. Count of AS learners receiving each scaffold across two rounds of data collection.
2.2.2. Data analysis
After the trace data were coded for SRL processes via the protocol, we analyzed the data using ONA to compare the SRL processes between AS and CN groups, as well as between AS and FS groups. A detailed technical description of ONA is beyond the scope of this study. For more information, see the study by Tan et al. (2023), which describes ONA, as well as the study by Shaffer et al. (2016), which describes epistemic network analysis (ENA), the widely used learning analytic technique on which ONA is based.
Briefly, ONA builds on ENA to measure and visualize the frequency of transitions between coded events in the data. Transitions are represented as points in a low-dimensional space (i.e., embeddings) and as network diagrams, whose nodes correspond to the codes and whose edges correspond to the relative frequency of transitions between codes. We chose ONA over other common approaches such as process mining to analyze SRL processes because previous studies had demonstrated data analytic and visual advantages of ONA. For example, Fan et al. (2023) applied ONA in analyzing SRL tactics used by MOOC learners and found that ONA revealed insights about the frequency, continuity, sequentiality, and role of different learning actions in learning tactics that other techniques such as process mining failed to fully represent.
We conducted the analysis using the ONA package for the programming language R (Marquart et al., 2023). The codes described in Table 4 are represented as nodes in the resulted ONA networks. To measure transitions between codes, ONA constructs ordered networks to represent the directed and weighted co-occurrence among coded events within pre-defined segments of data. Any transitions that occur within these segments are counted and contribute to the weight of the resulting network edges.
Table 4. SRL processes that were measured in the current study were based on the coding scheme proposed by Bannert et al. (2014).
To address RQ1, we conducted ONA analysis on the task-level of SRL processes, meaning all transitions within the task were counted. To address RQ2, only transitions that occurred within the same task segment were counted. Although the length of segmented sessions that are involved in the analysis varied from 12 to 36 minute, learners were working on an independent writing task in a structured learning environment without external interruptions. As explained in a previous section, it was plausible for AS learners to not receive one or more scaffolds if they had already demonstrated all expected SRL processes before the corresponding scaffold was triggered. Consequently, in cases where AS learners did not receive a particular scaffold, their corresponding trace data for that segment was excluded from the analysis. For example, if an AS learner did not receive the third scaffold, their trace data from the third segment was removed.
In ONA networks, the directed transitions between codes are represented by tapered edges. The chevron on each edge indicates the direction in which the transition occurred most frequently. For example, a chevron on the edge for first-reading/monitoring pointed toward monitoring indicates that more individuals transitioned from first-reading to monitoring rather than the other way around. Thicker and more saturated edges indicate that the connection occurred more frequently. The size of a given node in the network is proportional to the number of occurrences of that code in the data. The larger the node is, the more times that code followed prior events. The colored circle inside a node represent self-transitions—i.e., repeating the same SRL process. A larger circle means more self-transitions.
The node placement in ONA is the same for each unit of analysis—here, individual learners—facilitating comparisons between networks. Networks can be compared by subtracting their edge weights to find the edges that are stronger in one network vs the other. Additionally, it is possible to average individual networks—by averaging their edge weights—to compare the overall transition patterns between subgroups in the data. In this study, we averaged the networks of individual learners in each experimental condition for comparison. The network edges shown in this study were scaled by multiplying the same constant with each network. This process retained the relative differences among connections and plots while making the network graphs more readable.
Finally, because ONA also creates low-dimensional embeddings for each network using dimensional reduction via singular value decomposition, statistical comparisons can be made between groups of networks. In this study, we compared the average embeddings for each condition using Mann-Whitney tests. These tests indicate whether the pattern of transitions each condition made were significantly different. These tests were conducting using the positions of the embeddings on the first and second dimensions of the embedding space. These dimensions account for the most variance among the units of analysis, and they can be interpreted using the positions of the network nodes in the space. Nodes—and the transitions they represent—that are on the extremes of the dimensions are the most influential at distinguishing between units of analysis.
3. Result
To address our research questions, we created ONA network subtractions that visually compared the mean network of the AS condition to the mean networks of the FS and CN conditions at the task and segment levels. We conducted Mann-Whitney U tests between the mean networks using their embedding values on the first and second dimensions of the ONA space. The statistical analyses carried out in this study, encompassing mean differences, p-values, effect sizes, and power calculations, are presented in Table 5. All tests were conducted using a Bonferroni correction to control for family-wise error, where each family consisted of the given level (e.g., task-level or segment 4) and four tests. Power analyses were conducted using the statistical software GPower 3.1.
Table 5. Statistical result for both task-level and segment-level models.
3.1. RQ1: Effectiveness of scaffolding at the task level
3.1.1. Comparison between AS and CN groups
The network subtraction for the AS and CN learners is shown in Figure 3. Blue edges represent more frequent transitions for the AS learners, while red edges indicate more frequent transitions for the CN learners. The network subtraction shows that CN learners made more frequent self-transitions to first-reading, suggesting a sequential reading approach during some parts of the task. They also made more frequent transitions from first-reading to monitoring and from re-reading to monitoring, suggesting that they were more engaged in monitoring their reading processes throughout the task, for instance, by checking the remaining time. Similarly, compared to AS learners, CN learners made more frequent transitions between monitoring and elaboration/organization. As elaboration/organization involves essay writing as recorded from keystrokes, this strong transition suggests a recurrent cycle of writing, time-checking, and returning to writing processes.
Figure 3. Subtracted ONA network of SRL process between the control (CN, in red) and adaptive scaffolding (AS, in blue) groups for the whole main task.
The figure shows that AS learners made more frequent transitions to orientation (self), from monitoring to orientation, and from orientation to first-reading. This suggests that they tended to use the catalog and navigation window to guide their reading and writing process, and they regularly incorporated task instructions/rubrics in their reading process. More frequent self-transitions to monitoring highlight AS learners' deeper engagement with actions such as navigating to specific pages, referring to previously created notes, searching through annotations, and checking the timer. This pattern points to a more layered, detailed approach to self-regulation among AS learners. In contrast, CN learners mainly exhibited transitions to the monitoring node but not self-transitions within it. This suggests that CN learners also engaged in monitoring their learning progress during their reading and writing processes—but this monitoring tended to occur on an as-needed basis rather than being a consistent, deeply engaged activity as seen in the AS learners. AS learners also made more frequent self-transitions to elaboration/organization, suggesting more involvement in writing and note-taking processes. Moreover, stronger transitions from first-reading to re-reading suggest that they tended not to read sequentially. Instead, they tended to revisit previously read information, indicating a deeper, more thoughtful engagement with the material. In contrast, the CN learners predominantly followed a linear, page-by-page reading strategy, suggesting less thorough engagement with the material.
The statistical test (second row of Table 5) indicates that the AS and CN learners differed significantly in their processes along the second dimension of the ONA space. CN learners tended to make more transitions that involved elaboration/organization, monitoring, and re-reading, whereas AS learners tended to make more transitions that involved orientation and first-reading. Taken together, the results suggest that CN learners typically engaged in a sequential and reactive learning approach, often monitoring their progress during reading, re-reading, and writing tasks. Conversely, AS learners demonstrated a deeper, more reflective learning process, regularly integrating task instructions/rubrics into their reading, revisiting previously read information, and partaking in a broad range of monitoring processes.
3.1.2. Comparison between AS and FS groups
The network subtraction for the AS and FS learners is shown in Figure 4. Blue edges represent more frequent transitions for the AS learners; green edges indicate more frequent transitions for the FS learners. The figure suggests that the FS learners had similar transition patterns to the CN group and that the differences between the AS and FS groups are similar to the differences described above. The one exception is that FS learners did not make more frequent transitions from first reading to monitoring—the thin and faint edge for this transition indicates that AS and FS learners had similar amounts for this transition.
Figure 4. Subtracted ONA network of SRL process between the fixed scaffolding (FS, in green) and adaptive scaffolding (AS, in blue) groups for the whole main task.
The statistical test (fourth row of Table 5) indicates that the two groups were significantly different along the second dimension of the space—FS learners tended to make more transitions that involved elaboration/organization, monitoring, and re-reading, while AS learners tended to make more transitions that involved orientation and first-reading. Similar to above findings, the results suggest that FS learners typically engaged in a sequential and reactive learning approach—monitoring their progress after re-reading and writing. AS learners, on the contrary, demonstrated a deeper, more reflective learning process, regularly integrating task instructions/rubrics into their reading, revisiting previously read information, and partaking in a broad range of monitoring processes.
3.2. RQ2: Effectiveness of scaffolding at the segment level
To further investigate how each scaffold was associated with learners' SRL processes, we segmented the data based on the timing of each scaffold. This resulted in five segments, five corresponding network subtractions between the AS and CN learners, and five corresponding network subtractions between the AS and FS learners. All visualizations are included in the Appendix at this link. To address RQ2, our analysis focuses on the segments occurring after the triggering of the fourth and fifth scaffolds. This selection assumes that later segments are likely to include more diverse SRL processes. As shown in Table 5, no comparisons between the scaffolding groups in these segments were statistically significant. However, the small-medium effect sizes for these results and low statistical power suggest that, if we had more data, these results would be significant. Thus, we still describe the network subtractions as they suggest differences that may become more salient in future studies.
3.2.1. Comparing CN and AS groups post the fourth scaffold
Figure 5 shows the network subtraction for the CN and AS learners at the end of the fourth segment (CN in red; AS in blue). For AS learners, this is the learning stage where they have just received the fourth scaffold, which depending on their prior actions, prompted them to commence writing and do so strategically by employing a range of self-regulated learning techniques.
Figure 5. Subtracted ONA network of the SRL process between the control (CN, in red) and adaptive scaffolding (AS, in blue) groups during the fourth segment of the study task.
CN learners made stronger self-transitions to first-reading and stronger transition from first-reading to monitoring. This suggests that, at this learning stage, there were still heavily engaged in continuous, page-by-page reading while frequently monitoring their reading progress by, for example, checking the remaining time. Furthermore, CN learners made stronger transitions from first-reading to elaboration/organization, indicating that they were creating notes and/or gradually starting writing based on the information they read page-by-page.
AS learners made stronger self-transitions to re-reading and elaboration/organization and stronger transitions from re-reading to elaboration/organization. These transitions suggest that upon receiving the fourth scaffold, AS learners initiated their essay writing process by referring back to previously read pages. This diligent reviewing, understanding, and organizing of their essay content aligned with the recommendations provided in the fourth scaffold, which encouraged learners to' paraphrase the main arguments that you've read and write in your own words'.
3.2.2. Comparing CN and AS groups post the fifth scaffold
Figure 6 shows the network subtraction for the CN and AS learners at the end of the fifth segment (CN in read; AS in blue). In the fourth segment, the results suggested that AS learners were engaged in a process of re-reading to gather useful information for writing, as indicated by the transition from re-reading to elaboration/organization in Figure 5. However, after receiving the fifth scaffold, the direction of transition reversed (from elaboration/organization to re-reading), suggesting that learners began to check their writing by referring back to previously read pages. Moreover, AS learners made stronger transitions from re-reading to orientation and from re-reading to monitoring, which were not predominant in the fourth segment. This suggests that, after receiving the fifth scaffold, which advised learners to “revise your writing based on the task instruction and marking rubric”, AS learners followed the suggestions and initiated a process of finalizing, revising, and refining their essay by referring back to the task instructions and rubric, as well as their notes.
Figure 6. Subtracted ONA network of the SRL process between the control (CN, in red) and adaptive scaffolding (AS, in blue) groups during the fifth segment of the study task.
In contrast, CN learners seemed to ramp up their writing activities as they neared the end of the task with time running short. This is evidenced by a stronger transition from monitoring to elaboration/organization, suggesting they became more actively engaged in writing after revisiting previously made annotations and keeping a close eye on the remaining time. It appears that the time pressure acted as a spur to their shift into more intensive monitored writing, suggesting that their activities were largely driven by time constraints as opposed to a systematic or methodical approach to learning exhibited by AS learners.
3.2.3. Comparing AS and FS groups post the fourth scaffold
Figure 7 shows the network subtraction for the FS and AS learners at the end of the fourth segment (FS in green; AS in blue). Differences between the two groups are highly similar to the differences we observed between the CN and AS learners in the fourth segment. In particular, the stronger transitions for the FS learners were from first-reading to monitoring, from first-reading to elaboration/organization, from monitoring to elaboration/organization, and from monitoring to re-reading. These transitions suggest that the FS learners were actively monitoring their reading and re-reading processes, utilizing the information gleaned from reading to inform their writing, and subsequently monitoring their writing process. On the contrary, the AS learners made more frequent transitions from re-reading to orientation, from monitoring to orientation, and from elaboration/organization to orientation. These transitions suggest that the AS learners' activities were primarily guided by an understanding of the task requirements before embarking on reading and writing tasks.
Figure 7. Subtracted ONA network of the SRL process between the fixed scaffolding (FS, in green) and adaptive scaffolding (AS, in blue) groups during the fourth segment of the study task.
3.2.4. Comparing AS and FS groups post the fifth scaffold
Figure 8 shows the network subtraction for the FS and AS learners at the end of the fifth segment (FS in green; AS in blue). AS learners made more frequent transitions from re-reading to orientation, from re-reading to monitoring, and from elaboration/organization to re-reading. Additionally, AS learners made more frequent self-transitions to re-reading. Together, these transitions suggest that AS learners were more engaged in a process of checking and refining their essay after receiving the fifth scaffold. Conversely, FS learners made more frequent transitions from first-reading to monitoring, from elaboration/organization to monitoring, and from elaboration/organization to first-reading. These transitions suggest that FS learners were engaged in writing while still reading new information (pages that were not previously read).
Figure 8. Subtracted ONA network of the SRL process between the fixed scaffolding (FS, in green) and adaptive scaffolding (AS, in blue) groups during the fifth segment of the study task.
The analysis of SRL process patterns for the fifth segment suggests parallels between AS-FS and AS-CN learner comparisons. AS learners, across both comparisons, showed a transitions-related essay refinement and revision processes—as indicated by more frequent transitions to elaboration/organization, which capture writing behaviors via keystrokes—while their counterparts, both FS and CN learners, primarily engaged in reading and writing activities, along with monitoring their task progress. This result to a consistent divergence in SRL processes between AS learners and the other two groups during the fifth segment.
4. Discussion
This study used the ONA technique to model learners' SRL processes at both task level and segmented level. By comparing learners across different scaffolding groups, a number of intriguing findings were revealed.
4.1. Research question 1: effectiveness of scaffolding at the task level
This study examined the extent to which the existence of scaffolding facilitates effective SRL processes by comparing the ONA visualizations between AS and CN learners at the overall task level, and the differences in the second dimension of ONA are found to be statistically significant. It is found that AS learners were primarily engaged in task-guided reading and writing, while CN learners were predominantly focused on reading and writing while monitoring their learning progress. Meanwhile, the ONA comparison is also conducted between AS and FS learners to examine how the adaptivity of scaffolding may affect the effectiveness of scaffolding in promoting SRL processes. The comparison of SRL processes between AS and FS learners yielded analogous visualizations to those observed between AS and CN learners, and the differences in the second dimension of ONA are found to be statistically significant. Similar to the CN learners when compared to the AS learners, FS learners exhibited a learning approach that emphasized reading and writing while intermittently monitoring their learning progress. This result parallels the ONA visualizations between AS and CN learners, further corroborating the finding that learners without adaptive scaffolding tend to involve themselves more intensely in the reading and writing processes, monitoring their learning progress as necessary.
The SRL processes exhibited by AS learners present a fitting illustration of the key components of Winne and Hadwin's COPES model (Winne and Hadwin, 1998; Winne, 2018). As indicated by the COPES model, SRL learners strategically select learning tactics based on the specific conditions of the learning environment and meticulously align these chosen tactics to fulfill task requirements (Winne and Hadwin, 1998; Winne, 2017; Fan et al., 2021a). Based on our findings at the task level, AS learners demonstrated these abilities prominently. They showcased a strategic approach in their learning processes, with a clear focus on understanding the requirements of the task before delving into their reading and writing activities. This behavior signifies a mindful and well-planned approach to learning that aligns with the theoretical tenets of a self-regulated learner. Hence, given the strategic SRL processes demonstrated by the AS learners, it can be inferred that the provision of adaptive scaffolding aligns with the promotion of strategic SRL processes.
Drawing upon the findings, it can be inferred that AS learners' deliberate choice of what to read and write, as well as their ongoing integration of task instructions, underscores a heightened level of metacognitive engagement in their learning process. As Butcher and Sumner (2011) concluded, metacognitive processes in an essay-writing task mainly involve three activities—critical analysis on existing representation (e.g., reading material and the essay constructed by the learners), active searching for relevant information from reading material, and active revising on the existing representation. In the current study, the task-guided SRL processes which were performed by AS learners were highly consistent with those metacognitive processes as posited in Butcher and Sumner (2011). AS learners' keen engagement in task orientation and reading resonates with Butcher and Sumner (2011)'s emphasis on the critical analysis of existing representations as a key metacognitive process in essay-writing. Furthermore, their pattern of revisiting previously read information, alongside reading new material, and subsequently organizing their essays aligns with the active search for relevant information and the proactive revision of existing representations, further emphasizing their metacognitive engagement in the task.
4.2. Research question 2: effectiveness of scaffolding at different learning segments
To address RQ2, this study carried out a segmentation analysis, aiming to uncover the extent to which each individual scaffold is associated with different SRL processes across different scaffolding groups. This approach is intended to offer nuanced insights into how immediate adjustments in SRL processes correspond with different scaffolding conditions.
The fourth scaffolding encouraged learners to not only start writing but also to write strategically by using various SRL tactics (e.g., as evidenced from the message in the fourth scaffolding, to review annotations, to check requirements, or to read selectively). The findings showed that the FS and CN learners primarily engaged in a more linear, reading-centric process, continuously progressing page-by-page through the reading material and extracting information for their essays. In contrast, the AS learners demonstrated a more strategic approach to writing, regularly referring back to previously read pages or annotations. In sum, compared to those who did not receive scaffolding and those who only received fixed scaffolding, the learners who received adaptive SRL scaffolding tended to engage in more strategic writing and reading processes. While the differences we observed at the segment level were not statistically significant—likely due to low power—they align with previous studies which found that scaffolding—especially adaptive scaffolding—is effective at encouraging strategic learning processes (Azevedo et al., 2004). Furthermore, given that ‘orientation' is classified as a metacognitive process according to the SRL model proposed by Bannert (2007), the current study's findings highlight that adaptive SRL scaffolding, when compared to control conditions and fixed scaffolding is more potent in fostering metacognitive learning processes (Sonnenberg and Bannert, 2016).
From the fourth segment to the fifth segment, we observed different SRL process transitions among different scaffolding groups. Specifically, in the comparison between the AS and CN groups, it was observed that those in the CN group failed to exhibit certain SRL processes, including orientation, monitoring, and re-reading. This lack of guidance may have led them to allocate an excessive amount of time to reading, consequently leaving insufficient time for writing and minimizing the opportunities to review and revise their written article. From the theoretical perspective, this can be explained by the phenomena of availability deficiency, which happens when a learner does not have the knowledge or is unaware of the available cognitive or metacognitive processes that can be used in learning (Veenman et al., 2006; Wirth, 2009). On the contrary, as AS learners exhibited a multitude of SRL processes , suggesting that the implementation of SRL scaffolding may benefit learners by making them aware of available SRL processes. Meanwhile, compared to the FS learners, AS learners still demonstrated earlier SRL processes in task-guided writing and revising their essays. This concludes that, despite the fact that learners in both AS and FS groups received scaffolding which made them being aware of available SRL processes, implementing adaptive and fixed scaffolding still led to different SRL patterns. The potential reason for this finding might be that fixed scaffolding is unable to address the unique needs of individual learners, which could lead to the noncompliance to the provided scaffolding (Guo, 2022). Hence, learners are more likely to be receptive to scaffolding when the content is tailored to meet their particular SRL needs. From a theoretical standpoint, a scenario where learners are aware of the existence of various SRL processes, yet refrain from actively utilizing them, aligns with what is typically referred to as ‘production deficiency' (a situation where a learner who is aware of certain learning tactics but failed to utilize them) (Winne, 1997; Veenman et al., 2006; Wirth, 2009). This can be observed in the ONA models in comparison between AS and FS learners, which revealed that although both AS and FS learners received scaffolding, they nevertheless demonstrated distinct SRL processes. Thus, it could be surmised that the adaptivity inherent in scaffolding might play a crucial role in mitigating the phenomenon of production deficiency, thereby enhancing the effectiveness of promoting SRL processes. Overall, our findings lend support to the premise that adaptive scaffolding is potentially the most advantageous approach to support learners' SRL, by fostering an awareness of available SRL resources and concurrently encouraging early utilization of SRL processes.
5. Conclusion
In conclusion, this study utilized the ONA technique to explore varying SRL processes among higher education students participating in a two-hour reading and writing task under three different conditions: no scaffolding, fixed scaffolding, and adaptive scaffolding. Moreover, our investigation extended to both the overall task level and segmented levels. Findings illuminated the profound influence of adaptive scaffolding in fostering learners to be more task-oriented and metacognitively engaged, thus enabling more effective and strategic reading and writing processes. Conversely, learners under fixed scaffolding and no scaffolding conditions tended to delve more into the reading and writing processes, while concurrently monitoring their progress. These findings highlight the potential benefits of incorporating adaptive scaffolding in the learning context to bolster learners' self-regulation.
5.1. Research implication and future practice
At least two research implications and one practical implication can be concluded from our study. First, this study focused on the effectiveness of scaffolding using segmentation to analyze the immediate adjustments in SRL processes after the introduction of each scaffold by segmenting the learning task according to when the scaffolding was provided. Thus, segmentation analysis allowed an in-depth and detailed analysis of each scaffold. Future research should continue using segmentation analysis to deepen the understanding of learners' SRL process and the effectiveness of scaffolding at a segmented level. Second, this study offers the first insights of using the ONA technique to model learners' SRL processes in relation to scaffolding. Compared to other widely-adopted analytical techniques in understanding learners' SRL processes and the effects of scaffolding (e.g., process mining), the ONA technique is advantageous to the extent that it can address four dimensions of learning processes at once, including frequency, continuity, sequentiality, and role of actions (i.e., the function or functions that a learning action plays, which can be different in different learning contexts), which are aspects that other predominately-used techniques alone cannot (Fan et al., 2023). Moreover, ONA's deterministic node position layout supports the creation of subtracted networks to visualize differences in SRL processes between groups of learners. As such, future studies are recommended to continue in utilizing the ONA technique to model learners' SRL. For example, a promising direction could be modeling learners' use of learning tactics (e.g., highlighting) by using ONA and exploring if different transitions among learning tactics visualized on ONA can inform different learning strategy patterns. Moreover, we successfully identified the manifestation of distinct SRL processes in learners under different scaffolding conditions. An intriguing direction for future studies would be to probe whether these SRL processes, and the extent to which they are employed, correlate with variations in learning performance. This could yield a deeper understanding of the extent to which adaptive scaffolding could promote learning outcomes. Lastly, the results of this study may also provide some suggestions for practical and instructional improvement. Because we found that adaptive scaffolding can be effective in mitigating the phenomena of availability and production deficiency by not only making learners aware of available SRL resources but also promoting early SRL actions, educational instructors can take advantage of this positive effect by embedding adaptive scaffolding within the learning task. In addition, because we found that adaptive scaffolding was more closely related to more task-guided SRL processes, future instruction can leverage this advantage to design more adaptive scaffolding to further support the development of SRL.
5.2. Limitations
Our study has several limitations. The primary limitation constraint stems from the time limit set for the written task. With a 120-minute time limit, learners might have experienced pressure to complete the task, potentially amplifying the observed differences between the AS and FS conditions. However, we must bear in mind that this was not a tightly controlled laboratory study. Instead, it took place in a classroom setting, adhering to authentic course requirements. Future studies could address this limitation by allotting more ample time for task completion, which could help minimize the potential impact on the disparities between the FS and AS conditions. Second, in order to improve the readability of the ONA visualizations, the edge weights of each network were scaled up by using a consistent multiplier. While this maintains the relative differences between the examined connections, readers may perceive the differences as larger than they were. Moreover, our study did not find significant differences among the various scaffolding groups in terms of their SRL processes at the segmented level. However, our power analyses suggest that the lack of statistical significance for some comparisons may be due to a low N. A possible explanation for the small differences observed in some comparisons is the relatively short duration for each segment. Hence, it may be difficult for learners to significantly adjust their SRL processes just within a short period of time and encouraging more effective SRL processes should be proposed and implemented as a long-term process. We might expect the differences we observed to be more prominent for a longer learning task or a study with more participants. Relatedly, our ONA analysis only explored transitions between pairs of codes. Stronger differences may be observed for longer sequences, but this approach could reduce the interpretability of the results (Swiecki et al., 2019). Third, SRL is inherently contextual (Winne, 2010), and therefore, the research findings in the current study can only be referred to other similar learning tasks (i.e., read-and-write essay-writing tasks). As such, we suggest future studies investigating SRL learning processes in different learning contexts to test the generalizability of our findings. Lastly, our study encountered some technical difficulties that led to the exclusion of some participants from the data analysis, as illustrated in Table 1. This resulted in varying attrition rates across the different scaffolding groups. We recommend that future studies aim to replicate our research to verify the repeatability of our results.
Despite these limitations, our results suggest that adaptive scaffolds are associated with positive changes in SRL processes compared to providing fixed scaffolds or no scaffolds at all. Specifically, we found that adaptive scaffolds are effective at 1) encouraging learners to adopt metacognitively task-guided SRL processes and 2) bringing awareness of and facilitating early engagement in SRL processes. This study demonstrates significant novelty in not only deepening our understanding of the effects of scaffolding at the segmented task level but also in using a contemporary network analytic technique to evaluate the effects of adaptive scaffolding on learners' SRL processes.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by the Ethical Committee in the University of Chinese Academy of Sciences approved the present research. The patients/participants provided their written informed consent to participate in this study.
Author contributions
TL wrote and revised the draft of the manuscript. YF contributed to the conception and design of the study. YT, YW, and BY performed the ordered network analysis for all research questions. SS, XL, MR, JV, LL, IM, MB, and JM designed the learning platform from which data is collected for this study. ZS, Y-ST, DS, and DG provided guidance on the overall design and data analysis processes. All authors contributed to manuscript revision, read, and approved for submitted version.
Funding
This study was funded through Call 5 of the Open Research Area (ORA) which was jointly supported by Deutsche Forschungsgemeinschaft (BA20144/10-1), Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO464.18.104), and the Economic and Social Research Council of the United Kingdom (ES/S015701/1).
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
Azevedo, R., Cromley, J. G., Moos, D. C., Greene, J. A., and Winters, F. I. (2011). Adaptive content and process scaffolding: a key to facilitating students' self-regulated learning with hypermedia. Psychol. Test Assess. Model. 53, 106.
Azevedo, R., Cromley, J. G., and Seibert, D. (2004). Does adaptive scaffolding facilitate students” ability to regulate their learning with hypermedia? Contemp. Educ. Psychol. 29, 344–370. doi: 10.1016/j.cedpsych.2003.09.002
Bannert, M. (2009). Promoting self-regulated learning through prompts. Zeitschrift für Pädagogische Psychologie 23, 139–145. doi: 10.1024/1010-0652.23.2.139
Bannert, M., and Reimann, P. (2012). Supporting self-regulated hypermedia learning through prompts. Instruct. Sci. 40, 193–211. doi: 10.1007/s11251-011-9167-4
Bannert, M., Reimann, P., and Sonnenberg, C. (2014). Process mining techniques for analysing patterns and strategies in students” self-regulated learning. Metacognit. Learn. 9, 161–185. doi: 10.1007/s11409-013-9107-6
Bannert, M., Sonnenberg, C., Mengelkamp, C., and Pieger, E. (2015). Short-and long-term effects of students” self-directed metacognitive prompts on navigation behavior and learning performance. Comput. Human Behav. 52, 293–306. doi: 10.1016/j.chb.2015.05.038
Broadbent, J. (2017). Comparing online and blended learner's self-regulated learning strategies and academic performance. Intern. Higher Educ. 33, 24–32. doi: 10.1016/j.iheduc.2017.01.004
Broadbent, J., and Poon, W. L. (2015). Self-regulated learning strategies & academic achievement in online higher education learning environments: a systematic review. Intern. Higher Educ. 27, 1–13. doi: 10.1016/j.iheduc.2015.04.007
Butcher, K. R., and Sumner, T. (2011). Self-directed learning and the sensemaking paradox. Human-Comp. Interact. 26, 123–159. doi: 10.1080/07370024.2011.556552
Dorodchi, M., Benedict, A., Desai, D., Mahzoon, M. J., MacNeil, S., and Dehbozorgi, N. (2018). “Design and implementation of an activity-based introductory computer science course (cs1) with periodic reflections validated by learning analytics,”? in 2018 IEEE Frontiers in Education Conference (FIE). San Jose, CA: IEEE, 1–8. doi: 10.1109/FIE.2018.8659196
Duffy, M. C., and Azevedo, R. (2015). Motivation matters: Interactions between achievement goals and agent scaffolding for self-regulated learning within an intelligent tutoring system. Comput. Human Behav. 52, 338–348. doi: 10.1016/j.chb.2015.05.041
Fan, Y., Lim, L., van der Graaf, J., Kilgour, J., Raković, M., Moore, J., et al. (2022a). Improving the measurement of self-regulated learning using multi-channel data. Metacogn. Learn. 22, 1–31. doi: 10.1007/s11409-022-09304-z
Fan, Y., Matcha, W., Uzir, N. A., Wang, Q., and Gašević, D. (2021a). Learning analytics to reveal links between learning design and self-regulated learning. Int. J. Artif. Intell. Educ. 31, 980–1021. doi: 10.1007/s40593-021-00249-z
Fan, Y., Saint, J., Singh, S., Jovanovic, J., and Gac, D. (2021b). “A learning analytic approach to unveiling self-regulatory processes in learning tactics,”? in LAK21: 11th International Learning Analytics and Knowledge Conference (New York, NY: Association for Computing Machinery (ACM)), 184–195. doi: 10.1145/3448139.3448211
Fan, Y., Tan, Y., Raković, M., Wang, Y., Cai, Z., Shaffer, D. W., et al. (2023). Dissecting learning tactics in mooc using ordered network analysis. J. Comput. Assist. Learn. 39, 154–166. doi: 10.1111/jcal.12735
Fan, Y., van der Graaf, J., Lim, L., Raković, M., Singh, S., Kilgour, J., et al. (2022b). Towards investigating the validity of measurement of self-regulated learning based on trace data. Metacogn. Learn 17, 949–987. doi: 10.1007/s11409-022-09291-1
Fincham, E., Gašević, D., Jovanović, J., and Pardo, A. (2018). From study tactics to learning strategies: An analytical method for extracting interpretable representations. IEEE Trans. Learn. Technol 12, 59–72. doi: 10.1109/TLT.2018.2823317
Greene, B. A., and Land, S. M. (2000). A qualitative analysis of scaffolding use in a resource-based learning environment involving the world wide web. J. Educ. Comput. Res. 23, 151–179. doi: 10.2190/1GUB-8UE9-NW80-CQAD
Greene, J. A., and Azevedo, R. (2007). A theoretical review of winne and hadwins model of self-regulated learning: new perspectives and directions. Rev. Educ. Res. 77, 334–372. doi: 10.3102/003465430303953
Greene, J. A., and Azevedo, R. (2009). A macro-level analysis of srl processes and their relations to the acquisition of a sophisticated mental model of a complex system. Contemp. Educ. Psychol. 34, 18–29. doi: 10.1016/j.cedpsych.2008.05.006
Guo, L. (2022). Using metacognitive prompts to enhance self-regulated learning and learning outcomes: a meta-analysis of experimental studies in computer-based learning environments. J. Comput. Assist. Learn. 38, 811–832. doi: 10.1111/jcal.12650
Jovanović, J., Gašević, D., Dawson, S., Pardo, A., Mirriahi, N., et al. (2017). Learning analytics to unveil learning strategies in a flipped classroom. Inter. Higher Educ. 33, 74–85. doi: 10.1016/j.iheduc.2017.02.001
Knight, S., Wise, A. F., and Chen, B. (2017). Time for change: why learning analytics needs temporal analysis. J. Learn. Anal. 4, 7–17. doi: 10.18608/jla.2017.43.2
Lahza, H., Khosravi, H., Demartini, G., and Gašević, D. (2022). “Effects of technological interventions for self-regulation: A control experiment in learnersourcing,”? in LAK22: 12th International Learning Analytics and Knowledge Conference, 542–548. doi: 10.1145/3506860.3506911
Lim, L., Bannert, M., van der Graaf, J., Molenaar, I., Fan, Y., Kilgour, J., et al. (2021). Temporal assessment of self-regulated learning by mining students think-aloud protocols. Front. Psychol. 12, 749749. doi: 10.3389/fpsyg.2021.749749
Lim, L., Bannert, M., van der Graaf, J., Singh, S., Fan, Y., Surendrannair, S., et al. (2023). Effects of real-time analytics-based personalized scaffolds on students self-regulated learning. Comput. Human Behav. 139, 107547. doi: 10.1016/j.chb.2022.107547
Mahzoon, M. J., Maher, M. L., Eltayeby, O., Dou, W., and Grace, K. (2018). A sequence data model for analyzing temporal patterns of student data. J. Learn. Anal. 5, 55–74. doi: 10.18608/jla.2018.51.5
Maldonado-Mahauad, J., Pérez-Sanagustin, M., Moreno-Marcos, P. M., Alario-Hoyos, C., Munoz-Merino, P. J., and Delgado-Kloos, C. (2018). “Predicting learners” success in a self-paced mooc through sequence patterns of self-regulated learning,”? in Lifelong Technology-Enhanced Learning: 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK. Leeds: Springer, 355–369. doi: 10.1007/978-3-319-98572-5_27
Marquart, C. L., Tan, Y., Cai, Z., and Shaffer, D. W. (2023). ona: Ordered Network Analysis. R package version 0.1.1.
Milikić, N., Gašević, D., and Jovanović, J. (2018). Measuring effects of technology-enabled mirroring scaffolds on self-regulated learning. IEEE Trans. Learn. Technol 13, 150–163. doi: 10.1109/TLT.2018.2885743
Molenaar, I. (2014). Advances in temporal analysis in learning and instruction. Front. Learn. Res. 2, 15–24.
Molenaar, I., Chiu, M. M., Sleegers, P., and van Boxtel, C. (2011). Scaffolding of small groups” metacognitive activities with an avatar. Int. J. Comp.-Supported Collaborative Learn. 6, 601–624. doi: 10.1007/s11412-011-9130-z
Munshi, A., Biswas, G., Baker, R., Ocumpaugh, J., Hutt, S., and Paquette, L. (2023). Analysing adaptive scaffolds that help students develop self-regulated learning behaviours. J. Comput. Assist. Learn. 39, 351–368. doi: 10.1111/jcal.12761
Panadero, E. (2017). A review of self-regulated learning: Six models and four directions for research. Front. Psychol. 8, 422. doi: 10.3389/fpsyg.2017.00422
Pea, R. D. (2004). The social and technological dimensions of scaffolding and related theoretical concepts for learning, education, and human activity. J. Learn. Sci. 13, 423–451. doi: 10.1207/s15327809jls1303_6
Pieger, E., and Bannert, M. (2018). Differential effects of students' self-directed metacognitive prompts. Comput. Human Behav. 86, 165–173. doi: 10.1016/j.chb.2018.04.022
Pintrich, P. R., and De Groot, E. V. (1990). Motivational and self-regulated learning components of classroom academic performance. J. Educ. Psychol. 82, 33. doi: 10.1037/0022-0663.82.1.33
Raković, M., Fan, Y., van der Graaf, J., Singh, S., Kilgour, J., Lim, L., et al. (2022). “Using learner trace data to understand metacognitive processes in writing from multiple sources,”? in LAK22: 12th International Learning Analytics and Knowledge Conference (New York, NY: Association for Computing Machinery (ACM)), 130–141. doi: 10.1145/3506860.3506876
Saint, J., Fan, Y., Gašević, D., and Pardo, A. (2022). Temporally-focused analytics of self-regulated learning: a systematic review of literature. Comp. Educ. 3, 100060. doi: 10.1016/j.caeai.2022.100060
Saint, J., Whitelock-Wainwright, A., Gašević, D., and Pardo, A. (2020). Trace-srl: A framework for analysis of microlevel processes of self-regulated learning from trace data. IEEE Trans. Learn. Technol. 13, 861–877. doi: 10.1109/TLT.2020.3027496
Shaffer, D. W., Collier, W., and Ruis, A. R. (2016). A tutorial on epistemic network analysis: Analyzing the structure of connections in cognitive, social, and interaction data. J. Learn. Anal. 3, 9–45. doi: 10.18608/jla.2016.33.3
Siadaty, M., Gašević, D., and Hatala, M. (2016a). Measuring the impact of technological scaffolding interventions on micro-level processes of self-regulated workplace learning. Comput. Human Behav. 59, 469–482. doi: 10.1016/j.chb.2016.02.025
Siadaty, M., Gašević, D., and Hatala, M. (2016b). Trace-based micro-analytic measurement of self-regulated learning processes. J. Learn. Anal. 3, 183–214. doi: 10.18608/jla.2016.31.11
Sitzmann, T., Bell, B. S., Kraiger, K., and Kanar, A. M. (2009). A multilevel analysis of the effect of prompting self-regulation in technology-delivered instruction. Pers. Psychol. 62, 697–734. doi: 10.1111/j.1744-6570.2009.01155.x
Sonnenberg, C., and Bannert, M. (2015). Discovering the effects of metacognitive prompts on the sequential structure of srl-processes using process mining techniques. J. Learn. Anal. 2, 72–100. doi: 10.18608/jla.2015.21.5
Sonnenberg, C., and Bannert, M. (2016). Evaluating the impact of instructional support using data mining and process mining: A micro-level analysis of the effectiveness of metacognitive prompts. J. Educ. Data Mini. 8, 51–83. doi: 10.5281/zenodo.3554597
Sonnenberg, C., and Bannert, M. (2019). Using process mining to examine the sustainability of instructional support: how stable are the effects of metacognitive prompting on self-regulatory behavior? Comput. Human Behav. 96, 259–272. doi: 10.1016/j.chb.2018.06.003
Srivastava, N., Fan, Y., Raković, M., Singh, S., Jovanovic, J., van der Graaf, J., et al. (2022). “Effects of internal and external conditions on strategies of self-regulated learning: a learning analytics study,”? in LAK22: 12th International Learning Analytics and Knowledge Conference (New York, NY: Association for Computing Machinery (ACM)), 392–403.
Swiecki, Z., Lian, Z., Ruis, A., and Shaffer, D. W. (2019). Does Order Matter? Investigating Sequential and Cotemporal Models of Collaboration. Montral, QC: International Society of the Learning Sciences.
Tan, Y., Ruis, A. R., Marquart, C., Cai, Z., Knowles, M. A., and Shaffer, D. W. (2023). Ordered network analysis,? in Advances in Quantitative Ethnography: 4th International Conference. Copenhagen, Denmark: Springer, 101–116. doi: 10.1007/978-3-031-31726-2_8
van der Graaf, J., Lim, L., Fan, Y., Kilgour, J., Moore, J., Gašević, D., et al. (2022). The dynamics between self-regulated learning and learning outcomes: an exploratory approach and implications. Metacogn. Learn. 9, 1–27. doi: 10.1007/s11409-022-09308-9
van der Graaf, J., Lim, L., Fan, Y., Kilgour, J., Moore, J., Bannert, M., et al. (2021). “Do instrumentation tools capture self-regulated learning?, in LAK21: 11th International Learning Analytics and Knowledge Conference (New York, NY: Association for Computing Machinery (ACM)), 438–448. doi: 10.1145/3448139.3448181
Veenman, M. V., Van Hout-Wolters, B. H., and Afflerbach, P. (2006). Metacogn. Learn: Conceptual and methodological considerations. Metacogn. Learn. 1, 3–14. doi: 10.1007/s11409-006-6893-0
Winne, P., and Hadwin, A. (1998). “Studying as self-regulated learning” in Metacognition in Educational Theory and Practice, Dj Hacker, J. Dunlosky, A. C. (eds.). Mahwah: Lawrence Erlbaum Associates Publishers 277–304.
Winne, P. H. (1997). Experimenting to bootstrap self-regulated learning. J. Educ. Psychol. 89, 397. doi: 10.1037/0022-0663.89.3.397
Winne, P. H. (2010). Improving measurement of self-regulated learning. Educ. Psychol. 45, 267–276. doi: 10.1080/00461520.2010.517150
Winne, P. H. (2017). “Learning analytics for self-regulated learning,” in Handbook of Learning Analytics (Society for learning Analytics Research (SoLAR)), 241–249.
Winne, P. H. (2018). Theorizing and researching levels of processing in self-regulated learning. Br. J. Educ. Psychol. 88, 9–20. doi: 10.1111/bjep.12173
Wirth, J. (2009). Promoting self-regulated learning through prompts. Zeitschrift für Pädagogische Psychologie 23, 91–94. doi: 10.1024/1010-0652.23.2.91
Wong, J., Baars, M., de Koning, B. B., and Paas, F. (2021). Examining the use of prompts to facilitate self-regulated learning in massive open online courses. Comput. Human Behav. 115, 106596. doi: 10.1016/j.chb.2020.106596
Keywords: self-regulated learning, SRL process, SRL scaffolding, ordered network analysis, segmentation analysis
Citation: Li T, Fan Y, Tan Y, Wang Y, Singh S, Li X, Raković M, van der Graaf J, Lim L, Yang B, Molenaar I, Bannert M, Moore J, Swiecki Z, Tsai Y-S, Shaffer DW and Gašević D (2023) Analytics of self-regulated learning scaffolding: effects on learning processes. Front. Psychol. 14:1206696. doi: 10.3389/fpsyg.2023.1206696
Received: 16 April 2023; Accepted: 11 July 2023;
Published: 03 August 2023.
Edited by:
Valentin Riemer, University of Ulm, GermanyReviewed by:
Yang Jiang, Educational Testing Service, United StatesDirk Tempelaar, Maastricht University, Netherlands
Copyright © 2023 Li, Fan, Tan, Wang, Singh, Li, Raković, van der Graaf, Lim, Yang, Molenaar, Bannert, Moore, Swiecki, Tsai, Shaffer and Gašević. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Yizhou Fan, yizhou.fan@ed.ac.uk
†These authors share first authorship