Alexandra List
Patricia A. Alexander- 1Department of Educational Psychology, Counseling, and Special Education, The Pennsylvania State University, State College, PA, United States
- 2Department of Human Development and Quantitative Methodology, University of Maryland, College Park, MD, United States
This study examined undergraduates’ strategy use when learning about a complex and controversial topic (i.e., mass incarceration in the United States) based on information presented across multiple texts. Guided by the Integrated Framework of Learning from Multiple Texts, this study directed students to engage in one of three forms of strategy use while learning from multiple texts. In particular, students were asked to identify relevant and important information in texts (i.e., intratextual processing), to form relations or connections across texts (i.e., intertextual processing), or to identify easy or difficult to understand information in texts (i.e., metacognitive processing). In addition to receiving task instructions directing them to engage in these modes of processing, students were also provided with a highlighting tool to scaffold their strategy use (e.g., by allowing important and relevant information to be marked in green, in the intratextual processing condition). This highlighting tool also enabled researchers to collect log data of students’ manifest strategy use. Students were found to demonstrate differential patterns of strategy use in accordance with their assigned processing conditions. Moreover, students’ use of strategies directed toward multiple texts was found to predict multiple text task performance.
Introduction
This study examined whether prompting students to engage in different types of processing when learning from multiple texts impacted their strategy use and task performance. The multiple text task used in this study required learners to understand and write about a complex and controversial topic (i.e., mass incarceration in the United States) based on information presented across multiple texts. This task was designed to represent the types of academic assignments that undergraduate students are frequently asked to complete (Hendley, 2012; Datig, 2016; Weston-Sementelli et al., 2018). It was also devised to address the type of social issues, discussed in the popular press, that students may be driven to research or to learn more about on their own (Bazelon, 2019; Uhrmacher, 2020).
Similar tasks have been employed in prior research examining students’ learning from multiple texts (e.g., Wiley et al., 2009; Barzilai and Weinstock, 2015). This body of research has established that students need a variety of sophisticated skills and strategies to learn about complex and controversial topics from multiple texts. Such sophisticated strategies include being able to identify relevant content in texts (Potocki et al., 2017; McCrudden, 2018); synthesize and connect information introduced across disparate texts (Kobayashi, 2009; List et al., 2019b); and, make metacognitive judgments regarding comprehension quality and adequacy of task performance (e.g., Stadtler and Bromme, 2008; Wang and List, 2019).
Despite the need for students to demonstrate sophisticated and erudite strategy use when learning from multiple texts, relatively few studies have examined the nature of such strategy use during task performance (Wolfe and Goldman, 2005; Anmarkrud et al., 2014; Du and List, 2020). Those studies that are the exception have generally employed real-time methods like think-alouds to capture students’ strategic processing. In those studies, students’ more sophisticated text processing (e.g., use of evaluative and cross-textual linking strategies) has been associated with better task performance, particularly as assessed through writing (Goldman et al., 2012; Anmarkrud et al., 2014). This positive association notwithstanding, students’ strategy use when learning from multiple texts has rarely been experimentally manipulated. In this study, by comparison, we expressly altered the directions that students received regarding how to process an assigned set of four multiple texts, in order to examine the influence of such task manipulations on students’ strategy use and task performance. Specifically, we examined whether prompting students to (a) attend to relevant or important information in individual texts (i.e., intratextual processing); (b) relate or connect content across texts (i.e., intertextual processing); or (c) monitor the ease or difficulty of their understanding (i.e., metacognitive processing) impacted their demonstrated strategy use and task performance.
Integrated Framework of Learning From Multiple Texts
The design of this study was guided by the Integrated Framework of Learning from Multiple Texts (IF-MT; List and Alexander, 2019). Synthesizing much of the literature on students’ multiple text learning, the IF-MT depicts such learning as unfolding over three stages, preparation, execution, and production. In the preparation stage, students analyze the task guiding multiple text use based on its various objective characteristics (e.g., the topic or domain) and their subjective perceptions of these characteristics (e.g., students’ topic or domain interest). Students’ task analysis and subjective perceptions result in their adoption of a default stance or guiding orientation toward task completion. In adopting a default stance, students make decisions about their investment in and strategic approach toward task completion.
In the execution stage of the IF-MT, students engage in strategic processing consistent with the default stances they adopted in the preparation stage. Three categories of strategic processing characterize students’ interactions with multiple texts and predict students’ accomplishment of various learning outcomes. These three modes of strategic processing are behavioral, cognitive, and metacognitive. Behavioral strategies reflect students’ observable interactions with multiple texts, including text access and navigation. Cognitive strategies are defined as the internal operations or mental processes that students perform during reading. Finally, metacognitive strategies represent students’ efforts to monitor and regulate their own understanding during reading (i.e., comprehension monitoring), to appraise text quality (i.e., epistemic monitoring), and to judge the extent of their learning (i.e., monitoring of task outcomes).
In the present study, we targeted students’ cognitive and metacognitive strategy use. The cognitive strategies we examined included those involved in intratextual comprehension (i.e., understanding individual texts) and in intertextual integration (i.e., cross-textual linking). Intratextual strategies, like prior knowledge activation and elaboration, reflect the cognitive processes that students intentionally use before, during, and after reading. These strategies have been found to support single text comprehension in prior work (Woloshyn et al., 1994; McNamara, 2004; Dinsmore and Alexander, 2012; Parkinson and Dinsmore, 2018). By comparison, intertextual strategies, including organization and corroboration, involve students’ formation of cross-textual links in the service of developing an integrated and coherent representation of a central topic or issue discussed across multiple texts (Kobayashi, 2009; Bråten and Strømsø, 2011; Hagen et al., 2014). Behavioral strategy use was not examined in this study as we were focused on capturing the covert (i.e., cognitive and metacognitive) processes that students engaged during multiple text use. These have been examined to a more limited extent in prior work, as compared to behavioral strategies that are easier to capture, for instance, via students’ notes or log data (Hagen et al., 2014; List and Alexander, 2017). Although students’ behavioral strategy use was not the target of this investigation, students were nevertheless asked to use behavioral strategies (i.e., highlighting) to support the mode of cognitive or metacognitive processing that they were instructed to deploy when learning from multiple texts.
Although a variety of strategies are identified as important in the IF-MT, their differential impact on multiple text learning has yet to be established. Therefore, in this study, we set out to determine the extent to which providing students with various processing directives in the preparation stage of the IF-MT influenced their strategy use in the execution stage and, ultimately, their formation of various cognitive (mental) and external (i.e., tangible) outcomes in the production stage. Cognitive outcomes are defined in the IF-MT as the mental models or representations of complex topics or issues that students construct based on information introduced across multiple texts. Tangible or external outcomes reflect the physical products (e.g., written responses) that students compose based on the cognitive outcomes they generate. In part, these external outcomes are what allow learning from multiple texts to be evaluated and assessed. External outcomes are considered separately from their underlying cognitive bases in the production stage of the IF-MT. This is done to underscore that the external products that students develop are typically only selective or stylized representations of all the information that students may internalize and cognitively represent when processing multiple texts. For instance, when students write a summary based on multiple texts, they may only include main ideas in the external responses that they compose, while retaining additional salient details in their cognitive representations.
In this investigation, we assessed students’ cognitive representations of multiple texts and their correspondent external outcomes. In particular, two types of outcome measures were employed. First, we analyzed the quality of students’ research reports. Research reports were the external outcome that students were asked to produce based on the multiple texts they processed in this study. Second, we examined the quality of students’ responses to open-ended questions designed to probe the richness of students’ cognitive representations of the overlapping topics or issues discussed across four carefully crafted texts. By assessing these two outcome measures, in conjunction with the mode of text processing that students were asked to adopt (i.e., intratextual, intertextual, or metacognitive), we sought to achieve a deeper understanding of the relation between students’ manifest strategic processing and task performance when learning from multiple texts.
That is, the design of this study mirrored each phased of the IF-MT. In the preparation phase, students were instructed to engage in one of three modes of multiple text processing (i.e., intratextual, intertextual, or metacognition) while consulting multiple texts to compose a research report. It was our expectation that these strategy use directives would be incorporated into students’ task analysis and planning of task completion. In the execution phase, we expected students to engage in the intratextual, intertextual, or metacognitive processing of multiple texts, in accordance with their assigned task condition. We supported such strategy use by providing students with a highlighting tool, customized to their assigned condition. Finally, in the production phase, we assessed both students’ cognitive representations of the information introduced across texts (i.e., via the open-ended questions) and students’ manifest performance on the assigned multiple text task (i.e., composing a research report).
Strategy Use When Learning From Multiple Texts
The strategic processes that students engaged during the execution stage of the IF-MT were of particular interest in this study. Indeed, there has been a substantive and growing body of work documenting the various strategies that students may use when learning from multiple texts (e.g., Wineburg, 1991; Daher and Kiewra, 2016; Brante and Strømsø, 2018). In a recent taxonomy, the Comprehensive Strategy Framework (CSF), List (2020) suggests that these strategies may be differentiated according to two primary dimensions. That is, the strategies that students use when learning from multiple texts vary in their functions (i.e., goals for strategy deployment) and in their referents (i.e., informational foci).
Based in Cho et al.’s (2018) work, three possible strategy functions are identified in the CSF. According to Cho et al. (2018), when learning about complex topics using multiple texts, students engage in constructive-integrative, critical-analytic, and metacognitive-reflective processing. Constructive-integrative processing refers to students’ efforts to make meaning or to develop a single, coherent cognitive representation of information presented across multiple texts. Critical-analytic processing encompasses students’ efforts to establish source trustworthiness or information veracity during multiple text learning. Finally, metacognitive-reflective processing captures students’ efforts to deploy, monitor, and regulate their use of constructive-integrative and critical-analytic processing strategies, including metacognition and self-regulation.
List (2020) points out that each of the aforementioned functions may be directed toward at least three possible strategy referents or informational targets: (a) a single text, (b) multiple texts, or (c) learners’ prior knowledge and beliefs. For instance, when engaging in constructive-integrative processing during multiple text reading, students may elaborate the information introduced in a single text based on information included in that same text (i.e., single text referent), information explained in another text (i.e., multiple text referent), or based on their own experiences (i.e., prior knowledge and beliefs referent). In this case, students’ constructive-integrative processing may be thought of as both uniform in function and distinct in referent, with students’ efforts at meaning-making extended across single texts, multiple texts, and their own prior knowledge. Crossing the three strategy functions identified by Cho et al. (2018) with the three strategy referents from the IF-MT, List (2020) charts the landscape of students’ potential strategy use when learning from multiple texts.
This function by referent mapping of strategy engagement has been observed in prior work. For instance, in a think-aloud study, Anmarkrud et al. (2014) investigated students’ use of linking strategies (i.e., strategies connecting two or more texts) as a distinct strategy referent. They found linking to be disproportionately distributed across students’ constructive-integrative (47.1%), critical-analytic (36.3%), and metacognitive-reflective (16.7%) processing of multiple texts. Similarly, Wolfe and Goldman (2005) found that the elaborative strategies that students reported using differed according to whether these were associated with learners’ referencing of a single text, of multiple texts, or of their earlier generated think-aloud comments. In this study, we similarly investigate differences in students’ strategy use across the three different types of strategy referents identified in the CSF (List, 2020). In doing so, we build on prior work that has only documented the nature of students’ strategy use by explicitly directing students to engage in different modes of strategic processing when learning from multiple texts. Thus, in this study, we explicitly directed students to engage in intratextual, intertextual, or metacognitive processing during a multiple text task.
Task Assignment When Learning From Multiple Texts
Task instructions, or the directives that students receive prior to reading, have repeatedly been found to play an important role in students’ learning from multiple texts (Le Bigot and Rouet, 2007; Gil et al., 2010a, b; McCarthy and Goldman, 2015; List et al., 2019a). Task instructions specify the types of external products that students may be asked to produce from multiple texts and direct students’ attention and strategy engagement toward particular content in texts (McCrudden and Schraw, 2007; McCrudden et al., 2011). Nevertheless, to date, only the first of these task instruction functions has been well-investigated. That is, students asked to produce different external products based on multiple texts have been found to differ in the quality of their performance, with this association explained by features of students’ strategy use (Wiley and Voss, 1999; Cerdán and Vidal-Abarca, 2008; Stadtler and Bromme, 2008; Kobayashi, 2009; McCrudden and Sparks, 2014). In this study, rather than varying the types of task products that students were asked to produce we instead varied the modes of processing that students were asked to engage during multiple text use. We did this by directing students to engage in intratextual, intertextual, or metacognitive strategy use when learning from multiple texts and by facilitating such strategy use by asking students to highlight and explain information in texts that were consistent with their assigned task condition. For instance, students in the intertextual processing strategy condition received task instructions and a highlighting tool, with three different color options, to aid them in identifying similar, different, and otherwise related information introduced across four multiple texts. Students in the metacognitive processing strategy condition were instructed to identify content that was easy or difficult to understand and were provided with two highlighter options to aid them in doing so (i.e., a red highlighter to mark difficult to understand information and a green highlighter to mark easy to understand information). For this study, we posed the following research questions:
(1) Are there differences in strategy use among students prompted to engage in the intratextual, intertextual, or metacognitive processing of multiple texts?
We expected the nature of students’ strategy use across conditions to differ in both function and referent. In particular, we expect students in the intratextual condition to exhibit the greatest degree of constructive-integrative strategy use directed toward single texts. We expected students in the intertextual condition to manifest the greatest degree of constructive-integrative strategy use directed toward multiple texts. Finally, we expected students in the metacognitive processing condition to exhibit the most metacognitive-reflective strategy use, across referents.
(2) Are there differences in writing performance, citation use, and responses to open-ended integration questions among students prompted to engage in the intratextual, intertextual, or metacognitive processing of multiple texts?
Due to the important role that integration or cross-textual connection formation plays in students’ learning from multiple texts (Britt et al., 1999; Perfetti et al., 1999), we expected students assigned to the intertextual processing condition to demonstrate the greatest degree of task performance. Then, based on Stadtler and Bromme’s (2008) work, we expected students in the metacognitive processing condition to outperform students in the intratextual condition, across the outcome measures examined.
(3) Is there an association between students’ multiple text strategy use and their external outcomes (i.e., research report writing quality, citation use, and responses to open-ended integration questions) when learning from multiple texts?
We expected students’ greater engagement in strategy use directed toward multiple texts to be associated with research report writing quality and with open-ended integration performance.
Materials and Methods
Participants
Participants were 71 undergraduate students enrolled at a large university in the Mid-Atlantic region of the United States (age: M = 20.59, SD = 1.98; female: 57.89%, n = 33; male: 42.11%, n = 24). Students participating identified their race/ethnicity as White (42.59%, n = 23), Black/African-American (7.41%, n = 4), Asian (29.63%; n = 16), Hispanic/Latino (12.96%, n = 7), or as representing more than one racial or ethnic group (7.41% n = 4). Students represented a variety of class standings—freshman: 8.77% (n = 5); sophomores: 36.84% (n = 21); juniors: 21.05% (n = 12); seniors: 33.33% (n = 19). Ten students (14.08% of the sample) only completed the individual difference measures and did not complete the multiple text task, reducing our analysis sample to 61.
Procedures
This study proceeded in three main phases. First, students were asked to complete assessments of prior topic knowledge and topic interest. Then, students were asked to complete a multiple text task, including a reading and a writing phase. Students were randomly assigned to intratextual, intertextual, or metacognitive processing task conditions, as they did so. Finally, students were asked to respond to a post-task assessment. Students completed the study online, via the Qualtrics platform, at a time and location of their choosing. The study took students approximately 1 h to complete.
Overview of Study Measures
Three types of measures were collected in this study. First, individual difference measures were collected to use as controls. Second, process measures of students’ multiple text use were gathered. These data were collected using log indicators, namely students’ text highlights and associated explanations. Third, performance data were collected. Process and performance data were used to answer the focal research questions in this study.
Individual Difference Measures
Two individual difference factors, found to be associated with multiple text task performance in prior work, were assessed in this investigation (i.e., prior topic knowledge and topic interest, Bråten et al., 2014).
Prior Topic Knowledge
Prior topic knowledge was assessed via a term identification measure. In particular, students were asked to define eight terms, relevant to the task (i.e., mass incarceration) and taken directly from the experimental texts (i.e., cash bail, mandatory minimums, mass incarceration, misdemeanor, over-policing, parole, probation, and recidivism). Students were instructed to write N/A if they were unfamiliar with a particular term. Students’ definitions for each term were scored as 1 (correct) or 0 (incorrect or N/A). For instance, one student defined probation as: “a system set up to prevent incarceration and allow for some giving back to the community from the offender,” which received a score of 1. Another student wrote that probation was: “the conditional release of a convict into society,” which was scored as a 0, since this was the definition of parole. Cohen’s kappa inter-rater agreement for students’ prior knowledge was 0.90.
Topic Interest
Students were asked to rate their interest in each of five topics, related to mass incarceration (i.e., criminal justice, criminology, public policy, social issues, social justice). Students’ interest in each topic was rated on a seven-point scale from not at all interested to very interested. Cronbach’s alpha reliability for the five-item measure was 0.84. Students’ mean interest ratings were 4.16 (SD = 1.24), indicating that, on average, students were at least moderately interested in this study.
Multiple Text Task
Topic
Mass incarceration in the United States was selected as the topic of this study for several reasons. First, it represented a complex and multifaceted topic. Understanding mass incarceration required students to make sense of a number of difficult and interrelated concepts, including cash bail and mandatory minimums in sentencing. Second, mass incarceration was a topic about which conflicting, but comparably valid, points of view could be introduced. For example, while some experts consider parole to be an effective antidote to mass incarceration, others contend that parole increases recidivism by prolonging individuals’ contact with the carceral system. Third, mass incarceration constitutes an important societal issue addressed with some frequency in the popular press (Bazelon, 2019; Uhrmacher, 2020). Therefore, we expected students in this study to have some familiarity with this topic. Finally, mass incarceration was a topic about which much data were publicly available and readily accessible, facilitating our construction of texts that included relevant statistical information in support of various issues introduced.
Texts
Four texts were constructed for the purpose of this study. These were developed to be parallel in structure and overlapping in content, such that key issues were commonly introduced across texts, albeit from different perspectives. Structurally, each text consisted of three paragraphs, each introducing a key issue related to mass incarceration in the United States. Each key issue was supported by one piece of relevant statistical information, attributed to an embedded source cited in-text, such that there were three pieces of statistical data, and associated sources, included in each text. In terms of content, the texts were designed to include some complementary information (i.e., that agreed with information in another text) and some conflicting information (i.e., that disagreed with information in another text). For instance, two texts agreed that the United States incarcerated more individuals and a greater proportion of individuals than any other country in the world, while two texts expressed conflicting views. One of those conflicting texts suggested that the War on Drugs was responsible for increases in mass incarceration in the United States, whereas the other contended that only a minority of criminal convictions were for drug-related crimes.
All texts were created to appear trustworthy in nature by attributing them to faculty at prestigious post-secondary institutions in the United States. Texts were further presented as feature articles published in well-respected press outlets (e.g., Economist, Atlantic Magazine). Texts ranged from 253 to 258 words in length. The Flesch-Kincaid grade level readabilities ranged from 14.9 to 16.6, indicating that texts were appropriate for use with an undergraduate audience. Texts were presented in a random order and students could navigate backward and forward across texts. Information about study texts is summarized in Table 1.
Table 1. Overview of study texts.
Task Conditions
Students’ assignment to task condition had two phases. Fist, students received task instructions, consistent with their experimental condition, prior to completing the multiple text task. Second, students were provided with external supports (i.e., customized highlighting tools) to support their strategy engagement during processing.
Task Instructions
All students received the following task instructions prior to reading: We will ask you to read four texts to write a research report about mass incarceration in the United States. Your research report should connect information presented across texts and cite your sources. This general set of instructions was followed by one of three specific task directives, asking students to engaged in intratextual, intertextual, or metacognitive processing while learning from multiple texts. Students were randomly assigned to receive one of these three specific task directives.
External Supports
Additionally, students were asked to highlight information in the four study texts in accordance with the processing directives they received. Students were also asked to explain their highlights in a text-box provided for this purpose. Highlights and associated explanations were used both as a physical scaffold to support students’ strategy use, in accordance with their assigned strategy condition, and as a log-data indicator of what information students had attended to during reading and what type of processing was facilitated. While students across conditions may have highlighted the same sentence in text, the highlighter color and students’ associated explanations were used to determine what type of processing each instance of highlighting represented. For instance, students marking the same information may have done so in making a judgment of information relevance (i.e., engaging in intratextual processing), in forming of a cross-textual connection (i.e., reflecting intertextual processing), or in determining a sentence’s comprehension ease (i.e., corresponding to metacognitive processing). Figure 1 includes a screenshot of the highlighting tool available to students in association with each strategy condition.
Figure 1. Screenshot of highlighter tool for the intratextual strategy condition.
Before viewing and highlighting any of the four experimental texts, students were introduced to a practice text that they could highlight according to their assigned strategy condition. The goal of this practice text was both to familiarize students with the use of the highlighting tool and to define the construct of mass incarceration for readers. During this practice exercise students were also shown how to navigate forward and backward in the texts they read. This forward and backward navigation was specifically introduced to foster students’ potential connection formation across texts.
Intratextual Processing
In the intratextual condition, students were asked to identify the important or relevant information included within each study text. In particular, students were told: As you read, we will also ask you to highlight any information that you consider to be relevant or important in each text. Students were further provided with two highlighting colors allowing relevant (light green) and important (dark green) information to be differentially marked (see Figure 1). In this case, prompting students to attend to relevant and important information during reading was viewed as fostering an intratextual strategic approach in that students were prompted to attend to the (relevant and important) information included within each text. The intratextual processing condition served as a control or comparison group to which students’ more intertextually- or metacognitively-focused multiple text processing could be compared.
Intertextual Processing
The intertextual strategy condition asked students to identify connections or relations across texts: As you read, we will also ask you to highlight any information that you consider to be related or connected across texts. Students in this condition were provided with three highlighting colors to mark similar (green), different (red), or otherwise related (blue) content across texts. See Figure 2. This condition was intended to direct students’ attention toward the connections or links that could be formed across multiple texts.
Figure 2. Screenshot of highlighter tool for the intertextual strategy condition.
Metacognitive Processing
Metacognitive strategy use was elicited by asking students to highlight the easy or difficult to understand information in each text: As you read, we will also ask you to highlight any information that is easy or difficult for your to understanding in each text. “Easy to understand” content was highlighted in green and “difficult to understand” content was marked in red. See Figure 3. Prompting students to identify text-based information as either easy or difficult to understand was expected to cue students’ engagement in comprehension monitoring during reading.
Figure 3. Screenshot of highlighter tool for the metacognitive strategy condition.
Processing Measures
Text Highlighting
Students’ highlights and highlight explanations were coded in terms of their quantity and content. Quantitatively, the number of sentences students highlighted, across texts, was totaled. Inter-rater agreement for the number of highlights in students’ responses was 100%, based on 23 responses scored (32.39% of the sample).
With regard to content, students’ highlight explanations were coded per List’s (2020) Comprehensive Strategy Framework, according to their strategy functions (i.e., constructive-integrative, critical-analytic, or metacognitive-reflective processing) and referents (i.e., a single text, multiple texts, or students’ prior knowledge/beliefs). This 3 × 3 taxonomy resulted in students receiving nine separate scores to capture their reported strategy use across all four study texts. For example, students received three separate scores reflecting their engagement in constructive-integrative processing directed toward a single text, toward multiple texts, and toward their own prior knowledge and beliefs. As an example, one student explained the information they highlighted as follows: “The article is easy to understand…The author gives detail and support to his or her thesis very well.” This explanation was coded as reflecting metacognitive-reflective and critical-analytic processing, both directed toward a single text. Another student explained one of their highlights as follows: “This text expresses the failure of the community correction programs, similar to the previous text,” with this explanation coded as reflecting constructive-integrative processing directed toward multiple texts. It is important to note that while students’ highlights and explanations could correspond to their assigned task condition, this was not always the case. For instance, students directed to engage in intratextual processing often identified important and relevant information in texts, however, students assigned to the intertextual processing condition also, at times, explained their highlights as reflecting relevance determinations (see Table 2 for additional coding examples). Inter-rated agreement for strategy coding was 80.19%, based on our coding of all student responses.
Table 2. Examples of strategy explanation codings.
Research Report
Following the reading phase, students were asked to confirm that they were ready to compose their research reports on the topic of mass incarceration in the United States, with students able to return to the four study texts if they wanted. In composing their research reports, students were asked to “identify connections across the texts you read” and to include citations in their writing.
The research reports students composed were scored using a six-point rubric. The rubric was designed to award points for (a) the number of key issues related to mass incarceration that students discussed, (b) the extent to which key issues were explained or elaborated in students’ writing, and (c) the extent to which key issues were described in an integrative fashion, based on information introduced across two or more texts. Students’ responses were assigned a 1 when they introduced a single issue discussed anywhere in the study texts and a 2 when a single issue was not only introduced, but also discussed in an elaborated fashion, with associated evidence, examples, or explanations introduced. Responses assigned a 3 discussed multiple issues introduced within the study texts, while a 4 was assigned to responses that both discussed multiple issues and elaborated on at least two of these, with additional evidence or explanations provided. Finally, responses assigned a 5 contained one instance of intertextual integration, or discussed one issue related to mass incarceration, based on information introduced across two or more texts. Responses assigned a 6 included the integrated discussion of two or more issues based on information introduced across multiple texts. See Table 3 for a rubric with sample responses. Inter-rater agreement for the scores assigned to students’ written responses was Cohen’s κ = 0.75 (exact agreement: 78.94%). The number of unique citations included in students’ research reports was also totaled. Exact agreement for the number of citations included in students’ responses was 92.45%.
Table 3. Rubric for scoring students’ research reports.
Post-task Assessment
Although the rubric used to score research reports was designed to capture both the breadth (i.e., number of issues discussed) and depth (i.e., elaborated and integrated discussion of issues) of students’ understanding of mass incarceration, we were interested in probing this understanding further. As such, students were asked to respond to a series of open-ended questions designed to assess their integrative understanding of various key issues discussed across the four study texts. That is, while students could choose whether or not to write about the controversial issue of the War on Drugs in the research reports that they composed, students’ understanding of this issue was directly assessed in the open-ended questions that students were asked to answer. In particular, students were asked to: Think about the four texts you read. Please summarize what the texts said about each of these main points. Please be sure to think about the information presented across all four texts in the summaries you compose. Students were then asked to summarize information related to each of four key issues discussed across multiple texts (i.e., the number of incarcerated individuals in the U.S., the War on Drugs, the “tough on crime” culture in the U.S., and the advantages and disadvantages of community corrections). Students’ responses to each open-ended question were assigned a score of 0, 1, or 2, according to whether these were incorrect or incomplete (0), reflected information only from a single text (1), or considered information provided in more than one study text (2). Students’ scores were totaled across all four of the key issues that they were asked to summarize, based on information introduced across multiple texts. Sample responses are included in Table 4. Exact agreement for students’ open-ended response scores was 88.32%.
Table 4. Sample open-ended responses.
Results
Research Question 1: Differences in Strategy Use by Task Condition
Our first research question examined differences in students’ manifest strategy use across task conditions. A series of one-way ANOVAs were conducted, with alpha adjusted to 0.007 for multiple comparisons (i.e., α = 0.05/8). Because so few students exhibited critical analytic processing directed toward their prior knowledge, this aspect of strategy use was not analyzed. Descriptive information for strategy use across conditions is presented in Table 5.
Table 5. Descriptives.
To start, students’ use of constructive-integrative strategies directed at multiple texts differed significantly across task conditions [F(2,56) = 12.48, p < 0.001, η2 = 0.31], indicating a large effect. Post hoc analyses, using Tukey’s HSD, determined that students assigned to the intertextual condition employed significantly more constructive-integrative strategies directed at multiple texts (M = 3.60, SD = 3.98) than students assigned to either the intratextual (M = 0.42, SD = 0.84) or the metacognitive (M = 0.25, SD = 0.44) strategy conditions, ps < 0.001.
Moreover, students’ use of metacognitive-regulatory strategies directed at single texts differed by task condition [F(2,56) = 21.71, p < 0.001, η2 = 0.44], reflecting a large effect. Post hoc analyses using Tukey’s HSD found students assigned to the metacognitive strategy condition to use significantly more metacognitive-reflective strategies directed at single texts (M = 4.05, SD = 3.73) than students assigned to either the intratextual (M = 0.16, SD = 0.50) or the intertextual (M = 0.00, SD = 0.00) strategy conditions, ps < 0.001. Likewise, students’ metacognitive-reflective strategy directed toward their prior knowledge differed across conditions, F(2,56) = 6.61, p < 0.007, η2 = 0.19. In particular, this approach to strategic processing was only manifest by students in the metacognitive processing condition (M = 0.55, SD = 0.94), ps < 0.01.
No other strategy categories were found to significantly differ across conditions, ps > 0.02. The amount of information that students highlighted also did not differ by task condition, p = 0.52.
Research Question 2: Performance Differences by Task Condition
For the second research question, we used three one-way ANOVAs to examine whether students’ quality of research report writing, citation use, or responses to open-ended questions tapping integration differed by task condition. However, writing quality, citation use, and open-ended response scores did not differ by condition (ps > 0.26).
Research Question 3. Association Between Processing Differences and Task Performance
For our third research question, we examined the role of strategy use in students’ performance on the external outcomes examined in this study (i.e., the quality of students’ research reports, citation use, and responses to open-ended integration questions). For each model, prior topic knowledge was controlled for in Step 1. Experimental condition, indicator coded relative to the intratextual strategy use condition, was entered in Step 2, and the total volume of information that students highlighted, across texts, as well as students’ manifest strategy use, were entered as predictors in Step 3. Because of the volume of strategy types examined in this study, and because our interest was specifically focused on students’ cross-textual linking or integration-focused strategy use, only variables reflecting students’ multiple text-directed strategy use, including constructive-integrative, critical-analytic, and metacognitive-reflective processing, were included in Step 3. Table 6 includes correlations among key variables.
Table 6. Correlation among key indicators.
Research Report Scores
The model predicting students’ research report writing quality was not significant, p = 0.34.
Citations
The model predicting the number of citations included in students’ written responses was not significant, p = 0.07.
Open-Ended Integration
The model predicting students’ open-ended integration scores was overall significant [F(7,45) = 2.72, p < 0.05, R2adj = 0.19] corresponding to a medium effect. However, only students’ engagement in critical-analytic strategy use directed toward multiple texts was an individually significant predictor in the model (β = 0.43, p < 0.01). See Table 7 for a model summary.
Table 7. Model summary predicting open-ended integration performance.
Discussion
Guided by the IF-MT, this study examined whether directing students to engage in intratextual, intertextual, or metacognitive processing in the preparation stage of multiple text learning, resulted in variable strategy use during execution, and in differences in production, or in learners’ task performance. Two key findings emerged in this study. First, students’ manifest strategy use was found to differ in association with the processing directives that they received prior to reading. Second, students’ engagement in constructive-integrative processing directed at multiple texts was found to predict open-ended integration performance, one of the outcome measures examined in this study. We discuss each of these main findings, in turn. As a whole, results from this study align with theoretical insights from the IF-MT. In particular, using an innovative methodological approach, we establish that (a) modes of strategic processing can be instantiated via task instructions, (b) students direct strategic processing toward a variety of referents when learning from multiple texts, and (c) strategy use is associated with integration performance.
Differences in Processing by Strategy Condition
In this study, we asked students to engage in intratextual, intertextual, or metacognitive processing when completing a multiple text task. We then tracked such processing, or students’ manifest strategy use, by asking learners to highlight particular information in texts, in accordance with their strategy condition, as well as to explain their highlights. When the association between assigned mode of processing and manifest strategy use was examined, students assigned to the intertextual processing condition were found to use more constructive-integrative strategies directed at multiple texts than students asked to engage in intratextual or metacognitive processing. Likewise, directing students to engage in metacognitive strategy use during reading resulted in their significantly higher deployment of metacognitive-regulatory strategies direct at both single texts and at their prior knowledge or beliefs, as compared to students in the other two conditions. We draw four key conclusions based on these results.
First, as suggested by the IF-MT, the preparation and execution stages of multiple text learning were, indeed, found to be linked in this study. Prior work on learning from multiple texts, has long found task assignments asking students to produce various types of external outcomes to be associated with differences in task performance (Wiley and Voss, 1999; Gil et al., 2010a, b; List et al., 2019a). Here we demonstrate that task assignments can further be used to specify a desired mode of processing for students to engage during reading. Instructing students to engage in particular types of processing during task completion may be a particularly effective approach to use in instances when students have inaccurate or incomplete schema for what strategies different tasks may require (Wiley et al., 2018; List et al., 2019a). Instructing students to employ particular forms of processing may also increase the frequency with which students engage in deep-level strategy use (e.g., evaluation, metacognition), with such strategy use rarely spontaneously reported (Gerjets et al., 2011; Du and List, 2020).
Indeed, in this study we were encouraged to find that asking students to engage in intertextual processing resulted in their increased strategy use directed toward multiple texts. This constitutes a key contribution of this study. In effect, while prior work has recognized the importance of students’ engagement in intertextual processing, students have been found to manifest such processing to varying extents and often only in accordance with the degree of support for such processing provided by the study design (e.g., Britt and Sommer, 2004; Cerdán and Vidal-Abarca, 2008). In this study, we found a rather large volume of processing to be directed toward multiple texts, with such processing including constructive-integrative, critical-analytic, and metacognitive-reflective modes of strategy use. This suggests that the provision of task instructions to cue processing, in addition to various other physical scaffolds (e.g., highlighting; backward/forward navigation across texts), may increase students’ engagement in intertextual processing during reading.
As a third point, we found it fruitful to compare the relative prevalence of the various forms of strategic processing that students exhibited in this study to those documented in prior work. In examining similar categories of strategic processing, Anmarkrud et al. (2014) found students’ linking strategies (i.e., directed toward multiple texts) to most commonly reflect efforts to identify and learn important information (47.1%) and to evaluate sources and information (36.3%), with these categories generally corresponding to constructive-integrative and critical-analytic processing, respectively. In this study, too, strategies directed at multiple texts were most commonly constructive-integrative in nature, with critical-analytic and metacognitive-reflective strategies directed toward multiple texts to a considerably more limited extent. In this study, these somewhat reduced rates of critical-analytic and metacognitive-reflective strategy use, directed at multiple texts may be partly explained by the task directives we employed. That is, because students assigned to the intertextual processing strategy condition were directed to focus on multiple texts and to identify the connections among these, it seems logical that constructive-integrative strategy use dominated other strategy functions.
Among students directing any strategy functions toward multiple texts (52.54%, n = 31), 70.24% of the multiple-text directed strategies used were focused on construction-integration, with only 19.89% of such strategies focused on metacognition-reflection and 9.87% of these focused on critical-analytic processing. This suggests that when a particular approach to processing is cued, learners’ focus on such processing may come at the cost of a broader or more varied repertoire of strategy use. Alternatively, particularly in the case of critical-analytic processing, such processing may have been particularly limited both because it was not explicitly cued in any of the task conditions and because all of the texts used in this study were designed to be trustworthy in nature. Nevertheless, we were heartened by some students’ efforts to corroborate and compare information across texts, as demonstrated in responses such as: “This highlight shows a different statistic that only 20% of crimes are drug related,” reflecting critical-analytic processing or efforts to corroborate statistical information across texts.
When examining strategy use across conditions, a number of additional patterns emerged. For one, the majority of students’ strategy functions were directed toward single texts and constructive-integrative processing. This dominance is understandable given that, at its heart, this task involved students trying to learn about a complex and controversial topic, based on information presented within a series of individual, albeit conceptually connected, texts. It therefore follows that strategies aimed, fundamentally, at constructing meaning dominated students’ learning. Likewise, it seems logical that strategies directed at engaging students’ prior knowledge or beliefs were relatively under-represented in this study. This may reflect the relatively low prior knowledge of our sample. At the same time, we were somewhat surprised by these results given that the topic of mass incarceration is a controversial one in the United States and, in this study, was described across texts presenting partially conflicting information. As such, we expected the controversial nature of this topic to potentially elicit students’ strategy use directed toward their prior beliefs. Finally, students’ metacognitive-regulatory strategy use was found to be comparatively well-represented in this study, whereas prior work has found students to engage in metacognition only to a limited extent (Du and List, 2020). Of course, this may be in large-part attributable to the task instructions that students in the metacognitive processing condition received, prior to reading. Nevertheless, results from this study seem to be an encouraging indicator that metacognitive monitoring during reading can be cued via the task instructions provided, as was previously done by Stadtler and Bromme (2008).
The fourth and final conclusion is methodological in nature. In this study, we used students’ highlights and associated explanations as indicators of strategy use during reading. We found doing so to be an effective method of assessing processing. Indeed, capturing the nature of students’ strategy use during task completion has long proven to be a formidable challenge (Fryer and Dinsmore, 2020). On the one hand, think-aloud procedures have been effective at capturing students’ online processing. On the other hand, think-alouds are data and labor-intensive procedures that may be overly taxing for some learners (Van Gog et al., 2005; Muñoz et al., 2006). In this study, rather than using a think-aloud approach, we asked students to highlight particular information in texts, as an indicator of strategy use, and to explain these highlights. To the extent that the information students highlighted and their corresponding explanations were found to differ across task conditions in the ways expected, this approach to capturing strategic processing may be a promising one. Students’ highlights and associated explanations, in this study, seemed to be effective at providing information regarding students’ attendance to specific content in texts (i.e., according to the information highlighted) and associated cognitive processes (i.e., through the explanations provided). Moreover, our use of highlighting allowed us to capture students’ strategic processing in an accessible and scalable way. As such, using the highlighting tool, with associated explanations, may constitute a useful method for capturing strategic processing during reading in future work.
Predicting Task Performance Based on Strategy Use
A second finding to emerge from this study is that while students’ overall task performance did not differ by strategy condition, differences in manifest strategic processing, and multiple text directed critical-analytic strategy use, in particular, were predictive of students’ responses to open-ended questions tapping integration. This latter finding seems logical given that these strategies most reflected deep-level intertextual processing, while the open-ended questions used in this study assessed students’ integrative understanding of complementary and conflicting issues discussed across texts.
Still, we were somewhat surprised to find that strategy use was not significant in predicting students’ research report quality. In part, this lack of findings may be attributable to some limitations in sample size. Nevertheless, interpreted through the lens of the IF-MT, it may be that while the nature of students’ strategic processing was associated with the cognitive outcomes that students generated, such strategy use did not carry forward to the external products (i.e., research reports) that students composed. This constitutes an explanation for why open-ended integration performance, capturing students’ cognitive outcomes, was predicted by strategic processing during reading, while research report writing quality, considered to be an external outcome, was not. As suggested by List and Alexander (2019), in their description of the IF-MT, the external products that students compose based on multiple texts, in addition to reflecting a set of cognitive outcomes, also demand that students employ a variety of writing skills, not expressly specified in the IF-MT. Still, the link between students’ cognitive outcomes and writing performance is clearly exhibited in this study via the strong association among these two outcome measures of interest (see Table 6).
As a more general theory of the case, our understanding of these results is that the mode of processing prompted in students in the preparation stage of the IF-MT, impacts their strategic processing during execution. This strategic processing, in turn, then results in particular differences in task performance (i.e., the types of cognitive outcomes that students construct as a result of learning from multiple texts). Such an explanation is consistent with our not finding significant difference in task performance to manifest across strategy conditions (Research Question 2) but, nevertheless, our determining strategy use to differ by task condition (Research Question 1) and multiple text directed strategy use to predict task performance (Research Question 3). Validating such an understanding requires replicating this study and exploring mediation analyses, as we aim to do in future work. Implications for the IF-MT are, in part, that even if task assignments can be used to engender particular forms of strategic processing during execution, the degree to which such strategies are engaged and their quality are the ultimate determinants of students’ production, or resultant task performance.
Implications
There are at least four implications for theory and practice associated with this study. To start, this study is among the first to expressly use the IF-MT as a framework for understanding students’ learning from multiple texts. In this study, we were able to link aspects of the task assignment, to students’ processing during execution, to the quality of the external products that students developed after reading a set of multiple texts. Second, in this study we adopted an innovative and analytic approach to capturing students’ processing when learning from multiple texts. That is, we were able to decompose the nature of students’ strategic processing both in terms of its functions and referents. Indeed, and thirdly, we did this by adopting a novel methodological approach to capture the nature of students’ strategic processing, namely the use of a highlighting tool with associated explanations. In doing so, we demonstrated that strategic processing, when captured in this manner, corresponded to the task assignment that students received prior to reading. Finally, and in line with much of the literature, we demonstrated that learners’ engagement in multiple-text directed strategies, in particular, had benefits for students’ integration-related task performance when learning from multiple texts.
Limitations
Despite the strengths of this study, at least four limitations must be acknowledged.
First, in this study, we assigned students to engage in intratextual, intertextual, or metacognitive processing when learning from multiple texts. This was done to isolate the effects of each mode of strategy engagement on students’ multiple text task performance. And, indeed, task assignment was found to be associated with differences in strategy engagement, as captured via the information that students highlighted and their associated explanations. Nevertheless, within the context of real-world multiple text tasks, students are likely to need to engage a variety of strategies, including all three of these types, for successful task completion. In other words, learning from multiple texts simultaneously requires students to identify relevant and important information, to connect information across texts, and to monitor text quality and their own understanding. To the extent that cuing students’ use of all of the strategies that they may need for successful task completion is unreasonable and that strategic processing should, by its very nature, be deliberately and dynamically engaged by learners, this study only demonstrates the association between particular types of strategy engagement (i.e., directed toward multiple texts) and task performance. More work is still needed to understand how such strategy engagement may be best fostered in learners.
Second, our coding of students’ strategy use in this study was based on the information that students marked, using the highlighting tool, and their associated explanations for the information highlighted. While we considered this to be an effective and unobtrusive way to collect data on processing, this approach carried with it a number of limitations. For instance, there was not always a one-to-one mapping between the information that students highlighted and the associated explanations that they wrote. This requiring us to generalize that, for example, when students reported that they highlighted relevant information, this explanation pertained to all of the sentences that they indicated in-text. Further, we, as researchers, interpreted students’ explanations of strategy use as serving particular functions and as directed toward specific referents. However, these interpretations, although supported by the information that students highlighted in text, should be validated with behavioral measures, like eye-tracking, in future work. Finally, asking students to type their explanations for information highlighted in association with each text rather than to report strategic processing continuously (i.e., as during a think-aloud) may have resulted in incomplete or overly-crystalized strategy reports. That is, students may have either under-reported all of the processing that they engaged during reading or refined their explanations, perhaps to better comport with task demands. Both of these possibilities are suggested by prior work (Van Gog et al., 2005).
Third, the two performance outcomes examined in this study were scored in such a way that prioritized students’ multiple text integration. Although content integration, or connection-formation across texts, has been identified as a central outcome in students’ learning from multiple texts (Britt et al., 1999; Perfetti et al., 1999), additional factors (e.g., writing quality, organization) were not well-captured by our rubric, as aspects of external product composition. A broader range of multiple text learning outcomes, scored in a more comprehensive fashion, should potentially be considered in future work. As an added point, the emphasis on integration reflected in the rubrics used to score both students’ research reports and open-ended responses may have unduly benefited students belonging to the intertextual processing strategy condition. Still, these effects were somewhat mitigated both by the truly essential role of integration in students’ learning from multiple texts (i.e., we considered our prioritizing of integration to be appropriate) and by the lack of differences in task performance identified across conditions. Further, asking students to compose a research report is not a task assignment that has frequently been examined in prior work, with directing students to engage in argument composition being much more common (Wiley and Voss, 1999; Anmarkrud et al., 2014). Nevertheless, research report writing was the task assignment used in the present study both because we wanted to encourage students’ comprehensive discussion of the various key issues introduced across texts (List and Alexander, 2019; List et al., 2019a) and because the experimental texts used in this study did not have a clear, two-sided argument structure.
Finally, students completed this study online, at a time and location of their choosing and in the midst of the COVID-19 pandemic. These factors may have resulted in lower than desired recruitment and performance in this study. As such, replicating results, in both lab-based and classroom settings, constitute important areas for future work.
Conclusion
In this study, we sought to contribute new insights into undergraduate students’ ability to learn about a complex and controversial topic (i.e., mass incarceration) through their engagement in a multiple text task. The design of this investigation was theory-based, reflecting the phases of multiple text learning and the strategic processes articulated in the Integrated Framework of Learning from Multiple Text. In keeping with the goals of this special issue on information processing assessment and online thinking and reasoning in higher education, we presented the undergraduate students in our study with three varied task directives intended to orient their processing of information contained in four carefully orchestrated texts. Moreover, to externalize students’ thinking and reasoning during task completion, without disrupting or distorting processing too much, we asked students to highlight information in texts corresponding to their particular task condition (i.e., intratextual, intertextual, and metacognitive). We then created a unique system for scoring these highlights based on the strategy functions and referents represented by each highlighted segment of text.
To extend what is known about college students’ multiple text task performance, we also incorporated several measures of learning. Specifically, we assessed the quality of the research reports that students composed based on multiple texts, as well as students’ responses to a series of open-ended questions specifically created to capture their integrated understanding of content introduced across the four study texts. In terms of the IF-MT, we expected that the varied processing directives that students had been given in the preparation stage, and the specific highlighter tools that students were asked to use during execution, would translate into differential research report quality and responses to open-ended integration questions in the production stage. As hypothesized, the three directives, indeed, resulted in changes in learners’ processing and task performance.
All in all, what this investigation has contributed to the literature on information processing and online thinking and reasoning assessment is clear evidence that even mature readers can benefit from scaffolds that serve to orient their text processing in facilitative ways. In addition, the current study has offered alternative ways that students’ thinking and reasoning can be effectively captured during the course of task completion, along with innovative methods for scoring such thinking. Without question, there is much more to be learned about university students’ information processing in online contexts and the thinking and reasoning that give rise to learning within those contexts. Nonetheless, we regard this study as a step in the right direction.
Data Availability Statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics Statement
The studies involving human participants were reviewed and approved by Penn State IRB: STUDY00008166. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.
Author Contributions
Both authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
Anmarkrud, Ø, Bråten, I., and Strømsø, H. I. (2014). Multiple-documents literacy: strategic processing, source awareness, and argumentation when reading multiple conflicting documents. Learn. Individ. Differ. 30, 64–76. doi: 10.1016/j.lindif.2013.01.007
Barzilai, S., and Weinstock, M. (2015). Measuring epistemic thinking within and across topics: a scenario-based approach. Contemp. Educ. Psychol. 42, 141–158. doi: 10.1016/j.cedpsych.2015.06.006
Brante, E. W., and Strømsø, H. I. (2018). Sourcing in text comprehension: a review of interventions targeting sourcing skills. Educ. Psychol. Rev. 30, 773–799. doi: 10.1007/s10648-017-9421-7
Bråten, I., Anmarkrud, Ø, Brandmo, C., and Strømsø, H. I. (2014). Developing and testing a model of direct and indirect relationships between individual differences, processing, and multiple-text comprehension. Learn. Instruct. 30, 9–24. doi: 10.1007/s11145-018-9868-z
Bråten, I., and Strømsø, H. I. (2011). Measuring strategic processing when students read multiple texts. Metacogn. Learn. 6, 111–130. doi: 10.1007/s11409-011-9075-7
Britt, M. A., Perfetti, C. A., Sandak, R. L., and Rouet, J. F. (1999). “Content integration and source separation in learning from multiple texts,” in Essays in Honor of Tom Trabasso, ed. S. R. Goldman (Mahwah, NJ: Lawrence Erlbaum Associates, Inc), 209–233.
Britt, M. A., and Sommer, J. (2004). Facilitating textual integration with macro-structure focusing tasks. Read. Psychol. 25, 313–339. doi: 10.1080/02702710490522658
Cerdán, R., and Vidal-Abarca, E. (2008). The effects of tasks on integrating information from multiple documents. J. Educ. Psychol. 100, 209–222. doi: 10.1037/0022-0663.100.1.209
Cho, B. Y., Afflerbach, P., and Han, H. (2018). “Strategic processing in accessing, comprehending, and using multiple sources online,” in Handbook of Multiple Source Use, eds J. L. G. Braasch, I. Bråten, and M. T. McCrudden (New York, NY: Routledge), 133–150. doi: 10.4324/9781315627496-8
Daher, T. A., and Kiewra, K. A. (2016). An investigation of SOAR study strategies for learning from multiple online resources. Contemp. Educ. Psychol. 46, 10–21. doi: 10.1016/j.cedpsych.2015.12.004
Datig, I. (2016). Citation behavior of advanced undergraduate students in the social sciences: a mixed-method approach. Behav. Soc. Sci. Librar. 35, 64–80. doi: 10.1080/01639269.2016.1214559
Dinsmore, D. L., and Alexander, P. A. (2012). A critical discussion of deep and surface processing: what it means, how it is measured, the role of context, and model specification. Educ. Psychol. Rev. 24, 499–567. doi: 10.1007/s10648-012-9198-7
Du, H., and List, A. (2020). Researching and writing based on multiple texts. Learn. Instruct. 66:e0101297. doi: 10.1016/j.learninstruc.2019.101297
Fryer, L. K., and Dinsmore, D. L. (2020). The Promise and pitfalls of self-report. Front. Learn. Res. 8, 1–9. doi: 10.14786/flr.v8i3.623
Gerjets, P., Kammerer, Y., and Werner, B. (2011). Measuring spontaneous and instructed evaluation processes during Web search: integrating concurrent thinking-aloud protocols and eye-tracking data. Learn. Instruct. 21, 220–231. doi: 10.1016/j.learninstruc.2010.02.005
Gil, L., Bråten, I., Vidal-Abarca, E., and Strømsø, H. I. (2010a). Summary versus argument tasks when working with multiple documents: which is better for whom? Contemp. Educ. Psychol. 35, 157–173. doi: 10.1016/j.cedpsych.2009.11.002
Gil, L., Bråten, I., Vidal-Abarca, E., and Strømsø, H. I. (2010b). Understanding and integrating multiple science texts: summary tasks are sometimes better than argument tasks. Read. Psychol. 31, 30–68. doi: 10.1080/02702710902733600
Goldman, S. R., Braasch, J. L., Wiley, J., Graesser, A. C., and Brodowinska, K. (2012). Comprehending and learning from internet sources: processing patterns of better and poorer learners. Read. Res. Q. 47, 356–381. doi: 10.1002/RRQ.027
Hagen, ÅM., Braasch, J. L., and Bråten, I. (2014). Relationships between spontaneous note-taking, self-reported strategies and comprehension when reading multiple texts in different task conditions. J. Res. Read. 37, 141–157. doi: 10.1111/j.1467-9817.2012.01536.x
Hendley, M. (2012). Citation behavior of undergraduate students: a study of history, political science, and sociology papers. Behav. Soc. Sci. Librar. 31, 96–111. doi: 10.1080/01639269.2012.679884
Kobayashi, K. (2009). Comprehension of relations among controversial texts: effects of external strategy use. Instruct. Sci. 37, 311–324. doi: 10.1007/s11251-007-9041-6
Le Bigot, L., and Rouet, J. F. (2007). The impact of presentation format, task assignment, and prior knowledge on students’ comprehension of multiple online documents. J. Literacy Res. 39, 445–470. doi: 10.1080/10862960701675317
List, A. (2020). “Six questions regarding strategy use when learning from multiple texts,” in Handbook of Strategies and Strategic Processing, eds D. L. Dinsmore, L. K. Fryer, and M. M. Parkinson (New York, NY: Routledge), 119–140. doi: 10.4324/9780429423635-8
List, A., and Alexander, P. A. (2017). Text navigation in multiple source use. Comput. Human Behav. 75, 364–375. doi: 10.1016/j.chb.2017.05.024
List, A., and Alexander, P. A. (2019). Toward an integrated framework of multiple text use. Educ. Psychol. 54, 20–39. doi: 10.1080/00461520.2018.1505514
List, A., Du, H., and Wang, Y. (2019a). Understanding students’ conceptions of task assignments. Contemp. Educ. Psychol. 59:101801. doi: 10.1016/j.cedpsych.2019.101801
List, A., Du, H., Wang, Y., and Lee, H. Y. (2019b). Toward a typology of integration: examining the documents model framework. Contemp. Educ. Psychol. 58, 228–242. doi: 10.1016/j.cedpsych.2019.03.003
McCarthy, K. S., and Goldman, S. R. (2015). Comprehension of short stories: effects of task instructions on literary interpretation. Discourse Process. 52, 585–608. doi: 10.1080/0163853X.2014.967610
McCrudden, M. T. (2018). “Text relevance and multiple-source use,” in Handbook of Multiple Source Use, eds J. L. G. Braasch, I. Bråten, and M. T. McCrudden (New York, NY: Routledge), 168–183. doi: 10.4324/9781315627496-10
McCrudden, M. T., Magliano, J. P., and Schraw, G. (2011). The effect of diagrams on online reading processes and memory. Discourse Process. 48, 69–92. doi: 10.1080/01638531003694561
McCrudden, M. T., and Schraw, G. (2007). Relevance and goal-focusing in text processing. Educ. Psychol. Rev. 19, 113–139. doi: 10.1007/s10648-006-9010-7
McCrudden, M. T., and Sparks, P. C. (2014). Exploring the effect of task instructions on topic beliefs and topic belief justifications: a mixed methods study. Contemp. Educ. Psychol. 39, 1–11. doi: 10.1016/j.cedpsych.2013.10.001
McNamara, D. S. (2004). SERT: self-explanation reading training. Discourse Process. 38, 1–30. doi: 10.1207/s15326950dp3801_1
Muñoz, B., Magliano, J. P., Sheridan, R., and McNamara, D. S. (2006). Typing versus thinking aloud when reading: implications for computer-based assessment and training tools. Behav. Res. Methods 38, 211–217. doi: 10.3758/BF03192771
Parkinson, M. M., and Dinsmore, D. L. (2018). Multiple aspects of high school students’ strategic processing on reading outcomes: the role of quantity, quality, and conjunctive strategy use. Br. J. Educ. Psychol. 88, 42–62. doi: 10.1111/bjep.12176
Perfetti, C. A., Rouet, J.-F., and Britt, M. A. (1999). “Towards a theory of documents representation,” in The Construction of Mental Representations During Reading, eds H. van Oostendorp and S. R. Goldman (Hillsdale, NJ: Lawrence Erlbaum Associates, Inc), 99–122.
Potocki, A., Ros, C., Vibert, N., and Rouet, J. F. (2017). Children’s visual scanning of textual documents: effects of document organization, search goals, and metatextual knowledge. Sci. Stud. Read. 21, 480–497. doi: 10.1080/10888438.2017.1334060
Stadtler, M., and Bromme, R. (2008). Effects of the metacognitive computer-tool met. a. ware on the web search of laypersons. Comput. Hum. Behav. 24, 716–737. doi: 10.1016/j.chb.2007.01.023
Uhrmacher, K. (2020). Where 2020 Democrats Stand on Criminal Justice. Washington, DC: The Washington Post.
Van Gog, T., Paas, F., Van Merriënboer, J. J., and Witte, P. (2005). Uncovering the problem-solving process: cued retrospective reporting versus concurrent and retrospective reporting. J. Exper. Psychol. Appl. 11, 237–244. doi: 10.1037/1076-898X.11.4.237
Wang, Y., and List, A. (2019). Calibration in multiple text use. Metacogn. Learn. 14, 131–166. doi: 10.1007/s11409-019-09201-y
Weston-Sementelli, J. L., Allen, L. K., and McNamara, D. S. (2018). Comprehension and writing strategy training improves performance on content-specific source-based writing tasks. Intern. J. Artific. Intellig. Educ. 28, 106–137. doi: 10.1007/s40593-016-0127-7
Wiley, J., Goldman, S. R., Graesser, A. C., Sanchez, C. A., Ash, I. K., and Hemmerich, J. A. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. Am. Educ. Res. J. 46, 1060–1106. doi: 10.3102/0002831209333183
Wiley, J., Jaeger, A. J., and Griffin, T. D. (2018). “Effects of instructional conditions on comprehension from multiple sources in history and science,” in Handbook of Multiple Source Use, eds J. L. G. Braasch, I. Bråten, and M. T. McCrudden (New York, NY: Routledge), 341–361. doi: 10.4324/9781315627496-20
Wiley, J., and Voss, J. F. (1999). Constructing arguments from multiple sources: tasks that promote understanding and not just memory for text. J. Educ. Psychol. 91, 301–311. doi: 10.1037/0022-0663.91.2.301
Wineburg, S. S. (1991). Historical problem solving: a study of the cognitive processes used in the evaluation of documentary and pictorial evidence. J. Educ. Psychol. 83, 73–87. doi: 10.1037/0022-0663.83.1.73
Wolfe, M. B., and Goldman, S. R. (2005). Relations between adolescents’ text processing and reasoning. Cogn. Instruct. 23, 467–502. doi: 10.1207/s1532690xci2304_2
Keywords: multiple text comprehension, multiple text processing, strategic processing, integration- and synthesis- oriented strategies, metacognition
Citation: List A and Alexander PA (2020) Strategy Use in Learning From Multiple Texts: An Investigation of the Integrative Framework of Learning From Multiple Texts. Front. Educ. 5:578062. doi: 10.3389/feduc.2020.578062
Received: 30 June 2020; Accepted: 22 September 2020;
Published: 22 October 2020.
Edited by:
Lawrence Jun Zhang, The University of Auckland, New ZealandReviewed by:
Hossein Bozorgian, University of Mazandaran, IranTiefu Shaun Zhang, University of Electronic Science and Technology of China, China
Copyright © 2020 List and Alexander. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Alexandra List, azl261@psu.edu