Nathaniel J. Hunsu
Olusola Adesope
Matthew T. McCrudden- 1College of Engineering, University of Georgia, Athens, GA, United States
- 2Educational Leadership, Sport Studies, Educational and Counseling Psychology, Washington State University, Pullman, WA, United States
- 3College of Education, Educational Psychology, Counseling, and Special Education, Pennsylvania State University, CEDAR Building, University Park, PA, United States
This study investigated the effect of refutation text structure on reading processes and learning. Undergraduate biology students (n = 116) who had high or moderate misconceptions were randomly assigned to read either a non-refutation text, a simple refutation text, or an elaborated refutation text that addressed commonly held misconceptions about genetic biology. Participants’ were prompted to think-aloud and type their thoughts as they read the text. Typed verbal responses were coded for the use of paraphrasing, bridging inferences, and elaborations. The results showed that the simple and elaborated refutation texts promoted the use of bridging inferences, and the elaborated refutation text promoted the use of elaborations compared to the non-refutation text. Neither text type nor misconception status had a significant effect on post learning outcomes. These results suggest that refutations with illustrative examples can increase the use of cognitive processes that support comprehension.
1. Introduction
A reader constructs a coherent mental model of a text by creating a network of interrelated ideas that reflects information stated explicitly in a text, and by drawing inferences between ideas from the text and with one’s prior knowledge (Kintsch, 1998). When text information is consistent with a reader’s prior knowledge, information from a text is more easily integrated with one’s prior knowledge (Diakidoy et al., 2011). However, such integration during reading is more challenging when there is a conflict between text information and one’s prior knowledge (Sinatra and Broughton, 2011; Kendeou et al., 2014). Further, a conflict between to-be-learned content and a student’s prior knowledge can impede learning (Braasch et al., 2013). For example, a biology student who believes that an organism’s trait, such as eye color, is determined by a single gene, may experience cognitive conflict upon reading that traits are determined by the combination of different genes and may struggle to use this information later.
Refutation texts can help students revise or update their knowledge. A refutation text is written to explicitly acknowledge an alternative conception, directly refute it, and provide a more satisfactory explanation (Hynd, 2001). Refutation texts can support knowledge updating by not only explicitly signaling a discrepancy between a reader’s prior knowledge and text information, but by providing a new information that readers can use to update their knowledge about a topic. However, more research is needed to understand how refutation text affects comprehension processes and learning. The present study primarily seeks to investigate the effects of different refutation text types on readers’ cognitive processes during reading. In addition, the study seeks to understand the different measures of learning after reading for students with varying levels of prior knowledge. Specifically, we investigated whether elaborated refutation statements would affect cognitive processes and learning above and beyond simple refutation statements and compared to a control group that did not receive refutation statements.
The knowledge revision components framework (KReC) provides an account for understanding how knowledge revision can occur during reading (Kendeou and O’Brien, 2014). Two key assumptions undergird this framework. The first assumption is that information encoded in long-term memory cannot be deleted (encoding assumption). This assumption is important because the persistence of inaccurate information in long-term memory has the potential to interfere with comprehension and subsequent learning. The second assumption is that information in long-term memory is activated via passive memory processes (passive activation assumption). This assumption is important because information from long-term memory that is related to the reader’s focus of attention can be activated independently of whether it helps or hinders comprehension. Thus, the permanent availability of inaccurate information in long-term memory and the activation of this information when it is related to information in a reader’s focus of attention (e.g., the sentence currently being read) can interfere with comprehension and learning.
Given that inaccurate knowledge cannot be deleted and that it has the potential to interfere with comprehension and learning, the KReC framework identifies three conditions that are needed for knowledge revision to occur. The first condition is the co-activation of new information from text and inaccurate information from long-term memory (co-activation principle). This ensures that inaccurate information comes into contact with accurate information. The second condition is that the new information is integrated with the inaccurate information (integration principle). This leads to a modification or updating of the long-term memory representation. Lastly, according to the competing activation principle, an increase in the amount of newly encoded information will lead to an increase in the activation of that information relative to the inaccurate information. As a result, the new information begins to overpower the inaccurate information in the memory representation.
The KReC framework can be used to explain why refutation texts can promote knowledge revision. As indicated above, a refutation text explicitly acknowledges an alternative conception, directly refutes it, and provides a more satisfactory explanation (Hynd, 2001). A refutation statement is designed to promote co-activation. New information is held in the reader’s focus of attention as the statement is being read, and incorrect information is activated from long-term memory; thus, the correct and incorrect information come into contact with each other (co-activation principle). The process of co-activation increases the likelihood that the two elements of information will be associated or integrated with each other because they are simultaneously in the reader’s focus of attention (integration principle). However, for the correct information to overpower the incorrect information, correct information must draw increasing amounts of activation toward itself and away from the incorrect information (competing activation principle).
One way for correct information to overpower incorrect information during reading is by elaborating upon a refutation statement. In a series of experiments, Kendeou et al. (2014) used the contradiction paradigm to demonstrate how a refutation statement can minimize the effect of outdated information on comprehension processes during reading. In this paradigm, participants read a text in which target sentences contradict earlier sentences. For instance, participants read a narrative text in which character information (e.g., Mary is a vegetarian) was provided at the beginning of the text and later in the text the character did an action that was inconsistent with this description (e.g., Mary orders a cheeseburger). Participants read the target sentence (e.g., Mary orders a cheeseburger) more slowly when the target sentence contradicted the character description (e.g., Mary is a vegetarian) than when the target sentence was consistent with the character description (e.g., Mary loves junk food). When a reader slows down at a target sentence that reveals a conflict with previously stated information, it demonstrates how previously encountered information can exert a continued influence on comprehension.
Importantly, Kendeou et al. (2014) investigated whether refuting character information would reduce or eliminate the influence of character information on later comprehension. For example, in the narrative that states Mary is a vegetarian, this information was then refuted (i.e., She was not getting enough vitamins because of her diet so her doctor said she had to start eating meat). Results showed that refuting character information was effective at minimizing or eliminating disruption in comprehension at the target sentences that contradicted the character information. Thus, competing activation from a refutation can be used to overpower contradictory information that is already in memory. Subsequent research using the contradiction paradigm has since replicated the finding that refutations can reduce the effects of misconceptions on comprehension when students read science content (e.g., Kendeou et al., 2014).
The contradiction paradigm is useful for identifying possible increases in reading time at sentences that conflict or contradict previously encountered information; however, it does not provide insights into the contents of readers’ thoughts when they encounter contradictory information. Proficient readers use many automatic processes during reading; however, they use active processes when they experience comprehension problems (e.g., van den Broek et al., 2005). Reading for comprehension requires deliberate and directed use of cognitive processes geared toward making meaning of the text.
Think-aloud protocols can provide more direct insights into readers’ moment-by-moment cognitive processes (Ericsson and Simon, 1993; Trabasso and Magliano, 1996; Magliano et al., 1999). Although many cognitive processes that occur during reading are automatic or are not codable in language (e.g., passive activation), proficient readers are typically aware of explanatory processes (Graesser et al., 1994) and commonly report them when thinking aloud (Trabasso and Magliano, 1996; Magliano et al., 1999; Magliano and Millis, 2003).
When reading scientific texts, students who have misconceptions about the content may have comprehension difficulties, which impede a reader’s ability to construct a coherent mental representation of a text (Diakidoy et al., 2011). Motivated readers use both superficial cognitive processing activities (e.g., paraphrasing) and deeper cognitive processing activities (e.g., generating inferences) to address comprehension difficulties (Gilliam et al., 2007). For example, readers may use paraphrasing in order to understand unfamiliar words or concepts (McNamara et al., 2004). Similarly, readers may generate bridging inferences by connecting a sentence they are currently reading with a sentence they previously read earlier in the text (Diakidoy et al., 2011), or generate elaborative inferences by using prior knowledge to enrich the meaning of a sentence they are currently reading (Gilliam et al., 2007).
Refutation texts can be used to induce cognitive conflicts in order to promote knowledge revision processes by motivating readers to engage more deeply with the text (Sinatra and Broughton, 2011; Diakidoy et al., 2016). Previous research has shown that think-alouds can reveal some of the cognitive activities that readers use to resolve cognitive conflicts when they read refutation text, including the co-activation of misconceptions and accurate information from texts, and strategic processes they use to resolve such discrepancies (Kendeou and van den Broek, 2007; Kendeou et al., 2014; McCrudden and Kendeou, 2014; Diakidoy et al., 2016; Lassonde et al., 2016; Van Boekel et al., 2017; Kendeou et al., 2019). For example, refutation texts have been shown to influence the number of valid inferences that readers make, especially for students with low and inaccurate prior knowledge (Diakidoy et al., 2011).
Despite the fact the refutation texts can promote knowledge revision processes during reading, previous research has shown that refutation texts have mixed effects on knowledge revision in science learning (Kendeou and van den Broek, 2007; Braasch et al., 2013; Van Loon et al., 2015; Lombardi et al., 2016; Mason et al., 2017). This has led researchers to investigate the features of refutation text to determine whether certain features are more likely to promote knowledge revision (Kendeou et al., 2014).
1.1. The present study
The purpose of this study was to investigate the effects of two types of refutation text on readers’ moment-by-moment cognitive processes during reading. In addition, the study sought to examine different measures of learning after reading for students with high and moderate levels of misconceptions. We sought to extend previous research by investigating whether simple and elaborated refutation text affect deliberate cognitive processes differently during reading and knowledge revision after reading. University students enrolled in a biology course completed a conceptual knowledge test on genetics. Scores on this test were used to categorize students based on their level of the misconception about genetics. Then, the participants were randomly assigned to read one of three texts about genetics: non-refutation text, a simple refutation text, or an elaborated refutation text. The non-refutation text provided basic information about genetics. Both the simple and elaborated refutation texts included the same three basic components (Hynd, 2001): (a) explicitly acknowledge a commonly held misconception (in this case about genetics), (b) directly refutes the misconception, and (c) provides an alternative perspective that is consistent with normative scientific conceptions. However, only the elaborated refutation text included an example to further elaborate upon the refutation. We included the elaborated refutation to investigate whether increasing the amount of newly encoded information would more effectively overpower the inaccurate information in the memory representation. Participants did a think-aloud task while they read their assigned texts and completed post-reading tests on knowledge of genetics.
We had two main research questions. Our first question was: Do text type and misconception status independently or interactively affect online comprehension processes? Previous research has shown that refutation texts can affect cognitive processes during reading (Kendeou and van den Broek, 2007; Kendeou et al., 2014; Diakidoy et al., 2016; Lassonde et al., 2016). However, it is unclear whether an elaborated refutation will affect reading comprehension processes differently than a simple refutation. For instance, it is unclear whether an elaborated refutation (compared to a simple elaboration) will prompt greater use of bridging inferences. One way that readers attempt to resolve discrepancies between text information and prior knowledge is by drawing inferences between refutation information and previously read sentences (e.g., Kendeou and van den Broek, 2007). Providing elaborations may promote the use of bridging inferences as readers attempt to make meaning from previously read sentences in the text. Similarly, it is unclear whether providing elaborations will impede reader-generated elaborations. In the absence of text-provided elaborations, reader may be more likely to generate the own elaborations. Alternatively, text-provide elaborations may be helpful if readers are unfamiliar with or have misconceptions about the topic.
Our second question was: Do text type and misconception status independently or interactively affect offline comprehension products? We measured participants’ performance on cued recall and conceptual knowledge tests. We predicted no differences on the cued recall measure because text type should not affect recall if the texts are not substantively different overall with respect to content (Diakidoy et al., 2011). The cued recall items pertained to information that was included across all three texts (i.e., did not assess information from the refutation statements). However, we predicted that students with misconceptions who read the simple and elaborated refutation texts would show greater knowledge revision compared to students with misconceptions who read the non-refutation text (e.g., Braasch et al., 2013). By explicitly refuting misconceptions, we predicted that the co-activation of the correct and incorrect information during reading would increase the likelihood that both would be associated, which in turn would enable them to use the information on the conceptual knowledge items. However, it was unclear whether there would be differences between the simple and elaborated refutation text conditions. On one hand, the elaborated text provided more information to support the refutation statement, which might be more beneficial than simple refutation. On the other hand, the refutation statement itself might be sufficient to promote knowledge revision, in which case the elaboration might not provide any added value.
We sought to extend previous research by investigating the effects of text type and prior knowledge on three cognitive processes associated with comprehension and two learning outcomes. Specifically, we used typed think-aloud responses to measure the extent to which participants used paraphrasing, bridging inferences, and elaborations during reading. Previous research has shown that refutation statements can improve knowledge revision processes. However, it is less clear how differences in refutation statements affect deliberate cognitive processes that support knowledge revision.
2. Methods
2.1. Participants and design
Participants included 116 undergraduates in College of Science who were enrolled in eight laboratory sections of a first-year cell biology course at a public research university in the northwestern part of the United States. Participation was voluntary and students were free to opt-out at any point during the study.
The study used a 3 (text type: non-refutation, refutation, or elaborated refutation) X 2 (misconception status: high or moderate) between-subjects experimental design. The non-refutation text presented information about genetics in a traditional expository-style common to textbook writing. The simple refutation text explicitly acknowledged and directly refuted commonly held misconceptions about genetics. The elaborated refutation text explicitly acknowledged and directly refuted commonly held misconceptions about genetics but included elaborative examples to illustrate the normatively correct ideas. Misconception status was determined by performance on the conceptual knowledge test during the pre-intervention phase. We classified participants who missed three or all four items on the conceptual knowledge test (which focused on misconceptions) as high misconception status (M = 0.75 out 4), while those who missed one or two items were grouped as moderate misconception status (M = 2.22 out of 4). An independent samples t-test indicated participants in the moderate misconception status group had higher scores on the conceptual knowledge test (and thus fewer misconceptions) than participants in the high misconception status group, t(1,114) = 18.84, p < 0.01).
2.2. Materials
2.2.1. Conceptual knowledge and cued recall tests
All of the test items were drawn from pools of standardized genetic concept tests (Lanie et al., 2004; Bowling et al., 2008). Sample of items from each test are provided in Appendix A. The conceptual knowledge test consisted of four multiple choice that assessed commonly held misconceptions about genetics among undergraduate biology students. Students completed the conceptual knowledge test during the pre-intervention phase and again during the post-intervention phase. All items were scored as correct or incorrect; scores could range from 0 to 4. The cued recall test consisted of eight multiple-choice items that assessed memory for factual information from the texts. These items were adapted from the Genetics Literacy Assessment Instrument (GLAI; Bowling et al., 2008). All items were scored as correct or incorrect; scores could range from 0 to 8.
2.2.2. Experimental texts
We used the GLAI to develop the three texts (non-refutation, simple refutation, and elaborated refutation) to reflect the different genetics concepts tested on the knowledge tests and to target three misconceptions. The texts focused on basic concepts in genetic biology such as: (1) relationships between DNA, genes, chromosomes and proteins, (2) the basic functions of genetic materials; (3) and how they determine gene expression. The text used in this study were adapted from a biology text with help from the course instructor, who is a content-knowledge expert, to ensure accuracy, relevance to the students’ course, and comprehensibility before the study. The topic was chosen from a list of topics in the course syllabus for the semester.
The non-refutation text (390 words) presented information about genetics in a traditional expository-style common to textbook writing. For example: “Deoxyribonucleic acid (DNA) is the genetic material in humans and all other organisms.” In the simple refutation text (450 words), commonly held misconceptions about genetics were explicitly acknowledged and directly refuted. For example: “Some people think that only higher-order organisms have deoxyribonucleic acid (DNA). This is not true however; DNA is the genetic material in humans and all other organisms.” The first segment explicitly refuted the misconception (This is not true however); the second segment provided the correct information (DNA is the genetic material in humans and all other organisms). The elaborated refutation text (527 words) contained the same refutation statements as the simple refutation text, but with an elaborative example. For example: “Some people think that only higher order organisms have deoxyribonucleic acid (DNA). This is not true however; DNA is the genetic material in humans and all other organisms – which includes plants and animals, as well as lower organisms like protists, archaea and bacteria.”
2.3. Procedure
Participants attended 3-h lab sessions once a week as a requirement for their undergraduate biology course. Data collection occurred in three phases, occurring during the first hour of each of the three lab sessions. In the pre-intervention phase, participants completed the prior knowledge test (which also included conceptual knowledge items intended to assess misconceptions about genetics). Then, participants were randomly assigned to read one of the three text types.
In the intervention phase, which took place 3 weeks later, groups were assigned to one of three experimental conditions using the Qualtrics® survey platform and were oriented to the reading task procedure, which was adopted from previous research (Gilliam et al., 2007; Magliano et al., 2011). Similar to how think-aloud procedures are implemented, participants were prompted to stop and reflect on segments of the texts and type out their thoughts after reading couple of sentences. As shown in Figure 1, the platform included read-only pages and read-plus-prompt pages. Both the read-only and read-plus-prompt pages presented sentences from the text; however, only the read-plus-prompt pages provided an entry box for participants to type their thoughts in response to the prompt, “What are you thinking about now?” Participants were informed that they had to type a response before they could progress to the next page. Participants viewed a practice text to familiarize themselves with the typed verbal (think-aloud) protocol. Further, they were shown examples of better and worse typed responses to help them understand their task. Participants received a practice prompt after they viewed better and worse examples. The instruction pages were programmed to disable the “Next” button for a few seconds to discourage participants from simply scrolling through without reading the instructions and completing the practice tasks.
Figure 1. Participant score on the conceptual knowledge test across the three text groups.
The text was presented one sentence at a time per screen as has been done in previous studies and we recorded reading time per segment (e.g., Kendeou and van den Broek, 2005). Participants progressed through the text by clicking the “Next” button provided on the bottom of the screen. Each paragraph was introduced by the phrase “New Paragraph” to signal the text’s organizational structure.
Once participants had completed the practice section, they began reading their respective texts. They were prompted to think-aloud after reading the three segments that pertained to the misconceptions; we refer to these three prompts as target prompts to differentiate them from the other five prompts. Two of the target prompts occurred in the first paragraph and the third occurred in the third paragraph. The five non-target prompts were included for experimental control. One of the non-target prompts occurred at the end of the first paragraph, one each in the second and third paragraphs, and two in the last paragraph. Thus, the three targeted segments pertained to information related to refutation statements and the five non-targeted segments pertained to information common to all three texts. Participants could not progress through the task if they did not provide a typed response when prompted. Immediately after completing the reading task, participants completed the cued recall and conceptual knowledge tests. The intervention and testing lasted approximately 50 min.
2.4. Coding
2.4.1. Think-aloud typed protocol
Students responses were analyzed using the Reading Strategy Assessment Tool (R-SAT) scoring protocol, which is a computer program that is used to analyze the verbal protocol that readers generate as they read (Magliano et al., 2011, 2016). Skilled readers use several strategies to enhance comprehension (McNamara et al., 2004). Magliano et al. (2011) found that it is possible to determine the comprehension strategies readers use while reading using Latent Semantic Analysis to analyze overlaps between sentences in a text read and the verbal protocol generated when prompted to respond to the text.
The RSAT protocol estimates the use of comprehension strategies (bridging inferences, elaborations, and paraphrases) using series of generated scores that examine using a computer algorithm. The algorithm determines comprehension strategies by comparing words that show up in the verbal protocols elicited from participants’ response when prompted to words in the text they had read. Bridging inferences scores is deduced from the total sum of content words in respondents’ verbal protocol that were from the sentence immediately prior to a targeted sentence in the text; elaborations score is deduced from a sum of the number of words in participants’ typed responses that were not present in the text passage; and paraphrases is deduced from the sum of similar words from the assigned reading text that appeared in participants’ typed responses. The theoretical underpinnings and algorithmic procedures of the RSAT is extensively discussed in Gilliam et al. (2007) and Magliano et al. (2011).
2.4.2. Reading time
We collected reading time data to control for overall time spent reading. Each screen of the platform was programmed to capture the time spent reading each text segment and the time spent typing in response to the think-aloud prompts. Reading time on the read-only pages was computed as the difference between the time they entered the page and the time they exited the page. When participants finished reading a text segment, they clicked on the “Next” button, and the next page would appear. Reading time on the read-plus-prompt pages was computed differently. When participants entered a read-plus-prompt page, only the text segment was presented. When participants finished reading the text segment, they clicked on the “Next” button. When they did this, the think-aloud prompt and response box would appear below the text segment. After they finished tying, they would click the “Next” button again and the next text segment would be presented. Thus, reading time for the read-plus-prompt pages was computed as difference between the time they entered the page and they time they clicked the “Next” button and the think-aloud prompt and response box appeared. We did not analyze the amount of time participants spent typing. We converted the raw reading time data to seconds per word for each text. This enabled us to control for differences in the number of words across the three text types.
3. Results
3.1. Do text type and misconception status independently or interactively affect online comprehension processes?
Preliminary analysis indicated that assumptions of homogeneity of variance and non-multicollinearity between variables were satisfied. We conducted 3 (text type: non-refutation, refutation, or elaborated refutation) X 2 (misconception status: high or moderate) multivariate analyses of variance (MANCOVA) with text type and misconception status as between-subject variables on participants’ typed verbal protocols for the targeted and non-targeted segments with comprehension activities (paraphrases, bridging and elaborations) as the dependent variables. The targeted segments pertained to information related to refutation statements; the non-targeted segment pertained to information common to all three texts. Reading time was used as covariate to account for differences in text length.
The first analysis focus on targeted segments. The main effect for misconception status was not significant, F(3, 108) = 1.53, p = 0.21, Wilk’s Λ = 0.96, nor was the interaction effect, F(6, 216) = 0.745, p = 0.614, Wilk’s Λ = 0.96. However, the main effect for text type was significant, F(6, 216) = 7.39, p < 0.001; Wilk’s Λ = 0.69 (see Table 1). After controlling for reading time due to differences in the number of total words across text types, the between-subject effects analysis showed that text type had a significant effect on participants’ use of bridging inferences, F(2, 109) = 15.98, p < 0.001, η2p = 0.23. Pairwise comparisons indicated that participants who read the elaborated refutation text used more bridging inferences than those who read the non-refutation and the simple refutation texts (p <. 001). There was no statistically significant difference between the non-refutation and simple refutation groups (p = 0.29). Text type also had a significant effect on participants’ use of elaborations, F(2, 109) = 3.84, p = 0.025, η2p = 0.07. Pairwise comparisons indicated that participants who read the simple and elaborated refutation texts used more elaborations (p > 0.031, d = 0.55; and p > 0.01, d = 0.61) than those who read the non-refutation text. Text type did not have a significant effect on participants’ use of paraphrases, F(2, 114) = 2.97, p = 0.056, η2p = 0.05.
Table 1. Means and standard deviations of online processing strategies by text type and misconception status for targeted text segments.
The second analysis focused on non-targeted segments. The non-targeted segments of the texts were identical across the three different text types. This analysis was used as an experimental control. The main effects for misconception status (F(3, 108) = 0.47, p < 0.71) and text type (F(6, 216) = 1.81, p = 0.10) were not significant, nor was the interaction effect (F(6, 216) = 0.45, p = 0.85).
Thus, text type affected online comprehension processes. When reading targeted segments, participants who read the elaborated refutation text used more bridging inferences than those who read the non-refutation and the simple refutation texts, and participants who read the simple and elaborated refutation texts used more elaborations than those who read the non-refutation text. However, text type did not affect use of paraphrases. Conversely, there were no differences in the use of bridging inferences, elaborations, or paraphrases when participants read the non-targeted segments.
3.2. Do text type and misconception status independently or interactively affect offline comprehension products?
3.2.1. Cued recall
We conducted a 3 (text type) X 2 (misconception status) ANOVA to examine the effects of text type and misconception status on cued recall. Descriptive statistics of recall scores are report in Table 2. The main effects for text type and misconception status on cued recall were not significant, F(2, 110) = 0.13, p = 0.88) and F(1, 110) = 0.53, p = 0.47) respectively, nor was the interaction effect significant, F(2, 110) = 0.064, p = 0.94).
Table 2. Mean frequency and standard error by text type and misconception status for Cued Recall test.
3.2.2. Conceptual knowledge test
We conducted a 2 (time: before reading and after reading) x 3 (text type: non-refutation, refutation, or elaborated refutation) X 2 (misconception status: high or moderate) mixed model ANOVA with time (pre-intervention and post-intervention) as a within-subject variable and text type and misconception status as between-subjects variables. Descriptive statistics of conceptual knowledge scores are report in Table 3. The main effect for time was significant, F(1, 110) = 116.12, p < 0.01, η2p = 0.51, such that conceptual knowledge scores increased from pre-test (M = 1.45) to post-test (M = 2.59) across all groups (see Figure 1). The main effect for misconception status was significant F(1, 110) = 18.86, p < 0.01, η2p = 0.15, such that participants with moderate misconception status (M = 2.55) had higher scores than participants with high misconception status (M = 1.54). Neither the main effect for text type, F(2, 110) = 0.66, p = 0.52, η2p = 0.15, nor the interaction effect were significant, F(2, 110) = 0.08, p = 0.92, η2p = 0.002.
Table 3. Mean frequency and standard error for text type and misconception status for Transfer test.
4. Discussion
The purpose of the study was to investigate the effects of refutation text structure on readers’ moment-by-moment processing during reading and measures of learning after reading for students with high and moderate levels of misconceptions. The think-aloud data indicated that the refutation text affected readers’ cognitive processes when they read the targeted segments independently of whether they had high or moderate levels of misconceptions. Specifically, participants who read the elaborated refutation text used more bridging inferences and elaborations than participants who read the non-refutation text. Similarly, participants who read the simple refutation text used more elaborations than participants who read the non-refutation text–consistent with earlier finds that text structure affects how readers process text (Trevors and Muis, 2015). Further, there were no difference among the three conditions on the use of paraphrases. These findings replicate previous research which has shown that readers with misconceptions detect discrepancies between their misconceptions and normatively correct conceptions when they read refutation texts and attempt to explain the current text segments based on previously read text segments and/or background knowledge (e.g., Kendeou et al., 2019).
We sought to extend previous research by investigating whether readers would process elaborated and simple refutation text differently. The think-aloud data indicated two main findings. First, participants who read the elaborated and simple refutation texts did not differ in their use of elaborations, yet they both generated more elaborations than participants who read the non-refutation text. One possible explanation for this finding is that readers who read the refutation texts detected discrepancies between their conceptions and the information in the text and then attempted to resolve these discrepancies by explaining the current text based on previous text and background knowledge. Participants who read the non-refutation texts may not have detected such discrepancies, and, thus, did not attempt to address them. Second, participants who read the elaborated refutation text used more bridging inferences than students who read the simple refutation text. One possible explanation for this finding is that participants who read the elaborated refutation text may have used the provided elaborations to make sense of the previously read sentences in order to resolve discrepancies between their conceptions and the information in the text.
The post-reading data indicated that neither text structure nor misconception status affected performance on the cued recall or conceptual knowledge measures. Previous research has shown that refutation text structure may not affect performance on recall measures (Diakidoy et al., 2016; Lassonde et al., 2016; Kendeou et al., 2019). This outcome was expected because the texts were not substantively different with respect to overall content. That is, the cued recall items pertained to information from all three texts (i.e., did not assess information from the refutation statements). Further, previous research has shown that misconception status does not affect performance on recall measures (Diakidoy et al., 2011). Again, given that the cued recall measure did not assess information from the refutation statements, it could be expected that misconception status would not affect performance on this measure.
It is unclear why neither text structure nor misconception status affected performance on the conceptual knowledge test. There was a main effect for time, which suggests that reading about these topics promoted students’ understanding of the topics. Further, Figure 1 suggests that participants who read the simple or the elaborated refutation texts performed better on the conceptual knowledge test than those who read the non-refutation text (d = 0.27 and d = 0.11, respectively), although this finding should be interpreted with caution given that the main effect for text type was not significant. Online and offline measures sometimes provide divergent data about comprehension (Rapp and Mensink, 2011). Although text structure may affect moment-by-moment processing, it may not necessarily affect post-reading measures of learning. For instance, readers might attempt to resolve discrepancies between text information and prior knowledge, but unsuccessfully resolve prior misconceptions.
4.1. Theoretical and practical implications
The main conclusion from the present study is that refutation text structure promotes the use of cognitive processes that support text comprehension. This has both theoretical and practical implications. From a theoretical perspective, refutation statements can promote deliberate attempts to resolve discrepancies between text information and prior knowledge. Further, elaborated refutations appear to be more effective at promoting comprehension processes than simple refutations. It may be the case that elaborated refutations are more readily integrated into the reader’s mental model of the text, which in turn may amplify the refutation effect and help readers overpower inaccurate information that rise to the surface in their working memory because of the competing activation principle. For a practical perspective, it appears that providing illustrative examples in refutations can help readers as they attempt to comprehend refutation statements. If readers have inaccurate understandings, such illustrations might promote meaningful information that can help readers update their knowledge about a topic.
4.2. Limitations and future research directions
There were several limitations to the study. First, the learning measures had marginal to low reliability. It is plausible that these measures were not sufficiently sensitive to participants’ knowledge about the topic. Thus, the we found no significant difference in the effect of the text structures on the offline learning measures observed. Future research should include measures that have greater reliability. Alternatively, the think-aloud procedure may have introduced unintended generative learning effects that could have potentially overshadowed any text-based effects on offline learning measures among participants. Participants were intermittently prompted to think-aloud and write out their thoughts about the text. This could have unintended meta-cognitive learning benefits than if they read the texts silently and uninterrupted. Third, verbalizing one’s thoughts during reading can change how readers normally process text, nor does it reveal covert reading processes (Kendeou et al., 2014). In the same vein, requiring students to type out their thoughts could have gotten in the way of naturalistic learning. Future research could investigate online processes in more naturalistic reading context, such as with the use of a reading time or eye-tracking methodology. Lastly, we only used a short and very narrowly scoped text, which raises questions about how well our findings are generalizable to other similar reading contexts. Future studies could investigate the effects of modifying text structure on cognitive processes when students read texts with word lengths that reflect realistic reading exercises they get in class, or that examine other topics.
In conclusion, the present study contributes new insights into how refutation text affects the moment-by-moment cognitive processes when students read educationally-relevant text. In particular, refutations with illustrative examples can increase the use of cognitive processes that support comprehension more than refutations without illustrative examples or the absence of refutation statements.
Data availability statement
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
Ethics statement
The studies involving human participants were reviewed and approved by Washington State University. The patients/participants provided their written informed consent to participate in this study.
Author contributions
NH initiated the study idea whilst being involved in the design, instrument development, data collection, analysis, and final report writing. OA was involved in designing the study reported in this article, developing the material used for data collection, interpreting study results, and revising the manuscript. MM was involved in refining the study design, interpreting results of data analysis, and revising the manuscript.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/feduc.2023.1112804/full#supplementary-material
References
Bowling, B. V., Acra, E. E., Wang, L., Myers, M. F., Dean, G. E., Markle, G. C., et al. (2008). Development and evaluation of a genetics literacy assessment instrument for undergraduates. Genetics 178, 15–22. doi: 10.1534/genetics.107.079533
Braasch, J. L. G., Goldman, S. R., and Wiley, J. (2013). The influences of text and reader characteristics on learning from refutations in science texts. J. Educ. Psychol. 105, 561–578. doi: 10.1037/a0032627
Diakidoy, I. A. N., Mouskounti, T., Fella, A., and Ioannides, C. (2016). Comprehension processes and outcomes with refutation and expository texts and their contribution to learning. Learn. Instr. 41, 60–69. doi: 10.1016/j.learninstruc.2015.10.002
Diakidoy, I. N., Mouskounti, T., and Ioannides, C. (2011). Comprehension and learning from refutation and expository texts. Read. Res. Q. 46, 22–38. doi: 10.1598/RRQ.46.1.2
Ericsson, K. A., and Simon, H. A. (1993). Protocol Analysis: Verbal Reports as Data (revised edition). Cambridge, MA: Bradford books/MIT Press.
Gilliam, S., Magliano, J. P., Millis, K., Levinstein, I., and Boothum, C. (2007). Assessing the format of the presentation of text in developing a Reading strategy assessment tool (R-SAT). Behav. Res. Methods 39, 199–204. doi: 10.3758/BF03193148
Graesser, A. C., Singer, M., and Trabasso, T. (1994). Constructing inferences during narrative text comprehension. Psychol. Rev. 101, 371–395. doi: 10.1037/0033-295X.101.3.371
Kendeou, P., Butterfuss, R., Kim, J., and Van Boekel, M. (2019). Knowledge revision through the lenses of the three-pronged approach. Mem. Cogn. 47, 33–46. doi: 10.3758/s13421-018-0848-y
Kendeou, P., and O’Brien, E. J. (2014). “The knowledge revision components (KReC) framework: processes and mechanisms” in Processing Inaccurate Information: Theoretical and Applied Perspectives from Cognitive Science and the Educational Sciences. eds. D. N. Rapp and J. L. G. Braasch (Cambridge, MA: MIT Press).
Kendeou, P., and van den Broek, P. (2005). The effects of readers’ misconceptions on comprehension of scientific text. J. Educ. Psychol. 97, 235–245. doi: 10.1037/0022-0663.97.2.235
Kendeou, P., and van den Broek, P. (2007). Interactions between prior knowledge and text structure during comprehension of scientific texts. Mem. Cogn. 35, 1567–1577. doi: 10.3758/BF03193491
Kendeou, K., Walsh, E. K., Smith, E. R., and O'Brien, E. J. (2014). Knowledge revision processes in refutation texts. Disc. Process. 51, 374–397. doi: 10.1080/0163853X.2014.913961
Kintsch, W. (1998). Comprehension: A Paradigm for Cognition. Cambridge, UK: Cambridge University Press.
Lanie, A. D., Jayaratne, T. E., Sheldon, J. P., Kardia, S. L. R., Anderson, E. S., Feldbaum, M., et al. (2004). Exploring the public understanding of basic genetic concepts. J. Genet. Couns. 13, 305–320. doi: 10.1023/B:JOGC.0000035524.66944.6d
Lassonde, K. A., Kendeou, P., and O'Brien, E. J. (2016). Refutation texts: overcoming psychology misconceptions that are resistant to change. Scholarsh. Teach. Learn. Psychol. 2, 62–74. doi: 10.1037/stl0000054
Lombardi, D., Danielson, R. W., and Young, N. (2016). A plausible connection: models examining the relations between evaluation, plausibility, and the refutation text effect. Learn. Inst. 44, 74–86. doi: 10.1016/j.learninstruc.2016.03.003
Magliano, J. P., and Millis, K. K. (2003). Assessing reading skill with a think-aloud procedure and latent semantic analysis. Cogn. Instr. 21, 251–283. doi: 10.1207/S1532690XCI2103_02
Magliano, J. P., Millis, K. K., The RSAT Development Team, Levinstein, I., and Boonthum, C. (2011). Assessing comprehension during Reading with the Reading strategy assessment tool (RSAT). Metacogn. Learn. 6, 131–154. doi: 10.1007/s11409-010-9064-2
Magliano, J. P., Ray, M., and Millis, K. K. (2016). “The reading strategy assessment tool: a computer-based approach for evaluating comprehension processes during reading,” in Adaptive Educational Technologies for Literacy Instruction. Routledge, 282–287.
Magliano, J. P., Trabasso, T., and Graesser, A. C. (1999). Strategic processing during comprehension. J. Educ. Psychol. 91, 615–629. doi: 10.1037/0022-0663.91.4.615
Mason, L., Baldi, R., Di Ronco, S., Scrimin, S., Danielson, R. W., and Sinatra, G. M. (2017). Textual and graphical refutations: effects on conceptual change learning. Contemp. Educ. Psychol. 49, 275–288. doi: 10.1016/j.cedpsych.2017.03.007
McCrudden, M. T., and Kendeou, P. (2014). Exploring the link between cognitive processes and learning from refutational text. J. Res. Read. 37, S116–S140. doi: 10.1111/j.1467-9817.2011.01527.x
McNamara, D. S., Levinstein, I. B., and Boonthum, C. (2004). iSTART: interactive strategy training for active reading and thinking. Behav. res. meth. instrum. comput. 36, 222–233.
Rapp, D. N., and Mensink, M. C. (2011). “Focusing effects from online and offline reading tasks” in Text Relevance and Learning from Text. eds. M. T. McCrudden, J. P. Magliano, and G. Schraw (Charlotte, NC: IAP Information Age Publishing)
Sinatra, G. M., and Broughton, S. H. (2011). Bridging reading comprehension and conceptual change in science education: the promise of refutation text. Read. Res. Q. 46, 374–393. doi: 10.1002/RRQ.005
Trabasso, T., and Magliano, J. P. (1996). Conscious understanding during comprehension. Disc. Process. 21, 255–287. doi: 10.1080/01638539609544959
Trevors, G., and Muis, K. R. (2015). Effects of text structure, reading goals and epistemic beliefs on conceptual change. J. Res. Read. 38, 361–386. doi: 10.1111/1467-9817.12031
Van Boekel, M., Lassonde, K. A., O'Brien, E. J., and Kendeou, P. (2017). Source credibility and the processing of refutation texts. Mem. Cogn. 45, 168–181. doi: 10.3758/s13421-016-0649-0
van den Broek, P., Rapp, D. N., and Kendeou, P. (2005). Integrating memory-based and constructionist processes in accounts of reading comprehension. Disc. Process. 39, 299–316. doi: 10.1080/0163853X.2005.9651685
Keywords: refutation text, text processing, conceptual change, verbal protocols, comprehension strategies
Citation: Hunsu NJ, Adesope O and McCrudden MT (2023) The effects of text structure on students’ use of comprehension strategies and cognitive outcomes during science text processing. Front. Educ. 8:1112804. doi: 10.3389/feduc.2023.1112804
Edited by:
Joseph Magliano, Georgia State University, United StatesReviewed by:
Jason Braasch, Georgia State University, United StatesSarah Carlson, Georgia State University, United States
Copyright © 2023 Hunsu, Adesope and McCrudden. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Nathaniel J. Hunsu, bmF0aGFuaWVsLmh1bnN1QHVnYS5lZHU=