Text quality and changing perceptions of teacher feedback and affective-motivational variables: a study with secondary EFL students

Siekmann, Lea; Parr, Judy; Van Ophuysen, Stefanie; Busse, Vera

doi:10.3389/feduc.2023.1171914

ORIGINAL RESEARCH article

Front. Educ., 11 May 2023

Sec. Educational Psychology

Volume 8 - 2023 | https://doi.org/10.3389/feduc.2023.1171914

This article is part of the Research Topic Student and Teacher Writing Motivational Beliefs View all 18 articles

Text quality and changing perceptions of teacher feedback and affective-motivational variables: a study with secondary EFL students

Lea Siekmann¹^*

Judy Parr²

Stefanie Van Ophuysen¹

Vera Busse¹

¹Institute of Education, Faculty of Education and Social Science, University of Münster, Münster, Germany
²School of Curriculum and Pedagogy, Faculty of Education and Social Work, The University of Auckland, Auckland, New Zealand

Introduction: Feedback can support students’ writing and has the potential to enhance writing motivation and reduce writing anxiety. However, for feedback to fulfill its potential, it has to be accepted by students and perceived as motivating.

Methods: In this study, we investigate changes in less proficient English as a foreign language (EFL) students’ (N = 53) writing motivation and affect, as well as their perceptions of teacher feedback and how these relate to students’ argumentative text quality. Measurements were taken before EFL teachers attended a professional learning intervention on feedback (T1) and 8 months later (T2).

Results: From T1 to T2, students felt that general feedback quality improved, their writing self-efficacy increased, and their writing anxiety decreased. However, no significant changes in text quality could be observed between T1 and T2, and students continued to struggle with creating structure and coherence in their texts. Regression analyses revealed that feedback perceptions and affective-motivational variables did not predict students’ text quality at T1. Yet at T2, students’ perception of general feedback quality and the effect of feedback on writing motivation were significant predictors of text quality; self-efficacy and writing anxiety were not.

Discussion: Our results suggest that more attention needs to be paid to feedback’s motivational impact, especially among less proficient EFL writers.

1. Introduction

Writing plays a vital role in communication. However, developing the ability to write texts that adequately convey the writer’s intention to a target audience is a time-intensive and demanding process (Kellogg, 2008) and may be even more challenging when writing in a foreign language (FL) (Hyland, 2003; Galbraith, 2009). The challenges FL writers face may not only be based on text knowledge that develops through time and teaching but can additionally be rooted in motivational problems, such as low competence beliefs, a lack of writing enjoyment, or the presence of writing anxiety (e.g., Teimouri et al., 2019; Zumbrunn et al., 2019; Sun et al., 2021).

Feedback can be a powerful educational tool to support students’ writing development (Parr and Timperley, 2010; Biber et al., 2011; Graham et al., 2015; Busse and Scherer, in press) and writing motivation (Bruning and Horn, 2000; Camacho et al., 2021). Hattie and Timperley (2007) define feedback as information about one’s performance or comprehension and emphasize that feedback should answer three questions to enhance learning: Where am I going? How am I going? Where to next? Teachers, thus, need to make learners aware of the learning goals (feed up), make progress toward the learning goals visible to students (feed back), and explain how to move forward to close the gap between current performance and the desired goals (feed forward). This has been referred to as the old feedback paradigm where feedback is considered an information-sharing process aiming to improve student learning (see Winstone and Carless, 2020).

Current literature, however, often moves beyond an information-based approach and frames feedback as an interactive process in which teachers and learners engage in meaningful dialog (Henderson et al., 2019; Carless and Winstone, 2020; Lee, 2021). By placing the student at the center of feedback, the student’s role in generating, making sense of, and using feedback is emphasized. This new feedback paradigm broadens the view toward students’ perceptions, motivation, and understanding of feedback. For instance, students need to be motivated and able to regulate their emotions to act upon the feedback, make sound judgments, and use it for improvement (see also Carless and Boud, 2018).

Given that writing motivation is receptive to change (Graham, 2018a), one may also assume that changes in motivation attributable to feedback may generate different writing outcomes. As students’ writing motivation declines throughout the course of schooling (Boscolo and Hidi, 2007; De Smedt et al., 2020), exploring how feedback could be used to foster writing motivation seems particularly relevant. Yet, little is known about students’ perceptions of feedback’s quality and its motivational impact, and there is a general scarcity of research on how feedback perceptions and affective-motivational beliefs are associated with individual differences in writing quality. This study addresses existing research gaps and investigates the effect of feedback, self-efficacy and anxiety on text quality in English as a foreign language (EFL).

2. Theoretical and empirical background

2.1. Affective-motivational variables related to writing

Writing is perceived as motivationally challenging for many students, with EFL learners being no exception (Lee et al., 2018). For writing to be successful, special attention needs to be paid to students’ motivation (Bruning and Horn, 2000). Reconciling different definitions of the past 40 years, Abdel Latif (2021) frames writing motivation as “an umbrella term encompassing learners’ liking or disliking of writing situations and perceived value of writing, the situational feelings they experience while writing and the way they regulate them, the beliefs about their writing ability and skills, and their desired goals for learning to write” (p. 3). This definition illustrates that writing motivation is a multidimensional construct (see also Graham, 2018b) subsuming several concepts. Although research on writing motivation is still in its early stages (Lee et al., 2018), a systematic review by Camacho et al. (2021) offers insight into this research area and identifies 24 motivation-related constructs. Writing self-efficacy appears to be the most studied construct (n = 37), while relatively few studies explore affective variables such as writing anxiety (n = 2) or enjoyment of writing (n = 7). In the following paragraphs, we look at what exactly constitutes these constructs and how they relate to writing achievements.

2.1.1. Self-efficacy in writing

Self-efficacy beliefs can be understood as the confidence to perform successfully in a particular domain (Bandura, 1997). It is assumed that four factors contribute to self-efficacy beliefs: mastery experience, vicarious experience, verbal persuasion, and physiological arousal (Bandura, 1997). Self-efficacy can thus stem from having successfully managed a similar situation in the past (mastery experience), from knowing that people with similar abilities are capable of managing the situation (vicarious experience), from gaining self-confidence in one’s own abilities through positive affirmation by others (verbal persuasion), or from successfully dealing with physical tension and turning it into relaxation (physiological arousal). Usher and Pajares (2008) compiled a review of the role each source of self-efficacy plays in different domains. Mastery experience is reported to be the most impactful source of self-efficacy for various academic fields, while vicarious experience and social persuasions appear to be less associated with self-efficacy. This finding, however, needs to be viewed with caution given that measures of the two latter sources are inconsistent across studies. While the first three sources generally are related positively to self-efficacy, physiological arousal was found to predict self-efficacy negatively. In the context of writing, Pajares et al. (2007) investigated writing self-efficacy among 1,256 students at elementary, middle, and high school. Similar to the general results presented above, the largest proportion of variance in students’ self-efficacy was explained by the experience of mastering writing, regardless of school type, while vicarious experience had no predictive power. For elementary and middle school students, physiological indices (operationalized by Pajares et al., 2007 in terms of anxiety/stress) also significantly predicted self-efficacy, with middle school students showing a quadratic relationship between anxiety and self-efficacy. That is, while low and high anxiety scores predicted self-efficacy beliefs, moderate anxiety did not. In contrast, for high school students, social beliefs instead of anxiety were significant for self-efficacy beliefs. Therefore, Pajares et al. (2007) suggest focusing on writing skill development to facilitate students’ mastery experience and, thereby, strengthen their writing self-efficacy. Looking at the authors’ findings on high school students, one may also argue that self-persuasion methods or praise related to specific aspects of students’ work or progress (for a detailed discussion see Hattie and Timperley, 2007; Hattie et al., 2016) can foster students’ writing self-efficacy which can, in turn, also affect their writing achievements.

Literature reviews from the early twenty-first century report that students’ beliefs in their L1 writing capabilities are usually positively associated with writing outcomes (Klassen, 2002; Pajares, 2003). Studies show that students with higher levels of self-efficacy often tend to perform better in writing (e.g., Pajares et al., 2000) and that students with high writing proficiency possess higher levels of self-efficacy (Raoofi et al., 2017). Self-efficacy was found to positively predict writing quality of students in grade 4 (Graham et al., 2017), up to grade 10 (Troia et al., 2013) and in grade 11 (Yilmaz Soylu et al., 2017). However, single studies also report no or opposite relations between self-efficacy and writing performance. For example, in a study by Braaksma et al. (2018), self-efficacy and text quality were positively correlated among students in grade 11, but there were no significant correlations between those variables among students in grade 10. Similarly contradicting the literature presented above, Wijekumar et al. (2019) reported that self-efficacy in L1 writing did not independently predict writing quality of students in grade 5. Such findings may be explained partially by the fact that not all students succeed at evaluating their performance adequately. Although it appears reasonable that confidence to perform well in writing coincides with actual writing performance, some findings also show a mismatch between self-efficacy beliefs and writing achievements among less proficient writers with learning disabilities in grades 4 to 10 (Graham et al., 1993; Klassen, 2002). Particularly, these writers tend to overestimate their performance, revealing an illusion of competence (Kruger and Dunning, 1999), a phenomenon also found in other studies with less proficient writers (Anastasiou and Michail, 2013; Busse et al., in press). Such a mismatch between self-efficacy beliefs and actual performance is also referred to as low calibration (Schunk and Usher, 2012; Schunk and DiBenedetto, 2016) which students may especially encounter when feeling efficacious about performing difficult tasks without actually being aware how to complete them successfully (Wigfield et al., 2012; Chen and Zhang, 2019).

To what extent these results can be transferred to L2 writers in general and EFL writers in particular needs to be further investigated. A meta-analysis by Sun et al. (2021) revealed that self-efficacy had an even higher impact on L2 than L1 writing. The studies compiled in the meta-analysis were often conducted among adult learners. For example, Sun and Wang (2020) found that writing self-efficacy contributed significantly to college students’ scores in EFL essay writing. Similarly, Zabihi (2018) reported that writing self-efficacy positively predicted complexity, accuracy, and fluency in university students’ EFL narrative texts. While these studies imply high calibration between students’ self-efficacy and writing performance, a study by Chen and Zhang (2019) that investigated the relation between self-efficacy beliefs about surface and deep-level text revision and the frequency of such revisions in EFL university students’ argumentative writing showed no significant relation between beliefs and performance. Thus, in the EFL context, there is evidence for a positive relationship between self-efficacy and writing performance as well as for a mismatch as described above. However, the available studies mainly provide insight into possible associations among adult learners, while findings for school students are scarce. Closing this research gap by exploring ways to improve the low EFL writing proficiency of secondary school students (see Harsch et al., 2008; Siekmann et al., 2022) through self-efficacy development might therefore be beneficial.

2.1.2. Writing anxiety

Another motivation-related construct that has received little attention in the L1 writing context (see Camacho et al., 2021), but even less attention in L2 and FL research, is writing anxiety. Anxiety in language learning contexts has often been referred to with notions of tension or apprehension (see MacIntyre and Gardner, 1994; Cheng, 2002). As seminal work by Cheng (2004) suggests, FL writing anxiety can be seen as a three-dimensional construct. The author established and validated a scale to measure writing anxiety (the Second Language Writing Anxiety Inventory, SLWAI) which uses three subscales: somatic anxiety, cognitive anxiety, and avoidance behavior. Firstly, somatic anxiety refers to the increased physiological arousal learners may encounter when writing in a FL. Secondly, cognitive anxiety represents the individuals’ perception of arousal and also their worry or fear of negative evaluation. Lastly, avoidance behavior addresses learners’ tendencies to avoid FL writing.

Cheng’s (2004) scale has been widely used by researchers to investigate FL writers’ anxiety (see Tahmouresi and Papi, 2021). In general, studies report negative relations between students’ writing anxiety and their L2 writing performance. For example, a meta-analysis by Teimouri et al. (2019) investigating L2 and FL anxiety focusing on different language skills found that L2 and FL writing anxiety negatively impacted learners’ engagement in writing and their writing performance. Their analyses also show differences between students of different educational levels. L2 language anxiety and achievement seem to be closely related among elementary students, but this effect decreases up to junior high school. In senior high school and college, the relationship between students’ anxiety and writing achievements increases again. However, these findings must be considered with caution given that studies focusing on junior high learners are limited (see Teimouri et al., 2019). Interestingly, Teimouri and colleagues also report that the negative relationship between anxiety and achievement is less pronounced when English is the target L2 or FL. Teimouri et al. (2019) explain this with English’s status as lingua franca; due to its presence in daily life, students may be more familiar and less anxious when learning EFL. These findings, however, may not refer directly to EFL writing given that most studies in the meta-analysis investigate anxiety when speaking in a L2 or FL.

Focusing on anxiety in EFL writing, Tahmouresi and Papi (2021) also found anxiety to predict university students’ writing course grades negatively. Similarly, Zabihi (2018) showed that anxiety negatively predicted complexity, accuracy and fluency in EFL university students’ narrative texts. Although these effects seem to be unambiguous, some studies also suggest that writing anxiety interacts with other motivation-related constructs when affecting students’ writing; self-efficacy beliefs seem to mediate writing anxiety and negative effects of anxiety on writing performance may disappear when students’ self-efficacy is controlled for Pajares et al., 1999 and Pajares, 2003. For example, Han and Hiver (2018) found that EFL writers at middle school with elevated levels of writing anxiety still performed successfully on writing tasks, if they also displayed moderate to strong levels of self-efficacy. Interestingly, Busse et al. (in press) also found that anxiety was positively related to text quality in low-efficacious students with a migration background. These studies thus suggest that anxiety in L2 and FL writing may impact students’ writing performance in a more nuanced way than reported in other studies. Therefore, further studies investigating the effect of different motivational constructs and its effect on FL writing of high school students seem necessary (see also Camacho et al., 2021).

2.1.3. Enjoyment of writing

While research has already begun to examine self-efficacy and anxiety in L2 writing, studies of positive emotions associated with motivation such as enjoyment have long been neglected (see Dewaele, 2022). In general, enjoyment can be understood as feelings of pleasure one encounters during an activity (Tahmouresi and Papi, 2021). Transferring this to the context of L2 learning, enjoyment is also described as “positive emotions that language learners experience in the process of learning or using the target language” (Teimouri, 2017, p. 689). Similar to other motivation-related constructs, enjoyment is reported to affect students’ performance. For instance, more advanced and proficient language learners at secondary school experience higher levels of FL enjoyment (Dewaele et al., 2018; Dewaele and Alfawzan, 2018; Mierzwa, 2018). While enjoyment was found to be positively correlated with FL achievement among high-achieving high school students, no such relations were found among low-achieving students (Li et al., 2020).

Considering that the interest in researching enjoyment is only gaining momentum, it is not surprising that studies on enjoyment of writing are relatively scarce. The few existing studies tend to examine the effect of different interventions on students’ enjoyment of writing. For instance, single studies investigated to what extent writing in digital contexts had an impact on students’ enjoyment of writing and found positive effects (Beck and Fetherston, 2003; Lan et al., 2011). How students’ enjoyment of writing is related to their performance remains rather unclear. Initial insights are presented in a study by Zumbrunn et al. (2019), in which the authors investigated how elementary students’ enjoyment of writing was related to their quarterly writing grades. Using structural equation modeling, the authors found that students with higher writing enjoyment tended to receive higher writing grades. Arguably, more research investigating students’ FL writing enjoyment in general and, particularly, its effect on writing performance and text quality is needed.

2.2. The impact of feedback on text quality and affective-motivational variables

Formative feedback has proven effective in enhancing teaching and supporting students’ learning progress (see Shute, 2008; Hattie, 2009; Brookhart, 2018). Likewise, feedback was shown to have beneficial effects for students’ writing (see meta-analyses by Biber et al., 2011; Graham et al., 2015). Feedback is particularly useful because it can be provided during the writing process and thereby enhance learners’ writing development. Beginning writers tend to connect their ideas associatively without adapting them to the reader or to certain text purposes. As their writing develops, however, learners increasingly succeed in organizing their texts coherently and adapting them to the audience (Bereiter, 1980). In order to promote such writing development, feedback should not only address surface but also deep-level features of texts. On the surface level, high-quality texts may feature linguistic accuracy in terms of grammar, spelling, and punctuation. On the deep level, high-quality texts will be meaningfully organized and include coherently linked ideas so that readers can discern the information and meaning of the text. Accuracy on a surface level is important and might be demanding especially when writing in a FL (Manchón et al., 2009). However, focusing on deep-level features in FL writing might better serve the purpose of prioritizing higher-order goals of communication (see Lee, 2021), according to which writing aims at conveying meaning. Based on this communicative goal, we follow a definition of writing quality as “coherently organized essays containing well-developed and pertinent ideas, supporting examples, and appropriate detail” by Graham and Perin (2007, p. 14), referencing Needels and Knapp (1994).

Studies have shown that producing coherently organized texts poses challenges to students writing in a FL. For instance, texts of college students writing in EFL compared to students writing in their L1 seem to be of simple structure and are less coherent, given that ideas necessary to be included may be lacking (Silva, 1993). An older nationwide study in Germany showed that many students in Year 9 (N = 10,639) struggle with writing well-structured and comprehensive texts in English (Harsch et al., 2008). This finding was confirmed in a more recent study, where 56.2% of EFL students (N = 166) in German middle and low performance track schools reached half of the maximum score for text coherence and only 4.2% achieved doing so regarding text structure (Siekmann et al., 2022). Based on these studies one can conclude that many adolescent EFL students struggle with writing organized and comprehensible texts (Harsch et al., 2008; Siekmann et al., 2022) and may particularly need formative feedback regarding deep-level features.

In general, one may assume that if teachers provide high-quality feedback, particularly incorporating feedback on deep-level features, students can make significant progress in their writing (Parr and Timperley, 2010). A study by Brooks et al. (2021) further examined the potential of a new student-centered feedback model in influencing writing achievement of fourth graders (N = 1,060). Teachers and principals participated in a six-month professional learning intervention in which they were introduced to the relevance of a student-centered feedback culture and feedback that promotes learning based on Hattie and Timperley’s (2007) model. Before and after the intervention, students’ writing achievements and their perceived helpfulness of several feedback strategies were assessed. Increases in students’ perceived helpfulness regarding teachers’ feedback strategies including clarifying success criteria, checking in on progress, and promoting improvement through specific comments or use of models as well as possibilities for students to talk with peers and enact feed up, back, and forward were positively associated with gains in writing achievement.

Feed up, feed back, and feed forward are generally perceived as helpful by students (Brooks et al., 2019). Therefore, one may also expect these aspects of feedback to enhance affective-motivational variables related to writing. However, the evidence in this regard is more nuanced and suggests that single aspects of feedback may contribute differently to variables such as students’ writing self-efficacy and anxiety. While information on students’ progress (feed back) seems necessary to increase students’ writing self-efficacy, information on learning goals only (feed up) can lead to increases in students’ writing anxiety (Zarrinabadi and Rezazadeh, 2020). Therefore, combining the three aspects of feedback seems beneficial (Zarrinabadi and Rezazadeh, 2020). Additionally, providing these aspects in the right balance might also be relevant. Providing information on how to move forward is arguably important for learning improvement (Brooks et al., 2019), however, focusing too heavily on such feed forward might signal to students with low self-efficacy in writing that there is still a lot to be improved (Duijnhouwer et al., 2012). This could ultimately result in these learners believing themselves to have lower writing capabilities than initially thought, which may explain results from our pilot study where students’ self-efficacy in writing decreased after a feedback intervention (Busse et al., 2020).

While the above-mentioned findings on the positive impact of formative feedback on writing self-efficacy and anxiety are promising, studies focusing on the impact of feedback on enjoyment of writing remain scarce. In general, researchers have already called for further intervention research on FL enjoyment in the classroom (see Dewaele, 2022). Considering that FL enjoyment is particularly salient when students perceive themselves as autonomous and empowered instead of being passive learners (Dewaele and MacIntyre, 2014), formative feedback that places learners at the center of the learning process could have a particularly positive effect in this respect.

To sum up, feedback that enables students to derive specific information on learning goals (feed up), their progress toward these goals (feed back), and how to move forward to close the gap between their current performance and the desired goals (feed forward) can be beneficial for students’ writing quality and motivation. Feed up, back, and forward should address text deep-level features to help students successfully communicate their thoughts through organized and coherent texts. To unlock the potential of sustaining students’ self-efficacy, decreasing their writing anxiety, and possibly increasing their enjoyment of writing, additionally, all three aspects of feedback should be provided in a balanced way. Yet, not many studies have yet addressed feedback’s potential for enhancing students’ writing performance and motivation.

3. Materials and methods

3.1. Aims and research questions

Our study is part of a larger project aimed at promoting writing among secondary school students through a professional learning intervention (PLI). Teachers participated in a PLI on providing formative feedback on writing and were then asked to implement feedback in language classes. In this study, we investigate how EFL students’ (N = 53) perceptions of feedback and their writing self-efficacy and anxiety affect their argumentative text quality. Measurements were taken before EFL teachers attended the PLI (T1) and 8 months later (T2). First, we analyze EFL students’ text quality, their feedback perceptions, and affective-motivational variables at T1 and T2. We then examine the extent to which students’ feedback perceptions and affective-motivational variables predict text quality. We address the following research questions in particular:

RQ1: Are there changes in students’ perceptions of feedback and affective-motivational variables from T1 to T2?

In general, feedback needs to activate students to be effective. Still, various researchers have highlighted that students may not always perceive teachers’ feedback as useful and motivating and, thus, fail to act on it (e.g., Carless and Boud, 2018; Brooks et al., 2019). As teachers participated in the PLI on how to provide formative feedback, we expect students to perceive teachers’ feedback to be more useful and more motivating in terms of writing enjoyment at T2 (H1a).

As studies indicate that teachers’ feedback to students’ writings can positively influence affective-motivational variables (e.g., Duijnhouwer et al., 2012; Zarrinabadi and Rezazadeh, 2020), we further assume that students’ self-efficacy increases and that their writing anxiety decreases at T2 (H1b).

RQ2: Are there changes in text quality from T1 to T2?

Although there are little data on the effect of feedback on deep-level text development, particularly for secondary students, one could assume that text quality improves due to the PLI. We therefore hypothesize that students are better able to establish structure and coherence in their texts (H2).

RQ3: Are feedback perceptions and affective-motivational variables predictors of text quality?

Based on findings regarding the relevance of affective-motivational variables for students’ writing achievements (see Sun et al., 2021; Tahmouresi and Papi, 2021), self-efficacy can be expected to be a positive predictor and writing anxiety a negative predictor of students’ text quality both at T1 and T2 (H3a). Assuming that the PLI will lead to students perceiving the feedback as more useful and motivating, we also expect feedback perceptions to positively predict text quality at T2 (H3b).

3.2. Design and participants

Our article examines data from a quasi-experimental study with a pre-post test design (for an overview of the intervention project design see Figure 1) involving 53 EFL students (18 females, 33 males, two did not reveal their gender; mean age = 15.04 years, SD = 0.55 at T1) from three Year 9 classes at secondary schools (Realschule and Hauptschule) in North-Rhine Westphalia, Germany. The majority of students (45.3%) started learning EFL in Year 1. Students’ English grades from their last report card indicated medium levels of achievement in our sample (M = 3.38, SD = 0.80, on a six-point scale with 1 being the highest and 6 the lowest grade awarded in the German schooling system).

FIGURE 1

Figure 1. Overview of intervention project design.

3.3. Procedure

We informed secondary schools in North-Rhine Westphalia about our intervention project to solicit teachers and their EFL students. Six teachers from five different schools consented to participate in the project. In the first phase of the intervention, teachers participated in a PLI on how to implement formative feedback on students’ writing (for more information on the PLI content see section 3.4). In the second intervention phase, teachers were asked to incorporate feedback on writing in their EFL classes for 8 months. To facilitate teachers’ implementation of the PLI content in class, teachers received a logbook including a summary of the PLI content and materials to be used in class. Teachers were asked to document the methods and materials they used within a chart in the project’s logbook as a fidelity measure. However, as the feedback implementation period coincided with pandemic-induced partial and full home learning, teachers stopped documenting their writing and feedback practices after the first 2 weeks of the term when schools closed for the first time.

Before teachers participated in the PLI (T1) and after 8 months of the in-class feedback intervention (T2), students indicated via questionnaire to what extent teacher feedback was effective in emphasizing learning goals, progress and areas of improvement as well as motivating students in writing, that is, to what extent students’ enjoyment of writing was enhanced. Students also revealed how self-efficacious and anxious they were in writing. Following the questionnaire, students wrote an argumentative text within 20 min. The full assessment (questionnaire, writing tasks, test on general cognitive ability, see section 3.5 Instruments) was conducted during 90 min of regular school hours.

3.4. Teacher PLI content and materials

In the two-day PLI, the researchers presented five different modules on evidence-based feedback methods and writing exercises that teachers then discussed and practiced using exemplary students’ texts from a pilot study and materials designed for the project.

On day one, we covered general criteria of formative feedback (module 1) by introducing teachers to the feedback model of Hattie and Timperley (2007), that is, teachers learned about the importance of making learning goals (feed up), progress (feed back), and improvement information (feed forward) transparent to students. We analyzed and discussed feedback samples similar to the following: “The goal of the assignment was to write a pro and con discussion (feed up). There are already many arguments that support your thesis statement, which you improved on compared to your last draft. However, there is only one counter-argument (feed back). Can you think of further counter-arguments? If you need help, you can check the mind-map we prepared in our previous lesson (feed forward).” We also highlighted the relevance of being sensitive to students’ needs. In this regard, we discussed feedback and its possible effects on student motivation and engagement. Teachers then practiced giving feedback on deep- and surface-level features of texts (modules 2 and 3). On day two, teachers extended their knowledge of general criteria of formative feedback (module 4) and learned how to implement feedback in larger learning groups in a time-efficient manner (module 5). Here, we concentrated on working with criteria-based rubrics, peer feedback, exemplars/text models, and modeling process-oriented writing tasks in class (for more details on the methods and materials discussed in module 5, see Siekmann et al., 2022). Table 1 shows an overview of the PLI content, following the recommendations for reporting writing interventions given by Bouwer and de Smedt (2018).

TABLE 1

Table 1. Teacher professional learning intervention content.

3.5. Instruments

We assessed students’ self-efficacy for evaluating and revising, adapting a scale by Busse (2013). Students were asked to indicate to what extent they felt able to identify strengths and weaknesses and to revise their texts. Writing anxiety was assessed using adjusted items of the SLWAI by Cheng (2004) which measured to what extent students displayed cognitive and somatic anxiety as well as avoidance behavior In addition, we used a scale adapted from Rakoczy et al. (2005) to measure students’ perceived general feedback quality with items referring to the feedback model of Hattie and Timperley (2007), for instance, students had to state to what extent they were informed about the learning goals (feed up) and received improvement information (feed forward) in their EFL classes. We also examined students’ perceived effect of text feedback and their perceived effect of feedback on writing motivation using Harks et al.’s (2014) scales that we had previously adjusted in another study (Busse et al., 2020). The perceived effect of text feedback related to possible cognitive and behavioral effects of feedback on writing; students had to indicate the extent to which the feedback helped them identify where they could improve or whether they should prepare better. Students’ perceived effect of feedback on writing motivation included items addressing the enjoyment of writing, for example, the extent to which feedback made students look forward to future writing assignments and enjoy revising their writing more. All scales were based on a six-point Likert scale ranging from 1 (strongly disagree) to 6 (strongly agree) and showed satisfactory Cronbach’s Alpha at or above 0.70. For an overview of sample items and internal consistency values see Table 2.

TABLE 2

Table 2. Overview of scales on perceptions of feedback and affective-motivational variables.

We used an independent writing task from the TOEFL iBT^® writing assessment, publicly available on the TOEFL website and used in other studies (e.g., Keller et al., 2020), to measure students’ argumentative text quality. Students were asked to agree or disagree with the statement “A teacher’s ability to get along well with students is more important than excellent knowledge of the subject” and give reasons for their opinion. Following a definition of writing quality focusing on deep-level features of writing (see section 2.2), two raters analyzed students’ text structure and coherence using analytic rubrics (for a detailed description of the rating and rubrics see Siekmann et al., 2022). The raters evaluated all texts via common negotiation to guarantee consistency and a shared understanding and application of criteria (Trace et al., 2016). If there was disagreement whether the criterion was fulfilled, they discussed reasons for both options and referred back to benchmark texts selected from a pilot study until they reached a consensus. Students could reach a maximum score of eight points for structuring their text into an introduction (two points for providing and embedding an opening statement), a main body (one point), and a conclusion (two points for providing and embedding a concluding statement), and by setting appropriate paragraph breaks (three points). Regarding coherence, students could achieve a maximum score of nine points for providing a thesis statement (three points for providing a thesis statement and adhering to it throughout the main body to the end of the text), developing arguments (three points for providing an argument, examples and a closing sentence), and creating a common thread (three points for connection of ideas, more logical connection of ideas with mostly correct usage of linking words, and meaningfully connecting the introduction, main body, and conclusion with suitable linking words).

In addition, students did one subtest on figural analogies from the Cognitive Ability Test for 4th to 12th Grades, Revision (KFT 4–12 + R, Heller and Perleth, 2000). We included this measure of cognitive abilities to account for another individual student feature that was found to be related to text quality in other studies (e.g., Hajovsky et al., 2018; Köller et al., 2019).

3.6. Data analysis

To examine differences in students’ writing scores, affective-motivational variables, and feedback perceptions between T1 and T2, we calculated paired t-tests using SPSS v.26. Drawing on Cohen (1988), we calculated effect sizes by dividing the mean difference by the standard deviation of the difference $d = \frac{m e a n_{D}}{S D_{D}}$ and interpreted effect sizes of d ≥ 0.2 as generally small, d ≥ 0.5 as medium, and d ≥ 0.8 as large effect sizes for t-tests. In addition, we conducted multiple regression analyses to explore the predictive validity of affective-motivational variables and feedback perceptions for writing competence at T2 when controlling for students’ writing scores at T1 and cognitive ability.

4. Results

4.1. RQ1: Are there changes in students’ perceptions of feedback and affective-motivational variables from T1 to T2?

4.1.1. Feedback perceptions

On average, we observed moderate values among students’ questionnaire data; that is, the mean values were centered around the midpoint of the scales. Students perceived teachers’ general feedback quality as moderately positive both at T1 and T2 (see Table 3). A t-test showed a significant difference between both time points with a small effect [d = 0.43], indicating an increase in the perceived general feedback quality after teachers participated in the PLI (see Figure 2). Students also perceived teachers’ text feedback to be moderately positive, but they perceived feedback to be less beneficial for their writing motivation in terms of enjoyment of writing. T-tests showed no significant differences between students’ perceptions before and after the PLI regarding these two scales.

TABLE 3

Table 3. Students’ perceptions of teacher feedback, self-efficacy for evaluating and revising texts, and writing anxiety: means and standard deviations at T1 and T2.

FIGURE 2

Figure 2. Students’ perceptions of teacher feedback, their self-efficacy for evaluating and revising texts, and writing anxiety. Lines in the boxes represent median scores, and the crosses represent mean scores; boxes range from the 25th to the 75th percentile; vertical lines range from the minimum to the maximum score, with the symbol ° representing outliers.

4.1.2. Affective-motivational variables

Students indicated they were moderately self-efficacious in evaluating and revising their texts. A t-test showed a significant increase in students’ self-efficacy after the PLI with a small effect size [d = 0.28] (see Figure 2). The lowest values were found in the items on writing anxiety, with the scale mean values at both time points being minimally below the scale mean. Although differences between T1 and T2 were not very pronounced, t-tests showed a significant decrease with a small effect size [d = −0.35].

4.2. RQ2: Are there changes in text quality from T1 to T2?

Our analysis shows that students struggled with establishing text structure and coherence at both time points (see Table 4). Only 7.2% of students reached half of the maximum score for structure at T1, and even fewer students (3.8%) did so at T2. We identified an introduction in only 9.4% of students’ texts at T1 and 7.5% at T2. Moreover, only 22.6% of students wrote a conclusion at T1 and 28.3% at T2. Paragraph breaks were also largely missing at both T1 and T2.

TABLE 4

Table 4. Performance on an argumentative writing task: means and standard deviations at T1 and T2.

Regarding coherence, 47.9% of students reached half of the maximum score at T1, but only 35.9% did so at T2. While most of the students’ texts (T1: 81.1%, T2: 84.9%) stated their position concerning the statement prompt at the beginning of their texts and most texts (T1: 67.9%, T2: 60.4%) also referred back to this thesis in the main body, only a few students (T1 + T2: 15.1%) returned to their thesis at the end of their texts. Most students provided arguments for their thesis (T1: 84.9%, T2: 86.8%), with a total of two arguments appearing most frequently in students’ texts at T1 (37.7%) and T2 (39.6%). However, students mostly failed to elaborate on their arguments, with examples present in less than half of the students’ texts (T1: 47.2%, T2: 39.6%). Regarding the common thread, ideas were at least loosely connected in most argumentative texts (T1: 84.9%, T2: 77.4%). However, students widely failed to use linking words correctly. Students often picked up new thoughts unexpectedly, and they logically connected their ideas in a broad common thread in only 24.5 and 18.9% of the argumentative texts at T1 and T2, respectively. Thus, students had problems establishing structure and coherence in their texts at both T1 and T2, and t-tests confirmed no significant difference between the two time points.

4.3. RQ3: Are feedback perceptions and affective-motivational variables predictors of text quality?

We calculated Pearson correlations to investigate the relationship between all variables (see Table 5). There were no significant correlations between feedback perceptions and text quality at T1. However, at T2, we found medium positive correlations between text quality and not only the perceived general feedback quality (r = 0.422, p = 0.002) but also the perceived effect of text feedback (r = 0.451, p < 0.001), and the perceived effect of feedback on writing motivation (r = 0.488, p < 0.001). That is, the extent to which students perceived they received not only feed up, feed back, and feed forward (general feedback quality), but also feedback on writing that helped them identify whether they should prepare better (effect of text feedback) and made them enjoy writing (effect of feedback on writing motivation) was correlated with students’ text quality at T2. Contrary to our expectation, self-efficacy and writing anxiety were not significantly correlated with text quality either at T1 nor T2.

TABLE 5

Table 5. Pearson correlations between students’ feedback perceptions, affective-motivational variables related to writing and their text quality at T1 and T2.

In a multiple regression analysis, we included feedback perceptions, self-efficacy, and writing anxiety to find out to what extent they predicted text quality when controlling for cognitive abilities. All assumptions for multiple regression analysis were met.

As a lack of significant correlations between variables at T1 indicated, the variables mentioned could not significantly predict students’ writing at T1 [F(6, 44) = 1.485, p = 0.206]. Yet at T2 our Model 1 was significant [F(6, 43) = 9.199, p < 0.001] (see Table 6) and general feedback quality (β = 0.502, p < 0.001), the perceived effect of feedback on writing motivation (β = 0.368, p = 0.009), and students’ general cognitive abilities (β = 0.345, p = 0.003) made significant contributions. Surprisingly, however, students’ self-efficacy and writing anxiety made no significant contribution to the model; neither did the perceived effect of text feedback. In total, Model 1 explained 56% of the variance in students’ writing at T2. When adding students’ writing scores at T1 as another controlling variable in Model 2, the regression coefficients of students’ perceived general feedback quality and the perceived effect of text feedback remained similar. In Model 2, students’ perceived general feedback quality (β = 0.411, p < 0.001), the perceived effect of feedback on writing motivation (β = 0.328, p = 0.007), and text quality at T1 (β = 0.387, p < 0.001) significantly predicted students’ writing score at T2. In contrast to Model 1, general cognitive abilities did not significantly predict students’ writing in Model 2, while students’ perceived effect of text feedback, self-efficacy in evaluating and revising, and writing anxiety still made no significant contribution to the model. Overall, Model 2 explained 68% of the variance in students’ writing scores at T2.

TABLE 6

Table 6. Regression coefficients for argumentative writing score (structure and coherence) at T2.

5. Discussion

Our study first compared secondary EFL students’ argumentative text quality, feedback perceptions, as well as self-efficacy and writing anxiety before (T1), and 8 months after teachers participated in a professional learning intervention (PLI) on how to provide effective and motivating text feedback (T2). Second, we analyzed to what extent feedback perceptions, self-efficacy, and writing anxiety accounted for variance in students’ text quality.

Regarding students’ perceptions of teacher feedback (RQ1), at T1 and T2, students perceived teachers’ text feedback to be moderately positive, but they perceived feedback to be less beneficial for their writing motivation in terms of enjoyment of writing with no differences between the two time points. However, students reported teachers’ general feedback quality (based on the feedback model of Hattie and Timperley, 2007) to be better after the PLI than before, thus partially confirming our hypothesis (H1a).

Looking at affective-motivational variables related to writing, we found that students’ self-efficacy in evaluating and revising was high and further increased from T1 to T2, while their writing anxiety decreased significantly. Therefore, our hypothesis was confirmed (H1b). We emphasized the role of regular writing activities and praise related to specific aspects of students’ work and progress in our PLI, therefore, teachers possibly focused on a combination of providing opportunities to gain mastery experience and social persuasion to enhance students’ writing self-efficacy which resulted in increased writing self-efficacy (see Bandura, 1997; Pajares et al., 2007). However, students’ high self-efficacy beliefs do not align with their text quality as measured in our study, indicating low calibration between beliefs and performance also observed in other studies (Schunk and Usher, 2012; Schunk and DiBenedetto, 2016; Chen and Zhang, 2019), which may particularly affect less proficient writers (Graham et al., 1993; Anastasiou and Michail, 2013; Busse et al., in press). In general, however, while feedback should help students make self-evaluative judgments (Carless and Winstone, 2020), it can be assumed that higher self-efficacy and lower writing anxiety are beneficial for further writing development (Camacho et al., 2021). Considering that studies also indicate that students’ self-efficacy in writing only improves if feed up, feed back, and feed forward are equally distributed (Duijnhouwer et al., 2012; Zarrinabadi and Rezazadeh, 2020), future studies should further investigate the relationship between these aspects of feedback and affective-motivational variables.

Students’ ability to write well-structured and coherent texts (RQ2) was relatively low at T1 (see also Siekmann et al., 2022) and did not improve over the course of 8 months despite the feedback intervention. Therefore, our hypothesis (H2) was not confirmed. Our results thus contradict older studies reporting improvement in students’ texts quality in Year 9 (Harsch and Schröder, 2008; Schoonen et al., 2011). However, in these studies different measurements were applied to assess writing competence. Although our results are of concern and suggest that even more support is needed to help students in EFL text composition, it is also imperative to contextualize our findings. For example, it may take time for teachers to transfer and implement fully the feedback practices learned in the PLI (see also Brooks et al., 2021) to actually improve students’ writing. One may also consider that feedback implementation coincided with pandemic-induced partial and full home learning. Indeed, other studies reported learning losses in writing during the pandemic (see the overview by Helm et al., 2021). Therefore, our results could be interpreted, tentatively, as indicating that teacher feedback may have counteracted a loss in terms of writing performance. This would tie in with other studies indicating that feedback may be imperative in times of pandemic-induced school closures to promote students’ writing (see Jiang and Yu, 2021). Irrespective, future studies are needed to explore to what extent stagnating literacy can be attributed to the exceptional learning circumstances caused by COVID-19 or whether the plateauing observed was rather an indicator of little development in text composition in general.

Regarding the effect of feedback perceptions, self-efficacy, and writing anxiety on students’ text quality (RQ3), we found differing results between the two time points. While students’ feedback perceptions and text quality were not correlated at T1, we observed medium correlations at T2. In contrast, neither self-efficacy nor writing anxiety correlated with text quality at T1 or T2, thus contradicting other studies showing significant relationships between writing achievement and self-efficacy (see the meta-analysis by Sun et al., 2021) or writing anxiety (Tahmouresi and Papi, 2021). Accordingly, feedback perceptions, self-efficacy in evaluating and revising, and writing anxiety did not predict students’ text quality at T1.

At T2, we similarly found that writing self-efficacy and writing anxiety did not contribute to students’ text quality. Therefore, our hypothesis (H3a) was not confirmed. Missing associations of writing self-efficacy with text quality might be explained by the fact that students’ writing self-efficacy and their performance were mismatched in our sample. While students perceived themselves to be capable of evaluating and revising their texts, their performance data revealed an illusion of competence (Kruger and Dunning, 1999). Missing associations between writing anxiety and performance may be related to the fact that students in our sample displayed only moderate levels of anxiety, while high levels of anxiety were particularly found to be indicative for students’ writing performance (for similar results see Han and Hiver, 2018; Busse et al., in press). However, students’ perceived general feedback quality and the perceived effect of feedback on writing motivation significantly predicted text quality at T2 after controlling for students’ general cognitive abilities. These effects remained stable even after controlling for students’ T1 text quality scores. Therefore, our hypothesis was confirmed (H3b). It should be noted that the effect of cognitive abilities diminished when adding students’ text quality from T1 as another controlling variable. Thus, our findings indicate that students’ cognitive abilities might not be as robust a predictor of EFL writing development as shown in other studies (see Köller et al., 2019) but rather students’ previous writing quality may be a better predictor.

While further studies including control groups are needed to corroborate our findings, our study suggests that feedback that provides transparency regarding learning goals, information about students’ performance, and feed forward that closes the gap between students’ level of performance and learning goals, plays an essential role for students’ text development (see also Parr and Timperley, 2010; Gadd and Parr, 2017). Although FL enjoyment and achievement in general may not be associated (Li et al., 2020), writing enjoyment can be related to student achievements (Zumbrunn et al., 2019) and be predictive for EFL students’ text quality if enjoyment is fostered through teacher feedback. Our results thus underline that students’ perceived usefulness of feedback is associated with student achievement (Brooks et al., 2021) and that teachers might pay particular attention to the motivational impact of feedback to facilitate student uptake (Carless and Winstone, 2020). Arguably, there may also be mediating effects of feedback perceptions on affective-motivational variables for students’ text quality which future studies with larger sample sizes could further explore.

Certainly, there are other limitations to our study that must also be acknowledged. Our study’s major limitation is that the intervention period coincided with pandemic-induced partial and full home learning. The latter resulted in less shared class time and a drop-out of our control groups. The increased pandemic-induced demands also explained why teachers stopped documenting their writing and feedback practices in the project logbook. Thus, it is uncertain to what extent teachers implemented feedback and used material from the PLI as envisioned by the research team. Although no direct evidence on teachers’ feedback practices is available, additional questionnaire data collected from the teachers after the PLI indicated that teachers perceived the feedback methods discussed as valid and practicable for fostering students’ writing in everyday EFL teaching (Siekmann et al., 2022). In future studies, researchers might wish to collect additional information through classroom observations, considering different aspects of feedback. Another limitation is the length of the PLI (a 2-day workshop) which is, admittedly, a short time for teachers to learn about feedback literacy and how to sustainably implement PLI content in class (see Jesson and Parr, 2019; Lee, 2021). However, the length of the PLI is also shaped by a systemic problem in German professional learning. Although PL courses are mandatory for teachers, only a few federal states require evidence of attendance, which can influence participation in PL courses (Kuschel et al., 2020). Teachers’ further work involvement, disengagement, and perceived quality of PL also negatively affect participation in PL courses (Richter et al., 2018). Therefore, to make PL courses more attractive, it is advisable to offer them in a condensed form. Nevertheless, it is still promising that despite the brevity of the PLI, we observed that the predictive power of feedback for text quality was substantial after teachers attended the PLI indicating that the feedback content provided helpful input for participating teachers.

Lastly, in this paper we focused on coherence and structure and did not explore changes in surface-level features although these are also part of overall text quality. As we also addressed feedback on these features in the PLI, we acknowledge that teachers may have provided feedback on these features as well which we did not capture with our writing measures. To address this limitation, we currently also explore surface-level features in students’ texts. With regard to the lack of research addressing feedback on deep-level features, however, our study offers valuable insights into an under-represented topic and highlights the need to examine the influence of feedback on deep-level features in more detail, in terms of both research and practice.

6. Conclusion

The study responds to the paucity of research on EFL writing among secondary students, particularly regarding text deep-level features and affective-motivational variables related to writing. Our data suggest that feedback perceptions play an important role for students’ writing. Feedback perceived to adhere to quality criteria by providing information on learning goals (feed up), progress (feed back), and further improvement (feed forward) was positively related to students’ text quality which ties in with findings by Brooks et al. (2021). Interestingly, students’ writing self-efficacy and anxiety did not predict text quality, which could be related to the fact that we also found a mismatch between writing self-efficacy and actual text quality in our sample of less proficient EFL writers. Yet feedback perceived as motivating was positively associated with students’ text quality. Thus, our findings suggest that future research should further investigate the value of high-quality motivating feedback. We also advocate raising practitioners’ awareness of the necessity to provide information on learning goals, progress, and further improvement (Hattie and Timperley, 2007) and the motivational power of feedback (Carless and Winstone, 2020), particularly when working with less proficient writers.

Data availability statement

The datasets presented in this article are not readily available because the authors do not have permission to share data. Requests to access the datasets should be directed to leasiekmann@uni-muenster.de.

Ethics statement

The studies involving human participants were reviewed and approved by the Local Ethics Committee of Department 5 of the University of Koblenz-Landau. Written informed consent to participate in this study was provided by the participants’ legal guardian/next of kin.

Author contributions

VB conceived the project and supervised the study. JP served as an external advisor. VB, JP, and LS jointly contributed to final design of the study. LS executed the study, conducted the PLI, collected the data, organized the database, performed the statistical analysis, and wrote the first draft of the manuscript. VB and JP contributed to manuscript revision. SV advised on statistical analyses. All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

This study was funded by Stiftung Mercator (grant number 1111109), project WeLiKe Feedback. We also acknowledge financial support from the University of Münster for publishing this article.

Acknowledgments

We would like to thank Stiftung Mercator and the University of Münster for the funding and all the students and teachers participating in the “WeLiKe Feedback” project. We would also like to thank our project colleague Nora Müller and the (former) student assistants Katrin Peltzer, Maxime Kops, and Tanne Stephens for their help and support.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Abdel Latif, M. M. M. (2021). Writing motivation research, measurement and pedagogy. London: Routledge.

Google Scholar

Anastasiou, D., and Michail, D. (2013). Exploring discordance between self-efficacy and writing performance among low-literate adult students. Learn. Disabil. Contemp. J. 11, 53–87.

Google Scholar

Bandura, A. (1997). Self efficacy: the exercise of control. New York: Freeman.

Google Scholar

Beck, N., and Fetherston, T. (2003). The effects of incorporating a word processor into a year three writing program. Inf. Technol. Child. Educ. Annu. 1, 139–161.

Google Scholar

Bereiter, C. (1980). “Development in writing” in Cognitive processes in writing. eds. L. W. Gregg and E. R. Steinberg (Hillsdale, NJ: Erlbaum), 73–96.

Google Scholar

Biber, D., Nekrasova, T., and Horn, B. (2011). The effectiveness of feedback for L1-English and L2 writing development: a meta-analysis. TOEFL IBT Research Report No. TOEFLiBT-14. Princeton, NJ: Educational Testing Service.

Google Scholar

Bitchener, J., and Ferris, D. R. (2012). Written corrective feedback in second language acquisition and writing. New York: Routledge.

Google Scholar

Boscolo, P., and Hidi, S. (2007). “The multiple meanings of motivation to write” in Writing and motivation. eds. G. Rijlaarsdam (Series Ed.) and P. Boscolo and S. Hidi (Volume Eds.) (Oxford: Elsevier), 1–14. doi: 10.1163/9781849508216_002

Google Scholar

Bouwer, R., and de Smedt, F. (2018). Considerations and recommendations for reporting on writing interventions in research publications. J. Writ. Res. 10, 115–137. doi: 10.17239/jowr-2018.10.02.01