Recently I have come across an interesting article regarding automated writing evaluation (AWE) systems. The thing is, despite their promising nature, the adoption of automated writing evaluation systems by educators has been marginal at best. Since the extant research on the issue is scarce, prominent scholars from the University of Pittsburg and RAND Corporation – Lindsay Clare Matsumura, Elaine Lin Wang, Richard Correnti, and Diane Litman – have looked into teachers' perspectives on the use of AWE systems in educational settings. Following the implementation of AWE software in 32 American classrooms, the researchers shared what educators want to see in their feedback provision systems in the article on eschoolnews.com. This blog post reports the findings of their study and discusses the technology's implications for students.
Corrective Feedback and AWE Technology
The provision of corrective feedback is essential to students' writing development. However, teachers find it difficult to make enough time for feedback, which can only be effective if it is regular, timely, and thorough. Therefore, many educators are hesitant to engage students in tasks requiring formative feedback.
The promising solution to the challenge is offered by AWE systems that can lessen the burden of feedback provision. The software evaluates written prose, scores it, and provides meaningful suggestions for improving sentence structure, style, and grammar. Since their inception, AWE systems have undergone multiple modifications and currently offer scores based on the hundreds of intrinsic characteristics of student writing.
To better understand the implementation of scoring and feedback software, consider its inclusion into a basic cycle of writing assignments in a classroom, as described by Glower and Brown (2015).
- The instructor assigns a topic.
- Students submit their writing via the Internet.
- An AWE system scores the papers and provides feedback. This step may be repeated if needed.
- The instructor grades students' writing and provides additional comments.
What Educators Want to See in AWE Technology
Here's what teachers expect to see in AWE solutions:
Timely Feedback
For the feedback to be effective, its provision must be timely and regular, which can consume quite a lot of time. Since the adoption of writing assessment technology is a time-saving measure, there is little surprise that the participants of the study report that "having access to AWE systems generally would increase the number of assignments they gave to students that require writing across drafts" (Matsumura et al., 2020).
Active Participation
Another aspect of human-AWE communication teachers would like to see improved is their role in feedback provision. Specifically, educators want to supply automatically-generated feedback with their comments, thereby participating more actively in feedback provision. The particular importance of this desire lies in the fact that the improvement suggestions of AWE systems are too generic to be of particular use to students.
Alignment with Common Core Standards
The educators participating in the study suggest that AWE systems can be markedly improved through the alignment with grade-level writing standards. Thus, they will be able to demonstrate consistently to young learners what constitutes a satisfactory level of writing.
More Choice
Another common sentiment that merits discussion is the addition of a wider variety of texts in the language processing core of AWE technology. The educators believe that the change would make their instruction more flexible and "emphasize student engagement with both narrative and informational texts" (Matsumura et al., 2020), which is currently lacking.
Highlighting Feature
The educators argue that they would have a more favorable opinion of the technology if allowed them to point to certain parts of students' papers that need to be revised. They feel that the software should be able to underline arguments lacking evidence. The localized guidance would make it much easier for students to apply the feedback to their paper writing.
Pinpoint Success
In addition to allowing students to understand their errors, teachers want AWE software to help them celebrate their writing successes. The teachers would like the systems to acknowledge when students have made an incisive argument or used gripping language. Expanding on that point, researchers state that "marking students' success is important to building motivation and internalizing the skills needed to critically evaluate one's own writing" (Matsumura et al., 2020).
Progress Monitoring
According to the teachers who participated in the study, AWE systems cannot be complete without the progress monitoring functionality (Matsumura et al., 2020). The implementation of this feature would let young learners to better understand their educational trajectory and study at their own pace. The inclusion of a progress dashboard in AWE packages would serve this purpose nicely.
Implications of the Use of AWE Technology for Students
Having discussed the teachers' perspective on the use of AWE systems and how they can be improved, I want to focus on possible implications of the technology for students. For starters, the adoption of automated evaluation of essays can improve their writing skills due to the immediacy of the provided feedback. While it takes quite a lot of time to provide an insightful pedagogical assessment of writing performance for human graders, AWE systems can do it almost instantaneously. And as has been stated before, the timely feedback is essential for promoting academic growth.
Another implication of AWE adoption is the improvement of the depth of provided feedback, which will, inevitably, find reflection in student performance. The software can acknowledge an error, correct it, and explicitly state the reason why the corrective advice was provided. Needless to say, it all can happen in a split second.
Finally, the use of AWE technology in educational settings would provide students with a shared understanding of assessment criteria. When acted upon, it can serve as a means of improving writing competency.
