With increasingly large student numbers, providing personalized teacher feedback becomes untenable. On the other hand, providing students feedback about their work is an integral part of ensuring student support throughout their learning trajectory. Fortunately, Learning Analytics now makes it feasible to automatically deploy feedback to many students at once. However, the design of effective feedback still remains an area of investigation
Joshua Weidlich, Aron Fink, Ioana Jivet, Jane Yau, Tornike Giorgashvili, Hendrik Drachsler, and Andreas Frey‘s recently published paper in the Journal of Computer-Assisted Learning focused on one key design feature: the reference frame. Any feedback content must be formulated in reference to some performance level, be it the average of the student group (social comparison), the desired performance level (criterion-referenced comparison), or past performance. A longstanding literature on this topic has concluded that, usually, social comparison is less desirable than other reference frames. While some students like the competition, most students find it demotivating, especially if they are not among the top performers. Further, it can lead to some negative emotions, like frustration.
However, some open questions remain. First, do these findings also apply to automated and personalized feedback? Second, do they apply in the same way on a collaborative learning task, where social dynamics already play a key role? Third, what is the effect of combining reference frames? Finally, is it possible to provide more nuanced feedback design recommendations by considering the performance of individual students?
Weidlich et al. conducted a randomized field experiment with four feedback conditions, (1) pass/fail feedback, (2) social comparison, (3) criterion-referenced feedback, and (4) combined feedback. Students in teacher education worked on a computer-supported collaborative learning task for one week, and, at the end of this task, each student received one of the four feedbacks.
Results suggest complex emotional and motivational effects of the feedback reference frames. The key insights are: All feedback types were perceived as useful relative to the pass/fail feedback condition. Norm-referenced feedback showed detrimental effects for motivational regulation, whereas combined feedback led to more desirable motivational states. Further, criterion-referenced feedback led to more positive emotions for overperformers and to more negative emotions for underperformers.
This study was part of the HIKOF-DL project funded by Distr@l.
Suggested Citation: