Teacher dashboards can help secondary school teachers manage online learning activities and inform instructional decisions by visualising information about class learning. However, when designing teacher dashboards, it is not trivial to choose which information to display, because not all of the vast amount of information retrieved from digital learning environments is useful for teaching. Information elicited from formative assessment (FA), though, is a strong predictor for student performance and can be a useful data source for effective teacher dashboards. Especially in the secondary education context, FA and feedback on FA, have been extensively studied and shown to positively affect student learning outcomes. Moreover, secondary teachers struggle to make sense of the information displayed in dashboards and decide on pedagogical actions, such as providing feedback to students.

Objectives:

To facilitate the provision of feedback for secondary school teachers via a teacher dashboard, this study identifies requirements for designing a Learning Analytics Cockpit (LA Cockpit), that is, (1) a teacher dashboard that provides teachers with visualisations of results from formative assessment (FA) and (2) a feedback system that supports teachers in providing feedback to students.

Methods:

This study was conducted in the context of STEM classes and is based on semi-structured co-design interviews with German secondary school teachers. In these interviews, we first explored challenges teachers encountered in monitoring students’ learning and providing feedback. Second, in the ideation phase, teachers were asked to define features an LA Cockpit for FA should have. Finally, in the evaluation phase, we provided teachers with a design template for an LA Cockpit, the LAC_Template, which was built upon our previous work and feedback theory, and asked them to evaluate and improve it. Further design requirements were derived based on the evaluation of the LAC_Template and teachers’ suggestions for improvement.

Results:

We derived 16 requirements for designing an LA Cockpit for FA in secondary schools. Findings from the interviews indicated that the feedback system of an LA Cockpit should address teachers’ time limitations in giving students individualised feedback. It should therefore be designed to minimise the steps required to deliver feedback. To reduce workload, teachers requested an automated reminder to send feedback, but with the ability to adjust feedback to the learning context. Such a semi-automated feedback system can help teachers support students individually but also underline the importance of actively involving teachers in the feedback loop and giving them control when using such technologies in secondary school practice. A challenge for future teacher dashboard designs could be to find a balance between technology and teacher control that utilises the strengths of both in a beneficial combination.

What is already known about this topic

  • Despite the potential of teacher dashboards to aid instruction, their designs often result in teachers struggling to derive insights from dashboards
  • Most teacher dashboards are designed to display student information but not to facilitate the provision of feedback from teachers to students
  • Results from formative assessment (FA) serve as a strong predictor for student performance, making them a valuable data source for teacher dashboards

What this paper adds

  • This paper provides guidelines to develop a Learning Analytics Cockpit, functioning as a teacher dashboard with an integrated feedback system for FA
  • A Learning Analytics Cockpit has the potential to mitigate teachers’ lack of time in providing individualised feedback to students
  • While automating feedback can reduce workload, teachers prefer active involvement over complete automation.

Implications for practice and/or policy

  • We show how teacher dashboards can go beyond just visualising students’ information and enhanced to enable the provision of feedback
  • When incorporating automated features into teacher dashboards, it is crucial to actively involve the role of teachers to ensure beneficial cooperation between teachers and technology

Citation:
Karademir, O., Di Mitri, D., Schneider, J., Jivet, I., Allmang, J., Gombert, S., Kubsch, M., Neumann, K., & Drachsler, H. (n.d.). I don’t have time! But keep me in the loop: Co-designing requirements for a learning analytics cockpit with teachers. Journal of Computer Assisted Learning, n/a(n/a). https://doi.org/10.1111/jcal.12997