April 16, 2021
Award, Conference, open access, Publication

Learning analytics dashboards (LADs) are designed as feedback tools for learners, but until recently, learners rarely have had a say in how LADs are designed and what information they receive through LADs. To overcome this shortcoming, we have developed a customisable LAD for Coursera MOOCs on which learners can set goals and choose indicators to monitor. Following a mixed-methods approach, we analyse 401 learners’ indicator selection behaviour in order to understand the decisions they make on the LAD and whether learner goals and self-regulated learning skills influence these decisions. We found that learners overwhelmingly chose indicators about completed activities. Goals are not associated with indicator selection behaviour, while help-seeking skills predict learners’ choice of monitoring their engagement in discussions and time management skills predict learners’ interest in procrastination indicators. The findings have implications for our understanding of learners’ use of LADs and their design.

This work investigated what data learners find meaningful on LADs and whether their goals
and SRL skills affect this judgement. We did not find evidence that the way learners formulate their goals is related to the indicators that they monitored. SRL skills, namely help-seeking and time management, predict whether learners will choose to monitor engagement in discussions and a procrastination indicator, respectively. Thus, our results demonstrate that designing ‘one-size-fits-all’ dashboards puts certain learners at a disadvantage, as skilled learners are more inclined to monitor behaviours associated with higher achievement. Designing impactful LADs need to address learners’ skills levels and also support novice learners in recognising the benefit of monitoring their learning behaviour and use of learning strategies. Findings of the current study give considerable impetus to work towards defining targeted feedback and its equivalence in the potential indicators that LA can provide.

The paper its freely accessible at ACM:
Ioana Jivet, Jacqueline Wong, Maren Scheffel, Manuel Valle Torre, Marcus Specht, and Hendrik Drachsler. 2021. Quantum of Choice: How learners’ feedback monitoring decisions, goals and self-regulated learning skills are related. In LAK21: 11th International Learning Analytics and Knowledge Conference(LAK21). Association for Computing Machinery, New York, NY, USA, 416–427. DOI:https://doi.org/10.1145/3448139.3448179