Empirical Study Archives - EduTec Science

Oct232024 by Heather WehnerNo Comments

New Pub: How Can Learning Analytics Dashboards Help Improve Students’ Self-Regulated Learning?

Conference, Empirical Study, Learning Analytics, Publication, School, Self-Regulation

Learning Analytics Dashboards (LADs) are important and widely-used tools used to give feedback to students and to aid them in their self-regulating learning process. Much has been done to investigate the design of LADs, but there is still a lack of knowledge regarding how students interpret the information shown on LADs and how they actually use these tools while learning. In a newly published study, we try to fill this gap. In an experimental study, we compared two groups of students. One group was given personalized self-regulared learning (SRL) feedback on their interactions and learning advances. The control group was only given minimal feedback calculated from the average class scores. After reviewing their feedback, students submitted written reflections on how they would adjust their study behavior. The researchers then analyzed…

Sep202024 by Daniele Di MitriNo Comments

New Pub: Preserving Privacy in Multimodal Learning Analytics with Visual Animation of Kinematic Data

Empirical Study

A recent study has been published that addresses the growing concern of data privacy in multimodal learning analytics (MMLA). The research investigates the potential of using visual animations as an alternative to traditional video recordings for analyzing sensitive data, particularly in educational settings. MMLA involves collecting and analysing data from various sources, including video recordings, to gain insights into learning behaviours and outcomes. However, the use of video can raise significant privacy concerns, especially when it contains identifiable information about individuals. This has led to ethical dilemmas regarding using such data in research. The study, based on the master thesis of Aleksandr Epp, introduces the Kinematic Animation Tool (KAT) to address these privacy issues. This tool allows researchers to visualise kinematic data without relying on video footage, thereby mitigating privacy…

Jun102024 by Joshua WeidlichNo Comments

New Pub: Emotional and motivational effects of automated and personalized feedback

Computer-supported collaborative learning, Empirical Study, Feedback, Higher Education, Journal, Learning Analytics, New Pub, Open access

With increasingly large student numbers, providing personalized teacher feedback becomes untenable. On the other hand, providing students feedback about their work is an integral part of ensuring student support throughout their learning trajectory. Fortunately, Learning Analytics now makes it feasible to automatically deploy feedback to many students at once. However, the design of effective feedback still remains an area of investigation Joshua Weidlich, Aron Fink, Ioana Jivet, Jane Yau, Tornike Giorgashvili, Hendrik Drachsler, and Andreas Frey's recently published paper in the Journal of Computer-Assisted Learning focused on one key design feature: the reference frame. Any feedback content must be formulated in reference to some performance level, be it the average of the student group (social comparison), the desired performance level (criterion-referenced comparison), or past performance. A longstanding literature on this…

May242024 by Heather WehnerNo Comments

FoLA supports Simulation-based Training for High-risk Clinical Situations

Assessment, Empirical Study, Feedback, Further Education, Higher Education, Learning Analytics, Learning Design, medical education, Multimodal Learning Analytics, Workshop

From May 16th to 17th, 2024, Hendrik Drachsler and Marcel Schmitz (TETRA AI, ZUYD University of Applied Science) had the pleasure of providing a FoLA workshop at the College of Anaesthesiologists of Ireland (CAI) in Dublin, Ireland, under the support of the Insight research centre. Together with members from CAI and from the ASSERT Centre, College of Medicine and Health, University College Cork (UCC), they used the FoLA tool to plan a simulation-based training for high-risk clinical situations using highly informative feedback. Data from these trainings will be collected and analyzed to determine how effective this training is in improving the performance of the attendees. The study is being planned to take place with around 100 doctors in training at two simulation training centers: at ASSERT Centre UCC and at…

May92024 by Onur KarademirNo Comments

New Pub: I don’t have time! But keep me in the loop: Co-designing requirements for a learning analytics cockpit with teachers

Empirical Study, Learning Analytics, New Pub, Research Methods, School

Teacher dashboards can help secondary school teachers manage online learning activities and inform instructional decisions by visualising information about class learning. However, when designing teacher dashboards, it is not trivial to choose which information to display, because not all of the vast amount of information retrieved from digital learning environments is useful for teaching. Information elicited from formative assessment (FA), though, is a strong predictor for student performance and can be a useful data source for effective teacher dashboards. Especially in the secondary education context, FA and feedback on FA, have been extensively studied and shown to positively affect student learning outcomes. Moreover, secondary teachers struggle to make sense of the information displayed in dashboards and decide on pedagogical actions, such as providing feedback to students. Objectives: To facilitate the…

Apr162024 by Heather WehnerNo Comments

New Pub: Feedback sources in essay writing: peer-generated or AI-generated feedback?

Artificial Intelligence, Empirical Study, Feedback, Further Education, Journal, Publication

A newly published article discusses the use of peer feedback as a learning strategy, particularly in large classes where teachers face heavy workloads. For complex tasks like writing argumentative essays, peers may struggle to provide high-quality feedback due to the cognitive demands involved. The emergence of Artificial Intelligence (AI) tools, like ChatGPT, raises the question of whether AI can serve as a new feedback source for such tasks. To investigate this, a study compared ChatGPT-generated feedback with peer feedback on argumentative essays written by 74 graduate students from a Dutch university. The study collected essay data, peer feedback and ChatGPT-generated feedback, and then analyzed them using coding schemes. Results showed significant differences between ChatGPT and peer feedback, with ChatGPT offering more descriptive feedback while peers focused on identifying essay problems.…

Jan292024 by Sebastian GombertNo Comments

New Pub: From the Automated Assessment of Student Essay Content to Highly Informative Feedback: a Case Study

Artificial Intelligence, Assessment, Computational Psychometrics, Empirical Study, Feedback, Higher Education, Journal, Publication, Special Issue, Technical paper

How can we give students highly informative feedback on their essays using natural language processing? In our new paper, led by Sebastian Gombert, we present a case study on using GBERT and T5 models to generate feedback for educational psychology students. In this paper: ➡ We implemented a two-step pipeline that segments the essays and predicts codes from the segments. The codes are used to generate feedback texts informing the students about the correctness of their solutions and the content areas they need to improve. ➡ We used 689 manually labelled essays as training data for our models. We compared GBERT, T5, and bag-of-words baselines for both steps. The results showed that the transformer-based models outperformed the baselines in both steps. ➡ We evaluated the feedback with a learner cohort…

Jan152024 by Dana KubeNo Comments

New Article – Gender diversity dynamics in a Computer Supported Collaborative Learning

Artificial Intelligence, Computer-supported collaborative learning, Digitalisation, Empirical Study, Gender, Higher Education, Journal, Learning Design, New Pub, Open access, Publication, Special Issue, Team

🎉 Exciting News! Our article has just been published in the magazine of Computer Assisted Learning! 📰 We delved into the fascinating world of online group learning among adults, unravelling the mysteries of emergent team roles and their intricate connection to gender dynamics in communication. 🌐👥 Have you ever wondered how team roles subtly surface and evolve in online group learning discussions? We did, too! Our research explores the subtle nuances of team roles and their subversive emergence, especially when viewed through the lens of gender diversity, in order to understand how to support more productive learning for all participants. Gender and gender diversity are group features affecting social interaction and are critical for gender-inclusive and equitable education. As such, the role of gender and gender diversity is of particular…

Nov132023 by Joshua WeidlichNo Comments

New Pub: Social Presence and Psychological Distance

Computer-supported collaborative learning, Empirical Study, Higher Education, Journal, Learning Design, New Pub, Publication

Social presence –the sense that others are 'real' and 'there'– is a key variable in understanding interpersonal dynamics in online learning environments. As students are separated in time and place, social cues are diminished and communication is affected. This is particularly relevant for social learning scenarios like computer-supported collaborative learning. Despite its relevance and decades of research, there are still many gaps in our understanding of social presence. In order to arrive at a more holistic understanding of social presence, it would be valuable to better understand how this experience fits within larger psychological frameworks. One particularly well-established psychological framework is Construal Level Theory by Trope & Liberman (2010). It posits that our mental representations of objects, events, and persons (i.e. construals) are affected by the psychological distance between us…

Oct232023 by Joshua WeidlichNo Comments

New Pub: Development and initial validation of an instrument to measure student feedback literacy

Empirical Study, Feedback, Higher Education, Journal, Open access, Publication

To ensure quality higher education, students should routinely receive feedback on their academic endeavors. Alongside the question of what makes feedback effective, there is also an emerging research literature about empowering students to understand and utilize that feedback effectively. These abilities and attitudes of students have recently been subsumed under the concept of feedback literacy. The concept of feedback literacy was conceived by Carless and Boud (2018) as “the understandings, capacities, and dispositions needed to make sense of information and use it to enhance work or learning strategies.” Since then, a vibrant research literature has developed theoretical frameworks, explored dimensions of feedback literacy, and investigated whether feedback literacy can systematically be enhanced, etc. However, what is still missing are larger-scale rigorous investigations of the extent to which feedback literacy actually…