The fourth Committee meeting with industry members of the research project “HIKOF-DL” took place on March 31, 2023. HIKOF-DL is a project funded by the Hessen Ministerium für Digital Strategie und Entwicklung, which aims to provide high-informative feedback for a large number of university students, and automatic assessment, in order to reduce the lecturers’ workload. The meeting yielded exciting results and discussions, showcasing the progress and potential of this innovative study.
The meeting kicked off with an introduction by Prof. Dr. Hendrik Drachsler, who emphasized the significance of artificial intelligence (AI) in education. Dr. Joshua Weidlich, a key researcher in the project, then presented the research design of the study. The study population was divided into an experimental group and a control group to investigate two aspects: how students respond to new tasks and whether these tasks are conducive to learning. The primary focus of the study is on providing high-informative feedback to students, which is expected to have a positive impact on their learning outcomes.
One of the unique tasks given to the students during the pilot study was to formulate and submit learning tips in an essay after reviewing learning tips from a youtube video “It’s Lenaa.” The control group was tasked with checking whether the learning tips were correctly applied in the essay. Another task involved a forum discussion, where students had to engage in collaborative discussions and develop a position on a topic in the forum before submitting it. The third task was to create a concept map, where students had to work out a master solution in the Moodle environment, allowing for tracking of their approach using building blocks.
The results of the groups were presented by Sebastian Gombert (doctoral student with focus on machine learning) who shared insights from the dataset of 700 essays from the “Introduction to Teacher Education” lecture. Ten codes per study tip were created, totaling 100 codes, using deep learning models such as BERT. Templates were also established to determine discrepancies. Manual checks were performed for formal aspects like APA formatting and spelling errors, and feedback was provided to all students to gauge their perceptions.
The evaluation results showed promising outcomes, with the automated classification of sentences to the study tips achieving high accuracy. This indicates the potential of HIKOF as a feedback tool to improve learning outcomes for students.
Next, the data from the forum discussion was analyzed and presented by Dr. Joshua Weidlich. The data recorded different types of feedback to assess the achievement of learning goals and changes in learning behavior, including motivation and emotions. The results were complex, with intrinsic and extrinsic motivation showing different effects. It was observed that extrinsic motivation showed a linear decline, and performance within the groups was also analyzed, revealing variations among participants. The feedback was found to strongly influence the hope for progress and trigger emotions such as dissatisfaction and anger, particularly among underperformers who perceived feedback related to learning goals more strongly. However, preliminary results indicated that Group 3 accepted the feedback positively and found it helpful for further learning, showing the potential impact of high-informative feedback.
Overall, the fourth meeting with industry members of the research project “HIKOF” was productive and presented exciting results and discussions. The current project phase is dedicated for data analysis and dissemination and will conclude in May 2024. The use of data analytics and machine learning models in analyzing student performance and providing feedback holds promise for enhancing learning outcomes in education. The insights gained from this study have the potential to contribute to the field of educational research and inform the development of effective feedback strategies for students.The project team is also working on sustainability of generating feedback and automatic assessment for lecture assignments, without requiring a large amount of staff hours, after the project ends.
