FoLA supports Simulation-based Training for High-risk Clinical Situations

FoLA supports Simulation-based Training for High-risk Clinical Situations

Assessment, Empirical Study, Feedback, Further Education, Higher Education, Learning Analytics, Learning Design, medical education, Multimodal Learning Analytics, Workshop
From May 16th to 17th, 2024, Hendrik Drachsler and Marcel Schmitz (TETRA AI, ZUYD University of Applied Science) had the pleasure of providing a FoLA workshop at the College of Anaesthesiologists of Ireland (CAI) in Dublin, Ireland, under the support of the Insight research centre. Together with members from CAI and from the ASSERT Centre, College of Medicine and Health, University College Cork (UCC), they used the FoLA tool to plan a simulation-based training for high-risk clinical situations using highly informative feedback. Data from these trainings will be collected and analyzed to determine how effective this training is in improving the performance of the attendees. The study is being planned to take place with around 100 doctors in training at two simulation training centers: at ASSERT Centre UCC and at…
Read More
Team led by Sebastian Gombert wins one of two tracks at BEA 2024 shared task on predicting Item Difficulty and Item Response Time

Team led by Sebastian Gombert wins one of two tracks at BEA 2024 shared task on predicting Item Difficulty and Item Response Time

Artificial Intelligence, Assessment, Award, Computational Psychometrics, Conference, Higher Education, New Pub, Workshop
For standardized exams to be fair and reliable, they must include a diverse range of question difficulties to accurately assess test taker abilities. Additionally, it's crucial to balance the time allotted per question to avoid making the test unnecessarily rushed or sluggish. The goal of this year's BEA shared task (competition) was to build systems which could predict Item Difficulty and Item Response Time for items taken from the United States Medical Licensing Examination (USMLE). EduTec member Sebastian Gombert designed systems which are able to predict both variables simultaneously. These placed first out of 43 for predicting Item Difficulty and fitfth out of 34 for predicting Item Response Time. They use modified versions of established transformer language models in a multitask setup. A corresponding system description paper titled Predicting Item…
Read More
CORE Roundtable in Munich

CORE Roundtable in Munich

Assessment, Critical Online Reasoning, Multimodal Learning Analytics, Project, Project meeting
Hendrik Drachsler, Sebastian Gombert and Gianluca Romano participated at the Roundtable in Munich for the CORE project (Critical Online Reasoning in Higher Education) from 04.03.-05.03.2024. In those two days, our team had the chance to recapitulate on how the infrastructure stood strong during the first survey from December 2023 to February 2024, and pave the way for next steps and surveys. In summary, the infrastructure performed well. It dealt with approximately 10Mio. requests per seconds and the majority of hurdles participants reported were out of our authority. Requests from participants were dealt with quickly in a few days on average. Even though we are proud of our achievements there is still a lot to be done for future surveys. For the agenda we split into smaller groups, each of us…
Read More
New Pub: AI program doesn’t make kids better at math, but it makes them more independent

New Pub: AI program doesn’t make kids better at math, but it makes them more independent

Assessment, Journal, Publication, School
Students who receive math tutoring from an artificial intelligence (AI) program perform no better than students who are taught by a "real" teacher. These students do, however, need less help learning. This is the conclusion of Rashmi Khazanchi from the Open University of the Netherlands together with Hendrik Drachsler and Daniele Di Mitri. Math Lessons with AI The researchers examined the effectiveness of the Assessment and Learning in Knowledge Spaces (ALEKS) tutoring program, called Intelligent Tutoring Systems (ITS). Previous studies have shown that students learn math better using software than traditional teaching methods. Previous studies on ALEKS have also shown that, thanks to this program, students memorize more knowledge, perform better, experience more engagement in mathematics and drop out less. The advantage of an ITS like ALEKS is that it…
Read More
New Pub: From the Automated Assessment of Student Essay Content to Highly Informative Feedback: a Case Study

New Pub: From the Automated Assessment of Student Essay Content to Highly Informative Feedback: a Case Study

Artificial Intelligence, Assessment, Computational Psychometrics, Empirical Study, Feedback, Higher Education, Journal, Publication, Special Issue, Technical paper
How can we give students highly informative feedback on their essays using natural language processing? In our new paper, led by Sebastian Gombert, we present a case study on using GBERT and T5 models to generate feedback for educational psychology students. In this paper: ➡ We implemented a two-step pipeline that segments the essays and predicts codes from the segments. The codes are used to generate feedback texts informing the students about the correctness of their solutions and the content areas they need to improve. ➡ We used 689 manually labelled essays as training data for our models. We compared GBERT, T5, and bag-of-words baselines for both steps. The results showed that the transformer-based models outperformed the baselines in both steps. ➡ We evaluated the feedback with a learner cohort…
Read More
16th eLearning Netzwerktag: An Insightful Recap of the Fast-Paced Year

16th eLearning Netzwerktag: An Insightful Recap of the Fast-Paced Year

Artificial Intelligence, Assessment, Augmented Reality, Competence development, Computational Psychometrics, Computer-supported collaborative learning, Conference, Event, Feedback, Gender, Higher Education, Learning Analytics, Learning Design
The annual eLearning Netzwerktag was a highly anticipated one-day event where the eLearning community of Frankfurt and the surrounding areas gathered to present the highlights of the past year to the public. On November 21, 2023, the event took place at Campus Westend, Goethe University Frankfurt am Main. Among the speakers, the Prof. Dr. Maren Scheffel, Prof. Dr. Franziska Matthäus , CIO of Goethe University Ulrich Schielein, Prof. Dr.Hendrik Drachsler, Director of studiumdigitale, delivered an opening speech that reflected on an incredible year, with a particular focus on the advancements in generative Artificial Intelligence applications. Hendrik Drachsler's speech highlighted the significant developments in the field of digital learning. At the previous Netzwerktag, applications like ChatGPT, Midjourney, Stablediffusions, and open language models (LLMs) such as LAMA were relatively unknown to most…
Read More
New pub: Measuring Efficacy of ALEKS as a Supportive Instructional Tool in K-12 Math Classroom with Underachieving Students

New pub: Measuring Efficacy of ALEKS as a Supportive Instructional Tool in K-12 Math Classroom with Underachieving Students

Assessment, Journal, New Pub, Publication, School
In a recent quasi-experimental research study, the effectiveness of Assessment and Learning in Knowledge Spaces (ALEKS), an Intelligent Tutoring System (ITS), took center stage in the realm of 8th-grade mathematics education. The study aimed to determine whether ALEKS could bring a statistically significant improvement in students' mathematics achievement compared to traditional teacher-led instructions. The research involved 158 8th-grade students categorized as 'underachieving students, with 60 in the teacher-led group and 98 in the ALEKS-led group. The study used a non-randomized approach to compare the outcomes of teacher-led instructions to ALEKS-led instructions over two consecutive years. In the first year McGraw's curriculum "Reveal" was used exclusively without ALEKS. In the second year ALEKS was incorporated as a supplemental tool in a math support class. The study incorporated a rigorous methodology, utilizing…
Read More
New pub: Digitalisierung und Diagnostik in Schulen – Herausforderungen für Bildungspraxis und Bildungsforschung

New pub: Digitalisierung und Diagnostik in Schulen – Herausforderungen für Bildungspraxis und Bildungsforschung

Assessment, Book chapter, Computational Psychometrics, Digitalisation, Learning Analytics, Research Methods, Research topic, School
In the spring of 2020, schools faced an unprecedented challenge as the pandemic disrupted traditional modes of instruction and school development. With on-site learning replaced by digital formats and distance communication, educators had to quickly adapt to the new normal. Amidst these changes, the field of education encountered specific challenges related to digital school management, digital learning, and assessing learning progress. Particularly, computer-aided diagnostics emerged as a valuable tool for gaining insights not only into learning outcomes but also into the learning process itself. Researchers became intrigued by the potential of digital media in shaping learning experiences and how the resulting data could be effectively utilized in educational practice. This paper explores the current challenges and potentials in computer-based, learning-accompanying diagnostics. The primary hurdles involve implementing suitable instruments within schools…
Read More
New research program: Towards Highly Informative Learning Analytics

New research program: Towards Highly Informative Learning Analytics

Artificial Intelligence, Assessment, Book, Computational Psychometrics, Computer-supported collaborative learning, Feedback, Further Education, Higher Education, Keynote, Learning Analytics, Learning Design, Research Methods, School
On May 12, 2023, the Highly Informative Learning Analytics (HILA) research program of the EduTec@DIPF, studiumdigitale@Goethe University Frankfurt and the Open Learning and Instruction group@Open Universiteit was presented by Hendrik Drachsler at the main campus of the Open University of the Netherlands. The release of the HILA research program marks a significant milestone for the collaboration in the field of learning analytics between of the Dutch-German research collective.  The HILA research program is focused on developing new tools and methods to collect, analyze, and interpret data that can help educational institutions to understand the learning process better. As part of the program's launch, a keynote by Ioana Jivet on student-facing learning analytics was provided. Ioana reported on two empirical studies investigating the effect of data-driven feedback on students. [pdf-embedder url="https://edutec.science/wp-content/uploads/2023/05/2023_05-Keynote-Symposium-Hendrik.pdf"…
Read More
CORE: New project investigates students’ critical use of digital media

CORE: New project investigates students’ critical use of digital media

Assessment, Digitalisation, Higher Education, Project, Research topic, Target group
Students are increasingly using information from the Internet to learn for their studies. According to recent surveys, they even cite the Internet as their main source when searching for learning materials. Research shows that students often rely on inappropriate, irrelevant and scientifically unsupported sources when searching the web. Little is known about exactly how self-directed learning takes place on the Internet and how information is selected and processed. This will now be addressed by the new research group "Kritisches Denken in Online-Lernumgebungen in der Hochschulbildung (CORE)” (Critical Thinking in Online Learning Environments in Higher Education), which is being funded by the German Research Foundation (DFG) with more than 5 million euros over the next four years. The CORE project will investigate the critical use of digital media and information among…
Read More