Publication Archives - EduTec Science

Jul312026 by Longwei CongNo Comments

[New Pub] Estimating LLM Grading Ability and Response Difficulty in Automatic Short Answer Grading via Item Response Theory

New Pub

On July 4, 2026, Longwei Cong virtually presented the paper “Estimating LLM Grading Ability and Response Difficulty in Automatic Short Answer Grading via Item Response Theory” at the ACL SIGEDU 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026). The work investigates automatic short answer grading with large language models from a psychometric perspective. Instead of relying only on aggregate evaluation metrics such as accuracy or macro-F1, the study applies Item Response Theory to jointly estimate the grading ability of individual LLMs and the difficulty of student responses. The results show that LLM graders differ not only in their overall performance, but also in their robustness when evaluating more difficult responses. In addition, highly difficult responses were found to induce systematic grading errors, including a tendency…

Jul312026 by Longwei CongNo Comments

[New Pub] Confidence Estimation in Automatic Short Answer Grading with LLMs

New Pub

On June 30, Longwei Cong presented the work “Confidence Estimation in Automatic Short Answer Grading with LLMs” at the 27th International Conference on Artificial Intelligence in Education (AIED 2026) in Seoul. The work investigates how confidence estimates can make automatic short-answer grading with large language models more reliable in practice. In addition to commonly used model-based confidence signals, it also considers uncertainty originating from the data itself. In particular, aleatoric uncertainty is estimated from the semantic heterogeneity of student responses and combined with several model-side confidence signals. The study evaluates these uncertainty estimates with a focus on practical use: whether they can support selective grading and help identify responses for which automated grading is less reliable and human review would be preferable. The paper is available via the following…

Jul242026 by Heather WehnerNo Comments

New Pub: The Dependency Dilemma of Machine Learning Aids

Artificial Intelligence, Journal, New Pub

AI and machine-learning decision aids can be a great support for employees and help them improve their work performance. An overreliance on algorithmic recommendations, on the other hand, may result in a long-term reduction of the employees’ ability to develop and maintain their own decision-making skills. This could be especially problematic when such AI and machine-learning decision aids are not available. Instead of overly relying on such AI-based recommendations, organizations should train their employees to continue developing their critical thinking and decision-making skills. Hendrik Drachsler and his research partners address this problematic dependency on AI tools and the conflict of interests in the workplace in their newly published article “The Dependency Dilemma: How Machine Learning Decision Aids can Undermine Skill Growth”. Using a controlled experiment, the authors found that participants…

Jul162026 by Heather WehnerNo Comments

New Pub: Recommendations for Higher Education in the Age of Generative AI

Artificial Intelligence, Higher Education, Publication, Report

Generative AI cannot be treated as just another digital tool that has come along. As it becomes more and more embedded in higher education, universities face the challenge of responsibly navigating the many challenges and opportunities that generative AI brings with it. One central guiding principal for institutions and stakeholders engaged with generative AI is intellectual sovereignty, which the German Science and Humanities Council (Wissenschaftsrat, WR) highlights in its newly published position paper, which was presented in a digital press release on 06.07.2026. Intellectual sovereignty refers to the ability to think independently, exercise critical judgment and maintain autonomy in the creation and evaluation of knowledge. Rather than relying uncritically on AI-generated outputs, this concept encourages students, educators and institutions to actively question, assess and contextualize information. By placing intellectual sovereignty…

Jul102026 by Sebastian GombertNo Comments

[New Pub] Report on the BEA 2026 Shared Task on Rubric-based Short Answer Scoring for German

New Pub

At BEA 2026, we organized the first shared task on rubric-based short answer scoring for German, introducing a new benchmark dataset spanning multiple STEM domains and designed to evaluate both in-domain performance and generalization to previously unseen questions. The goal was to better understand how modern NLP systems can interpret and apply textual scoring rubrics—an ability that closely mirrors how human assessors evaluate student responses. The shared task attracted multiple teams, who explored a wide range of approaches, including fine-tuned large language models, retrieval-augmented prompting, hybrid symbolic–neural systems, and ensemble methods. Across all four evaluation tracks, systems that explicitly incorporated rubric semantics consistently achieved the strongest performance, while scoring previously unseen questions remained the biggest challenge. We hope this benchmark provides a foundation for future research on more robust, interpretable,…

Jul102026 by Sebastian GombertNo Comments

[New Pub] Rubrics as Semantic Subspaces: A New Perspective on AI Assessment

New Pub

How can AI not only score student answers accurately, but also explain why it assigned a particular score? In our latest paper, we introduce AGRAA (Aspect-Grounded Rubric–Answer Alignment), a new framework for rubric-based assessment that represents rubric criteria as semantic subspaces. Instead of treating scoring as a standard classification problem, AGRAA measures how strongly a student's response aligns with the latent semantic aspects defined by each rubric level. We evaluate the approach on both short-answer and essay scoring benchmarks, where it consistently achieves performance highly competitive with strong transformer-based baselines. More importantly, the model naturally produces interpretable explanations by showing which rubric-defined aspects contributed to each scoring decision. We believe this work is a step toward automated assessment systems that are not only accurate, but also transparent, trustworthy, and aligned…

Jun252026 by Prof. Dr. Hendrik DrachslerNo Comments

Keynote: From Experiment to Infrastructure: Reflections on AI in Higher Education

Academy, Artificial Intelligence, Conference, Higher Education, Keynote, Keynote

Today, Hendrik Drachsler gave a keynote at the VHB annual Conference in Bamberg, speaking to an audience of over 33 AI project leaders from higher education institutions in Germany. The topic was "From Experiment to Infrastructure: Scaling, Evaluation, and Governance of AI Systems in Higher Education," reflecting his thinking on various grassroots projects and how to scale them across the whole university. The gap between performance and learning Hendrik opened with a provocation: Is AI the new calculator? Both technologies automate cognitive processes, both faced early scepticism, and both promised efficiency gains. But the analogy breaks down quickly. A calculator takes over arithmetic, not problem-solving. AI takes over the formulation, analysis, and argument. That is a qualitative difference, and it demands a different institutional / governance response. Mixed results on…

Jun102026 by Heather WehnerNo Comments

New Pub: Competent Usage of AI and Digital Technology in Education

Artificial Intelligence, New Pub, Publication

[caption id="attachment_8403" align="alignright" width="300"] Modified TPACK model as Level 2 of the AIEDTEC-CDM. Note. TPK = technological pedagogical knowledge (TPK), TCK = technological content knowledge, PCK = pedagogical content knowledge, TPCK = technological pedagogical content knowledge.[/caption] How can teachers and learners use AI and digital technology competently, critically and safely in education? Simply using AI tools and digital technology in educational settings does not necessarily mean that learners will profit from these tools. Some tools may have a positive effect on learning, while others may not. A newly published paper addresses this issue and formulates a theoretical framework for the “Competence to Use Artificial Intelligence and Digital Technology in Educational Processes” (AIEDTEC competence). Combining insights from psychology and computer science, it aims to provide a theoretical definition of the competent…

Jun102026 by Sebastian GombertNo Comments

[New Pub] Analyzing students’ conceptual understanding over the course of a teaching unit: Tracking changes in knowledge structures over time

New Pub

In a recent study, we explored how students’ conceptual understanding develops throughout a digitally supported chemistry unit on chemical kinetics. Working with approximately 300 upper-secondary students, we combined automated scoring of students’ written responses and other learning artifacts with network analysis techniques to model the growth of individual knowledge structures over time. The results showed that transformer-based language models can reliably identify and score chemistry-related knowledge elements from classroom data, enabling the construction of longitudinal knowledge networks that reflect how students connect scientific concepts. Several network characteristics were associated with students’ posttest performance, suggesting that these representations capture meaningful aspects of learning progress. The findings highlight the potential of combining automated assessment and learning analytics to provide teachers with real-time insights into students’ developing understanding and to support adaptive instructional…

May292026 by Sebastian GombertNo Comments

Workshop on Educational NLP at the 20th EATEL Summer School

New Pub

At this year's JTEL Summer school, Sebastian Gombert and Amir Rajabi from DIPF hosted a workshop on Natural Language Processing (NLP) for Education and Research. Participants explored how modern language technologies can support both educational practice and scientific inquiry. The session introduced core NLP concepts and discussed applications ranging from automated assessment and feedback generation to classroom discourse analysis, literature synthesis, and retrieval-augmented generation. Following the introductory presentation, participants worked collaboratively in small groups to design NLP-based solutions for real-world challenges. Drawing on both provided examples and their own research interests, attendees developed concepts for systems such as educational recommender tools, feedback assistants, and research support applications. Interactive collaboration was supported through discussion, sketching, and digital whiteboarding. The workshop highlighted not only the growing potential of large language models and…