Category: Publication

At BEA 2026, we organized the first shared task on rubric-based short answer scoring for German, introducing a new benchmark dataset spanning multiple STEM domains and designed to evaluate both in-domain performance and generalization to previously unseen questions. The goal was to better understand how modern NLP systems can interpret and apply textual scoring rubrics—an ability that closely mirrors how human assessors evaluate student responses. The shared task attracted multiple teams, who explored a wide range of approaches, including fine-tuned large language models, retrieval-augmented prompting, hybrid symbolic–neural systems, and ensemble methods. Across all four evaluation tracks, systems that explicitly incorporated rubric semantics consistently achieved the strongest performance, while scoring previously unseen questions remained the biggest challenge. We hope this benchmark provides a foundation for future research on more robust, interpretable,…

How can AI not only score student answers accurately, but also explain why it assigned a particular score? In our latest paper, we introduce AGRAA (Aspect-Grounded Rubric–Answer Alignment), a new framework for rubric-based assessment that represents rubric criteria as semantic subspaces. Instead of treating scoring as a standard classification problem, AGRAA measures how strongly a student's response aligns with the latent semantic aspects defined by each rubric level. We evaluate the approach on both short-answer and essay scoring benchmarks, where it consistently achieves performance highly competitive with strong transformer-based baselines. More importantly, the model naturally produces interpretable explanations by showing which rubric-defined aspects contributed to each scoring decision. We believe this work is a step toward automated assessment systems that are not only accurate, but also transparent, trustworthy, and aligned…