Two papers from EduTech have been accepted to the main track of the 2026 Empirical Methods in Natural Language Conference (EMNLP) and were presented at the conference in Budapest, Hungary.
Paper1:Alice: A Large-Scale German Benchmark for Rubric-Based Multi-Dimensional Automatic Short Answer Scoring
This paper presents the full Alice Dataset, a large-scale German student answer scoring benchmark. The novelty of the benchmark lies in the multidimensional scoring, where each student answer is scored according to three subtasks: Learning Performance, which measures how well the student answers the question directly; Knowledge Elements, where each answer is scored by how well the student masters and utilises the target concepts for each question; and Skill, which is the cognitive concepts required by each question. In addition, each available level in three subtasks is annotated with rubrics, which are natural language descriptions of the scoring criteria.
Paper2: Rubric Spans are Label Representations: Joint LLM Encoding for Short Answer Scoring
This paper proposes a universal architecture called Ruspan for any rubric-based scoring tasks, and can be easily generalised to other discriminative natural language understanding (NLU) tasks. Inspired by token classification, we input the entire rubric sequence into an LLM, and use the last token activations of the rubric text as representations for scoring. This way, a variable number of available rubrics (i.e., labels) can fit in without assuming a fixed classification head.
During the experiments, we found that the architectures easily overfit to the rubric structures seen during training. To tackle this, we apply an additional mask to the attention matrix such that tokens of the rubrics cannot attend to tokens in other rubrics. This way, the overfitting is removed, and this masks the architecture better, allowing it to robustly generalise to new benchmarks and tasks.