[New Pub] Automatic Short Answer Grading with LLMs: From Memorization to Reasoning

On 01. Mai, Longwei Cong presented his paper “Automatic Short Answer Grading with LLMs: From Memorization to Reasoning” at the 16th International Conference on Learning Analytics and Knowledge.

The paper examines the performance of fine-tuned PLMs and LLMs across different dataset sizes and compares them with prompt-based approaches for automatic short answer grading. The results show that fine-tuned LLMs and rubric-based prompting can match or even exceed the performance of BERT-based models. In particular, rubric-based prompting with open-weight models can deliver competitive results without requiring annotated training data or hardware-intensive fine-tuning, while also helping to address data protection concerns.

This work provides empirical evidence for the role of LLMs in automatic short answer grading and opens up future research directions on resource-efficient, interpretable, and reasoning-driven grading. You can find the paper here: https://dl.acm.org/doi/full/10.1145/3785022.3785031