Education in a digitalized world opens the door for observing actual learning behavior with a fine-grained resolution. The proposed network of excellence investigates what is needed to release “computational psychometrics” (cf. von Davier, 2017) for formative assessment. We therefore will investigate how online-trace data as compared to standardized psychometric measures can be used to shed light on the learner’s knowledge, skills, and attributes that are in operation when using digital learning environments in higher education. For instance, timing behavior may be a useful indicator to evaluate individual learning engagement (cf. Nguyen, Huptych, & Rienties, 2018), and learning trajectories (e.g., semantic coherence of selected texts) may reflect self-regulation (see also Aleven, Roll, McLaren, & Koedinger, 2010; Winne, 2017).

 

For this purpose, the pedagogy concept and the design of the learning environment need to be closely aligned. That means that after specifying the learning outcomes of a course, we do not only need to define the assessment thereafter, but also need to think about potential learning analytics indicators that provide valuable insights into the state of the learner (Schneider et al., 2018). For this, the student’s learning process is inherently tied to the use of the learning environment for receiving, producing and exchanging information. This allows observing meaningful learning behavior that can be used to make inferences about the learner status and for providing feedback (formative assessment). To validate inferences based on learning analytics indicators well-proven classical standardized assessment instruments can be used as criteria. Thereby the DiFA project integrates methodological perspectives from educational assessment and learning analytics to develop new kinds of non-invasive assessment (“stealth assessment”, Shute, 2011) by exploiting digital trace data. Findings of this research are highly relevant for a better understanding learner’s behavior, learning outcomes and for providing individual feedback automatically to learners in digital environments.

 

The educational context of proposed project is initial and further teacher education. Within the DiFA project, we aim to develop an online qualification program for German teachers (students and professionals) on digital education. We can therefore reuse various learning materials that the consortium developed over the past years. This online qualification program will include various interactive online learning opportunities that will produce vast amounts of digital trace data.

 
Funding: Leibniz Kooperative Excellence