Learning Analytics for Programming Competencies

Learners and Learning Traces

Students taking an introductory computer programming class in an online environment, or in any course with a significant digital component, have access to learning activities associated with an array of media, including slideshows, reading material, audio and video lectures, interactive tutors, and automated tests. As they work through the course material in this digital environment, students leave behind digital records, or traces, of their learning activity that track their interaction with the course material, much like a Web analytics system tracks visitors to a website. Similarly, as students complete the associated computer programming assignments, software development traces are generated that can be used to track the student success and coding competency. The same traces can then be used to assess the effectiveness of instruction.

Learning analytics is the science of analysis, discovery, and utilization of learning traces in emergent and related levels of granularity. Learning trace data refers to observable raw data of study activities such as reading, writing, conceptualizing, critically thinking, solving problems, storytelling, and visualizing, where a network of study activities leads to a measurable chunk of learning. For instance, the types of sentence openers used by a learner, the range of errors that the student can confidently correct, the level of trust exhibited by the student in sharing information in a forum, and the depth of understanding in a set of concepts are examples of learning traces where one could measure learning over time. In learning analytics, these data are expected to arrive continuously, typically in an interleaved fashion, subject to interpretation at various levels of granularity.

In general, learning traces translate raw data into incoming data for learning analytics, where incoming data are typically big, unstructured, unrelated, and fit multiple models and possibly multiple theories. Importantly, learning traces capture highly personalized study experiences. Although each learning trace is measurable, there is no standard scale of measurement that applies across different types of traces. A learning trace is the least common denominator for a measure of learning.

Analysis using learning analytics may include techniques ranging from data mining to machine learning to big data analysis. This may uncover correlations in highly unconventional data, for example, inherent economic drivers influenced by a curriculum, or functional magnetic resonance imaging/electroencephalogram (fMRI/EEG) indicators for learning capacity, or rate of changes in motivation levels of students with respect to weather. Relations of interest to learning analytics include the emotions, beliefs, and goals among learners across collaborating groups; inter-institutional credit transfer policies among institutions; and mutual respect among instructors.

Learning analytics aims to operate on large volumes of data as well as on large volumes of models produced from the data. These data are not confined to formal and informal data sets, but also include ad hoc data sets with little relationship to learning. Ad hoc data sets include browsing patterns, reading habits, writing style (including freehand writing), coding, posture analyses, collaboration drivers, thinking protocols, chats, domain-specific tool traces, brain-activity traces, recording of cognitive and metacognitive traces, and knowledge traces.

...