Cognitive Processes: Current Status

Introduction

Cognitive process assessment is not a specific, universally agreed upon approach to assessment, but rather refers to a general orientation concerning mostly what kinds of knowledge, skills, and abilities ought to be assessed, and to a lesser extent perhaps, to how they ought to be assessed. Cognitive process assessment is often defined at least partly by what it is not – it is not ‘behaviourism’ and it is not ‘psychometrics’. It is not behaviourism, because behaviourist [Page 238]approaches focus on observable behaviour, which can be recorded on checklists. In contrast, cognitive process approaches focus on internal thoughts, feelings, strategies, orientations, predispositions, and other attributes that can only be inferred, based on patterns of behaviour. It is not psychometrics, because psychometric approaches are typically driven by correlational findings rather than by theory. Psychometric approaches are characterized as primarily empirical, involving the administration of a variety of tests, followed by an exploratory factor analysis to identify test clusters, the analysis of which might reveal common processes underlying test performance. In comparison, cognitive processes assessment is said to be more theoretical based on our understanding from cognitive and brain science of how the mind works. These characterizations are, of course, often overstated and, in practice, the distinction between the approaches is frequently blurred, but there are differences in emphases.

Cognitive Processing Frameworks

There are many models and theories of cognitive processes, but one useful distinction might be made between macro- and micro-theories, or between cognitive architectures (a macro-theory) and models of cognitive tasks (micro-theories). Much of the cognitive process work originated as micro-theories of particular tasks, such as the kinds of tasks that routinely appear in intelligence tests. For example, analyses have been conducted on inductive and deductive reasoning tasks, spatial relations tests, vocabulary tests, and so on. From this work, important concepts and distinctions emerged. Some examples are the differentiation between short-term, or working, memory and long-term memory, the distinction between declarative and procedural memory, the concept of automaticity, the distinction between imaginal and verbal processing, the delineation of stages of processing (e.g. apprehension, encoding, retrieval, decision, etc.). Other examples are the identification of task-specific strategies, the positing of metacognitive skills, such as planning, self-monitoring, and self-regulation, and many more. Also, from this work, we now know how to manipulate item difficulty levels on some tasks, particularly the more arid ones found on intelligence tests. Such a capability has implications for item design and automatic item generation (see Irvine & Kyllonen, 2002; Kyllonen, 2002).

However, it was recognized, early on, that the micro-theory approach was severely limited, and that real progress would only be made when grander theories were attempted that incorporated what we know about cognition. Consequently, there have been several macro-theories of cognitive processes, formulated along these lines, including ACT-R (Anderson, 1993), SOAR (Newell, 1990), and EPIC (Kieras & Meyer, 1994). It is useful to note that these all contain some common elements. An important distinction, for example, in all is the one between the current focus of thought, and long-term memory, the current focus of thought usually being called ‘working memory’. Another characteristic is the simultaneous-sequential distinction, in which some processes, such as vision, and memory retrieval, are assumed to be simultaneous, while others, such as problem solving, are assumed to be more deliberative, and sequential.

...