Instructional Design and Evaluation

Like program evaluation, instructional design emerged as a distinct field of study in the mid-20th century. However, although large government programs drove the application of evaluation, it was the very modest accomplishments of behaviorism's teaching machines and programmed instruction that suggested there is much to be learned about instructional design.

Although piecemeal at first, a knowledge base examining varying methods of instruction and their effects on goals, content, and learners has produced a thriving and stable field of study whose reach extends into K-12 education, community education, higher education, and training in business and industry. Like evaluation, instructional design considers itself a transdiscipline, blending the processes of curriculum development, instructional materials development, and instructional management and delivery. The general aim of instructional design is to construct optimal “blueprints,” or knowledge about what methods of instruction will produce desired learning outcomes under the variety of conditions that may exist.

Literally hundreds of instructional design models bear similar characteristics by being reciprocal in nature and generally bearing comparable components. Every model also identifies evaluation as one of its key steps, as the seminal Dick and Carey model (Figure 1) illustrates. Instructional design also employs and makes very clear distinctions between formative and summative evaluation.

The role of formative evaluation in instructional design is for course or curricular improvement. Frequently, formative evaluation is internal, but it may involve subject matter experts or others external to the design process. Various evaluation models are also applied at this stage of design and include but are not limited to connoisseur, decision-oriented, and objectives-driven approaches. Field or pilot testing is also considered a strategy of formative evaluation. After formative evaluation, instructional designers hope to have as close as is possible to an error-free version of their instructional designs and materials for delivery to their target audience.

Summative evaluation in instructional design matches learning outcomes to learning goals. Not all summative evaluation is goal based, but instructional [Page 204]objectives are the basis for key evaluation questions. Related elements of the summative evaluation are institutional impacts, learner and content congruence, and programmatic costs. Summative evaluation is generally reserved for the external evaluator, who applies both qualitative and quantitative data collection and analysis methods. Although learner assessment data is used in both forms of instructional design evaluation, summative evaluation makes more frequent use of standardized assessments in education.

Figure 1Dick and Carey Design Model

Recently, instructional designers have adopted “confirmative evaluation” as an evaluative stage aimed at the terminal objectives for a stated program or curriculum. Although timing plays the key role from an instructional design perspective, confirmative evaluation takes up where summative evaluation leaves off. Smaller samples of target learners and more indirect measures may be used at this point, but the emphasis is still on performance in a realistic context. For example, confirmative evaluation of a substance abuser prevention program will focus on participant quality of life and citizenship indicators. Confirmative evaluation of a primary grade literacy program will focus on middle school competencies in reading, creative writing, and verbal expression.