Skip to main content icon/video/no-internet
icon backReturn to Entries

Assessing Learning in Simulation-Based Environments

Since simulations approximate real-world task environments, they offer the potential of increased face validity when compared to a conventional written knowledge test. Online simulations can capture a rich stream of click-by-click data from each performer, creating the possibility of capturing a complete dynamic record of the whole task performance. Since simulations may support multiple collaborative users as well as single users, there is the possibility of using simulations to assess collaborative performance as well as single performers. Potential advantages of simulations for assessment over conventional tests are widely recognized. Potential advantages of simulation-based assessment over performance assessment using observational rubrics include scalability, improved reliability and validity, and lower per-assessment cost when implemented at scale. This entry discusses ways that simulation-based assessment is used, the design of simulation-based assessments, and the future of simulation-based assessment.

Uses of Simulation-Based Assessment

Use of simulations for assessing procedural skills is common in many technical and medical professions. The procedures may be purely cognitive (such as mathematical computation), or they may also embody an important psychomotor component (e.g., flying an airplane or medical procedures, such as cardiopulmonary resuscitation). The procedures may be performed by individuals or teams, such as cockpit crews or emergency room staff. Typically, these procedural assessments assess correct performance of each step in a sequential procedure (accuracy) and correct completion of the procedure (attainment of the goal state). In addition, the assessment may examine correct sequencing and choice among the branches of a nonlinear procedure, timely and uninterrupted performance of the procedure, reliable performance of the procedure under varying conditions (such as high stress, emergency or combat conditions, fatigue, high cognitive load, uncertainty, distraction, etc.), and correct selection of the procedure among alternatives (conditions of performance).

More recently, interest has grown in the potential of simulations for assessing complex problem-solving skills as a complement to what can be accomplished with conventional testing. Closely related constructs for this domain include metacognitive skills, higher order thinking skills, 21st-century skills, and expertise. Complex problem-solving skills have in common the lack of needed information to solve the problem, the lack of a clear goal, or the need to invent a procedure to attain the goal. These skills are found in a very wide range of occupations and may include skills such as troubleshooting and diagnosis, negotiation and strategy in almost any context, or engineering, research, design, and creative activities.

Simulation-Based Assessment Design

Unfortunately, it is easy to build the wrong simulation. It is quite possible—perhaps even common—to build simulations that allocate too many development resources to the superficial representation of the task (physical fidelity), while ignoring or even misrepresenting the information representations and decision-making process that underlies the task performance (cognitive fidelity). To avoid this design error and create simulations that are sufficiently valid and reliable for assessment purposes in training or education contexts requires particularly rigorous application of an appropriate design process. Simulations of sufficient quality to be useful for formative or summative assessment thus should always have high cognitive fidelity to the target task, but high physical or superficial fidelity to the target task may or may not be required. This is an important cost consideration in development of simulations.

...

  • Loading...
locked icon

Sign in to access this content

Get a 30 day FREE TRIAL

  • Watch videos from a variety of sources bringing classroom topics to life
  • Read modern, diverse business cases
  • Explore hundreds of books and reference titles

Read next

More like this

Sage Recommends

We found other relevant content for you on other Sage platforms.

close

Have you created a personal profile? Login or create a profile so that you can save clips, playlists and searches

close
close
close Signed In to profile You are signed in as:
close
Logged in Institution Institution