Agent Technologies for Evaluation

The Evaluation Domain

Evaluation objectives are to determine program outcomes, strengths, and effectiveness. Complex educational program evaluation processes are characterized by the need to access distributed information sources and expertise to perform collaborative tasks, as well as the need to efficiently design integrated online applications for flexibility and reliability. Any improvement mechanism intended to measure student knowledge and behavior as a result of a program should be able to operate without a time lag. Focused and systematic collection and analysis of information used for the purpose of documenting the effectiveness and impact of programs is essential. Strategic plans, policies, needs assessments, and national standards and guidelines need to be available for access. Results are communicated in multiple methods depending on the audience and analyzed and reused for reports, journal articles, conferences, brochures, websites, and television and newspaper interviews.

Utilization of Agent Technology

Agent technology can assist in various phases and activities of program evaluation efforts. Agents can be used, for example, in task allocation, team planning, data capture and analysis, and presentation activities needed to support evaluation. Multiple disciplines are now being leveraged toward the definition of agent technology applications. Disciplines related to agent technology include artificial intelligence, software engineering, distributed system and computer networking, game theory, and economics. Application domains in addition to education where agents can be utilized include finance, health care, workflow management, network management, air traffic control, electronic commerce, e-mail, digital libraries, and command and control. Agent technology is now considered as an integral part of the Internet development environment, although the technology has not yet been integrated into the mainstream.

Agent Technologies for Program Evaluation

As in any development process, the problem domain, program evaluation, must be well understood and defined. Goals, environments, states, and agent behaviors must be defined and documented. Agents can be classified in multiple ways, according to different types of tasks to be performed. The properties of agents enable them to function in a way that can accelerate [Page 17]complicated tasks that occur in program evaluations such as near real-time data capture and analysis. The basic properties of agents are autonomy, proactiveness, reactivity, and social ability (ability to communicate). Additional properties including mobility, veracity, benevolence, and rationality have been proposed to enable stronger agent capabilities.

...