Entry
Reader's guide
Entries A-Z
Subject index
Experimental Designs
The validity of inferences stemming from empirical research in industrial and organizational psychology and allied disciplines is a function of a number of factors, including research design. Research design has to do with the plan, structure, or blueprint for a study. The literature indicates that among the components of such a plan are (a) the experimental design type (i.e., randomized experiment, quasi-experiment, and nonexperiment); (b) the study setting (e.g., created for the purpose of doing research); (c) the numbers and types of study participants; (d) the way in which the variables considered by the study are operationally defined; and (e) the techniques that will be used to analyze the data produced by the study. The focus here is on the randomized experiment. It differs in important ways from both nonexperimental and quasi-experimental design types.
Prior to considering the nature of randomized experiments, we consider several issues concerning the validity of inferences stemming from empirical research. Note that the same issues apply not only to randomized experiments but also to quasi-experiments and nonexperiments.
Factors Affecting the Validity of Research-Based Inferences
The overall correctness of inferences (e.g., research-based conclusions, recommendations for practice) stemming from a study is a function of its design and the manner in which it is actually conducted. Four facets of validity are critical: construct validity, statistical conclusion validity, internal validity, and external validity, as noted by Thomas Cook and Donald Campbell in 1979, and by W. R. Shadish, Cook, and Campbell in 2002.
Construct Validity
Construct validity is a function of the degree of correspondence between the constructs dealt with by a study and their realizations. It has to do with not only the operational definitions of variables (e.g., manipulations, measures), but also the empirical realizations of other features of a study (e.g., types of participants, research settings). Construct validity inferences are threatened by a number of factors, including inadequate preoperational definitions of constructs, study procedures that lead participants to guess a study's hypotheses and behave in ways that confirm them, operational definitions that underrepresent focal constructs, and a lack of correspondence between the type of participants in a study and the way the participants are labeled by a researcher.
Statistical Conclusion Validity
Statistical conclusion validity has to do with the correctness of inferences about relations between variables that stem from the results of statistical tests. Among the factors that threaten this facet of validity are testing statistical hypotheses with data that violate relevant assumptions (e.g., homogeneity of variance), implementing treatments unreliably within study conditions, conducting research in settings having random irrelevancies (e.g., fluctuations in noise, temperature, illumination), and sampling too few units (e.g., participants) to provide for adequate statistical power in hypothesis testing.
Internal Validity
Internal validity is the degree to which inferences about causal connections between variables are correct. Among the factors that detract from the validity of such inferences are history, maturation, instrumentation, testing, selection, and mortality.
External Validity
External validity deals with the correctness of inferences about the generalizability of a study's findings to and across populations of settings, participants, time periods, and so forth. External validity is threatened by such factors as interactions between settings and treatments, interactions between history and treatments, and interactions between selection and treatments.
...
- Foundations: History
- Army Alpha/Army Beta
- Hawthorne Studies/Hawthorne Effect
- History of Industrial/Organizational Psychology in Europe and the United Kingdom
- History of Industrial/Organizational Psychology in North America
- History of Industrial/Organizational Psychology in Other Parts of the World
- Human Relations Movement
- Project A
- Scientific Management
- Scientist-Practitioner Model
- Unions
- Foundations: Ethical and Legal Issues
- Adverse Impact/Disparate Treatment/Discrimination at Work
- Affirmative Action
- Age Discrimination in Employment Act
- Americans with Disabilities Act
- Bona Fide Occupational Qualifications
- Civil Rights Act of 1964, Civil Rights Act of 1991
- Comparable Worth
- Corporate Ethics
- Corporate Social Responsibility
- Employment at Will
- Equal Pay Act of 1963
- Ethics in Industrial/Organizational Practice
- Ethics in Industrial/Organizational Research
- Family and Medical Leave Act
- Glass Ceiling
- Labor Law
- National Institute for Occupational Safety and Health/Occupational Safety and Health Administration
- Race Norming
- Sexual Discrimination
- Sexual Harassment at Work
- Stereotyping
- Test Security
- Uniform Guidelines on Employee Selection Procedures
- Workplace Accommodations for the Disabled
- Foundations: Research Methods
- Benchmarking
- Case Study Method
- Competency Modeling
- Content Coding
- Critical Incident Technique
- Cross-Cultural Research Methods and Theory
- Experimental Designs
- Focus Groups
- Lens Model
- Linkage Research and Analyses
- Longitudinal Research/Experience Sampling Technique
- Meta-Analysis
- Naturalistic Observation
- Nonexperimental Designs
- Organizational Surveys
- Policy Capturing
- Program Evaluation
- Qualitative Research Approach
- Quantitative Research Approach
- Quasi-experimental Designs
- Sampling Techniques
- Simulation, Computer Approach
- Survey Approach
- Verbal Protocol Analysis
- Foundations: Measurement Theory and Statistics
- Classical Test Theory
- Confidence Intervals/Hypothesis Testing/Effect Sizes
- Construct
- Criterion Theory
- Descriptive Statistics
- Differential Item Functioning
- Factor Analysis
- Generalizability Theory
- Incremental Validity
- Inferential Statistics
- Item Response Theory
- Measurement Scales
- Measures of Association/Correlation Coefficient
- Moderator and Mediator Variables
- Multilevel Modeling
- Multilevel Modeling Techniques
- Multitrait–Multimethod Matrix
- Nomological Networks
- Normative versus Ipsative Measurement
- Reliability
- Statistical Power
- Structural Equation Modeling
- Utility Analysis
- Validation Strategies
- Validity
- Industrial Psychology: Understanding and Assessing Individual Differences
- Affective Traits
- Big Five Taxonomy of Personality
- Biographical Data
- Cognitive Abilities
- Cognitive Ability Tests
- Computer Assessment
- Core Self-Evaluations
- Emotional Intelligence
- Employment Interview
- Genetics and Industrial/Organizational Psychology
- Graphology
- Gravitational Hypothesis
- Hardiness
- Impression Management
- Individual Assessment
- Individual Differences
- Integrity Testing
- Job Knowledge Testing
- Letters of Recommendation
- Locus of Control
- Machiavellianism
- Motivational Traits
- Need for Achievement, Power, and Affiliation
- Optimism and Pessimism
- Personality
- Personality Assessment
- Physical Performance Assessment
- Practical Intelligence
- Protestant Work Ethic
- Self-Esteem
- Situational Judgment Tests
- Standardized Testing
- Stereotype Threat
- Trainability and Adaptability
- Type A and Type B Personalities
- Work Samples
- Work Values
- Industrial Psychology: Employment, Staffing, and Career Issues
- Dictionary of Occupational Titles
- Applicant/Test-Taker Reactions
- Banding
- Career Development
- Careers
- Compensation
- Credentialing
- Dirty Work
- Drug and Alcohol Testing
- Electronic Human Resources Management
- Employee Selection
- Executive Selection
- Exit Survey (Exit Interview)
- Expatriates
- Gainsharing and Profit Sharing
- Gay, Lesbian, and Bisexual Issues at Work
- Human Resources Strategy
- Job Advertisements
- Job Analysis
- Job Analysis Methods
- Job Choice
- Job Description
- Job Evaluation
- Job Search
- Job Typologies
- Occupational Information Network (O*NET)
- Older Worker Issues
- Person–Environment Fit
- Person–Job Fit
- Person–Organization Fit
- Person–Vocation Fit
- Placement and Classification
- Prescreening Assessment Methods for Personnel Selection
- Realistic Job Preview
- Recruitment
- Recruitment Sources
- Retirement
- Selection Strategies
- Selection: Occupational Tailoring
- Succession Planning
- Underemployment
- Industrial Psychology: Developing, Training, and Evaluating Employees
- 360-Degree Feedback
- Assessment Center
- Assessment Center Methods
- Distance Learning
- Diversity Training
- Electronic Performance Monitoring
- Employee Assistance Program
- Executive Coaching
- Feedback Seeking
- Frame-of-Reference Training
- Leadership Development
- Mentoring
- Organizational Socialization
- Organizational Socialization Tactics
- Performance Appraisal
- Performance Appraisal, Objective Indexes
- Performance Appraisal, Subjective Indexes
- Performance Feedback
- Rating Errors and Perceptual Biases
- Self-Fulfilling Prophecy: Pygmalion Effect
- Socialization: Employee Proactive Behaviors
- Training
- Training Evaluation
- Training Methods
- Training Needs Assessment and Analysis
- Transfer of Training
- Industrial Psychology: Productive and Counterproductive Employee Behavior
- Contextual Performance/Prosocial Behavior/Organizational Citizenship Behavior
- Counterproductive Work Behaviors
- Counterproductive Work Behaviors, Interpersonal Deviance
- Counterproductive Work Behaviors, Organizational Deviance
- Creativity at Work
- Customer Satisfaction with Services
- Cyberloafing at Work
- Innovation
- Integrity at Work
- Job Performance Models
- Organizational Retaliatory Behavior
- Theft at Work
- Time Management
- Violence at Work
- Whistle-Blowers
- Withdrawal Behaviors, Absenteeism
- Withdrawal Behaviors, Lateness
- Withdrawal Behaviors, Turnover
- Workplace Incivility
- Industrial Psychology: Motivation and Job Design
- Action Theory
- Control Theory
- Empowerment
- Expectancy Theory of Work Motivation
- Goal-Setting Theory
- Human–Computer Interaction
- Incentives
- Intrinsic and Extrinsic Work Motivation
- Job Characteristics Theory
- Job Design
- Job Involvement
- Job Rotation
- Job Sharing
- Need Theories of Work Motivation
- Path–Goal Theory
- Positive Psychology Applied to Work
- Self-Concept Theory of Work Motivation
- Self-Efficacy
- Self-Regulation Theory
- Social Cognitive Theory
- Telecommuting
- Theory of Work Adjustment
- Two-Factor Theory
- Work Motivation
- Workaholism
- Industrial Psychology: Leadership and Management
- Abusive Supervision
- Behavioral Approach to Leadership
- Charismatic Leadership Theory
- Employee Grievance Systems
- Global Leadership and Organizational Behavior Effectiveness Project
- Implicit Theory of Leadership
- Judgment and Decision-Making Process
- Judgment and Decision-Making Process: Advice Giving and Taking
- Judgment and Decision-Making Process: Heuristics, Cognitive Biases, and Contextual Influences
- Leader–Member Exchange Theory
- Leadership and Supervision
- Least Preferred Coworker Theory
- Life-cycle Model of Leadership
- Normative Models of Decision Making and Leadership
- Reinforcement Theory of Work Motivation
- Situational Approach to Leadership
- Spirituality and Leadership at Work
- Trait Approach to Leadership
- Transformational and Transactional Leadership
- Trust
- Industrial Psychology: Groups, Teams, and Working with Others
- Conflict at Work
- Conflict Management
- Diversity in the Workplace
- Group Cohesiveness
- Group Decision-Making Quality and Performance
- Group Decision-Making Techniques
- Group Development
- Group Dynamics and Processes
- Groups
- Groupthink
- Input–Process–Output Model of Team Effectiveness
- Intergroup Relations
- Interpersonal Communication
- Interpersonal Communication Styles
- Justice in Teams
- Meetings at Work
- Negotiation, Mediation, and Arbitration
- Networking
- Social Exchange Theory
- Social Loafing
- Social Norms and Conformity
- Social Support
- Team Building
- Team Mental Model
- Team-Based Rewards
- Virtual Teams
- Workplace Romance
- Industrial Psychology: Employee Well-Being and Attitudes
- Affective Events Theory
- Attitudes and Beliefs
- Boredom at Work
- Emotional Burnout
- Emotional Labor
- Emotions
- Eustress
- Job Satisfaction
- Job Satisfaction Measurement
- Job Security/Insecurity
- Mood
- Morale
- Organizational Commitment
- Organizational Cynicism
- Organizational Justice
- Psychological Contract
- Quality of Work Life
- Role Ambiguity
- Role Conflict
- Role Overload and Underload
- Stress, Consequences
- Stress, Coping and Management
- Stress, Models and Theories
- Theory of Reasoned Action/Theory of Planned Behavior
- Union Commitment
- Work–Life Balance
- Industrial Psychology: Organizational Structure, Design, and Change
- Attraction–Selection–Attrition Model
- Automation/Advanced Manufacturing Technology/Computer-Based Integrated Technology
- Balanced Scorecard
- Compressed Workweek
- Downsizing
- Entrepreneurship
- Flexible Work Schedules
- Globalization
- High-Performance Organization Model
- Learning Organizations
- Mergers, Acquisitions, and Strategic Alliances
- Organizational Behavior
- Organizational Behavior Management
- Organizational Change
- Organizational Change, Resistance to
- Organizational Climate
- Organizational Communication, Formal
- Organizational Communication, Informal
- Organizational Culture
- Organizational Development
- Organizational Image
- Organizational Politics
- Organizational Sensemaking
- Organizational Structure
- Outsourcing
- Shiftwork
- Sociotechnical Approach
- Strategic Planning
- Survivor Syndrome
- Terrorism and Work
- Theory of Action
- Total Quality Management
- Virtual Organizations
- Workplace Injuries
- Workplace Safety
- Professional Organizations and Related Fields
- Loading...
Get a 30 day FREE TRIAL
-
Watch videos from a variety of sources bringing classroom topics to life -
Read modern, diverse business cases
-
Explore hundreds of books and reference titles
Read next
More like this
Sage Recommends
We found other relevant content for you on other Sage platforms.
Have you created a personal profile? Login or create a profile so that you can save clips, playlists and searches