Needs Assessment: Analysis and Prioritization
Publication Year: 2010
“There is plenty of sound advice for beginners, an update for individuals with more background, and pragmatic tools for everyone in the Needs Assessment process.”—Valerie Larsen, University of VirginiaThis book shows how to analyze and combine qualitative and quantitative data. The authors also explain how to put results together to facilitate making empirically based needs decisions. Although this book can be used in a stand-alone fashion, it is part of the Needs Assessment KIT—five interrelated and sequenced books that take the reader through the needs assessment process (ISBN: 978-0-7619-2595-8).
- Front Matter
- Back Matter
- Subject Index
- Chapter 1: The Data and Prioritization Mess: What is the NAC Getting into?
- Chapter 2: Dealing with Qualitative Needs Data
- Chapter 3: Coping with Quantitative Needs Data
- Chapter 4: Getting the Results Together for Prioritization
- Chapter 5: Prioritizing Identified Needs
- Chapter 6: Some Nagging Afterthoughts and Caveats
Copyright © 2010 by SAGE Publications, Inc.
All rights reserved. No part of this book may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system, without permission in writing from the publisher.
SAGE Publications, Inc.
2455 Teller Road
Thousand Oaks, California 91320
SAGE Publications Ltd.
1 Oliver's Yard
55 City Road
London EC1Y 1SP
SAGE Publications India Pvt. Ltd.
B 1/I 1 Mohan Cooperative Industrial Area
Mathura Road, New Delhi 110 044
SAGE Publications Asia-Pacific Pte. Ltd.
33 Pekin Street #02-01
Far East Square
Printed in the United States of America
Library of Congress Cataloging-in-Publication Data
Altschuld, James W.
Needs assessment. Analysis and prioritization (book 4)/James W. Altschuld, Jeffry L. White.
Includes bibliographical references and index.
ISBN 978-1-4129-7557-5 (pbk.)
1. Strategic planning. 2. Needs assessment. I. White, Jeffry L., 1954-II. Title.
This book is printed on acid-free paper.
09 10 11 12 13 10 9 8 7 6 5 4 3 2 1
Acquisitions Editor: Vicki Knight
Associate Editor: Lauren Habib
Editorial Assistant: Ashley Dodd
Production Editor: Brittany Bauhaus
Copy Editor: Melinda Masson
Typesetter: C&M Digitals (P) Ltd.
Proofreader: Victoria Reed-Castro
Indexer: Diggs Publication Services, Inc.
Cover Designer: Candice Harman
Marketing Manager: Stephanie Adams
At initial glance it may seem that this book does not quite fit with the first one in the KIT, which is an overview of the needs assessment process, and the other three that parallel the phases of assessment. Numerous useful tables are embedded in them, as are ways to analyze and portray the data and information that have been collected. This book may appear to be redundant and a bit of overkill.
There is some overlap, not an undue amount. Our firmly held impression is that the two main topics of this text tend to be glossed over when needs assessments are conducted, prioritization in particular. The senior author never ceases to be amazed when he queries supposedly knowledgeable needs assessors about ways in which needs-based priorities are determined. Usually when this question is brought up in group settings, followed by others involving how one need was selected over another and upon what criteria such decisions were made, a pregnant pause (silence) is noticeable. It seems that the outcomes were sort of decided by an invisible mechanism without well-defined rules or even semiapparent structures to guide choices. For relatively small needs this isn't a problem, but in large organizations with competing needs, the absence of criteria and decision rules can have a serious effect on priorities.
Indeed, one example in Book 1 is about an organization that funded work in an area that did not prove to be in accord with the needs of its clientele. A structured prioritization strategy would have prevented this wasteful expenditure of funds. All of the above observations underscore the perception that a separate book devoted to analysis and prioritization is a valuable addition to the four others in the KIT.
[Page x]It is our sincere hope that this volume provides a good overview of how to analyze two distinct types of data, pull them together in a meaningful way, and derive priorities from the collation of the information that has been generated by the needs assessment. What should result is a stronger foundation for needs-related decisions and one that will stand the scrutiny of involved and questioning audiences. If that foundation is not there and the priorities are challenged, they are difficult to defend. Having a basis for coming to final decisions is a step forward.
By the same token, it is recognized that data obtained from multiple groups and methods may not fall easily or simply into alignment and even may be contradictory. Therefore, ultimate priorities often result from negotiations with key stakeholders and groups. The text offers guidance rather than absolute solutions to help needs assessment committees (NACs) and their facilitators work through the complexities of analysis and subsequent prioritization.A Few Notes about Content
The content in the text is an overview of analytic and prioritization procedures and not a comprehensive treatment of all strategies in this regard. If the local circumstances require specialized indices and other aspects of the two processes, seek out sources in the literature.
As indicated in its title this is Book 4 in the Needs Assessment KIT. The other books are:
- Book 1: Needs Assessment: An Overview
- Book 2: Needs Assessment Phase I: Getting Started
- Book 3: Needs Assessment Phase II: Collecting Data
- Book 5: Needs Assessment Phase III: Taking Action for Change
We would like to express our appreciation to Yi-Fang Lee with whom we worked (2003–2005) in a complex evaluation project that utilized surveys with needs types of questions in them. Her research, which is frequently cited, benefited us, and thanks are extended for her diligence and insights into what can happen with data. We would also be remiss if we omitted Belle Ruth Witkin's work of 25 years ago that still resonates and had a significant influence on the current text. We would also like to thank the following reviewers:
- Stephanie Brzuzy, Xavier University
- Valerie Larsen, University of Virginia
- Kui-Hee Song, California State University, Chico
Last, we express gratitude to all others whose endeavors have enriched our understandings of needs.—James W.Altschuld—Jeffry L.White[Page xii]
About the Authors[Page xiii]
James W. Altschuld, PhD, received his bachelor's and master's degrees in chemistry from Case Western Reserve University and The Ohio State University (OSU), respectively. His doctorate is from the latter institution with an emphasis on educational research and development and sociological methods. He is now professor emeritus in the College of Education and Human Ecology at OSU after 27 years of teaching research techniques and program evaluation. In evaluation, he developed and taught a sequence of courses on theory, needs assessment, and design. He has coauthored three previous books (two on needs assessment and the other on the evaluation of science and technology education), has written many chapters on needs assessment as well as others on evaluation research and issues, and has an extensive list of publications, almost all in the field of evaluation. He has given presentations and done work in five countries outside of the United States. In his career he has been the recipient of local, state, and national honors including the Alva and Gunnar Myrdal Practice Award for contributions to evaluation.
Jeffry L. White, PhD, is assistant professor of educational foundations and leadership at the University of Louisiana at Lafayette. He teaches statistics, quantitative research methods, program evaluation, measurement and assessment. He received his doctorate from The Ohio State University in quantitative research, evaluation, and measurement. He has master's degrees in educational research evaluation, and public policy and management, and health services administration. He is a member of the American Evaluation Association and served as chair, cochair, and program chair of the needs assessment Topical Interest Group. He has published in a diverse set of journals.[Page xiv]
References[Page 111][Page 116]2004). Emerging dimensions of needs assessment. Performance Improvement, 43(1), 10–15. http://dx.doi.org/10.1002/pfi.4140430104(1997). Evaluation of the Eisenhower national clearinghouse for mathematics and science education. Columbus: The Ohio State University., , , , , & (1999). School-based educational innovations: Case illustrations of context-sensitive evaluations. Journal of Family & Community Health, 22(1), 66–79., , , & (2000). From needs assessment to action: Transforming needs into solution strategies. Thousand Oaks, CA: Sage., & (1990). Needs assessment based on household and key informant surveys. Evaluation Review, 14(2), 182–191. http://dx.doi.org/10.1177/0193841X9001400205, , & (2003). E-research: Methods, strategies, and issues. Upper Saddle River, NJ: Allyn & Bacon., & (Applied Survey Research. (1999). 1999 Monterey County homeless census and needs assessment. Watsonville, CA: Author.Applied Survey Research. (2002). Homeless census and homeless youth/foster teen study: Monterey County 2002. Watsonville, CA: Author.2003). Web-based surveys. Journal of Extension, 41(4). Retrieved February 9, 2006, from http://www.joe.org/joe/2003august/tt6.shtml(2006, September). What's happening in Hawaii-Pacific evaluation? Panel presentation to the annual meeting of the Hawaii-Pacific Evaluation Association, Waikiki, HI., , & (1980). Nutrition education needs assessment final report: 1980. Durham, NC: NTS Research Corporation.(1986). Maryland nutrition education needs assessment, final report and supplements. College Park, MD: State Department of Education., & (Centers for Public Health Preparedness Network of Evaluators. (2006). Lessons learned regarding education evaluation methods in the centers for public health preparedness network: A promising practices resource database. Washington, DC: Association of Schools of Public Health.[Page 112]2005). Imputation verses estimation of finite population distributions (Southampton Statistical Sciences Research Institute Monograph No. M05/06). Highfield, Southampton, United Kingdom: University of Southampton.(2007, August). Greater Longview United Way 2007 needs assessment. Retrieved December 15, 2008, from http://www.longviewunitedway.org/media/2007CNA.pdf, & (2001). Assessing the professional development needs of public health professionals. Journal of Public Health Management and Practice, 7(4), 23–27., , & (2003, May). Results of data analysis: NIC needs assessment on correctional management and executive leadership development. Washington, DC: U.S. Department of Justice.(Community Research Partners. (2005). Community indicators database report: A picture of trends and conditions in central Ohio. Retrieved February 3, 2009, from http://www.communityresearchpartners.org/uploads/publications//CommunityIndicatorsdb_v3.1.pdf1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16, 297–334. http://dx.doi.org/10.1007/BF02310555(1974, April). Magnitude estimation scaling in needs assessment. Paper presented at the annual meeting of the American Educational Research Association, Chicago.(1984, October). The use of multiple methods to assess continuing education needs. Paper presented at the joint meeting of the Evaluation Network and Evaluation Research Society, San Francisco., , & (2004). Night fall. New York: Hachette Book Group.(2003). Applied Social Research Method Series: Vol. 36. Scale development: Theory and application ((2nd ed.). Thousand Oaks, CA: Sage.2006). Cleaning up data and running preliminary analysis. In F. T. L.Long & J. T.Austin (Eds.), The psychology research handbook: A guide for graduate students and research assistants (, & (2nd ed., pp. 241–253). Thousand Oaks, CA: Sage. http://dx.doi.org/10.4135/97814129766262000). Mail and Internet surveys: The tailored design method. New York: John Wiley & Sons, Inc.(2004). Internet surveys: Back to the future. The Evaluation Exchange, 3, 6–7.(1993). Beyond equity in the delivery of alcohol and drug abuse treatment services. Journal of Drug Issues, 23(4), 559–577.(1999). After drug treatment: Are 12-step programs effective in maintaining abstinence?American Journal of Drug and Alcohol Abuse, 25(1), 93–116. http://dx.doi.org/10.1081/ADA-100101848(2008). Psychometrics: An introduction. Thousand Oaks, CA: Sage., & (2001). Principles, methods, and instruments for the creation, preservation and use of archival records in the digital environment. American Archivist, 64(2), 238–269.([Page 113]2002). Web-based surveys: Changing the survey process. First Monday. Retrieved August 13, 2009, at http://firstmonday.org/htbin/cgiwrap/bin/ojs/index.php/fm/article/view/1014(1997). The effects of instrument design and respondent characteristics on perceived needs. Unpublished doctoral dissertation, The Ohio State University, Columbus.(2002). Using formal needs assessment data in ongoing strategic planning. Paper presented to the annual meeting of the American Evaluation Association, Baltimore., , & (1991). An empirical study of the structure of needs assessment. Unpublished doctoral dissertation, The Ohio State University, Columbus.(1974). Statewide educational needs assessment: Results from selected model states. Silver Spring, MD: Hershkowitz Associates.(1998). Educational and psychological measurement and evaluation ((8th ed.). Needham Heights, MA: Allyn & Bacon.2008). Methodological and conceptual issues confronting a cross-country Delphi study of educational programs. Evaluation and Program Planning, 31(2), 191–198. http://dx.doi.org/10.1016/j.evalprogplan.2008.02.005, , & (1999). Seeking causal explanations in social epidemiology. American Journal of Epidemiology, 150(2), 113–120. http://dx.doi.org/10.1093/oxfordjournals.aje.a009969, & (1983). Needs assessment. In F. W.English (Ed.), Fundamental curriculum decisions (pp. 53–67). Alexandria, VA: Association for Supervisory and Curriculum Development.(1982). Evaluation basics: A practitioner's manual. Beverly Hills, CA: Sage., & (1999). Evaluation of interactive media in science education: A contextual approach. Journal of Science Education and Technology, 8(1), 55–65. http://dx.doi.org/10.1023/A:1009429504881, & (2007a). Effects of multiple stakeholders in identifying and interpreting perceived needs. Evaluation and Program Planning, 30(1), 1–9. http://dx.doi.org/10.1016/j.evalprogplan.2006.10.001, , & (2007b). Problems in needs assessment data: Discrepancy analysis. Evaluation and Program Planning, 30(3), 258–266. http://dx.doi.org/10.1016/j.evalprogplan.2007.05.005, , & (2000). Causal-utility decision analysis (CUDA): Qualifying SWOTs. In E.Biech (Ed.), The 2000 Annual: Vol. 2. Consulting (pp. 261–265). San Francisco, CA: Jossey-Bass/Pfeiffer.(2007). Factors influencing evaluation scope of coalitions on formative to summative levels. PhD dissertation, The Ohio State University, United States. Retrieved February 20, 2009, from ProQuest Digital Dissertations database (Publication No. AAT 3260171).(2005). Advanced and multivariate statistical methods: Practical applications and interpretation (, & (3rd ed.). Glendale, CA: Pyrczak.2006). Applied multivariate research: Design and interpretation. Thousand Oaks, CA: Sage., , & (2006). The basic practice of statistics (([Page 114]4th ed.). New York: W. H. Freeman & Co.1994). Needs assessment across disciplines. Performance Improvement Quarterly, 7(1), 60–79. http://dx.doi.org/10.1111/j.1937-8327.1994.tb00617.x, & (1990). Epidemiological analysis as a method of identifying safety training needs. Human Resource Development Quarterly, 1, 277–293. http://dx.doi.org/10.1002/hrdq.3920010309(2009). Understanding research methods: An overview of the essentials ((7th ed.). Glendale, CA: Pyrczak.2001). Qualitative research and evaluation methods ((3rd ed.). Thousand Oaks, CA: Sage.1997). Multiple regression in behavioral research: Explanation and prediction ((3rd ed.). Fort Worth, TX: Harcourt Brace College Publishers.2005, August). Keynote address to the Ohio Science and Engineering Alliance Summer Research Forum. University of Akron, Akron, OH.(2006). Tests and measurement for people who think they hate tests and measurement. Thousand Oaks, CA: Sage.(2006). Does marriage reduce crime? A counterfactual approach to within-individual causal effects. Criminology, 44(3), 465–508. http://dx.doi.org/10.1111/j.1745-9125.2006.00055.x, , & (1980). Results of a needs assessment strategy in developing a family practice program in an inner-city community. Journal of Family Practice, 10, 871–879., , & (2001). A case study of needs assessment in teacher in-service development. Educational Research and Development, 7(2–3), 137–160., , , & (2002). Using electronic surveys: Advice from survey professionals. Practical Assessment, Research & Evaluation, 8(1). Retrieved December 15, 2008, from http://pareonline.net/getvn.asp?v=8&n=1, , , & (1998). Program priorities, purposes, and objectives. In P. S.Cookson (Ed.), Program planning for the training and continuing education of adults: North American perspectives (pp. 273–300). Malabar, FL: Krieger.(2005). The reliability and validity of the item means and standard deviations of ordinal level response data. Management Dynamics. Retrieved December 12, 2008, from http://findarticles.com/p/articles/mi_qa5465/is_200507/ai_n21386649(2001). Using Internet/Intranet Web pages to collect organizational data. Organizational Research Methods, 4, 199–216. http://dx.doi.org/10.1177/109442810143002, & (2007). Intermediate statistics: A modern approach ((3rd ed.). New York: Lawrence Erlbaum Associates.2009). Developing a model comprehensive statewide needs assessment with corresponding training materials for state VR agency staff and SRC members (provisional document). Berkeley, CA: InfoUse., & (2006). Using multivariate statistics (, & (5th ed.). New York: Harper & Row.2007). The research methods knowledge base (, & ([Page 115]3rd ed.). Cincinnati, OH: Atomic Dog Publishing.2008). Applied statistics: From bivariate through multivariate techniques. Thousand Oaks, CA: Sage.(1983, July). Statewide needs assessment: A multi-methodological approach. Paper presented to Society for Nutrition Education, Denver, CO., & (2006a). Cultural dimensions in science, technology, engineering and mathematics. Journal of Educational Research and Policy Studies, 6(2), 41–59., , & (2006b). Persistence of interest in science, technology, engineering, and mathematics: A minority retention study. Journal of Women and Minorities in Science and Engineering, 12(1), 47–64. http://dx.doi.org/10.1615/JWomenMinorScienEng.v12.i1.40, , & (2007). Northwick Park Care needs assessment: Adaptation for inpatient neurological rehabilitation settings. Journal of Advanced Nursing, 59(6), 612–622. http://dx.doi.org/10.1111/j.1365-2648.2007.04344.x, , & (1996, November). Using indicators of the gap between need for service and available service capacity as the basis for needs assessment. Paper presented at the annual meeting of the American Evaluation Association, Atlanta, GA., , , & (1995). Planning and conducting needs assessment. Thousand Oaks, CA: Sage., & (2008). Multivariate statistics: Data screening to assess validity/quality of input. Retrieved August 13, 2009, from http://cfs.nrcan.gc.ca/subsite/wulder/screen(