- Home
- About us
- ILAC Activities
- Resources
- ILAC Publications
- ILAC Briefs
- ILAC Sourcebook
- Acknowledgement
- About the Sourcebook
- Acronyms
- Part 1 - Introduction and overview
- Part 2 - Tools and approaches
- Chapter 6: Introduction Part 2
- Chapter 7: Innovation histories
- Chapter 9: Culture study as a tool for change
- Chapter 12: Collaborative agreements
- Chapter 13: Facilitation as a foundation skill for ILAC
- Chapter 14: Learning alliances
- Chapter 15: Institutional histories
- Chapter 16: Engaging Scientists through Institutional Histories
- Chapter 18: Horizontal evaluation
- Chapter 19: Appreciative inquiry
- Part 3 - Cases and experiences
- Part 4 - Challenges and strategies
- About the authors
- Glossary
- References
- Working Papers
- Journal Articles
- Newsletter
- Reports
- ILAC Library
- Tools and methods for M&E
- Appreciative inquiry
- Biophysical measurements
- Case study
- Content analysis
- Contribution analysis
- Cost-Benefit Analysis
- Creative expression
- Diaries, journals and logs
- Dreams realised or visioning
- Expert review
- GIS mapping
- Graphing results
- Group assessment
- Historical trends and timelines
- Horizontal evaluation
- Impact evaluation
- Benefit-cost methods
- Case studies
- Cost-effectiveness analysis
- Counterfactual Impact Evaluation (CIE)
- Difference-in-difference
- Econometric methods
- Ex-post comparison of project beneficiaries with a control group
- Instrumental variables
- Integrated partial indicators
- Mathematical programming
- Modified peer review
- Partial indicators of impact
- Patent analysis
- Pipeline comparison
- Production function approach
- Propensity score matching
- Quasi-experimental design, involving the use of matched control and project groups
- Randomization
- Rapid assessment or review, conducted ex post
- Regression discontinuity design
- Simulation method
- User surveys
- Impact flow diagram
- Innovation histories
- Institutional history
- Institutional linkage diagram
- Interviews
- Learning alliances
- Learning-oriented evaluation
- M&E Frameworks
- M&E wheel (or "spider web")
- Mapping (sketch)
- Matrix scoring
- Most significant change
- Net-Map
- Non-random sampling
- Observation
- Outcome mapping
- Participatory Impact Pathways Analysis
- Participatory methods
- Performance indicators
- Photographs and video
- Problem and objectives trees
- Random sampling
- Ranking and pocket charts
- Rapid appraisal methods
- Relative scales or ladders
- Rich pictures (or mind maps)
- SWOT
- Seasonal calendars
- Semantic differentials
- Social mapping or well-being ranking
- Sociograms
- Stakeholder analysis
- Survey
- Systems (or inputs-outputs) diagram
- Theory-Based Evaluation
- Transects
- Evaluation studies and reports
- Evaluating capacity development
- Partnership
- Seminars
- Video Room
- Links to other sites
- E-learning courses
- ILAC Publications
- Contact
- Blog
How Legitimate and Justified are Judgments in Program Evaluation?
Tue, 02/23/2010 - 17:16 — Cristina Sette
Publication Type:
Journal ArticleSource:
Evaluation, Volume 15, Issue 3, p.307-319 (2009)Keywords:
Evaluation; ILAC Newsletter; judgment in program evaluation; legitimate justified judgmentAbstract:
The main function of program evaluations is to describe programs in order to generate judgments of value. To be considered credible, judgments should be both legitimate and justified. The research presented in this article posed the following question: do program evaluation practitioners generate legitimate and justified judgments? A meta-analysis of 40 program evaluation reports was carried out, which found that only 50 percent of the reports generated judgments. While these judgments seemed legitimate, they were rarely justified. However, the elements required to support legitimate and justified judgments were present in the reports in similar proportions, whether the reports generated a judgment or not.Sublibrary:
Evaluation