AfrEA-NONIE-3ie Impact Evaluation Conference in Cairo, April 2009

Wed, 07/30/2008 - 12:54 — Cristina Sette

Dear colleagues:

I’ve been invited to serve as a member of an advisory group for a conference being organized by the African Evaluation Association (AfrEA), the Network of Networks on Impact Evaluation (NONIE) and the International Initiative for Impact Evaluation (3ie) on Impact Evaluation for Effective Development: Perspectives, Approaches, Methods, to be held in Cairo in April 2009. The conference aims to move beyond methodological debates to share experiences and build capacity.
This event looks like an important opportunity for ILAC related issues particularly on the impact evaluation side, and we can strategize as to how we participate in the meeting beyond my serving on the advisory group.

In the first instance, the organizers have asked members of the advisory group for feedback on the following questions. If you can provide any ideas immediately, please send them now and I will prepare a response that I would want to send soon, probably Monday.

1. What would you see as the main issues or topics which must be included in a conference on this theme?

2. Who would you expect to see as keynote speakers at such a conference?

3. The organizers will invite other organizations (e.g. IDEAS, the Campbell Collaboration, DIME) to be partners who will be given responsibility for a stream of sessions in the pre-conference workshops or at the conference itself. Which organizations would you propose as potential partners?

So, be aware of the conference, start thinking about how ILAC should be involved and provide any thoughts on any of the above questions as soon as possible. Thank you.

All the best, Jamie

Bookmark/Search this post with:

Comments

Wed, 07/30/2008 - 13:10 — Cristina Sette

Patricia's thoughts

Hi all,

I'm also on the advisory group so I'll share the response I've already made, including suggesting the inclusion of ILAC.

FWIW I think we do need to be careful in suggesting the big names in evaluation - some of them have only worked on impact evaluation of tightly defined interventions and literally have no repertoire for evaluating complex and complicated interventions like most international development.

Here are my thoughts:

2. What would you see as the main issues or topics which MUST be included in a conference on this theme?

Methods for negotiating the impacts that will be the focus of impact evaluations, particularly for projects, programs and policies with multiple stakeholders and funders Methods for capturing important unanticipated impacts Efficient methods for getting empirical evidence of impacts and the factors that influence them, including participatory methods Methods for causal analysis Methods for reporting to government, civil society, funders, Methods for supporting uptake of evidence from impact evaluations, including appropriate translation to different contexts Strategies for avoiding false negatives and false positives Strategies for avoiding other risks, including data corruption, goal displacement.

3. Who would you expect to see as keynote speakers at such a conference?

I've actually answered this as who I'd like to see…

Carefully chosen keynotes that are explicitly linked and contrasted – so they might address different issues or take different approaches, but with some process for commenting on them and highlighting their different contributions.

Martin Ravaillon – best practice in using experimental and quasi-experimental designs Professor Robert Chambers, IDS – participatory data collection and use of data Professor Ray Pawson, Leeds University – realist approaches Professor Sandra Nutley, University of Edinburgh – issues in developing and using the evidence base to guide policy

Other possible speakers:
Ben Ramalingam, ODI - applying complexity concepts and methods to impact evaluation

4. The organizers will invite other organizations (e.g. IDEAS, the Campbell Collaboration, conference workshops or at the conference itself. Which organizations would you propose as potential partners?

I've identified some other organisations who are doing work in the area of development impact evaluation and might be constructive partners:

Global Environment Facility
PEP –Poverty and Economic Policy Research Network ILAC, CGIAR – Institutional Learning and Change initiative, Bioversity, Consultative Group on International Agricultural Research AusAid NORAD IOCE – the International Organisation for Co-operation in Evaluation IDEAS World Bank Institute

Kind regards,
Patricia

Wed, 07/30/2008 - 13:09 — Cristina Sette

Uninspiring title

Hi Jamie,

Here are my reactions to the request;

The title of the conference is pretty uninspiring because it is formulated in language that does not challenge participants to think. It's a bit like "Christina for President"

It should be more direct e.g. How has (or How can be) evaluation been used to make development projects and organizations more effective. Such a title would be more inkeeping with the purported aim -- 'to move beyond methodological debates to share experiences and build capacity" (taken from your message).

Of course it is unlikely that you can influence the title of the conference at this stage, but it certianly is a turn-off -- especially as the subtitle (Perspecitives, Approaches, Methods) seems to conflict with the aim.

If the conference does pursue its aim to "share experiences and build capacity" then some topics might be:

How IA has allowed program staff / actors

to improve project results

to make the development organization more effective

to help build the development skills of all actors

to employ evaluative thinking while engaged in project activities

to enhance understanding of the root causes of the development problem

to enhance understanding of how the project accomplishes its goals

to enhance understanding of the range of actors whose behaviour must change for results to be achieved

to enhance the development project team's understanding of the development problem and its resolution

to justify the project's expenditures to taxpayers, donors, funders

One topic in particular I would like to see treated would be The optimum role(s) of the evaluator (evaluation team) and evaluation activity in the development project. (Impact evaluation is a poorly-funded add-on undertaken at a point beyond which project staff are receptive to learning and are thinking about the follow-up or the next project.) One of the few projects I have heard of that employs an evaluation team from the outset is the CPWF. Learning from evaluation would be promoted more by actually having a full-time evaluator on project staff from the beginning of a development project.

Other topics I would like to see treated -- and in some depth -- would be The Evaluation Standards. They do not get enough attention -- sometimes none at all! Also the guiding principles for evaluators that some evaluation associations ave formulated -- the US principles are available at http://www.preval.org/documentos/00478.pdf

Also skills required by evaluators or put another, slightly different way -- skills required in evaluative thinking.

As to who might be invited -- One group might be evaluators who have in fact been hired at the start of the project to build evaluation and evaluation thinking into the project as it develops and to build evaluation capacity and enhance learning across the board. They might be accompanied by those who did the hiring and spent their budget on such an evaluator; another might be project staff and other actors who might have benefited from the presence and activity of the evaluator.

Another might be those who have authored or sat on the committee that developed the Evaluation Standards -- and those who have helped formulate principles for evaluators and offered profiles of the skills evaluators should possess.

Some of the names that come to mind are Rossi, Lipsey, Freeman, Lincoln, Guba, Donaldson, Weiss, Chen --

I hope this helps you to make suggestions to the organizers.

Ronald Mackay

Wed, 07/30/2008 - 13:07 — Cristina Sette

reaction to the three questions

Dear colleagues: If you have any reaction to the three questions below about the Cairo impact conference please send them as soon as possible. I need to draft a response and send it in. Thank you. Jamie

Institutional Learning and Change Initiative