Institutional Learning and Change Initiative

The following text is drawn from ISNAR (2003).

Integrated Partial Indicators/Weighted Multiple Criteria Analysis/Scoring Analysis

Conceptually, this method differs little from a checklist, or partial indicators of impact. The difference between these methods lies in the fact that there is some sort of system for “adding up” the partial indicators and arriving at a “bottom line score” for each potential R&D project or project area under consideration. The most common approach is to evaluate each project with reference to a specific set of criteria/questions (partial indicators). Each criterion is then assigned a numerical weight, which enables the array of R&D projects, or projects under consideration to be ranked in order of priority, according to the sum of the numerical values assigned to the various criteria.

Advantages of Integrated Partial Indicators

The major advantages of integrated partial indicators are:

• It forces the decision makers to evaluate all the significant factors which have bearing on the “worth” of the R&D and to make conscious trade-off’s among multiple goals;
• It forces R&D decision makers to determine the criteria for assessing what makes for a good R&D investment; and
• It compels decision makers to rank R&D projects in terms of their relative importance.

Problems with Integrated partial Indicators

Two problems with the integrated partial indicators method are:

• Potential arbitrariness and subjectivity in assigning weights to the various criteria; and
• It is not well suited for ranking R&D projects in significantly diverse research areas.

Suitability of Integrated Partial Indicators

The integrated partial indicators method is more suited for assessing R&D under consideration for the future. The method works best for research toward the applied/development end of the scale. The integrated partial indicators method is better suited for comparing projects within categories rather than across categories.
 

Source: ISNAR (2003) Monitoring, Evaluation, and Impact Assessment of R&D Investments in Agriculture, The Hague: International Service for National Agricultural Research.