Institutional Learning and Change Initiative

Keywords:

Abstract:

Conventional wisdom is that investments in agricultural R&D have yielded handsome dividends for society, more than enough to justify past investments and to support increased funding in the future. Many cite annual rates of return in the range of 40?60 percent as the norm, although some have suggested that these estimates are biased upwards or represent a partial and possibly biased sample of the overall rate-of-return evidence. Past reviews of the evidence on rates of return to agricultural research have been generally descriptive in nature, usually ad hoc, and always partial. The entire body of work has not been subjected to systematic, quantitative scrutiny of the types needed if we were to adequately answer various questions that have relevance to decisionmakers concerned with agricultural R&D. For example, do the returns to more recent investments match those of investments in earlier times; do investments in international R&D yield greater payoffs than investments in research conducted by national agencies; is there any evidence to support the view that research into crops yields higher rewards than livestock research? To interpret the empirical evidence properly also requires answers to some more subtle but equally important questions concerning the consequences of varying estimation techniques for the measured rates of return. Different studies in different locales at different time periods have used different evaluation methods. Do these differences in methods have implications for the interpretation of the evidence? Do some methods lead to a bias, a systematic difference between the actual and the measured rate of return? In this study we made a concerted effort to assemble all the available evidence on the returns to investments in agricultural R&D published since 1953. This was the publication year of the seminal study by Nobel laureate T. W. Schultz, who first introduced an economic approach to assessing the impacts of agricultural R&D. We searched comprehensively for all the subsequent literature, be it studies published in refereed journals or in less formal forms like book chapters, monographs, and discussion papers. We assembled 292 studies reporting a total of 1,886 rate of return estimates?an average of 6.5 estimates per published study. About one-third of the publications were in refereed journals. Few (21 percent) of the published rate of return estimates fall within the range of conventional wisdom of 40?60 percent per year. Excluding two extreme outlier observations (724,323 and 455,290 percent per year), the average rate of return was 100 percent per year for research, 85 percent for extension, 48 percent for studies that estimated the returns to research and extension jointly, and 81 percent for all the studies combined. However, these averages give an incomplete and in some important ways misleading picture. The rate of return estimates are widely dispersed around their respective averages. For example, studies of returns to research reported estimates of annual rates of return ranging from ?7.4 percent to 5,645 percent. To demonstrate the effects of skewness on measures of central tendency in the various distributions of estimates of rates of return, we report the mode (the value of the most frequent observation) and median (the central value when observations are arrayed by size), in addition to the mean (or simple average). The median of the rate of return estimates was 48.0 percent per year for research, 62.9 percent for extension studies, 37 percent for studies that estimated the returns to research and extension jointly, and 44.3 percent for all studies combined. This is almost half the corresponding average, indicating significant positive skewness in the distribution of rates of return. What accounts for the substantial variation in the reported rates of returns? We posited a number of factors, grouped into four broad categories: ? Characteristics of the rate of return measure (for example, real versus nominal measures, ex post versus ex ante, average versus marginal, private versus social) ? Characteristics of the analysts performing the evaluation (factors intended to reveal possible bias or differences in precision of the measures associated with the attributes of the person or group that generated the estimate, or differences in the methods and approaches used that are not revealed by our other proxies) ? Characteristics of the research being evaluated (for example, the field of science, commodity class, type of technology, time period and geographical location, and institutional scope of the research being evaluated) ? Features of the evaluation (details of the methodologies used to estimate the returns to research, like the structure and length of the lag between R&D spending and its productivity consequences) Some of these factors cause variation in the underlying, true rate of return, some affect only the measurement of that true effect, and others influence both the true rate of return and its measurement. Because of the importance of within-group variability, it was difficult to draw meaningful inferences from tabulations and simple pairwise comparisons. Thus, to identify not only the significance but also the magnitude of the effect of a particular factor on the reported rate of return, we used multiple regression techniques in a meta analysis of the rates of return evidence. To construct our meta?data set, all the rate of return studies were assigned an identification number and then scored according to characteristics deemed likely to influence the true ix or measured returns to R&D. Using statistical methods to discard extreme outliers that would unduly influence the analysis, and dropping 726 observations because of missing values for one or more of the characteristics and a further two extreme observations, left us with 1,128 observations in our regression analysis. We found no evidence to support the view that the rates of return have declined over time, but our results suggest that returns may be higher when the research is conducted in more-developed countries. The returns varied by problem focus, with lower rates of return for research on commodities and natural processes with longer production cycles. Characteristics of the research being evaluated matter?specifically, the measured rates of return were lower when the scope of the research being evaluated was broader and when studies measured the rate of return to research and extension jointly, compared with research only. Characteristics of the research evaluation matter too. In particular, in econometric studies, large rates of return were associated with truncated research lags. We could identify no effect of accounting for R&D spillovers or market distortions on measured rates of return. Our key finding is that the sample averages or representative ranges stressed by previous reviews reveal little meaningful information about the rate of return evidence. We show that the rate of return literature and the numerous rate of return estimates in that literature have a low signal-to-noise ratio that does not lend them to meaningful analysis by ad hoc inspection. Nonetheless, our formal meta-analysis using multiple regression techniques does reveal some systematic sources of variation in the rate of return to R&D that should prove useful to policymakers.