Assessing the Benefits of Public Research Within an Economic Framework: The Case of USDA’s Agricultural Research Service
by Paul Heisey
, John King, Kelly Day-Rubenstein
, Dale Bucks, and Rick Welsh
Economic Research Report No. (ERR-95) 90 pp, May 2010
Agricultural research managers continually seek to identify the benefits of research programs, both to demonstrate social and economic impact and to prioritize future research. Tight budgets for public agricultural research have increased the need to get the most out of these investments. Meanwhile, shifting policy goals have pushed public agricultural research in new directions. Although economic analysis can provide quantitative estimates of research benefits, it may not be amenable to all situations. However, economic reasoning may be useful even when formal economic methods are not used.
What Is the Issue?
USDA's Agricultural Research Service (ARS) accounts for the majority of Federal expenditures on agricultural research. The most common method of Federal research evaluation is peer review-widely used throughout the ARS prioritization, planning, and evaluation cycle. Peer review's strength is assessing scientific merit. However, it is not well-suited for quantifying market impacts or ultimate social benefits of research programs. Economic analysis, by contrast, can address these benefits, but often at considerable cost.
What Did the Study Find?
The standard techniques of economic evaluation-especially econometric and economic surplus techniques - have usually been applied retrospectively, and at aggregate levels of analysis. Three case studies of ARS research-bovine quantitative genetics/genomics, water quality/watersheds, and nutritional composition of food-illustrate opportunities and limitations of applying existing methods of economic evaluation to individual programs of research, and demonstrate the value of economic reasoning even in the absence of quantitative analysis. These case studies were selected to capture research diverse in its nature (basic versus applied), program scope, coordination roles, and relevance to USDA missions. As such, the case studies are not meant to represent the full range of issues addressed by ARS; rather, they focus on some of the more challenging aspects of research evaluation.
Case Study Specifics
• Bovine Quantitative Genetics and Genomics. ARS researchers conserve animal genetic resources, identify genes related to economically important production traits in beef and dairy cattle, and estimate the heritability of desirable genes. The benefits of ARS research can be measured by the value of the increased productivity engendered by the research, so standard economic analysis is plausible. Nonetheless, allocating attribution among the various contributors to increased productivity adds to the complexity of the problem.
• Water Quality and Watersheds. ARS research on water quality and watersheds examines physical and chemical properties of water and agricultural pollutants, their impact on agricultural production and water quality, and methods to improve agricultural and environmental outcomes. This research generates both private and social benefits. The benefits of research leading to the development of specific products or services, such as irrigation control equipment, are amenable to standard economic analysis. But the benefits of research on factors affecting water quality are harder to evaluate, requiring the valuation of nonmarket goods/services and linking research with public policies and regulations.
• Nutrient Data Laboratory. The Nutrient Data Laboratory (NDL) provides data on the nutrient composition of foods in the American diet. Its databases are the foundation of virtually all public and commercial nutrient databases used in the United States and a number of other countries. The primary social benefit of NDL's research is improved human health, but the link between nutrition information and health outcomes, while widely recognized, is not well
understood. Coupled with difficulty in assigning a dollar value to health outcomes, these measurement and attribution problems make standard cost/benefit analysis problematic.
Economic reasoning can provide qualitative analysis even when quantitative estimates of benefits are intractable. For instance, market failures and the presence of significant cross-State application of research findings may lead the private sector and State-funded institutions to underinvest in economically justifiable research. All of the case studies show that ARS research provides scientific results with few substitutes. Because of their "public goods" nature, the benefits of these research programs would be difficult to replicate by non-Federal research efforts. Thus, qualitative analysis can provide a clear indication of the public goods characteristics of a research program, even if it cannot rank multiple projects or programs, all of which demonstrably address public goods.
Furthermore, interviews with stakeholders such as food processors, natural resource managers, universities, other Federal agencies, and international research institutions suggest that ARS research facilitates numerous linkages between peers, contributors, and users. While it is difficult to put a dollar estimate on these benefits, interest on the part of a diverse group of stakeholders may indicate a broad set of benefits.
Finally, public research often aims to enhance the operations of Federal regulatory agencies and strengthen the scientific basis of government policies. Assessing research programs aimed at improving government regulations generally exceeds the scope of economic analysis because they work through the political process. However, economic reasoning is useful for tracing the demand for and performance of mission-related research. In these cases, ARS research often contributed to the regulatory and policy functions of the Federal Government in ways that other public sector research institutions did not. For example, with passage of the Clean Water Act, ARS adapted its soil movement models to examine the effects of sedimentation on downstream water quality. The findings of the three case studies provide additional insights about important issues in assessing research benefits.
• Basic research and nonmarket goods. In Federal research evaluation, benefit-cost analysis has been used primarily for Federal research programs that produce specific, near-market technologies. While some ARS research produces near-market technologies, two of the case studies (and much of the third) involved research producing significant nonmarket benefits or basic research with no immediate market applicability.
• Attribution. Research builds on previous findings, and numerous related efforts are often being performed by other institutions. Attributing benefits to one group of researchers is imprecise at best. All three case studies featured significant numbers of different researchers, with numerous horizontal and vertical linkages.
How Was the Study Conducted?
This report focuses on the feasibility of using economic and other evaluation methods to value ARS research, rather than the estimation of quantitative values. This study combined two principal methods-detailed literature reviews, and in-depth qualitative interviews with ARS research administrators, scientists, and stakeholders. The interviews used an interview guide approach that featured structured but open-ended questions, with information-rich respondents queried until answers confirmed working conclusions rather than revealing new important topics.