Navigation Links
Changes needed in how federal government evaluates efficiency of research at EPA, other agencies
Date:1/31/2008

WASHINGTON -- The White House Office of Management and Budget evaluates research at the U.S. Environmental Protection Agency and other federal agencies using the Program Assessment Rating Tool (PART), a set of questions that asks agencies about many aspects of their programs, including whether they can measure and demonstrate annual improvements in efficiency. Based on the answers, OMB rates research programs as effective, ineffective, or somewhere in between. An "ineffective" rating can have serious adverse consequences for a program or agency. After experiencing difficulty meeting OMB's requirements to demonstrate the efficiency of its research programs, EPA asked a National Research Council committee for guidance on how to measure efficiency.

The committee's new report recommends four changes in how the federal government assesses the efficiency of research at EPA and other agencies. First, the report emphasizes that assessing efficiency should be considered only one part of evaluating a program's quality, relevance, and effectiveness.

Second, the report introduces a novel distinction between "investment efficiency" and "process efficiency," and recommends that these aspects be evaluated in different ways. Assessments of investment efficiency should examine whether an agency's R&D portfolio, including the budget, is relevant, of high quality, matches the agencys strategic plan, and is adjusted as new knowledge and priorities emerge. These evaluations require panels of experts, which should comprise both scientists and other stakeholders. In contrast, evaluations of process efficiency should focus on "inputs" (the people, funds, and facilities dedicated to research) and "outputs" (the services, grants, publications, monitoring, and new techniques produced by research), as well as their timelines. Of nine measures currently used by R&D agencies, common examples are the number of grants awarded or publications produced annually, which could be assessed against appropriate benchmarks.

Third, the committee clarified the roles of outputs and outcomes. EPA's difficulties in answering PART's questions about efficiency have grown out of OMB's insistence that the agency find ways to measure the efficiency of its research based on outcomes, rather than outputs. Measuring research efficiency based on what the committee describes as "ultimate outcomes" -- for example, whether a program eventually results in cleaner air or fewer deaths -- is neither achievable nor valid, because such outcomes occur far in the future and are highly dependent upon actions taken by many other people who may or may not use the research findings. The committee's review of practices across government R&D agencies revealed that no agency has found a way to demonstrate efficiency based on ultimate outcomes.

The report does endorse evaluating efficiency based on "intermediate outcomes" -- for example, assessing whether a program has improved the body of knowledge for decision-making, or disseminated newly developed tools and models. These assessments would be conducted most effectively using expert review panels and not just formulas.

Finally, the committee recommended that the efficiency of EPA's research programs be evaluated according to the same standards used at other agencies. OMB has rejected some methods for measuring research efficiency when proposed by EPA, but accepted them when proposed by other agencies. OMB should train and oversee its budget examiners to make sure they implement the PART questionnaire consistently and equitably across agencies.


'/>"/>

Contact: Sara Frueh
news@nas.edu
202-334-2138
The National Academies
Source:Eurekalert

Related medicine news :

1. DuPont Announces Leadership Changes
2. EpicTide Changes Name to FairWarning(R)
3. XLHealth Corporation Announces Executive Changes
4. OPKO Health Announces Changes to Board of Directors
5. Small Lifestyle Changes Can Boost Longevity
6. QED International Associates Announces Changes to the HealthShares(TM) Emerging Cancer Index
7. URAC Calls for Comments on Changes to Standards for Health Web Site, HIPAA Privacy and Security Accreditation Programs
8. Significant changes in plastic surgery expected in 2008
9. Standard & Poors Announces Changes to U.S. Indices
10. Fish Give Clues to Human Skin Color Changes
11. Kidney Patients Health Is Jeopardized by Proposed Medicare Changes
Post Your Comments:
*Name:
*Comment:
*Email:
(Date:10/13/2017)... ... October 13, 2017 , ... Lori R. Somekh, founder of the ... a national organization of elder law and special needs planning attorneys. “Membership in ElderCounsel ... also provides a forum to network with elder law attorneys nationwide,” said Somekh. ...
(Date:10/13/2017)... ... 2017 , ... Ellevate Network, the leading network for professional women, brought together ... equality at their inaugural Summit in New York City in June. The event was ... of over 3 million. To watch the Mobilize Women video, click here . ...
(Date:10/13/2017)... VA (PRWEB) , ... October 13, 2017 , ... ... of DevOps and Agile Software Development, has been awarded a contract by the ... Blanket Purchase Agreement (BPA) aims to accelerate the enterprise use of Agile methodologies ...
(Date:10/12/2017)... San Francisco, CA (PRWEB) , ... October 12, ... ... and Dr. Cheng, are now treating sleep apnea using cutting-edge Oventus ... sleep apnea, a serious sleep disorder characterized by frequent cessation in breathing. Oral ...
(Date:10/12/2017)... ... ... The American College of Medical Informatics (ACMI) will present the 2017 Morris F. ... AMIA’s Annual Symposium in Washington, D.C. AMIA’s Annual Symposium is taking place ... the field of medical informatics, this prestigious award is presented to an individual whose ...
Breaking Medicine News(10 mins):
(Date:10/11/2017)... 11, 2017  True Health, a leader in ... effort during National Breast Cancer Awareness month to ... Research recently ... that more than 10 million American women are ... BRCA1 or BRCA2 and have not had testing. These ...
(Date:10/7/2017)... , Oct. 6, 2017   Provista, a ... than $100 billion in purchasing power, today announced a ... information. The Newsroom is the online home ... trends, infographics, expert bios, news releases, slideshows and events. ... to a wealth of resources at their fingertips, viewers ...
(Date:10/4/2017)... 4, 2017 OBP Medical , ... medical devices, today announced regulatory approval from ... (or Agência Nacional de Vigilância Sanitária (ANVISA)) to ... surgical retractor with integrated LED light source and ... and exposure of a tissue pocket or cavity ...
Breaking Medicine Technology: