Obtaining research funding is a critical part of science, but the process seems to be getting more challenging each year. Funding rates are decreasing. Applicant frustration is increasing and many grant applicants tell me they feel confused about the differences between funded and unfunded proposals.
The largest funder of biomedical research in the U.S. is the National Institutes of Health (NIH), which is divided into different institutes based on focus of research (e.g. cancer, neuro, cardio, etc). One NIH institute is a bit broader in its scope and that is the National Institute for General Medical Sciences (NIGMS).
Dr. Jeremy Berg (pictured at right) was Director of NIGMS for several years until last year when he moved to the University of Pittsburgh where he is now Associate Senior Vice Chancellor for Science Strategy and Planning and a faculty member in the Department of Computational and Systems Biology.
During his tenure at NIGMS Dr. Berg developed what I think is a well-deserved reputation for openness and a thoughtful approach to the policy questions embedded in funding decision-making processes.
Dr. Berg kindly agreed to an interview with me about his experiences at NIH. Today we have part one of that interview below. Stay tuned tomorrow or Wednesday for part two of the interview.
Specifically based on your experience as an institute director at NIGMS is there any specific advice you’d give faculty regarding obtaining NIH funding?
Answer: Submit the most carefully prepared applications that you can! Given the present highly constrained budget situation and policies such as the limitation to only 1 amended application, applicants cannot afford to submit applications that have avoidable flaws. Work with other faculty at your institution or elsewhere to refine your ideas and application to make it as strong as possible. Check with program officers at the institute where your application is likely to go to explore any programmatic preferences that could help increase your chances of being funded. This will require getting started on your application well before the desired deadline, refining the specific aims, and then carefully editing the proposal.
Would you say that the NIH funding system could be accurately described as a “meritocracy”? If not, how would you describe it? If yes, do you believe that meritocracy is the best system for NIH?
Answer: I do believe that the NIH funding system is fundamentally a “meritocracy” but with some caveats. It is certainly a meritocracy in the sense that decisions are fundamentally based on estimated scientific merit, as opposed to more political considerations. But here are some caveats. First, percentile scores from peer reviews are experimental measurements (estimates) of scientific merit. They are not terribly precise and they do suffer from some systematic biases. For example, comparing an application from an established well-funded investigator with one from a starting assistant professor is no simple matter. The established investigator is likely to present considerable preliminary data and have a track record to relieve many concerns about the feasibility of the proposed project whereas the starting assistant professor will have fewer preliminary data and a shorter track record, but may have a more creative idea. Which has more scientific “merit”: a solid idea that is almost certain to work or a more creative but risky approach? Study sections wrestle with such comparisons, but they are hard. Second, the mission of NIH is to support relevant research in both the short- and long-terms. If two applications have the same estimated scientific merit (within the uncertainty to the peer review system), then the NIH program officers and institute leaders should consider other aspects of the applications. Continuing the example above, if one comes from an investigator with significant other resources to continue his or her research program while the other comes from a promising young investigator, I feel that the NIH mission is better served by supporting the application from the young investigator as developing this individual’s research program is likely to have a large impact on the NIH’s mission in the long run. I think it is a false choice to describe considering factors other that percentile scores in making funding decisions as an alternative to meritocracy. Maintaining a robust and broad scientific enterprise is crucial to fulfilling the NIH mission and full consideration of merit with as level a “playing field” as possible is essential.
Can an NIH study section realistically distinguish between proposals that are 10th vs 15th percentile? If not, what are the consequences given the critical nature of that funding range these days?
Answer: Like many people who had served on study sections, I was skeptical about the ability of study sections to make such distinctions reliably. While I was at NIH, I did some analysis of productivity (measured by a wide range of metrics) as a function of percentile score. This analysis supported the view that such distinctions cannot be reliably made. This analysis supports the point that I made above that factors other than percentile score can be taken into account without funding applications that have demonstrably less scientific merit. I hope that NIH will take factors such as the availability of other funding into account to make decisions about the potential impact of applications that cannot be distinguished on the basis of scientific merit based on peer review scores.
I hear scientists complaining about how NIH institutes are so divergent in how they handle funding decisions. For example, one hears about colleagues with proposals at one institute getting a 25% scored proposal funded, while at another NIH institute an 11% proposal was unfunded. Another example is that one institute very often cuts budgets up to 25% and/or cuts a year from funding, while another more often funds budgets at requested levels and full time periods. What are the benefits of allowing institute diversity of this kind? Are there downsides to or risks associated with this lack of conformity across NIH institutes?
Answer: I understand why scientists can be frustrated by such divergences but, on balance, I think they are helpful for achieving the NIH mission. Each institute has its own mission and the institute leadership should have the flexibility to manage the available resources in what they believe in the most appropriate way. The examples you cite also clearly indicate that individual policies are not independent but, in fact, are quite interrelated. Institutes that tend to fund applications to higher percentile scores do so, in large part, by cutting budgets from requested levels. For example, at NIGMS when I was Director, requested budgets were frequently cut by approximately 20% from requested levels and the resulting funds were used to fund applications up to or above the 20th percentile. We felt that it was better to fund more applications rather than to fund applications at the full requested level. We realized that such cuts could result in a decrease in the rate of research progress but having more laboratories and projects active seemed to us to be a better strategy. For institutes supporting more clinical research and clinical trials, such a strategy might not make any sense. A significantly underfunded clinical trial runs the risk of being a complete waste of money if, for example, the number of subjects recruited to a trial is not sufficient to give the statistical power necessary to answer the question under study. Again, the leadership of each institute needs to have the flexibility to make such decisions. With that said, however, I think it is important for each institute to be as transparent as possible about its policies, their basis, and the associated outcomes and I tried to promote such transparency while I was working at NIH.
Tomorrow in part two of the interview we cover what most surprised Dr. Berg at NIH and what one thing he would change there if he had a magic wand (my words).