The Review Process

It is critically important to understand the review process before you apply for funding. Every funding agency has a process that is a bit different. However, the majority use a peer-review system to evaluate proposal quality. This means that your research proposal will be read and evaluated by your scientists in your field of study. Most agencies have well established review criteria whereby their reviewers must evaluate proposals. For example, the National Science Foundation, which provides funding to scientists and engineers whose efforts are primarily in fundamental research, bases its peer review around two review criteria:

Criterion 1: What is the intellectual merit of the proposed activity?

Under this criterion, reviewers are asked to evaluate the significance of the proposed work, the qualifications of the research team, the adequacy of the research facilities including instrumentation, the creativity and originality of the project, and the thoughtfulness and thoroughness of the research design articulated in the proposal.

Criterion 2: What are the broader impacts of the proposed activity?

Specific issues here include the benefits of the project to society, the likelihood that funding will enhance the infrastructure for research and education in terms of personnel, facilities, instrumentation, and/or dissemination of findings, and the quality of the training and training environment afforded student participants.

Individual grant programs at NSF often have additional specific review criteria that applicants must address in their proposals. These additional criteria are identified in the published announcement for that grant program. Wise applicants read these announcements before beginning to write their proposal and address these criteria in their grant application.

NIH uses a set of five review criteria to evaluate its proposals:

• Significance;
• Approach;
• Innovation
• Investigator; and
• Environment.

Note, however, that these are very similar to those criteria used by NSF (discussed above).

Examples of Different Review Mechanisms

A significant number of academic researchers receive their external funding from either the NIH or the NSF. In this section, we’ll look at the review process at these two federal agencies to better understand the importance of preparing a concise, clearly written and articulated grant proposal when seeking funding for your research efforts.

NIH

The NIH evaluates research proposals uses priority scores, a quantitative assessment, to determine the relative funding priority for individual proposals. Anonymous assessments are provided by panels of a dozen or more reviewers, called study sections. Study section panelists are scientists, engineers, and clinicians who have been selected on the basis of their specific scientific expertise. Panelists typically serve for a period of four years or more on a study section. The rosters of the 100 or so regular study sections at NIH are published on the world-wide-web. NIH panels meet regularly for two-to-three days at a stretch near NIH throughout the year to review proposals. Study sections may review as many as one hundred proposals at a study section meeting. This is not to suggest however that all members of the study section read and evaluate each and every proposal. In fact, study section members are assigned to review a finite subset of the proposals under review as either primary or secondary reviewers. Both primary and secondary reviewers are expected to read and evaluate the proposal in advance of the study section meeting and to be prepared to discuss their evaluation of the proposal at the section meeting. However, primary reviewers are expected to prepare and submit a written report as well. At the section meeting, primary reviewers initiate the discussion of the proposal, followed by the secondary reviewers. Afterward, based on the discussion, all panelists then by secret ballot score the application on a 5 point scale ranging from excellent (1.0) to poor (5.0). Expressed on a percentage basis, the average numerical rating (taken to three significant figures) given by the panel, referred to as its “priority score” is used to rank proposals for funding. Ultimately, funding of a proposal at NIH is determined by the Institute’s National Advisory Council based on the priority score, the specific program needs/funding priorities, and the availability of funds.

NSF

NSF uses a more qualitative assessment in which proposals are scored by reviewers as: excellent, very good, good, fair or poor. Different programs at NSF use a mixture of panels and/or reviews solicited from single investigators reading the proposal at their home institution. NSF does not use a standing panel system. Panelists are invited to serve on a panel. They may or may not serve again on that panel next year. Panels are typically much smaller than NIH study sections both in their composition and their work load. NSF panels typically consist of anywhere from five to twelve scientists and engineers who may read and review between ten and twenty grant proposals. Panels meet for a period of one to three days depending on the grant program. Unlike NIH, NSF does not publish the rosters of its panels in order to preserve the anonymity of the reviewers and the integrity of the peer review process. Another important difference is that depending on the program, panelists may or may not receive copies of the grant applications in advance of the actual review meeting. This is a very important point as panelists may have a very limited amount of time in which to read and evaluate each proposal. Consequently, a poorly written and poorly organized proposal may be at a significant disadvantage. NSF panelists reviewing a proposal are not required to reach any kind of consensus nor do they rank proposals as a general rule during their evaluations. Funding of a proposal at NSF is ultimately determined by the Division based in part on the reviews, the specific program needs/funding priorities, and the availability of funds to support the project.