Recommendations for writing successful proposals from the reviewer's perspective. (Shop Talk).

Siegel, Linda S.

Making a Match

All funding sources require an application or proposal with similar requirements. Good project ideas have to be well-expressed and must fit into the sponsor's high priority areas for funding. Proposal writing fits into the metaphor of the Diving Contest. First, just like great divers who must be ready to perform according to the rules, well-practiced and informed in their field, and with a strong desire to win, so too must proposal writers' principal investigator (PI) be well-prepared. It is essential to know which project ideas get the best marks, that is part of knowing the rules and recognizing what the reviewers' value. Which of the state-of-the art project ideas is the one that will generate the most enthusiasm? What are the reviewers looking for? Who are the reviewers? And where, strategically, should the most important material be placed? Should it be up-front or should it appear later? The strategic submission of a proposal to a funding agency and the strategic placement of essential information withi n a proposal are essential parts of creating a competitive proposal.

Not all agencies require the same types of applications. For example, proposals to some Canadian agencies are only six pages long. That is the good news. The bad news is that the six pages must describe the entire project, including methods, significance, references, and what the investigators have done in the past. Canadian proposals must be succinct to get the point across. The shorter Canadian applications have the advantage of increasing the feasibility of getting good reviewers because even busy people can read six pages. Reviewers find that the shorter proposals also result in better reviewing because the information in the proposal is more focused and authors do not have space to digress. But shorter proposals also place a lot of trust in the reviewers to determine whether the PI has the skills to be able to do the research since many of the details normally included in a proposal are simply not there. Although proposals to U.S. funding agencies are typically longer than six pages, the principle of wri ting a focused, well-organized and succinct proposal that follows the agency guidelines is a sound one.

The Abstract--Idea In a Nutshell

A proposal's central idea must connect with the agency and the reviewers. The proposal must represent a novel idea, a new approach, method or tool, or must address a critical issue that has not been well-studied in the past. There must also be a compelling reason to fund it. The importance of the project must be captured throughout the proposal but is especially critical in the abstract. Reviewers who are assigned as first, second, or third reviewers will read the entire proposal but other reviewers may read only the abstract. Therefore, the abstract needs to be carefully crafted. The abstract as well as the body of the proposal must hook the reader with an interesting and well-articulated idea and a feasible plan of study so that all the reviewers on the panel will be able to understand the merits of the proposal.

Project Design--A Slippery Slope

Reviewers often single out the quality of the project design as the main reason a proposal makes or misses the funding cut. Many proposal writers focus on the literature review and seem to attend less to the quality of the project design. The design must fit the project goals and be methodologically sound. Regardless of the type of proposal, the methods, measurements, procedures for evaluating the project's goals, hypotheses or research questions, and the time line for completing the project must be complete and carefully described. The reviewers, many of whom are experts in their areas, spend much of their time examining the project's design. Reviewers also examine the expertise of the investigators for executing the methods and the promise of the approach for revealing new information.

For U.S. agencies in general, but particularly for Canadian funding agencies, a great deal of emphasis is placed on the investigator's research record. Past publications count a great deal, so it is essential to include details on findings from past research projects that are relevant to the project. This information shows that the investigator has the capability to conduct the proposed research and assures the reviewer that findings will be translated into peer-reviewed publications. This is, of course, a disadvantage for new investigators or those switching to a new field but often special consideration or special grant competitions are available to new investigators.

Because the research design is so important, proposal writers need to get all the help they can in writing this section. For this reason, we suggest that proposal writers have their protocols reviewed by appropriate individuals or local committees before submission. This way, proposals may get helpful feedback for strengthening the methodology, avoiding a critical oversight that makes a project impossible to execute. Pre-submission reviews need to be timely and constructive and applicants should submit their proposal sufficiently before the deadline for the pre-proposal review.

Proposal Strategies to Avoid Annoying the Reviewer

Reviewers are busy people and are often faced with a large number of proposals to review in a short period of time. The review duties come on top of the other work reviewers have as part of their daily jobs. Therefore, efforts directed at making the review process easier are worthwhile. Several problem areas follow.

Non-professional appearance. One of us (TM) reviewed a proposal with a handwritten cover sheet and budget pages. This proposal did not look professional. Appearances are influenced by different font styles and sizes but in the end the reviewers must be able to easily read and understand the text. Many reviewers are at the age where they have to wear glasses to read; tiny letters serve to remind reviewers of their age. Making the proposal text easy to read is critical.

Disorganized proposals. Reviewers are not sympathetic toward proposals that do not follow the agency's format. Reviewers are looking for certain key sections or proposal elements, either because they are going to have to write a review that critiques those sections, or because they are key pieces of the proposal. For these reasons, headings and subheadings are very important. Page numbers and accurate references to pages in the Table of Contents, and references to figures and tables within the text are important. Finally, if you say "the information is in the Appendix," it actually should be there. Sometimes proposal writers get so rushed at the end of the proposal process that they can forget to change the text. Reviewers expect accurate information.

Incomplete proposals. It is hard for proposal writers to attend to every detail but reviewers do expect proposals to be complete. If the agency asks for a dissemination plan, one must be described beyond "I'll publish an article." Reviewers tend to be compulsive and they look for things such as whether all the references are included, especially if they read a section with an interesting citation. They note incomplete text, especially when the text notes "Bill, put something important in there." Reviewers examine the proposal in detail to determine that all information required is there.

Sparse justification. The entire proposal must be reasonable and logical. Reviewers are not impressed when: (a) important details are omitted or only vaguely specified, (b) the P1 devotes little time to the project, (c) the budget is not realistic because it reflects plans to completely equip a lab for the project, or (d) funds are not well-justified. Missing material, such as letters of support from consultants, collaborators, or other cooperating agencies is also noted by reviewers because it substantiates the project's feasibility.

Untested measures. A classic non-starter for reviewers is a lengthy list of measures to be developed or identified after the project is funded. Developing a valid and reliable instrument or implementing a complicated technique is difficult and time-consuming. Projects relying on complex techniques or critical measures are jeopardized when techniques and measures are unknown and the P1 lacks expertise in their use. Reviewers are not trusting about uncertainties in these areas.

No alternative approach as backup. Reviewers usually are looking for leading-edge approaches but they may also worry that the leading-edge approach will not work as proposed. If the proposal does not work out as planned, then what? Reviewers can be reassured when investigators combine their approach with a more standard approach so there is a fall-back position.

Reviewers as Human Beings

In general, reviewers are overworked, Type A but caring individuals, although they can also be malicious, vindictive, and self-serving. Thinking about the reviewers as individuals may help the process of successful proposal writing. Find out, if possible, which panel, study section, or subcommittee is likely to review the proposal and who the members are. Ideally, none of the members will be the intellectual or personal enemy of the investigators.

The composition of the review committee may influence how the proposal is prepared. For example, if mathematicians will be included on the review panel, the proposal should be written from their perspective. The word math is an abbreviation and it irritates mathematicians, so the word mathematics should be used throughout. The reviewers also are likely to be prominent in their fields and chosen for a review panel because their expertise is relevant to the proposals under review. Their work should be cited in the proposal where appropriate because they will look for citations to their work. Finally there may be differences of opinion between experts about theory, methods, and interpretations of findings. Knowing this, proposals should be written to explicitly consider these differences and to use evidence to justify the proposed approach. The reviewers may not change their opinions but at least they can not fault the proposal for ignoring opposing views.

Conclusion

Research administrators play an important role in the development of successful proposal writers. Few people are born with excellent proposal writing skills, but all people can learn to write successful proposals. Research administrators can be helpful in communicating information about the proposal review process, such as the information in this article, to proposal writers. Research administrators can identify proposal elements that are needed to support the proposed project. For example, ideas about budget elements (costs to include and exclude), reasonable bases for justification of budget elements, inclusion of a time line for the proposed work, and seemingly naive but helpful questions such as, "Does this work differ from that of others in the field?" "How does this project address the agency's funding priorities?" "What will we really learn if this project is successful?" Finally, research administrators can encourage proposal writers to resubmit an unsuccessful proposal using information contained in the review. Reviewers spend a lot of time on and give a lot of thought to their reviews. The intention of the reviewer is to provide a critique of the proposal and to also offer suggestions for improvements. Research administrators should encourage proposal writers to share their reviews and to continue working together to develop the proposal in light of the reviews. Many ultimately successful proposals were initially rejected by agencies. But when rewritten with consideration of the reviewers' comments, these proposals can result in grants. Furthermore, comments from reviewers even on proposals that are funded can be critical for future funding. Projects submitted for renewal are sometimes re-reviewed by the same people who reviewed the initial application. These reviewers remember what was suggested to improve the project during the initial review and if these suggestions are not followed, the renewal application could be jeopardized. Therefore, it is important for research administrators to help proposal writers to understand the importance and helpfulness of the reviewers' comments.

Author's Note: This article is a result of a presentation at the 2001 SRA Annual Meeting in Vancouver, British Columbia. Contact the corresponding author, Tori Molfese at the Center for Research in Early Childhood, Room 236, College of Education and Human Development, 1st and Cardinal Boulevard, University of Louisville, Louisville, KY 40292 or through Email: tori@louisville.edu

Victoria J. Molfese, PhD, is the Ashland/Nystrand Chair in Early Childhood Education, professor, and Director of the Center for Research in Early Childhood at the University of Louisville. She received her PhD in developmental psychology from The Pennsylvania State University and her research support is from foundations and federal agency grants. Publications span journal articles to books and include contributions to the journal. Dr. Molfese served as study section reviewer for two federal agencies and on the SRA board for seven years including SRA President in 2000. Karen S. Karp, EdD, Professor of Mathematics at Education University of Louisville, College of Education and Human Development received her doctorate in educational administration and policy studies from Hofstra University. Dr. Karp is the President Elect of the Association of Mathematics Teacher Educators and a SPA member. Linda S. Siegel, PhD, is Associate Dean of Graduate Programs and Research, the Dorothy C. Lam Chair in Special Education, a nd a professor at the University of British Columbia. Her doctorate is from Yale University. Dr. Siegel's research spans fields from language and cognitive development to identification of learning disabilities and resulted in extensive publications and several professional editor experiences.