This document provides guidance beyond the instructions provided with the application form. It is intended to be read by all those involved in UMGF applications: the applicants themselves, those writing letters of reference, and representatives of departments involved in processing the UMGF applications from students in their departments.
As a result, for the 2007 competition, the Faculty of Graduate Studies decided to allocate the UMGFs to individual faculties and ask each faculty to do its own, internal adjudication. Decentralization resulted in a system where responsibility for the UMGFs is, in some sense, shared among the various faculties and Graduate Studies. This change addressed some of the issues but also introduced new challenges.
Subsequently, the Faculty of Graduate Studies decided to further devolve handling of UMGFs (and MGSs now) to the “unit” level (using a formula based on graduate student enrolment and time in program).
Handling these awards at the department level had the potential to introduce a number of issues (mostly related to small departments and the limited number of UMGF awards available) so the constituent departments of the Faculty of Science agreed to continue with faculty-level adjudication of the fellowships based on individual merit. This system enables us to benefit all departments while avoiding disadvantaging any single department and still allocate awards to the most deserving applicants. (E.g. Handling situations such as “Computer Science has 4 awards allocated but only 2 truly excellent applicants while Statistics has 1 award allocated but 3 excellent applicants.”)
The applicants are ranked (as described in the next section) by a Science-local evaluation committee, and then the Associate Dean, Research communicates a list of “to be awarded” along with their award GPAs, to the Faculty of Graduate studies. A number of “alternate” candidates is maintained by the Faculty of Science in case awards are declined or more awards become available.
The Faculty of Graduate Studies confirms the selections, verifies eligibility and availability of funding and informs successful applicants of their awards. The Associate Dean, Research informs unsuccessful applicants that reviewed by the evaluation committee. The departments notify unsuccessful candidates whose applications were not forwarded for review to the Faculty of Science.
Each reviewer assigns the application a score out of 10 to create an initial ranking. The resulting scores are then discussed in committee to ensure that the scoring is consistent, that all information has been taken into account and that the relative ranking of applicants is appropriate. During this process, the committee member from the applicant’s department acts on behalf of the applicant to provide clarification as needed. Ultimately, the final ranking of applications is decided by committee consensus.
Scoring is based on each student's performance and potential as measured by research experience, departmental ranking and scholarly achievement (including GPA).
Indicators of performance in these areas include the quality and originality of the research statement, research accomplishments, awards held, publications, conference presentations, and letters of reference.
To address the issue of differences among departments, the committee evaluates an applicant’s performance in a discipline-specific way, making use of information regarding “departmental norms” describing expectations for excellent students (either MSc or PhD) in each department. The committee also takes special care in the assessment of GPA's for foreign students considering the quality of the institution and, when possible, the quality of the individual department.
The “big picture” problems fall into three areas: inconsistencies, lack of detail, and missing information.
There are two commonly occurring specific forms of inconsistency. The first is inconsistency by referees between their scoring of applicants on specific characteristics (e.g. “Top 10% in terms of Academic Ability”) and the subsequent description of their abilities and experience. The second is inconsistency in quality of the various sections of the application, likely resulting from more attentive proofreading by the proposed advisor of some sections than others.
Lack of detail is most commonly manifested in information provided by the head (or department awards committee) in Part III and, sometimes, in information provided by the referee in Part II. Occasionally, the applicants themselves also provide research descriptions that are “too thin”, particularly in discussing how they intend to approach their research problem.
Missing information is disappointingly prevalent. Small things such as the monetary value of awards and whether they were national or local may seem insignificant but can be important when differentiating between very closely ranked applicants. Also, what would appear to be obvious information in the research description is sometimes omitted. For example, saying what research is planned but nothing about why it is interesting to the target community is all too common.
Second, it is important to remember that the adjudication will be done by committee members who are not in the applicant’s own department. This means that the entire application must be written so it will be understood by a non-expert audience but is still scientifically significant. (This can be challenging!) In particular, it is very important to explain the significance and potential impact of the proposed research since this may not be obvious to non-expert reviewers. (Significance need not be some application of the proposed work. Instead, for example, it might be a description of how it will advance the field.) Also, the relevance of previous experience (especially industrial experience) to the proposed research needs to be explained.
Third, the entire application is to be written by the applicant including the research proposal. While it is entirely appropriate for the applicant to discuss the proposal with his/her supervisor, it is inappropriate to include material that is written by anyone else. Including such material is normally obvious to the committee and must be avoided.
Finally, applicants should be careful to choose appropriate referees who know them well and are able to write in detail about them, their experience and abilities.
Some more specific suggestions include the following:
Wherever possible, the cohort of “similar students” should be as specific as possible. Thus, for example, a cohort of “new MSc students” is much more useful than one of “graduate students” generally. It is also advantageous to convey information about the cohort selected to the evaluation committee. Very small cohorts should be avoided if possible. This is sometimes problematic with referees who are new academics. Please remember, however, that there is no reason why a cohort has to consist of only University of Manitoba students.
Comparison to students at a similar stage that a referee may have known at other institutions is perfectly reasonable. Again, the cohort should be described to the committee. Further, ranking someone in the top 2% in a cohort of size 10 does not make sense and casts doubt on the validity of other scores as a result. Finally, simply saying that a student is “excellent” has little value since at this level all students are, to some extent, excellent. It is far more useful to precisely explain why/how the student is excellent and compare him/her (anonymously) to other students at similar levels.
It is also very important for a referee’s report to be as detailed as possible. The bulk of referees’ reports provide specific information on the qualities, abilities and potential of an applicant. A report that does not provide specifics, suggests that either the referee does not care whether or not the student receives the award or that the referee does not know the applicant well. In most cases, the referee should know the applicant well.
In all cases, it is useful to describe the context in which the referee knows the applicant. If there is limited knowledge of the applicant then the referee should explain why s/he is acting as a referee and why the referee’s report should be considered to be accurate. A referee who does not know the applicant likely should not have agreed to serve as a referee in the first place. If one has agreed to serve, the committee must be made aware of the extenuating circumstances that lead to this decision.
The Head’s recommendation can also provide information that the referee’s letter cannot. In particular, it is very useful to explain why the Head or awards committee ranked the applicants in the order they did. It is not uncommon for the UMGF evaluation committee to independently rank applicants from a department in a slightly different order, and this divergence can be a cause for concern. Statements such as “This applicant was ranked ahead of applicant X because…” are very valuable. Sometimes the head can also provide information about the value of a candidate’s experience, etc. that a referee may not be able to provide. This may happen when a referee is external or new to the University of Manitoba.
Finally, it is extremely important for the Head or awards committee to do a good job of describing the “departmental norms” provided to the Faculty of Science UMGF adjudication committee prior to the adjudication. There is significant variance in expectations for students in different departments in terms of publications and experience, etc. There are also real differences within departments for MSc and PhD programs. Expectations for “typical” excellent MSc and PhD students need to be clearly articulated so that the evaluation committee can fairly compare and ultimately rank the applicants. As in the recommendations to referees, the heads/awards committee members are strongly encouraged to provide evidence supporting their comments (e.g. “why a student is brilliant) and to cover as many criteria for evaluation as are practically possible.
Steve Whyward, Biological Sciences
Battling Mosquitoes and the West Nile Virus
A fluorsecence image of non-biological material: science meets art in microscopy imaging. Photo courtesy of Erwin Huebner, Biological Sciences
A color 3-D projection of the DNA contained in nuclei of the female germ tissue in an ovary of a marine worm Ophryotrocha labronica. The high colorful peak shows the DNA amount in a nurse cell as compared with the low blue flatter one of the oocyte. Photo courtesy of Erwin Huebner, Biological Sciences.