Inclusion and Anti-Bias Awareness: Assessment of Candidates and Use of Rubrics
Modified from University of Washington Handbook of Best Practices for Faculty Searches, Part 4 Assessment
One of the best tools for mitigating potential bias in the hiring process is to establish evaluation criteria before the committee begins reviewing applications—ideally, before or while the committee drafts the job advertisement. An assessment rubric ensures that all applicants are subject to the same evaluation criteria, and that members of search committees apply selection criteria consistently.
An assessment rubric also helps the committee clearly weigh its selection criteria against hiring priorities. Some questions to consider:
- What are the goals for this hire in terms of research/creative practice, teaching, and leadership/service?
- How is a commitment to diversity, equity, and inclusion a factor in each goal?
- What types of evidence will demonstrate achievement or future potential in each area?
- Does the job ad request materials appropriate to the assessment criteria?
Committees should consider how many distinct criteria will be used in their assessment. Between 5 and 8 is a recommended range. They should also consider whether a single rubric will be adequate, or whether it will be useful to devise multiple rubrics for the multiple stages of a complex search. Some committees, for instance, find it useful to “scaffold” their rubrics so that they use 2 -3 criteria in the first round of assessment then add additional criteria in the subsequent rounds. And committees should consider what kind of scale to employ. Some typical scales include:
- A simple choice of “High,” “Medium,” and “Low” rankings (3-point scale). A simple choice may ensure greater consistency in how diverse committee members employ the scale.
- A more elaborate choice of “Excellent,” “Good,” “Fair,” “Deficient,” and “No Evidence” rankings. (Finer distinctions may require “norming” exercises to ensure diverse committee members employ the scale in similar ways.)
Sample candidate evaluation/rubric templates for general use and also generated by Parsons are available. A range of sample assessment rubrics are also available in the University of Washington’s Handbook of Best Practices for Faculty Searches Online Toolkit.
Using the Assessment Rubric as a Tool for Discussion
Committees may be tempted to use the assessment rubric to rank applications based on total scores. It is important to stress, however, that the assessment rubric is a tool to help maintain consistency and fairness in the evaluation process, that is, to minimize bias either in favor of or against particular applicants. The rubric is not a substitute for active committee deliberations. Committee members should discuss the relative merits of specific applicants, and the review process should allow committee members opportunities to discuss any applicants they find have merit, regardless of assigned scores or combined rankings.
Creating and Implementing an Assessment Plan
Before any applications are reviewed, the committee should agree on an explicit plan for how it will conduct its business in a fair and consistent manner. Some questions to ask:
- When will the committee begin reading and assessing applications? As applications come in? Or after the priority deadline?
- Given the anticipated size of the applicant pool, how many rounds or stages of assessment are likely to be needed? And will a single assessment rubric be appropriate, or will multiple rubrics be needed?
- Should all committee members read and assess the same materials at the same stage of the search process?
- How will committee members define and then handle potential conflicts of interest, such as a prior relationship with an applicant or with an applicant’s adviser? This issue can be especially challenging if the pool includes internal applicants.
- By what process will the committee come to a decision about its interview list? Will members vote, for example, or deliberate until they achieve consensus?
- At what point in the process will the committee review letters of recommendation or contact references? Research suggests that, although they can provide useful information, letters of recommendation often reflect their authors’ idiosyncracies and biases—rather than provide an “objective” assessment.
- Will the committee conduct preliminary interviews? If so, will these be in person, over the phone, by Zoom, or by some other means?
- By what process will the committee create its short list of candidates for final interviews/campus visits?
- How will the committee organize final interviews/campus visits—will they be conducted in person and on campus or in a virtual environment?
- By what process will the committee make its final assessments and recommendations?
- And most importantly, how will the committee work to minimize the potential impact of implicit bias at each stage of the process?