01
Step one: choose four to six criteria
Start from the role, not from a generic list. For a customer support lead, you might select written communication, conflict handling, process improvement, coaching ability and tool fluency. Each criterion should be observable in an interview and distinct from the others, so a candidate cannot be scored twice for the same strength. Write a one-sentence definition for each, such as coaching ability meaning evidence of helping others improve measurable results. Keep the list short; more than six columns invites shallow scoring and makes debriefs harder to run within a single meeting.
02
Step two: define the scale and anchor each point
Pick a simple scale, commonly one to four, and describe what each number looks like in practice. For conflict handling, a one might mean avoids the issue, a two means resolves with prompting, a three means resolves independently with a clear outcome, and a four means resolves and changes the process to prevent repeats. Anchors reduce drift between interviewers who would otherwise treat a three differently. Avoid a middle score that lets people hedge. Add a notes cell beside every rating, requiring a specific example or quote from the conversation to justify the number.
03
Step three: assign interviewers and run the debrief
Give each interviewer two or three criteria so questions do not overlap, and ask them to score independently before discussing. In the worked example, one panelist covers written communication and tool fluency, another covers conflict handling and coaching. Collect scores into the grid, with candidates as rows, then review the largest gaps first. A split of one versus four on a criterion signals a need to compare evidence rather than average the result. Record the final decision with the strongest supporting notes, then revisit criteria after the hire to see which ones proved useful.