How to Assess Senior Engineers With a Code Review Work Sample

AI-generated illustration. The review work sample asks candidates to distinguish release-blocking defects from minor concerns and questions that need more evidence.

Assess senior engineers with a code review work sample that asks them to prioritize risks, explain their reasoning, and help an author improve a change. Score those behaviors against shared criteria, not comment volume or conversational polish.

The hiring question is specific: Can the candidate make a consequential review decision and explain what would change their mind?

This is a proposed assessment framework, not a validated predictor of job performance. Use it alongside other hiring evidence.

Give every candidate the same review conditions

Use a small, fictional pull request with enough context to distinguish a defect from a reasonable design choice. Avoid making success depend on undocumented repository conventions.

Here is an illustrative 40-minute format:

• 5 minutes: Read the system brief and ask initial questions.
• 20 minutes: Review the change and write prioritized comments.
• 10 minutes: Discuss evidence and receive a standardized clarification.
• 5 minutes: Update the review recommendation.

These timings are design choices, not benchmarks. Adjust them to the exercise and accessibility needs. Do not quietly make reading or typing speed a seniority criterion.

Give candidates this instruction:

Review the change for release. Identify what should block it, what is non-blocking, and what you cannot decide without more information. Explain your highest-priority concern and suggest a test or revision. You are not expected to find every possible issue.

Include a shared context sheet covering:

• The change’s purpose and intended behavior.
• Authentication, authorization, and data boundaries.
• Relevant dependencies and guarantees.
• Available tests and how to run them.
• Known unknowns the candidate may ask about.
• Tool rules and the final deliverable.

State whether documentation, search, code execution, and AI assistance are allowed. Provide comparable access to permitted tools, and assess whether candidates can explain and verify their conclusions. Do not make tool use a hidden criterion.

Use an isolated fixture with synthetic data. No production access or live customer records should be necessary.

An illustrative pull request: caching a project summary

Hypothetical change: An author adds a cache to reduce repeated database reads for a project’s open-issue count.

The candidate receives these facts:

• Authentication supplies a trusted tenant_id.
• Every authenticated tenant member may read every project within that tenant.
• Project IDs are unique within a tenant, but may repeat across tenants.
• The cache is shared across tenants and stores values under exactly the supplied key, with no additional namespacing.
• The repository query is correctly tenant-scoped.
• The existing uncached path handles missing projects correctly; the fixture’s projects all exist.
• The acceptable age of a cached count has not yet been specified.

The proposed change is pseudocode, not tested application code:

function get_project_summary(tenant_id, project_id):
    key = "project-summary:" + project_id
    cached = cache.get(key)
    if cached is not null:
        return cached

    project = repository.find(tenant_id, project_id)
    summary = { open_issue_count: project.open_issue_count }
    cache.set(key, summary, ttl_seconds=300)
    return summary

The fixture contains two synthetic tenants:

If Alpha’s project is requested first, Beta’s request can receive Alpha’s count: both use project-summary:17. The tenant-scoped repository query cannot protect a cache hit that returns before the query runs.

That is the release-blocking defect.

A proposed regression test would start with an empty isolated cache, request Alpha/17, then request Beta/17, and assert that each response contains its own count. Repeat with a fresh cache in the reverse order. These are test suggestions, not reported results.

The five-minute cache lifetime is a separate question. Whether that delay is acceptable depends on the product contract, which has not been supplied. A reviewer should distinguish an established defect from a missing requirement.

Score review judgment with evidence anchors

Record an observed comment, question, or decision beside each rating. Use insufficient evidence for any dimension the session did not provide a reasonable opportunity to observe.

Insufficient evidence is not a failing score. If the interviewer never offers a clarification or time to respond, that dimension remains unassessed.

A missing comment is not automatically a demonstrated mistake, either. Record “did not mention cache isolation” separately from “said shared keys were safe after tracing both requests.” The latter provides stronger evidence of a reasoning error.

Do not mechanically total the ratings into a hiring threshold. Decide which behaviors the role requires and how this exercise contributes to the decision. One missed issue should not become a judgment about every aspect of seniority.

AI-generated illustration. In the article’s hypothetical example, matching project IDs collide in a shared cache because the key omits tenant identity. A cache hit can return another tenant’s count before the tenant-scoped database query runs.

What candidate comments reveal

These invented comments illustrate evidence, not required scripts.

Style policing does not establish correctness

“Rename cached to cached_summary. I would block until this is clearer.”

The naming suggestion may be reasonable. Blocking the release needs a substantive rationale, not a personal preference that outranks a demonstrated isolation defect.

A concise comment can be adequate

“Blocking: the key omits tenant_id, but project IDs can repeat across tenants. A cache hit can return another tenant’s count. Include tenant identity in the key and add a test using the same project ID in two tenants.”

This identifies the mechanism, consequence, and a practical correction. Fewer comments do not make it weaker evidence.

Strong feedback connects a claim to a check

“Blocking: request Alpha/17, then Beta/17 with the shared cache. Both requests use project-summary:17, so Beta receives Alpha’s cached count without reaching the scoped query. Please make the key tenant-scoped using an unambiguous encoding and add a regression test. Separately, what freshness requirement supports the five-minute TTL?”

The added value is a reproducible failure path and a clear separation between a defect and a requirements question, not extra length.

Use clarification to test revision, not compliance

Prepare a shared answer sheet so interviewers supply consistent facts. At a set point, or earlier if asked, provide this clarification:

The product requirement permits open-issue counts to be up to five minutes stale. Tenant isolation is mandatory.

Then ask:

• “Does this change your release recommendation?”
• “Which comment would you keep, revise, or remove?”
• “What evidence would make you comfortable approving the revised change?”

A supported response can drop freshness as a blocker while retaining the isolation concern. Candidates need not adopt the interviewer’s preferred implementation if their alternative preserves the required behavior.

If the candidate missed the cache problem, first check their understanding of the supplied context: “What guarantees does the cache provide about tenant separation?” Then use graduated prompts:

1. “Walk through Alpha/17 followed by Beta/17. Which branches execute?”
2. If necessary: “Both requests use the same cache key. What follows from that?”

Record the assistance. Finding a defect independently and recognizing it after a targeted hint are different observations. Both can inform the discussion; they are not equivalent evidence.

Avoid praising a specific answer before the follow-up. Observe reasoning under new information, not willingness to agree.

Keep an evidence record, not an impression summary

Use this record for each dimension:

Dimension:
Observed comment or action:
Relevant context available at the time:
Prompting or assistance provided:
Rating: weak / adequate / strong / insufficient evidence
Reason for rating:
Remaining uncertainty:

Have evaluators record ratings independently before discussing them. Resolve disagreements by returning to the candidate’s reasoning and the context available, not labels such as “senior presence.”

Before using the exercise, ask reviewers unfamiliar with its construction to try it. Check whether the brief is clear, the time allows meaningful review, and evaluators interpret the anchors consistently. This can expose design flaws. It does not establish predictive validity.

For candidates presenting their own work, the framework can guide an evidence packet: a permitted, sanitized change description, the risk identified, the evidence used, the proposed revision, and any verification actually performed. Separate personal contributions from team outcomes. Omit confidential material.

The useful result is a defensible statement: “We observed this behavior, under these conditions, with this assistance, and these questions remain.” That tells a hiring panel more than “They left lots of good comments.”

For a next step in presenting your engineering evidence, explore Skills Tech Network.