AI-generated illustration. Assess demonstrated engineering judgment without requiring access to confidential work; support for past achievements remains a separate question.
Assess engineers whose work is confidential by offering safe ways to demonstrate the same job-relevant skills: authorized artifacts, clearly labeled fictional examples, and structured technical questioning. Score the judgment you observe. Record claims about past delivery and impact separately as verified, partially supported, or unverified.
A public portfolio can supply useful evidence. Making it mandatory, however, can turn permission to publish into an unstated hiring criterion.
The alternative is one evidence standard with several acceptable routes.
Separate demonstrated capability from historical claims
Before requesting a portfolio, define the capabilities the role requires. For a senior engineer, these might include framing an ambiguous problem, identifying operational constraints, comparing designs, and planning validation.
Then keep two records:
Capability: What did the candidate demonstrate during the assessment? A fictional exercise can reveal reasoning under its stated conditions.
History: Which claims about prior ownership, deployment, adoption, or impact are supported, and what are the limits of that support?
Neither record substitutes for the other. A polished reconstruction does not establish that a system shipped. An impressive outcome does not, by itself, establish the candidate’s contribution or explain why their decisions were sound.
This separation supports a precise assessment: “Strong tradeoff reasoning demonstrated; claimed historical cost reduction remains unverified.”
Set disclosure boundaries before requesting material
Tell candidates that confidential work is not required. Offer the same alternatives to everyone, including those with public portfolios.
A usable invitation:
Please do not share protected employer or customer information. You may use material you are authorized to share, a clearly labeled fictional example, or our supplied exercise. We will assess the same job-relevant criteria whichever route you choose.
Removing names is not enough. Architecture, incident sequences, configurations, traffic patterns, and business rules can still reveal protected information. Sanitization is not a substitute for authorization.
Candidates should check applicable agreements and disclosure policies rather than ask interviewers to interpret them. If permission is unclear, use an independently constructed fictional example. Do not request private repository access, internal screenshots, live credentials, or customer data as proof.
Be ready to interrupt a disclosure: “We do not need those internal details. Let’s switch to a hypothetical version.”
Build a disclosure-safe evidence matrix
Link each candidate claim to permitted supporting material, its limits, and a follow-up. Here, authorized means the candidate has permission to share the material with this audience. A sanitized appearance is not proof of permission.
| Candidate claim | Supporting material | What it can establish | What remains unverified | Follow-up |
|---|---|---|---|---|
| “I designed a migration.” | An authorized decision record or a labeled fictional migration plan | Reasoning about dependencies, sequencing, and rollback | Occurrence and ownership unless the material supports them | Ask what would stop the rollout and how old and new versions would coexist |
| “I improved reliability.” | An authorized incident summary or a fictional incident exercise | Diagnosis, mitigation priorities, and learning from failure | Historical improvement and individual contribution unless separately supported | Ask which observations would distinguish competing failure hypotheses |
| “I reduced infrastructure cost.” | An authorized measurement summary or a synthetic cost model with explicit assumptions | Measurement design and understanding of cost drivers | Actual savings, attribution, and sustained results unless separately supported | Ask how workload changes could distort the comparison |
| “I led a difficult technical decision.” | An authorized decision record, a permitted non-sensitive account, or a supplied scenario | Reasoning about alternatives, stakeholder needs, and decision criteria | Actual influence, team agreement, and organizational outcome unless separately supported | Ask how a credible objection might change the decision |
| “I mentored other engineers.” | An authorized teaching artifact, an original explanation, or a fictional review exercise | Clarity, feedback quality, and ability to adapt an explanation | Historical mentee outcomes and their causes | Ask for a revised explanation for someone with less context |
Treat each row as a boundary on inference. Evidence of reasoning is not automatically proof of past delivery.
Public work needs the same scrutiny. A repository or published case study is inspectable, but publication alone does not establish authorship of every decision, production adoption, or business impact.
AI-generated illustration. In the fictional migration exercise, a write may reach only one store. Rolling back requires reconciling data, not merely switching the application.
Apply one rubric across evidence routes
Use the same dimensions for public artifacts, authorized private samples, and fictional exercises. Adjust the task to the role, not the scoring standard to the candidate’s access to publishable work.
This is a proposed rubric, not a validated predictor of job performance.
| Dimension | Limited demonstration | Sound demonstration | Strong demonstration |
|---|---|---|---|
| Problem framing | Jumps to a solution without defining the problem | States the goal, scope, and success conditions | Identifies ambiguity, tests assumptions, and distinguishes symptoms from the underlying need |
| Constraints | Leaves important constraints unexplored after prompting | Identifies relevant operational, security, resource, and time constraints | Explains which constraints dominate and how changing them changes the design |
| Tradeoffs | Names a preferred option without comparing alternatives | Compares credible alternatives and explains the choice | Identifies failure modes, reversibility, and conditions that would justify another option |
| Validation | Relies on “it should work” | Proposes relevant tests, measurements, and rollout checks | Defines disconfirming evidence, measurement limitations, and explicit stop or rollback conditions |
Add a separate status: Not observable.
Use it when the assessment did not offer a reasonable opportunity to demonstrate a dimension. It is not a low score and should not be averaged into one.
That differs from an observed limitation. If an exercise clearly asks for validation and the candidate cannot propose checks after clarification, record that limitation within the exercise.
For each rating, record a concrete behavior and any substantial interviewer assistance. “Identified retry-induced duplicate writes and proposed an idempotency test” is more useful than “seemed senior.”
If an essential dimension remains not observable, offer a targeted follow-up or state that the evidence is insufficient. The hiring bar stays intact. An evidence gap is not demonstrated weakness.
Worked example: assess a migration without the real system
Hypothetical candidate claim: “I led a migration that improved reliability, but the design documents and measurements are confidential.”
Do not negotiate for increasingly revealing details. Offer this fictional exercise instead:
Assume a service stores account preferences in Store A and must move to Store B. Requests may be retried. Both stores support reads and writes, but there is no transaction spanning them. Old and new application versions may run at the same time. The service must remain available during migration. Data-loss tolerance and acceptable read staleness have not yet been specified.
Sketch a migration plan. Identify the decisions you cannot make yet, compare alternatives, and explain validation and rollback.
If implementation is requested, use mock records in an isolated test environment. No employer schema or customer data is needed.
Ask structured follow-ups:
- What must be clarified before selecting a consistency strategy?
- What happens if one write succeeds and the other fails?
- How will backfilled records interact with concurrent updates?
- What evidence permits moving reads to Store B?
- If rollback occurs after new writes reach Store B, how does Store A become safe to use?
These questions probe decision-making under explicit constraints. A candidate can explain why rollback is a data-consistency problem, not merely a deployment switch, without revealing a former employer’s architecture.
An example assessment note:
Observed: Candidate identified missing consistency requirements, compared a write pause with a dual-write approach, and proposed reconciliation before switching reads.
Not yet observed: Implementation quality and operational execution.
Historical claim: Prior migration leadership and reliability improvement remain unverified.
The exercise does not establish performance during a real incident or the success of the previous migration. If implementation matters to the role, add a bounded test task rather than extending the inference.
Make the decision traceable
Before a hiring debrief, check the record:
- Which role requirement does each sample address?
- Is each artifact authorized for sharing, and is it historical or synthetic?
- What did the candidate demonstrate directly?
- Which historical claims remain unsupported?
- Which dimensions were not observable, and do they need follow-up?
- Would the same reasoning receive the same rating in a less polished format?
For candidates, the preparation task is to build an evidence map, not a disclosure-heavy portfolio. Choose a relevant claim, identify safe supporting material, and state what it cannot prove. Label fictional systems, data, and outcomes clearly.
Permission to publish should not stand in for engineering capability. A defensible assessment distinguishes what is demonstrated, what is supported, and what remains unknown.
As a next step, prepare your disclosure-safe evidence map before exploring Skills Tech Network.