ai-era interviews should test evidence, not leetcode muscle memory

Software engineering interviews have always had a strange theater problem.

We ask people to reverse a linked list, then hire them to spend six months arguing with YAML, logs, flaky tests, half-migrated services, and the undocumented billing job everyone fears.

Very normal industry. No notes.

Now AI is making the old ritual look even weirder.

Recent coverage from The Guardian, Business Insider, and ITPro all circles the same point: software engineering is not disappearing, but the visible shape of the job is changing. More code can be generated. More candidates will show up with AI in their workflow. Some companies already encourage it in interviews. At the same time, people are worried about juniors skipping fundamentals because the assistant gives them a plausible answer before they understand the question.

That does not mean interviews should become “prompt this chatbot while we watch.”

It also does not mean we should retreat into LeetCode nostalgia and pretend the last decade did not happen.

The better interview should test evidence.

Not vibes. Not memorized puzzle moves.

Evidence.

code generation made the old signal weaker

The classic algorithm interview was never perfect, but it had a theory behind it. If someone could solve a constrained problem under pressure, maybe they had enough programming fluency, decomposition, and correctness thinking to do real work.

Sometimes that signal was useful. Sometimes it filtered for people who had time to grind the format. Sometimes it selected for confidence more than judgment. We all know the game.

AI makes the signal weaker because the easiest parts to test are now the easiest parts to outsource.

Write a function from a clean prompt? Generate boilerplate? Remember syntax? Produce a first-pass implementation? Those are no longer rare skills. They are table stakes plus tool usage.

That does not make programming knowledge obsolete. The opposite, actually.

When the assistant produces code, the engineer still has to know whether the code is correct, maintainable, secure, idiomatic for that codebase, and compatible with the mess already in production. The assistant can write a migration. It cannot feel the cold sweat of being on call when that migration locks the wrong table.

So an interview that only tests raw code production is increasingly testing the part of the workflow that is getting commoditized.

The more valuable signal is what happens after the code appears.

give candidates a messy diff

One good AI-era interview is simple: give the candidate a generated pull request.

Not a perfect one. A realistic one.

The diff should include a small feature or bug fix with a few problems: maybe an edge case, a weak test, a security issue, a hidden coupling, or a migration with bad production behavior.

Then ask the candidate to review it.

What worries you?

Which tests would you add?

What would you ask the author?

Which part can ship today and which part needs another pass?

What evidence would make you comfortable approving this?

This tests something closer to the work.

A good engineer reads code skeptically without becoming theatrical. They separate style nitpicks from correctness issues. They notice when a change violates a boundary. They know when a test is useful and when it is just a decorative green checkbox.

That is much harder to fake with a memorized problem set.

It is also exactly where AI changes the job. If teams use coding agents, review load goes up unless the agent output is boringly good. The candidate who can inspect generated work, ask for evidence, and improve the change without turning every review into a personality contest is valuable.

That person saves production.

Or at least saves the team from another “tiny cleanup PR” that somehow rewrites authentication.

test debugging, not just building

Another better interview: give the candidate a failing test and an incident-shaped timeline.

The service started returning 500s after a dependency update. The logs show a weird serialization error. A teammate already asked an AI assistant and got a confident but probably wrong explanation. The candidate gets the code, the logs, and the failing test.

Ask them to debug out loud.

Not perform confidence. Debug.

What do they inspect first? Do they form hypotheses? Do they know how to reduce the problem? Do they distrust the first plausible answer? Do they know the difference between fixing the symptom and fixing the cause?

This is where fundamentals still matter.

Git matters because you need to understand what changed.

Tests matter because you need a reproducible signal.

Logs matter because production does not care that your local run was beautiful.

Architecture matters because a local bug often has system-level consequences.

AI can help with each step, but it cannot replace the engineer’s responsibility to verify. If anything, it makes verification more important because the assistant can produce a very clean wrong answer.

prompt fluency is useful, but judgment is the product

Yes, candidates should probably know how to use AI tools.

Pretending otherwise is silly. It is like interviewing a backend engineer in 2026 and acting surprised that they use an IDE, docs, and a terminal. Tool fluency is part of the job.

But “can prompt well” is not enough.

The useful question is whether the candidate can turn AI output into accountable engineering work.

Can they write a prompt that includes constraints from the existing system? Can they ask the model for alternatives instead of accepting the first answer? Can they force the assistant to produce tests, assumptions, and failure modes? Can they spot hallucinated APIs?

That last one matters more than people admit.

Sometimes the fastest AI workflow is closing the chat and opening the file.

This is also why companies should stop treating AI-assisted interviews as cheating by default. If the real job allows AI tools, the interview should observe how candidates use them. The interesting signal is whether the tool makes their work more inspectable or just more confident.

juniors still need the boring foundations

There is a real concern in the ITPro/O’Reilly-style discussion about fundamentals. If new developers skip Git, testing, debugging, code review, and architecture because AI gives them runnable-looking answers, they become productive enough to create change, not grounded enough to own it.

That is not a reason to ban AI from learning. It is a reason to design interviews and apprenticeships around ownership.

Ask junior candidates to explain a diff, a failing test, and a safe revert. Let them use AI, but require them to validate every claim against the codebase.

hiring should reward the work we actually need

The industry says it wants engineers with judgment, ownership, debugging skill, systems thinking, security awareness, and the ability to work with AI.

Then many interviews still reward speed-running toy problems.

This mismatch was annoying before.

Now it is expensive.

If the job is becoming more about reviewing generated work and deciding what evidence is enough, then the interview should look like that. Give candidates a generated patch, an ambiguous ticket, a flaky test, or an AI-produced explanation and ask them to verify it.

A strong candidate should be able to say what is correct, what is risky, which test is weak, which requirement is ambiguous, and where the AI answer sounds plausible but the code says something else.

That is not less technical than a puzzle. It is more technical because it connects code to consequences.

the punchline

AI did not make software engineering interviews easy to fix.

It made the old laziness harder to defend.

The question is no longer “can this person produce code in isolation while someone watches awkwardly?”

The question is “can this person turn code into reliable engineering evidence?”

That means review. Debugging. Tests. Logs. Git history. Architecture boundaries. Security thinking. Rollback plans. Clear explanations. The unglamorous stuff that keeps systems alive.

LeetCode muscle memory is not useless. It can still test some fluency.

But if it remains the center of the interview while the actual job moves toward AI-assisted delivery, companies will hire for the wrong reflexes.

The best interview now is not the one that proves a candidate can write code without help.

It is the one that proves they can be trusted when help produces code too quickly.

references

• The Guardian: how software engineers are adapting to AI
• Business Insider: software engineering jobs need more than coding skills in the AI era
• ITPro: developers, AI, Git, Agile, and fundamentals

To test my projects, I use Railway. If you want $20 USD to get started, use this link.