Post an "AI Engineer" search and get a hundred profiles with "LLM," "RAG," and "agents" in the title, and you still haven't solved anything. Those three words show up on almost every CV on the market today. The problem isn't finding candidates — it's knowing, within the hour an interview lasts, who actually built something real and who is repeating what they read in a blog post.

This guide brings together four things a technical hiring process for AI roles needs to cover: the questions that truly separate signal from noise, the keywords a senior candidate has to handle fluently, the landscape of models that exist today, and whether the candidate knows when a self-hosted model makes more sense than paying for an API call every time.

The questions that reveal whether someone built something real

Technical trivia questions ("what is RAG?") can be answered by googling. The ones that work are the ones that ask the candidate to explain a design decision along with its consequences:

• "What chunking strategy would you use for technical documentation, and what breaks if you get it wrong?" A senior candidate talks about semantic boundaries, not just token counts.
• "When does hybrid search (BM25 + vector) make more sense than embeddings alone?" This reveals whether they understand that keyword matching and semantic similarity are different problems.
• "Tell me about a time a RAG system or an agent failed in production. What happened, and what did you change?" This is the question that filters the most: a candidate who only studied the theory doesn't have a real story to tell.
• "When do you fine-tune instead of using RAG?" The correct answer distinguishes style and reasoning patterns (fine-tuning) from up-to-date knowledge (RAG) — confusing the two is a red flag.
• "How would you detect hallucinations in a production system, at scale?" Look for something more developed than "I use an LLM as a judge": citation validation, periodic checks, faithfulness metrics.

The most reliable signal isn't a memorized correct answer — it's whether the candidate can describe an explicit trade-off ("this is slower but works better for complex documents") instead of defending a single option as if it were the only one.

The keywords that have to show up, no exceptions

If none of these terms come up on the candidate's own initiative during a one-hour technical interview (not because you asked about them directly), that's a red flag:

Architecture and training: attention, KV cache, fine-tuning vs. RAG, LoRA, quantization (4-bit / 8-bit).

Agents and orchestration: tool calling, ReAct, function calling, MCP (Model Context Protocol) — the standard that's solving how agents connect to external tools without custom integrations for each one.

Production and cost: semantic caching, context pruning, model routing — the real levers for cutting token spend, not just "use a cheaper model."

A candidate who only handles the interface-level terms (prompt, chatbot, assistant) but not the architecture underneath probably built on top of pre-made tools without understanding what's happening inside. That doesn't disqualify them for every role, but if the position is an AI architecture role, it's a critical difference.

The model landscape a senior candidate needs to know

They don't need to master the fifteen models that come out every month, but they do need to understand the general map:

Proprietary models (GPT, Claude, Gemini, Grok): the frontier in reasoning and multimodality, but with per-token cost that scales with volume and dependency on an external provider. (Copilot doesn't belong on this list because it isn't a model itself, but a product built on top of these models — though Microsoft has been pushing its own in-house family, MAI, to depend less on OpenAI.)

Open-weight models (Llama, Qwen, DeepSeek, Mistral, Gemma, GLM, among others): can be self-hosted, fine-tuned, and run without sending data to a third party. Each family has a different focus — some prioritize reasoning and math, others code, others deployment on resource-constrained devices.

Asking a candidate "when would you choose an open model over a proprietary one?" reveals more about their real judgment than asking them to name the latest model release. A useful answer talks about data privacy, cost at scale, and the need for fine-tuning — not about which one "is better" in the abstract.

Can they set up a local model to avoid depending on a paid API?

This is the question that separates seniority levels fastest. Running your own model (self-hosted) isn't free — but past a certain volume of use, it comes out cheaper than paying per token to an external provider, and it keeps sensitive data from leaving your infrastructure.

A candidate who understands this terrain can talk about:

• Deployment tools: Ollama for fast prototyping, vLLM for real production workloads, llama.cpp for resource-constrained devices.
• Quantization: running a model in a 4-bit or 8-bit version drastically cuts memory requirements with a small quality loss — the difference between needing a high-end GPU or being able to run it on far more modest hardware.
• The economic break-even point: below a certain daily usage volume, paying per API call is still cheaper than maintaining your own infrastructure. A senior candidate knows how to calculate that crossover point, not assume that "self-hosting always pays off."

If the role you're filling handles a high volume of requests or sensitive data, this is one of the questions with the biggest direct impact on budget and project risk.

The problem isn't the question list — it's sustaining it

Having these questions isn't enough if you don't know how to recognize a good answer when you hear one, and that takes time most teams don't have available for every search.

At iBender, this is exactly the work we do before we ever show you a profile: every professional goes through a human validation process across three layers — logic, English, and AI architecture — before they reach you. It doesn't replace your own judgment in the final interview, but it saves you the most expensive part: filtering out who actually knows the subject from who just knows how to talk about it.

Browsing already-validated profiles is free. You only pay when you decide to hire.

Create your free account →