How to Prepare for an AI Engineer Interview: 5 Areas to Study

The title "AI Engineer" gives you almost nothing to plan around: the same job posting can hide a prompt-and-pipelines role, an ML engineering role, or something in between. But the interviews themselves have converged faster than the job descriptions. Across companies, the questions cluster into five areas, in a fairly stable priority order:

1. LLM fundamentals - how models generate text, and why they fail
2. RAG and retrieval - the default architecture question
3. Prompting and tool use - function calling, structured outputs, agents
4. Evaluation - how you know the system works
5. Production trade-offs - cost, latency, and the security that shipping brings with it

So the practical way to prepare for an AI Engineer interview is not to study "AI" broadly, but to work these five areas in order. This article covers what each one actually asks, with the questions it tends to produce, and closes with a 7-day plan. But before you read any of it, find out where you actually stand.

Start here: a 6-question self-check

Six questions from squizzu's question bank, the same ones you'd meet in the app. They span model mechanics, retrieval, tool use, evaluation, how agents handle real codebases, and the security review that gates a model into production. No sign-up: pick an answer and you get the explanation plus an in-depth breakdown. The point is not the score; it's that each miss tells you which of the sections below to read carefully instead of skimming.

The one that catches the most experienced engineers is the pass@k question. A high pass@10 sounds like reliability, but it measures best-of-ten capability, and even a genuine 95% per-run success rate completes ten runs cleanly only about 60% of the time. If temperature tripped you up, start with area 1 below; the search question maps to area 2; JSON Schema and the coding-agent question map to area 3; pass@k belongs to area 4; and the checkpoint review belongs to area 5. Whatever you missed, that's where your preparation time should go first.

What an AI Engineer interview is - and is not

An AI Engineer interview is not an ML research interview. Nobody will ask you to derive backpropagation or implement attention from scratch. The role builds reliable products on top of models. The questions therefore test whether you understand model behavior well enough to design around it: why outputs vary between runs, why the model confidently invents an API that doesn't exist, why your RAG system answers correctly on Monday and fails on Tuesday.

A typical loop looks like this:

• Recruiter screen - your projects, which usually seeds later questions, so only mention systems you can defend in depth.
• Technical conversation - concepts from the five areas above, with follow-ups until you reach mechanism.
• System design - "design a chatbot over our internal documentation" or "add AI summaries to our product," end to end.
• Coding - practical Python: call a model, parse structured output, handle a malformed response. Rarely algorithm puzzles.

Some companies - especially ones with a data-science heritage - add a classical ML round. If yours does, the fundamentals are a separate preparation track; 20 Google AI/ML interview questions covers exactly that set.

Area 1: LLM fundamentals

You need working knowledge, not research depth: what a token is, and why character-level questions like "how many r's in strawberry" are awkward for a model that never sees individual letters. Newer models mostly work around that by reasoning step by step, which patches the symptom rather than the representation. Then: what the context window limits in practice; what temperature and top-p change about sampling; what embeddings represent and why cosine similarity between them means anything; the difference between a base model and an instruction-tuned model.

The single most common question in this area:

Question: Why do LLMs hallucinate?

Short answer: A language model is trained to produce plausible continuations of text, not to verify facts. When the training data doesn't contain the answer - or contains it rarely - the most statistically plausible continuation can still be fluent, confident, and wrong. The model has no internal mechanism that distinguishes "I know this" from "this pattern looks right."

What the interviewer is testing: Whether you can name the mechanism instead of the symptom. Candidates who say "the model makes mistakes sometimes" fail the follow-up. Candidates who tie hallucination back to the training objective, plausible continuation and never verified fact, can then reason about which mitigations attack the cause and which only hide it.

Common follow-up: How would you reduce hallucinations in a production system? (Expected directions: grounding with RAG, asking for citations and verifying them, constraining output to retrieved context, temperature 0 for factual tasks - and knowing none of these eliminate the problem.)

Area 2: RAG and retrieval

RAG lets the model retrieve external information before generating an answer, which makes it the default architecture whenever the data is private or changes often - which is to say, in most real products. Expect at least one RAG question in nearly every AI Engineer loop.

Know the pipeline concretely: documents are split into chunks, each chunk is embedded into a vector, vectors go into an index, a user's question is embedded the same way, the nearest chunks are retrieved and placed into the prompt. Then know where it breaks, because that's where the questions live:

• Chunking too coarse or too fine. A 4,000-token chunk buries the relevant sentence in noise; a 50-token chunk loses the context that made it meaningful. Splitting along document structure (sections, headings) usually beats fixed-size windows.
• Retrieval misses. The answer exists in the corpus but isn't in the top-k results. Vocabulary mismatch between question and document is the classic cause, which is why hybrid search (vectors plus keyword/BM25) and reranking exist.
• The model ignores the context. Retrieval worked, but the model answers from its own training data anyway, or stitches together an answer from a half-relevant chunk.

Question: What is the difference between RAG and fine-tuning?

Short answer: RAG changes the information available to the model at query time; fine-tuning changes the model itself. RAG is the right tool for knowledge that changes or must be cited; fine-tuning is the right tool for consistent behavior - style, format, domain-specific patterns.

What the interviewer is testing: Whether you reach for the right tool for the right problem. The red flag they're screening for is "we'll fine-tune the model on our docs" as an answer to a freshness problem - retraining on every document change is slow, expensive, and still can't cite sources.

Common follow-up: When would you choose fine-tuning over RAG? (Good answers: enforcing an output format across thousands of calls, matching a brand voice, teaching a niche domain's conventions - cases about behavior, not facts. The strongest answer notes they combine: a fine-tuned model inside a RAG pipeline.)

Area 3: Prompting, structured outputs, and tool use

This area sounds soft and is not. The questions test whether you've built something real:

• Function calling - be precise about the mechanics, because interviewers use this to separate people who've built from people who've read: the model doesn't execute anything. It returns a structured request - a function name and arguments matching the JSON Schema you declared for the tool - and your application executes it and passes the result back. The model is a planner; your code is the runtime.
• Structured outputs - how you get reliable JSON out of a model: schema-constrained generation where the API supports it, validation plus a retry path where it doesn't. Then the real question, which is what you do when parsing fails in production.
• Agents - an agent is a model in a loop with tools: model decides an action, your code executes it, the result goes back into the context, repeat until done. The senior-signal answer includes when not to use one: each iteration compounds error and adds latency, so a fixed pipeline beats an agent whenever the steps are known in advance.

If you can sketch the agent loop on a whiteboard and name one failure mode (looping without progress, a wrong tool call cascading), you're ahead of most candidates in this area.

Question: When would you choose a fixed pipeline over an agent?

Short answer: Whenever the steps are known in advance. An agent buys you the ability to decide the next action at runtime, and you pay for it in latency, cost, and compounding error - every extra iteration is another chance to go wrong. If the sequence is "extract these fields, validate them, write the row," that is a function, not an agent.

What the interviewer is testing: Whether you reach for agents by default. Most production LLM features are pipelines with one or two model calls, and a candidate who proposes an autonomous agent for a deterministic task signals that they have built demos rather than systems.

Common follow-up: How would you keep a long agent run from drifting? (Good directions: cap the iterations, validate after every tool call rather than at the end, make actions idempotent so a retry is safe, and put a human checkpoint in front of anything irreversible. The strongest answers connect this to the arithmetic in area 4 - per-step reliability compounds, so a 95%-per-step agent finishes a 20-step task about a third of the time.)

Area 4: Evaluation

This is the area that decides senior-vs-junior impressions, and the one candidates prepare least. Every system-design answer eventually gets the same follow-up: "How do you know it works?" - and "we looked at the outputs and they seemed good" ends the conversation.

What a strong answer contains:

• A golden set. A fixed collection of real inputs with expected outputs or grading criteria, run against every prompt change and model upgrade. It is regression testing for behavior, because a prompt tweak that fixes one case silently breaks three others.
• Stage-separated RAG evaluation. Measure retrieval on its own: for questions with known source documents, do those documents appear in the top-k (recall@k)? Then measure generation on its own: is the answer faithful to the retrieved context, and does it address the question? Without the separation you can't tell whether to fix the retriever or the prompt.
• LLM-as-a-judge, with its caveats. Using a strong model to score another model's outputs scales far beyond human review. But judges have position bias, favor longer answers, and drift across model versions, so you calibrate them against a human-labeled sample rather than trusting the scores blindly.
• The right metric for the operating mode. pass@k asks whether at least one of k attempts succeeded, so it measures capability and rises with k. An automation that must work every time needs the opposite: the per-run success rate, and pass^k for all k runs succeeding, which falls with k. Quoting pass@k as a reliability number is the category error the self-check at the top of this article tests for.

That last point is worth seeing rather than reading, because the arithmetic is the part people refuse to believe:

The gap between the curves is the whole argument for chasing per-run reliability. Going from 95% to 99% per run barely registers as a number, but it turns a ten-run sequence from a coin flip into something you can put in front of customers. This is also why the answer to "our agent works great in the demo" is a question about how many steps the task takes.

Question: How would you evaluate a RAG system?

Short answer: Separately per stage. Retrieval: recall@k and precision@k against questions with known relevant documents. Generation: faithfulness to the retrieved context and relevance to the question, scored by human review on a sample or a calibrated LLM judge. Track both over time so you can attribute regressions.

What the interviewer is testing: Whether you treat an LLM system as an engineering artifact with measurable behavior, or as something you tweak until the demo looks good.

Common follow-up: Your retrieval recall is 95% but users say answers are wrong - where do you look? (The generation stage: faithfulness failures, context getting truncated, or the model preferring its parametric knowledge over the retrieved text.)

Area 5: Production trade-offs

The closing questions in most loops are about running these systems for real money and real users:

• Cost and latency. Know the levers: caching identical and near-identical requests, routing easy queries to a smaller model and hard ones to a larger one, and streaming tokens so perceived latency drops even when total latency doesn't. Trimming prompts belongs on the same list. Cost scales with tokens, so the verbose system prompt someone pasted in eight months ago is a real line item.
• Prompt injection. The interview version: user-supplied or retrieved content can contain instructions ("ignore your previous instructions and…") that the model may follow. Mitigations - separating instructions from data, restricting tool permissions, output filtering - reduce but don't eliminate it, and saying "this is unsolved, so we limit blast radius" is a better answer than claiming a fix.
• Supply chain. Third-party weights and datasets are untrusted input, not inert files. Pickle-based checkpoints (.pt, .bin) are archives of Python pickles, and the pickle format can import and call code as a normal part of loading. "Loading a model" can therefore execute whatever the publisher put there. That is why the ecosystem moved to safetensors, which has no code path at all, and why PyTorch 2.6 flipped torch.load to weights_only=True by default. The container is only half of it: the weights themselves can carry a backdoor that fires on a trigger phrase. Pin the revision, verify the publisher's hash, prefer a non-executable format, and scan before the first load. OWASP files this under LLM03: Supply Chain.
• Graceful failure. Model APIs time out, return malformed output, and change behavior between versions. Retries with backoff, output validation, fallback models, and a version-pinning strategy are the difference between a demo and a product.

Question: How would you defend an LLM feature against prompt injection?

Short answer: By limiting what a successful injection can reach, not by trying to make the model immune. The model cannot reliably distinguish instructions you wrote from instructions that arrived inside a document it retrieved. That makes the defense architectural: give the model the narrowest set of tools and permissions the feature needs, treat every tool call as untrusted input to your own code, and require confirmation before anything irreversible.

What the interviewer is testing: Whether you know this is unsolved. A candidate who claims a filter or a system-prompt instruction fixes prompt injection has not read seriously about it; a candidate who talks about blast radius has.

Common follow-up: Your RAG assistant can read internal documents and send email - what could go wrong? (The classic exfiltration chain: a poisoned document tells the model to email its context somewhere. It is the standard argument for splitting read and write capabilities across separate, differently-permissioned calls rather than handing one agent both.)

A 7-day preparation plan

This assumes you already write software and have called an LLM API at least once. With less runway, compress the reading rather than dropping days: day 3 and day 6 are the ones to protect, because the build is what gives you concrete examples and the mock is what turns knowing into saying. The other days can be skimmed to their headline points.

• Day 1 - Map the role, then LLM fundamentals. Reread the job posting and the company's product. Every ambiguous "AI Engineer" title resolves into a concrete system they are building, and your system-design round will be about it. Then cover tokenization, context windows, sampling parameters, embeddings, and the hallucination mechanism.
• Day 2 - RAG in depth. The pipeline, the three failure modes above, hybrid search, reranking. Practice saying the RAG vs. fine-tuning answer out loud until it's 30 seconds.
• Day 3 - Build a small RAG system. A few hours, not a project: take twenty markdown files, chunk them, embed them, retrieve, and ask ten questions. At least two answers will be wrong - diagnosing why gives you the concrete example every interviewer asks for.
• Day 4 - Evaluation. Golden sets, recall@k, faithfulness, LLM-as-a-judge and its biases. Then evaluate yesterday's build: write ten test questions and score it. Now you have a second story.
• Day 5 - Tool use and production. The function-calling mechanics, the agent loop and when to avoid it, cost and latency levers, prompt injection, and why a downloaded checkpoint gets a security review.
• Day 6 - Mock the interview. Answer out loud: definition, mechanism, trade-off, example - knowing something and producing it under pressure in two minutes are different skills, and this day converts one into the other.
• Day 7 - Close the gaps. Revisit whatever the opening self-check and day 6 exposed, and prepare your own questions about their stack - asking "how do you evaluate your models today?" signals more than most answers do.

How ready are you?

Reading a study plan and being able to answer under pressure are different things - the self-check at the top told you that in six questions, spread across all five areas.

Test yourself on the full AI Engineer question set on squizzu. Same format as the embedded quiz, with an explanation and an in-depth breakdown on every question, so each miss turns into a topic you can close before the interview.

If the role includes a classical ML round, work through 20 Google AI/ML interview questions next. And if you're still landing the interview, how to write an AI/ML resume in 2026 covers what screening models actually match on.