The mis-shaped hire
A mid-market company’s first AI hire is almost always the wrong shape. The pattern goes like this:
The CTO writes a JD that lists “experience with LLMs, RAG pipelines, vector databases, fine-tuning.” The recruiter finds candidates whose resumes list those things. The interview panel asks about transformer architecture, attention mechanisms, and model evaluation. The strongest candidate by interview is offered the role. Six months later they’ve built two interesting prototypes and shipped none of them to production. The CTO is frustrated. The engineer is bored. Both feel misled.
The shape mismatch is this: the company needs an engineer who can ship reliable systems on top of frontier models. They hired a researcher who can talk about models. These are different jobs.
The three signals that predict production capability
We’ve run dozens of AI engineering hires for clients and built our own team around it. The three things that actually predict whether someone will ship production AI systems:
1. They have shipped a system that survived first contact with users
Not a demo. Not a Kaggle notebook. Not a “weekend project that almost worked.” A system that real users used and that the candidate had to operate when it broke. The system can be small. The signal is in the operational experience — they know what bad inputs look like, they know what observability they wished they’d had, they have opinions about kill switches.
Without this signal you cannot tell research aptitude from engineering aptitude. The fastest way to identify it: in the interview, ask “tell me about the worst production incident in something you built. What happened, how did you find out, how did you fix it.” If they have a real answer, that’s the signal. If they don’t, no amount of model knowledge compensates.
2. They prefer boring solutions to clever ones
Watch how the candidate talks about architecture. Do they reach for the simplest pattern that could work, or the most sophisticated? Real production AI engineers default to boring. They use APIs instead of self-hosting unless there’s a reason. They use JSON instead of inventing protocols. They use SQL instead of vector databases when SQL would do.
This is countercultural in AI hiring. The field’s incentive structure rewards talking about complex techniques because that’s how you signal that you’ve kept up with the literature. Production-shipping engineers signal differently — they talk about latency budgets, retry logic, what they cut from scope, and why.
The interview probe: present a real architectural problem and watch how they decompose it. If their first instinct is to reach for the newest tool, that’s a flag. If their first instinct is “what’s the simplest thing that could work, and what’s the failure mode,” that’s the signal.
3. They can communicate with non-engineers
AI engineering is unusual in how much it touches non-engineering parts of the business. Product, ops, compliance, customer success — they all have opinions, all need to be heard, all need to understand the trade-offs the engineer is making. An AI engineer who cannot translate a technical decision into “here’s what this means for the customer experience and here’s why we’re doing it” will produce systems the rest of the business resents.
This is testable. In the interview, ask them to explain something technical they built to a customer-facing colleague who’s in the room. Watch the room. The bad sign is dense jargon and impatience. The good sign is a clean two-sentence framing and a follow-up question.
What to ignore on the resume
A few resume signals that aren’t predictive of production capability:
Specific model names. “Experience with GPT-4, Claude, Gemini.” Anyone serious has used all of them. The line item is decoration.
Listing vector databases. Pinecone, Weaviate, Chroma, Qdrant. The differences matter less than the candidate’s intuition for when not to reach for a vector database in the first place.
Open-source contributions to ML repos. Sometimes a real signal. Often not. A two-line PR to a popular repo is worse than no signal because it’s signal-shaped without being real signal. Read the actual PRs before counting them.
Online course completions. Andrew Ng’s course, the deeplearning.ai sequence, Hugging Face courses. They demonstrate willingness to learn. They do not demonstrate shipping ability.
Years of “AI/ML experience.” Almost universally inflated. Two years of real LLM application experience is a lot in this market because the field is two years old in its current form.
What to weight heavily
In contrast, these are worth disproportionate weight:
A working portfolio of small systems. Three real things they’ve shipped, even if small, even if for themselves. The act of shipping is the filter.
A blog or write-up where they explain what they built and what went wrong. Bonus points if the write-up is honest about failures. This is rare and valuable.
Operational experience with anything in production. Backend engineering, devops, SRE backgrounds — these transfer beautifully to AI engineering because the production discipline is the same.
Experience with a real ambiguous business problem. A candidate who’s worked on a product where the user requirements were unclear and the right answer wasn’t obvious will be far better than one whose work was always crisp and bounded.
The interview shape we use
A four-step loop. Each step has a clear pass/fail.
Step 1: 45 minutes, “tell me about your production work.” Open-ended. They walk through 1–2 systems they shipped. We ask about architecture, what went wrong, what they’d do differently. Pass = real ownership of real production work. Fail = demos, prototypes, “we were going to ship it.”
Step 2: 60 minutes, paired architecture problem. Realistic scenario from our actual practice. Whiteboard architecture together. Pass = they ask the right clarifying questions, simplify aggressively, name trade-offs explicitly. Fail = they reach for sophistication, can’t name failure modes.
Step 3: 60 minutes, take-home with debrief. Small build (3–4 hours) using a frontier model. We don’t grade the code; we grade the debrief. Pass = they explain why they made the choices they made and what they’d do with another day. Fail = “I just used X because it’s what I know.”
Step 4: 30 minutes, non-engineering panel. A product or ops colleague joins. Same question as the interview probe above: explain something technical. Pass = the non-engineer leaves thinking they understood what they need to know. Fail = blank stare from the non-engineer.
A candidate passes all four or they don’t get an offer.
What to pay them
This will shift with the market, but a reasonable 2026 anchor:
- Junior production-capable AI engineer: $140–180k base + equity.
- Mid-level: $180–240k.
- Senior: $240–340k.
- Staff/principal: $340–500k+.
These ranges are aggressive for non-tech-hub mid-market companies and slightly under tech-hub frontier-lab comp. The right candidate at the right comp is the one who chose your problem over a frontier lab’s problem, and you should respect that choice with comp that makes the decision durable.
What this looks like as a team
The pattern that works for mid-market is small and senior. One staff-level engineer who owns the architecture, one or two mid-level engineers who build, plus product/data/ops capacity loaned in from the rest of the org. Four people, fully loaded, can deliver more production AI than ten people without this shape.
If you’re hiring your first AI engineer, hire the staff-level one. The mid-level engineers come second. Reversing this order leads to the failure pattern at the top of the post.
The take
The market is full of people who can talk about AI and short of people who can ship AI. The rubric that filters is biased toward shipping experience, operational maturity, and translation skill — three things that don’t usually appear on AI-specific JDs. Re-shape the JD and the interview around the rubric and the hires improve immediately.
Helping companies make their first AI hire is part of Fractional CAIO engagements. If you’re staring at a stack of resumes and want a second opinion, schedule a call.
