A pattern across builds
After enough production agent deployments, a repeating timeline becomes visible:
- Demo week. The agent looks great. Stakeholders are happy.
- Week 1 of production. Works well enough. Some edge cases surface, the team patches them quickly.
- Week 2 of production. Quality drops. Output gets weirder. Users start complaining.
- Week 3+. Either the team intervenes substantially or the agent gets quietly turned off.
This pattern is not random. It happens because production agent behavior is determined by inputs that aren’t visible during demo or week 1, and those inputs shift over week 2.
The fix is mostly boring. It’s the operational discipline that turns a working demo into a working system. This post is what that discipline looks like.
Why week two, specifically
Three things change in week two that didn’t change in week one:
1. User behavior generalizes
In week 1, users are typically beta users or early adopters. They know how the agent is supposed to work. They use it the way the demo suggested.
In week 2, the user base widens. New users don’t know the implicit rules. They prompt the agent with inputs the demo never anticipated. The agent encounters its first real out-of-distribution traffic.
2. Edge cases accumulate
Week 1 produces a few edge cases the team patches. Week 2’s edge cases were waiting in the long tail — slightly different from the week 1 ones, common enough to matter, infrequent enough that the eval suite didn’t cover them. They show up cumulatively.
3. The model provider updates something
This one is the hidden killer. Frontier providers update their models continuously, often without explicit version bumps for the consumer API. A model that produced X output last week starts producing X’ this week. Small differences amplify across the agent’s reasoning chains.
The week-two diagnostic
When an agent starts to fail in week two, the diagnostic question is: which of the three is causing it?
If new users’ inputs are different shape from demo inputs: the problem is generalization. The eval suite didn’t cover the real distribution. Fix: expand the eval suite with real production inputs.
If specific edge cases are causing failures: the problem is the long tail. Fix: add the edge cases to the eval suite and patch the prompt or workflow.
If outputs have shifted across the board without input changes: the problem is the model. Fix: pin the model version if possible, or adjust prompts to compensate.
The first two are addressed by operational discipline. The third requires architectural readiness.
The operational discipline: the boring fix
Here is what week-two-proofing looks like in practice. It’s mostly unglamorous. It also is what separates the agents that survive from the ones that don’t.
Build the eval suite from real inputs, not synthetic ones
Most teams build their eval suite from inputs the team imagined. This is wrong. The eval suite should be built from a representative sample of real user inputs, ideally from a beta phase.
A 50-example eval suite of real inputs catches most week-two regressions. A 100-example eval suite of synthetic inputs catches almost none.
The cost of building the real-input eval suite is a few days of work. The benefit is enormous.
Run the eval suite continuously, not just on deploy
Run the eval suite hourly or daily, not just when you ship a change. This catches model-shift regressions that happen even when your code didn’t change.
When the eval score drops, you know within hours, not weeks. The remediation work starts before users complain.
Sample outputs for quality, every day
Pick 10 random production outputs each day. Have a human read them. Score them against quality criteria. Track the daily quality score over time.
This catches the slow drift that the eval suite might miss — the subtle ways outputs are getting worse that don’t show up as clear regressions on benchmarks.
Daily quality review takes 15 minutes once you have the tooling. The signal it provides is irreplaceable.
Pin the model version where possible
If your provider supports pinned model versions, use them. The auto-upgrade pattern is convenient for development and dangerous for production. You want to control when behavior shifts.
If your provider doesn’t support pinning, your eval suite is your only safeguard. Run it more often.
Have a rollback path
The agent should be re-deployable to a previous version within an hour. This sounds obvious. Most agents we audit can’t do it in less than a day.
The cost of building rollback is small. The cost of needing it and not having it is the entire second week.
The cultural piece
The boring fix only works if the team is set up to do it. Most teams aren’t, by default. The structural changes:
Treat “we shipped, it works” as a starting hypothesis, not a conclusion. Week 2 is when the hypothesis gets tested. The team has to be expecting and watching for this.
Allocate operational capacity for the first month. Plan for 30–50% of the build team’s time to go to operations for the first 4 weeks. Don’t immediately move them to the next build.
Decouple “shipping a feature” from “the feature is operationalized.” The latter takes weeks longer than the former and requires explicit work.
What goes wrong when teams skip this
Without the operational discipline, the typical second-week failure mode is:
- Users complain.
- The team scrambles to diagnose.
- The diagnosis is slow because nothing is instrumented to make it fast.
- The fix is uncertain because there’s no eval suite to validate it against.
- Each attempted fix breaks something else.
- Stakeholder confidence collapses.
- The agent gets pulled.
This timeline plays out over 2–3 weeks. The cost is enormous: the engineering time, the stakeholder trust, the customer impact, the morale damage. All of it preventable with a week of upfront operational work.
A week-two readiness checklist
If you have an agent shipping in the next month, run through this:
- Eval suite of 50+ real or realistic inputs, with expected outputs.
- Eval suite runs automatically on every deploy.
- Eval suite runs continuously (hourly or daily) on production.
- Daily sampling of 10 production outputs for human quality review.
- Model version pinned where the provider allows.
- Rollback path tested. Can revert to the previous version in <1 hour.
- On-call rotation includes someone familiar with the agent.
- Operational capacity allocated for the first 4 weeks post-launch.
- User feedback channel exists (thumbs up/down, support tickets categorized).
That’s the checklist. None of it is sophisticated. All of it is the difference between week-two-proof and week-two-vulnerable.
The take
Most AI agents fail in week two because user behavior generalizes, edge cases accumulate, and models shift. The fix is operational discipline — real-input eval suites, continuous eval, daily sampling, model pinning, rollback paths. None of it is glamorous. All of it is what separates the agents that scale from the ones that get quietly disabled. Build the operations before you launch the feature. Week one will be fine. Week two will be too.
Production readiness is part of every Custom AI Build. If you have an agent ready to launch and want a week-two readiness review, schedule a call.
