The pilot worked. Someone ran the cell, the model returned a clean answer, the program office clapped, and the slide deck wrote itself. Six months later the same capability is still a notebook on one analyst's laptop, and nobody can explain why it never shipped. We have walked into this room more times than we can count.
The demo is the easy 20 percent
A notebook that produces a good answer once proves the model can do the task on a good day, with curated inputs, under a human who knows what "right" looks like. That is genuinely useful information. It is also roughly a fifth of the work. The other four-fifths is the part nobody demos because it is not photogenic: the harness that catches the bad day, the controls that let an accreditor sign off, the plumbing that lets a second person run the same thing and get the same result.
Federal programs stall at exactly this seam. The pilot answers "can the model do it." Production answers "can we let it run unattended, prove what it did, and turn it off when it misbehaves." Those are different questions with different owners, and the second set never appears in a notebook.
What production actually requires
Here is the short list of what has to exist before a federal AI capability is allowed near a real mission. None of it is exotic. All of it is skipped in the pilot.
An eval suite, not a vibe check
A pilot is judged by a human reading outputs and nodding. That does not scale and it does not survive a model swap. Before a system runs unattended you need a held-out set of cases with known-good answers and a scoring function that runs on every change. When the underlying model is upgraded — and it will be, on the vendor's schedule, not yours — the eval suite is the only thing that tells you whether the upgrade quietly broke a task that used to work. We have seen a "minor" model version bump drop extraction accuracy on a specific document type by a third, with no other signal that anything had changed.
Version control over prompts and policy, not just code
Teams version their Python and treat the prompt as a string they paste in. The prompt, the system instructions, the tool definitions, and the policy that governs what the agent may do are the behavior. They belong in the repository, tagged, diffable, and tied to a release. When someone asks six months from now why a decision came out the way it did, the answer is "version 1.4.2 of the policy, running model X" — and you can check out that exact state and reproduce it.
Observability that survives the demo
The notebook shows you one run. Production runs thousands, most of which you never watch. You need traces of every step an agent took — which tool it called, what it got back, what it decided — plus token cost, latency, and error rates as first-class telemetry. Without this, the first production incident is unsolvable: you have an unhappy user, no record of what the system actually did, and no way to reproduce it.
Rollback that takes seconds
If the only way to undo a bad change is to redeploy and pray, you do not have rollback, you have hope. A production system can be reverted to the last known-good policy and model in the time it takes to flip a config — and the eval suite tells you the revert is actually safe. This is mundane release engineering. It is also the difference between a bad afternoon and a suspended program.
Human-in-the-loop where it belongs
Autonomy is not binary. The useful question is which actions an agent may take on its own and which require a human to approve before they commit. A read-only summary can run unattended. An action that writes to a system of record, sends something external, or changes an entitlement should pause for a person. Designing those checkpoints up front — instead of bolting on an approval queue after the first scare — is what lets you expand autonomy later with evidence rather than nerve.
An audit trail an accreditor will accept
For federal work this is not optional and it is not last. Every consequential decision needs a record of which version of which policy made it, on what inputs, using which model, and what a human did or did not approve. Build this in from the first commit. Reconstructing an audit trail after the fact is, in our experience, harder than building the original system.
Why this keeps happening
The incentives point the wrong way. Pilots are funded to prove feasibility, and a notebook proves feasibility. The work that turns feasibility into a running system is invisible on a demo day, expensive in engineer-hours, and easy to defer to "phase two" — which is frequently never funded because phase one already "worked."
The fix is to stop treating production engineering as a follow-on and start treating it as the definition of done. A capability is not done when it answers correctly once. It is done when a second person can run it, the eval suite is green, the traces are flowing, rollback is one command, and the audit trail would satisfy the accreditor. That bar is higher, and it is the only bar that produces something a mission can depend on.
What this looks like for a real team
We work with national laboratories and federal civilian agencies whose first instinct, reasonably, is to start in a notebook. The change we push is not "stop prototyping." Prototype freely. The change is to draw the production line early — write three eval cases before you write the prompt, put the prompt in the repo on day one, wire a trace logger before the second demo — so the prototype grows into a system instead of being thrown away and rebuilt.
Between the founders, we have put more than 150 agents into production across federal, SLED, and commercial environments. The pattern is consistent: the teams that ship are not the ones with the best model. They are the ones who treated evals, versioning, observability, rollback, and audit as the work — not the cleanup.
If your AI capability is stuck in a notebook and you need it running where the mission can depend on it, that is the work we do. Start a conversation, or look at how we train your team to run it after we leave.