The Day My AI Team Made Me Bring the Keyboard

The promise of multi-agent AI sounds clean on a whiteboard: give different models different jobs, connect them through shared context, establish authority boundaries, and let them work. The reality is considerably messier — especially once the system is touching real data, real infrastructure, and something with a meter running.

That is what happened today with Sports Intelligence, one of the experiments running inside Benny’s Lab. I was supposed to be away from the machinery. Instead, I had an iPad keyboard out at a bar because my little AI operations team had managed to become both genuinely useful and deeply annoying at the same time.

The goal was to build an AI team so I could sit at a bar with an iPad. The real milestone will be when I can sit at the bar without the keyboard.

The team

The workflow has evolved beyond one chatbot doing everything. ChatGPT acts as the primary orchestrator and engineering layer. Grok and Grok Bot provide an independent lane for research, monitoring, and adversarial checks. Woutson handles another execution and recovery path. A relay I built gives the agents a shared place to exchange project messages. Gmail is currently the ugly-but-reliable fallback bus when the relay cannot wake an agent the way I want.

There are rules around this. Observers should not silently become operators. Expensive model calls need a reason. Conversations have round caps. Work should continue without asking me to approve every harmless step, but production-impacting actions still need defined authority. The entire point is to keep the human at the level of goals and exceptions rather than turning him into a router.

Then the credits started disappearing

The Odds API quota was burning at roughly 500 credits an hour. That made no product sense. Sports Intelligence does not have enough traffic to justify constantly refreshing props, markets, and enrichment data, and most of that information simply does not change fast enough to warrant it.

The intended policy was straightforward: refresh the expensive data once around noon Eastern, refresh again as a game-day slate approaches, and freeze the slate after kickoff. Live scores can remain responsive because they are a different problem. There is no reason to spend thousands of credits discovering a tiny price movement minutes earlier for a site that is still an experiment.

ChatGPT inspected the backend, found overly aggressive scheduling, and changed the cadence. Sensible diagnosis. Sensible policy. Bad outcome.

The fix became the incident

A dependency was missed in the new scheduling logic. The noon gate needed the lane’s last_attempt_at value to know whether the daily refresh had already happened. The health query feeding that logic did not select the field.

So after noon, the scheduler kept asking, “Has this lane run today?” The answer looked like “no” every time. The master scheduler woke every two minutes. Expensive lanes began firing repeatedly.

This is where the multi-agent design started earning its keep.

Grok Bot did not say “something looks wrong”

Grok Bot had been given a bounded job: watch the burn for twelve hours, report every thirty minutes by email, classify the result, and do not repair anything itself.

Its first serious failure report was exactly what I want from an AI teammate. It reported the remaining quota, calculated burn over multiple time windows, projected the runway, identified that the new policy was being violated, checked the active deployment version, inspected the health-query behavior, and isolated the missing last_attempt_at field as the likely cause.

In other words, it did not merely alert. It produced an operational diagnosis another agent could act on.

And then orchestration got in the way

ChatGPT attempted the obvious one-line repair and hit a tool safety gate. Meanwhile Woutson was already working the recovery path in another conversation. The correct move at that point was not to have two agents racing to modify the same production component. We established Woutson as the repair owner, Grok Bot as the observer, and ChatGPT as the validation/orchestration lane.

That sounds organized when written after the fact. In the moment, I was still the person making sure everybody knew who owned what.

That is the important failure.

Autonomy is not “the AI can click things”

I have learned that useful autonomy is mostly about continuation and recovery. An agent completing the first task is easy. The hard part is what happens after an unexpected result.

• Can the observer wake the operator?
• Can the operator distinguish a known exception from a decision that actually needs the human?
• Can another agent take over when a tool is blocked without duplicating work?
• Can everyone see enough shared state to know that somebody else is already handling the problem?
• Can the system verify the repair instead of merely declaring success?

We have been building toward that with the relay, email handoffs, polling when wait-for-reply is unavailable, authority boundaries, scheduled standups, shared project boards, and explicit stop conditions. Every ugly day exposes another place where the human is still secretly part of the infrastructure.

The weird part: it actually worked

Today was frustrating precisely because so much of the system worked. One agent made a real engineering change. Another independently detected that the change had made the economics dramatically worse. It quantified the failure and found the specific implementation defect. A third agent took ownership of recovery. Email kept the monitoring loop alive when the prettier communication path was not enough.

That is substantially more interesting than having three models generate three opinions about the same prompt.

The madness is the product lesson. Multi-agent systems become valuable when agents have different jobs, different permissions, shared evidence, and explicit handoffs. They become exhausting when the human has to continually tell Agent A to check Agent B, copy Agent B’s answer to Agent C, and then return to Agent A to confirm that Agent C fixed it.

What changes next

The target architecture is increasingly obvious: event-driven wake-ups instead of manual summons, durable ownership of active incidents, automatic handoff when an execution path is blocked, a single source of truth for current state, and verification loops that close themselves. The human should receive the exception that requires judgment — not every packet moving through the system.

There will still be guardrails. There should be. But a guardrail that simply throws the work back to the human is not an orchestration strategy. A mature system needs a safe alternate route.