What AI Agent Frameworks Actually Do Versus What They Say They Do
Understanding the real mechanism helps you stop buying structure you don’t need and start building the structure that actually works.
The pitch for AI agent frameworks is genuinely compelling.
You describe your software in plain language.
Multiple independent AI agents research, plan, write code, and review each other's work.
They debate. They catch each other's mistakes. A finished, production-ready application emerges. Full autonomy. No micromanagement required.
That pitch is technically accurate in the same way a film trailer is accurate. The individual frames exist. The sequence doesn't.
I want to be clear: I am not here to dismiss these tools. Some of them are genuinely useful in specific contexts. But there is a gap between the legend and what actually happens when you build something complex with them, and understanding that gap is worth your time.
What's actually happening under the hood
Here is the real mechanism behind every multi-agent framework on the market right now.
One model. One context window. Different personas, taken at different steps. Sometimes those steps run in parallel across separate API calls.
That is the complete technical picture.
The "multi-agent" label describes the structure, not the underlying intelligence. There is no team of separate minds deliberating on your codebase.
There is one model, well-prompted, switching roles at defined points in the workflow.
Andrej Karpathy made this clearer than most people noticed when he wrote about his workflow shift in January 2026. He went from 80% manual coding to 80% agent-driven coding in weeks, and then listed, with equal clarity, the failure modes that came with it.
Agents make wrong assumptions without flagging them. They over-engineer things you didn't ask them to change.
They produce code that compiles and runs and quietly gives you the wrong output.
A few weeks later, developer Forrest Chang took those observations and turned them into a CLAUDE.md file: a plain markdown configuration file that embeds behavioral rules directly into your project, so the AI reads them at the start of every session. The repo crossed tens of thousands of GitHub stars almost immediately, which tells you something about how widespread the frustration was.
What the file actually does is interesting. It doesn't add capability. It adds constraint.
State your assumptions explicitly before acting. Use the minimum code that solves the problem. Only touch what you were explicitly asked to touch. Define what success looks like before you start, not after.
Those aren't prompting tricks. They are a command structure.
Where frameworks create the most risk
Here's where developers tend to go wrong.
The appeal of an agent framework is that it automates the role-switching.
You set it up, it runs, different agents handle different concerns, and your code comes out reviewed and tested on the other side. The automation is real. The black box is also real.
When a framework fails on complex terrain, you often don't know why. The switching happened automatically. The context was passed between roles in a way you didn't configure.
The assumption that broke your database query was introduced somewhere in the pipeline and you have no clean way to trace it.
This is the core tradeoff. Frameworks trade control for automation. For a standard service with a single user type and a well-scoped problem, that tradeoff is often worth it.
For a platform with three user types, shared state, row-level security, and live users depending on specific permission boundaries, the tradeoff tilts the other way fast.
The damage radius of a hidden failure expands with complexity. On a complex platform, you want the role-switching to be yours.
What manual orchestration actually looks like
The approach I use is called the Fleet Model. The principle is the same as what frameworks automate, but done by hand. One model, named roles, locked domains, consistent triggers.
Six roles, each with a defined domain and a standing question they ask before acting. One for mission alignment: is what we are building still what we set out to build?
One for architecture, specifically the rule that the same concept in two files without a shared contract is a structural fault, not a style preference.
One for security, which owns every SELECT statement, every INSERT verification, every credential that should not exist on a public page. One for logic precision, whose job is hunting the assumption we made and never said out loud. One for session continuity, so the next session doesn't start from scratch. One for end-user discoverability, evaluating everything as someone who has never seen the platform before.
You invoke them by name. "The Sentinel, look at this query." The model shifts into that role's domain. The constraint is the point. The name is the trigger.
The difference from a framework is visibility. I know when each role is active, what it has access to, and what it is allowed to do. When something goes wrong, the trace is clean.
The CLAUDE.md question
A common question when I describe this: why not just use a CLAUDE.md?
The answer is that you should. They are not in competition.
A CLAUDE.md in your project root holds the project-wide rules. The standing orders, the tech stack conventions, the security principles that never change.
It is read automatically at the start of every session. It does the work of a permanent set of behavioral guardrails.
The role-based approach holds the thinking-mode triggers. Who does what, when you call them, what question they always ask first. It does the work of specialised analysis deployed at the right moment.
Both layers together is what makes complex platform development with AI actually reliable.
The CLAUDE.md ensures the rules are always loaded. The roles ensure the right analysis runs at the right moment. Neither does the other's job.
A practical comparison
For solo developers or small teams building multi-role platforms, here is how the three approaches compare in practice.
Agent frameworks are high-setup, low-visibility, high-token-cost, and genuinely risky on complex terrain where interdependencies are dense. Useful for well-scoped, lower-stakes automation tasks where the black box is acceptable.
Manual role orchestration is medium-setup, full-visibility, efficient, and strong precisely where frameworks struggle: complex builds with multiple user types, shared state, and real permission boundaries. The cost is the discipline required to maintain it.
A CLAUDE.md alone is low-setup and immediately useful, but it covers the rules layer without the role layer. A good start. Incomplete on its own for serious platform work.
The combination of manual role orchestration plus a well-crafted CLAUDE.md is where the reliability actually lives.
What changed in practice
Before I had a structured method, my AI sessions were productive in bursts and destructive in ways I often found too late. Silent query changes that broke other dashboards. Assumptions about authentication states that produced permission failures in production. Features that technically worked but told users nothing when they succeeded or failed.
After building out the role structure and the standing orders, sessions became auditable. I could trace why a decision was made. I could identify which role had missed a check.
I could reproduce the discipline reliably across sessions rather than relying on a good day.
The tools were always capable enough. The discipline was the variable.
The full framework
I documented everything in a short book called The Fleet Method. Six roles, seven standing orders, a CLAUDE.md starter file, and a pre-launch cloud integrity checklist. It covers the full command structure for building complex platforms with AI: how to define roles, how to run sessions, how to close without losing context, and how to audit your cloud security layer before you deploy.
There is also a sister edition called Mission Critical, which covers the same framework using special ops terminology.
Same intelligence, different vocabulary. For developers who prefer thinking in terms of operators, objectives, and rules of engagement rather than ships and standing orders.
If you are building anything with multiple user roles and AI assistance right now, they will save you the incidents I had to learn from.