Last week a client sent us a 250-page specification. Beautifully structured, every section where a textbook says it should be, and in several places describing a product the client didn’t actually want. He hadn’t read it. His AI wrote it.
I hear a version of this story every time we hold a certain meeting at Brocoders. Every two to four weeks our developers, tech leads, project managers, our head of delivery, and I get on a call with no agenda beyond three questions: what’s new, what hurts, what did you learn since last time. No demos. Just people who ship code for clients every day, comparing notes on where the tools actually break. We call it the AI Insights club, and it has quietly become one of the most useful hours in my calendar.
This time the conversation was too useful to keep internal. Here it is, written for founders and product leaders rather than developers, because most of it turned out to be about you.
A fluent spec still needs verifying
The 250-page document landed on Viktor, who is leading a new large project for us. When he described it, everyone on the call recognized it immediately. Anna and Anton run two of our long-running client products, and they see the same thing week after week. A client describes a feature to their AI assistant in two sentences. The assistant expands it into six hundred lines of requirements and acceptance criteria. The document arrives in our inbox, unread by the person who sent it.
Then this happens. The acceptance criteria say “available to admins only.” We build it for admins only. The client asks why other roles can’t see it. We point at the criteria. “Oh, I never wrote that.” Of course not: the model did, it sounded reasonable, and nobody caught it. Recently one of our teams received a task whose description covered an entirely different piece of functionality from the one the client had in mind. What he actually wanted turned out to be a separate feature, about six hours of work that appeared nowhere in the document.
The uncomfortable insight: the bottleneck in software has moved. Writing code got dramatically cheaper. Knowing what to build did not, and the AI-generated spec makes that part look finished too. A fluent document still needs a human to verify it.
What we’re doing about it, in three layers:
- Marking the invented parts. Anton has trained himself to read a client spec with an eye for which sentences the client wrote and which the model padded in. Instead of sending a wall of clarifying questions, he annotates: “this looks like an AI addition, my assumption is X, shout if I’m wrong.” It cuts the email ping-pong drastically, and clients confirm or correct in one line.
- Giving the client’s AI a better brief. Clients are going to write tasks with an AI anyway, so we would rather shape how. We’re building a task-writing skill our clients can plug into their own assistant: it asks the right questions, sticks to the what and the why, and refuses to invent the how.
- A shared, current knowledge base. A skill is only as good as its context. So the next step is a shared source of truth per project, what exists and what each feature actually does, that the client’s assistant and ours both read from. We keep it deliberately user-facing rather than pointed at the codebase. Once a non-technical person’s AI can see the code, it starts proposing technical solutions, and the conversation gets harder.
Viktor put the underlying risk in one sentence. We had a case where the client’s AI wrote to our PM, our PM’s AI wrote the tasks, a developer’s AI built the frontend, and his questions were answered by another AI. “The AI ended up talking to itself.” Every layer of that chain needs a human who has actually read what they’re forwarding. We’re trying to make that easy rather than heroic.
The small changes that make no sense for anyone
I brought a problem I hear constantly from clients whose products we’ve launched. Move this block. Change this text. Swap these images. Tiny frontend tweaks, and the economics are absurd for both sides. The client raises a ticket, we find a developer who isn’t allocated to the project, a manager gets involved, everyone waits. For a ten-minute change, most of the cost is overhead, and it’s slow.
The discussion went somewhere I found exciting. Picture post-launch support working like this. The client describes a change in plain language to an AI interface we’ve configured. The system assesses the risk, executes the change if it’s a safe, visual, frontend-only edit, and says “this one is too complex, ask your team” when it isn’t. Something like a CMS, but with an AI editor instead of a page builder. Backed by mandatory backups of code and database, so anything can be rolled back.
Then the room did what I value most about it, it pushed back. Viktor: any guardrail can fail. Anna: don’t forget we hand over the codebase, and nothing stops a client from removing the limits. And the sharper point: clients may not even want this. Anna described a client with a weekly, fully templated data task that takes ten minutes. Our deploy takes longer than the AI does, and the client has zero interest in touching it. Another client came to us to extend a promo code by changing one digit. Wanting control and wanting to do things yourself are different appetites. So it’s a hypothesis rather than a product. For a specific class of clients and a specific class of changes, AI-mediated self-service support is possible, and it would change the economics of post-launch support for both sides. If you’re a client of ours reading this and you’d want it, or definitely wouldn’t, I’d like to hear why.
Wanting control and wanting to do things yourself are different appetites.
How we keep the machine on a leash
Everything above invites a fair question from any reader: do we actually control these tools ourselves? The last part of the call was about exactly that, how our engineers work with AI agents day to day, and what they’ve stopped trusting.
Vlad K., one of our senior engineers, admitted that he stopped reading every command the agent runs. It fires off twenty scripts per task, and no one can review that live. His answer was structural. Every project runs in its own isolated container, the agent works unattended inside it, and if it wipes the container, nothing of value is lost. He estimates it made him about 30% faster. Vlad R., consistently our most prolific engineer, works two projects in parallel. He moved test runs and reviews to a separate machine entirely, so a hung agent can’t take his laptop down with it.
Both of them said the same thing about output: review is non-negotiable. Control moved from the beginning of the task to the end, and it did not disappear. “Sometimes it produces nonsense, especially right after a new model version.” Several people described the same pattern. Same rules, same codebase, and one week the model starts skipping things it hadn’t skipped for a month. Usually a release is a few days away. A new model means re-checking everything you thought was settled.
Two smaller findings surprised me. Pushing the “thinking effort” setting to maximum made the current models over-engineer and second-guess the codebase’s own patterns, while going too low made them drop details. Most of the team has converged on “high, not maximum” for nearly everything. And the newest models have started editing code through generated scripts rather than visible edits, which is cheaper on tokens and worse for oversight. One of our engineers now keeps a standing rule in the project config that forces visible edits unless it’s a genuine mass change. The tools optimize for their own economics. We optimize for being able to see what happened.
What I take from it
If you’re a founder or a product leader working with a development partner right now, the practical version of all this is short.
Treat your AI-written spec as a draft. Read it before you send it, or tell your team which parts are yours. Ask your partner how they separate what you wrote from what the model padded. Ask them what runs unattended, what gets reviewed by a person, and where. And if small changes to your launched product feel absurdly expensive, you’re right. That’s a problem the industry hasn’t solved yet, and we’re working on it.
Our club meets again in a couple of weeks. If this was useful, I’ll keep writing them up
And a word about who “we” are, since I’ve said it a dozen times. We’re Brocoders: about fifty engineers, designers, and managers who have been building software for clients for over a decade, now working through this shift in the open rather than pretending we’ve finished it. If an AI-written spec is about to land on someone’s desk and you’d like a second pair of eyes that can tell what the model padded in, write to me. Same if you have a launched product and the economics of keeping it alive don’t add up. And if you disagree with anything above, even better, the club could use a few outside voices.
Frequently Asked Questions
A client describes a feature in two sentences, and the AI expands it into hundreds of lines of requirements and acceptance criteria. The document arrives unread. As a result, the team builds what the model wrote, not what the client actually wanted. For example, the criteria say “available to admins only,” the team implements it that way, and the client later asks why other roles can’t see the feature.
Yes. Writing code has become dramatically cheaper and faster thanks to AI. Knowing what to build has not. An AI-generated specification creates the illusion that this part is already done, when in reality it still requires human verification.
We use three layers:\n1. We mark the invented AI parts directly in the document (“this looks like a model addition, my assumption is X”).\n2. We give clients a better brief for their AI — a skill that asks the right questions and refuses to invent the “how.”\n3. We maintain a shared, up-to-date, user-facing knowledge base per project that both the client’s AI and ours can read from.
It’s the situation where the client’s AI writes to the PM, the PM’s AI writes the tasks, a developer’s AI builds the frontend, and the developer’s questions are answered by yet another AI. There is no human in the chain who has actually read and understood the content. Every layer needs a person who has genuinely reviewed what they are forwarding.
This is one of the hypotheses we are exploring. The idea: the client describes a change in plain language through an AI interface. The system assesses the risk and executes safe visual frontend-only edits automatically. More complex changes are escalated to the team. Mandatory backups of code and database allow any change to be rolled back. This is not a universal solution — not every client wants it.
Wanting control and wanting to do things yourself are different appetites. Some clients prefer to send a simple 10-minute templated task once a week rather than configure and oversee an AI tool. Others value the ability to quickly fix small things on their own.
Key practices:\n• Every project runs in its own isolated container — if the agent breaks something, nothing of value is lost.\n• Tests and reviews are often moved to a separate machine.\n• Review of the output is non-negotiable. Control has moved from the beginning of the task to the end.\n• Maximum “thinking effort” is avoided — models start to over-engineer.\n• Standing rules force models to make visible code edits instead of hidden scripts.
This is a common pattern. Same codebase, same rules, and the model suddenly starts skipping details it hadn’t skipped for a month. It usually coincides with a new model release. The solution is to re-check everything you thought was already settled after a model update.
- Treat your AI-written specification as a draft. Read it before sending it, or clearly mark which parts are yours and which were added by the model.
- Ask your development partner how they separate what you wrote from what the model padded.
- Ask them what runs unattended, what is reviewed by a person, and where.
- If small changes to your launched product feel absurdly expensive — you’re right. This is a problem the industry hasn’t fully solved yet.
Brocoders is a team of about fifty engineers, designers, and managers who have been building software for clients for over a decade. The AI Insights club is an internal meeting held every two to four weeks with no demos — just discussion of what’s new, what hurts, and what was learned. The most useful insights are later shared publicly for founders and product leaders.