Writing · Tales of Engineering Leadership
The Project Trap
Why projects, PRDs and RFCs feel safe but slow us down, and what Extreme Programming knew all along
As an Engineering Leader, I’ve seen that when teams feel most in control, they often learn the least. There’s the project kickoff meeting with a polished slide deck, the PRD1 signed off after weeks of comments, and the RFC2 that passes review with every objection addressed. These rituals make everyone feel safe. When I look back at the failed projects in my career, almost all of them had immaculate paperwork. So did the successful ones, and that’s the uncomfortable part: the documents never predicted the outcome. They just kept us confident while we waited to find out.
It took me years to make sense of that. We’ve built an entire way of working (the Project, with its PRD at the front and RFCs along the way) assuming the biggest risk in software is doing the work wrong. But, in my opinion, the biggest risk was never the work. It was deciding what to do with too little knowledge and defending that decision long after we knew better.
So the question I want to explore is uncomfortable: what if projects, PRDs, and RFCs aren’t neutral containers at all? What if the packaging is part of what’s slowing us down?
Projects: Borrowing Certainty We Don’t Have
The project is an idea borrowed from construction and finance: fixed scope, start date, end date, and temporary allocation of people. It works beautifully for bridges, but software isn’t a bridge.
Sriram Narayan points out that a project is fundamentally a funding model, not a delivery model, and that funding model ends up dictating behaviour more than we admit3. Because a project must end, teams are assembled for it and can be disbanded after. The awkward part is that a team understands the problem best right when it finishes shipping, which is exactly when we break it up. Mik Kersten describes this pattern in Project to Product: we dissolve teams at their point of maximum learning and then wonder why the next initiative starts from zero4.
I’ve written before about knowledge silos - https://talesofengleadership.substack.com/p/breaking-down-the-walls - and the challenges every time work crosses a boundary. Handovers aren’t a side effect of the project model, the model runs on them: the people who decide hand over to the people who build; the people who build hand over to the people who test, who then handover to the people who support and maintain. Ownership, in the “you build it, you own it, you run it” sense, becomes impossible because the organisational unit that built the thing no longer exists.
And a project asks for its biggest decisions on day one, which is the day we know the least, yet that’s exactly when we commit to scope, staffing, and dates.
PRDs: A Contract Where a Conversation Should Be
The PRD is the project’s opening move, and it inherits all of its problems in written form.
Donald Reinertsen spent a career showing that large batches kill product development: they delay feedback, hide errors, and make mistakes more expensive to fix5. A thick PRD is exactly that: a large batch of guesses frozen at the moment of least knowledge, then defended because so much effort went into producing it. The document records our assumptions, not our learning, and once something is in print it looks far more certain than it ever was.
To be clear, the problem isn’t writing. A short narrative that forces you to think in full sentences is one of the best tools we have, and I’d never take it away from a team. The problem is the social role the document plays. The people who wrote the Agile Manifesto (Beck and Jeffries among them) saw this twenty-five years ago: working software over comprehensive documentation, and customer collaboration over contract negotiation6. A PRD works like a contract between people, whatever we call it. It exists so that later, when reality differs from the plan, someone can point at a paragraph. Which is negotiation, really, just conducted in advance.
Extreme Programming offered a simpler alternative. Ron Jeffries compressed it into three words: Card, Conversation, Confirmation7. Write the need on something too small to hold details: a card. Let the details live in a conversation between the people building and the person who wants it, held just in time when knowledge is freshest. Then agree on the confirmation, and make it executable where you can. The spec doesn’t disappear; it becomes a test that runs. Picture a card that says: “Every Friday I need three numbers from this report in an email.” A requirements document would have translated that into an export feature by the time anyone wrote code, specified down to the file format. In a ten-minute conversation, the team learns the numbers sit in the same place every week, and a copy button ships that afternoon. The export feature would have been built in full, and it would have worked, and it would have been the wrong thing to be building.
This also removes the biggest handover of all, the one from the people who decide to the people who build. Marty Cagan has argued for years that real product discovery happens through prototypes and customer contact, not requirements documents8. XP went further and put the customer inside the team’s conversation. Honestly, the on-site customer was the hardest XP practice to sustain, and few businesses ever managed it in full. But a product manager who acts and feels part of the team and brings real users to the demo gets you most of the way there, and it leaves very little for a requirements document to do.
RFCs: Asking Permission Instead of Asking Questions
I’ll admit RFCs are the hardest of the three to criticise, because their intent is noble: open design, written thinking, feedback before commitment. I’ve written my share, and some of them helped.
But watch what actually happens in most organisations. The RFC becomes a gate. Work stops while the document circulates. Engineers debate hypotheticals in comment threads for two weeks. Hypotheticals, because no code exists yet to tell anyone the truth. And when the RFC is finally approved, it hardens into precedent: the design was decided, so discovering something better mid-build now requires re-litigating the document.
Kent Beck described the problem years ago: “Optimism is an occupational hazard of programming; feedback is the treatment.”9 Debating an RFC before any code is written is optimism dressed up in a document. XP’s answer was incremental design and spikes: short, throwaway experiments meant to turn opinions into evidence. I’ve watched two engineers argue a technology choice across a two-week comment thread, and I’ve watched the same kind of argument end in an afternoon because someone built the smallest version of both options and measured.
So where’s the line? Inside a single team’s domain, spike first, decide second, write third. Across team boundaries I want to be honest: shared API contracts, data migrations, and security decisions are useful, and partly a negotiation between teams rather than a question code can settle. There a written proposal earns its keep, and in highly regulated environments the written trail may not even be optional. Keep it to a page or two, attach the spike results, and write the decision record once the evidence is in. Write things down to remember, not to ask permission.
What XP Knew: Steering Beats Aiming
Rereading Extreme Programming Explained today, what strikes me is that its practices were never a grab-bag. They were one coherent answer to a single question: “how do we decide well when we know so little?”.
The project approach says: aim carefully, then fire. Spend weeks refining the aim with a PRD, check the trajectory with an RFC, and hope for the best.
XP says steer. Small releases put real software in front of real users, in weeks back then, in minutes with today’s continuous delivery pipelines, so decisions rest on evidence instead of guesses. And the continuous planning makes prioritisation a cheap, recurring conversation for the whole team10. This used to be framed as a negotiation between business people and technical people, each bringing what they knew. Product engineering has since dissolved even that line: engineers join discovery, product people join delivery, and the what and the how get shaped together. But notice the direction of travel. Every generation of this idea moves deciding closer to the people doing the work. A PRD moves it the other way: one document, written by one side, before the work begins.
The whole case for deciding everything up front rests on change being expensive, and XP’s technical practices, test-driven development, continuous integration, relentless refactoring, exist precisely to make change cheap. Once change is cheap, deferring a decision stops being reckless and becomes the rational move. Small batches without that discipline, though, is just thrashing.
This isn’t nostalgia either. DORA’s research in Accelerate identifies working in small batches as one of the capabilities that drive both throughput and stability, with no trade-off between the two11. The Poppendiecks, who brought lean manufacturing ideas to software, list handoffs among the seven wastes, because each one loses tacit knowledge that no document, however well written, can carry12.
What I Try to Do Instead: Fund the Team, Shrink the Batch
None of this means working without discipline or writing nothing down. The discipline moves from writing things up front to showing working software on a cadence. In the teams I lead, that has come to mean a few things, some further along than others:
Fund teams, not projects: Keep stable teams that own outcomes, and treat the roadmap as a list of bets the team shapes against the business goals, not a Gantt chart the team is assigned to. When the funding unit is the team, learning compounds instead of dissolving at every project boundary. And if you can’t change how funding works in your organisation, you can at least refuse to disband the team. Keep the same people together through the next initiative and you’ve kept the learning.
Structure teams around business domains: Most of the coordination we invent, the cross-team RFCs and dependency spreadsheets and alignment meetings, exists because team boundaries cut across the business instead of along it. Conway observed back in 1968 that a system’s design copies the communication structure of the organisation that built it13, so team boundaries are architecture decisions whether we admit it or not. Skelton and Pais built Team Topologies on that insight: align each stream-aligned team to a single, well-bounded domain, sized to the team’s cognitive load, so most changes start and finish inside one team14. When a team owns a whole domain, dependencies stop being a fact of life and become a design smell. Every dependency you remove is one less handover, and usually one less standing meeting too.
Replace the PRD with a one-pager and a cadence: A single page stating the problem, the customer, and how you’ll know it worked. The rest happens in short and continuous planning conversations with the whole team (including product manager, UX, data science, etc.) in the room, where stories are cards and no story leaves without an agreed confirmation. Usually a couple of sentences, sometimes a test.
Turn RFCs into decision records: Spike first, decide second, write third. A short record of what we chose and why, written after the evidence is in, preserves memory without becoming a gate.
Let the demo serve as the status report: If stakeholders see working software frequently, they stop asking for documents that try to predict it. Nothing creates trust like seeing the real thing.
A Word on AI: The Biggest Batch of All
There’s a new reason to care about this, and it’s right in your editor. Now that AI agents can build from a written brief, I see teams falling in love with PRDs again: the mega-prompt, the exhaustive spec, handed to the machine just like we used to hand it across the org chart. It’s the same mistake, only faster. The machine made typing nearly free, which means a wrong assumption no longer produces one bad function. It produces thousands of plausible-looking lines before anyone has talked to a user, and the cost of the mistake doesn’t shrink. It moves downstream, to whoever has to find it.
The answer is still the same. Shorten the feedback loop. Small stories, fast feedback, a human steering. XP, with much faster AI agent partners.
Conclusion: One Conversation at a Time
Projects, PRDs, and RFCs stick around because they make us feel responsible. They give us things to point to and a paper trail to defend. But feeling in control isn’t the same as actually being in control, and most of that paper records what we assumed back when we knew the least
Extreme Programming’s main insight wasn’t really about code. It was about humility: we make bad decisions at the start, so let’s work in a way where no decision has to survive too long. Fund the team, shrink the batch, keep the customer close, and let working software speak for itself.
Steer instead of aiming. One conversation, one small release at a time.
Footnotes
-
PRD - Product Requirement Documents - https://www.atlassian.com/agile/product-management/requirements ↩
-
RFC - Requests For Comments - https://www.ietf.org/process/rfcs/ ↩
-
Narayan, S. (2018). “Products Over Projects.” martinfowler.com. ↩
-
Kersten, M. (2018). Project to Product: How to Survive and Thrive in the Age of Digital Disruption with the Flow Framework. IT Revolution Press. ↩
-
Reinertsen, D. G. (2009). The Principles of Product Development Flow: Second Generation Lean Product Development. Celeritas Publishing. ↩
-
Beck, K., et al. (2001). Manifesto for Agile Software Development. ↩
-
Jeffries, R. (2001). “Essential XP: Card, Conversation, Confirmation.” ronjeffries.com. ↩
-
Cagan, M. (2018). Inspired: How to Create Tech Products Customers Love (2nd ed.). Wiley. ↩
-
Beck, K. (2004). Extreme Programming Explained: Embrace Change (2nd ed.). Addison-Wesley. ↩
-
Beck, K., & Fowler, M. (2000). Planning Extreme Programming. Addison-Wesley. ↩
-
Forsgren, N., Humble, J., & Kim, G. (2018). Accelerate: The Science of Lean Software and DevOps. IT Revolution Press. ↩
-
Poppendieck, M., & Poppendieck, T. (2006). Implementing Lean Software Development: From Concept to Cash. Addison-Wesley. ↩
-
Conway, M. E. (1968). “How Do Committees Invent?” Datamation, 14(4), 28-31. ↩
-
Skelton, M., & Pais, M. (2019). Team Topologies: Organizing Business and Technology Teams for Fast Flow. IT Revolution Press. ↩