You can build exactly what was asked for and still get the product completely wrong.
A customer requests a feature. A prospect says it's the one thing stopping them from buying. And when an agent can build it in an afternoon, why wouldn't we?
But asking for a feature isn't the same as needing it. And needing it isn't the same as actually using it.
Other times, the problem is real and the idea sound, but the finished experience is confusing, frustrating, or simply doesn't accomplish what the user needed.
In both cases, the code might work perfectly. The tests pass. The feature ships. And the product can still fail.
I've spent a considerable amount of time and energy answering the question "Can we build it?" over the years. AI makes the answer to that question almost always "yes". Which makes me wonder, have we spent too much time worrying about the middle of product development, rather than what we should question on either side?
The process now looks something like this, which is very similar to previous loops that were popularized (e.g. build, measure, learn):
Let's start with the first question...
1. Is it worth creating?
When building was expensive, implementation costs were the filter. Not necessarily a good one, but a filter nonetheless. If a feature required six engineers and three months of work, somebody probably had to make a reasonable case for pursuing it.
Today, an agent can produce a convincing implementation of the same idea in an afternoon. Which is great for experimentation, less so if we confuse being able to build something with having a reason to.
At Lenny's Summit, Claire Vo joked in her talk that we can now ship all our bad ideas. The temptation is real (and I'm definitely guilty of shipping a few).
When the cost of saying yes approaches zero, knowing when to say no becomes considerably more important.
The cost of building a feature may be close to zero now, but the cost of having it isn't. Every addition gives users something else to understand and the product team something else to maintain. Building it might be practically free, living with it less so. The opportunity is to understand which problems deserve solving, and which solutions are worth pursuing.
That doesn't mean six weeks of validation before writing code. Sometimes the fastest way to know if an idea is worth building is to build it. But we should know what we're trying to learn. The goal isn't necessarily to build fewer things. It's to get better at knowing which things are worth building.
2. Can we build it?
Almost certainly, yes. But that doesn't mean reliable software is suddenly trivial. Architecture, security, accessibility, and maintainability still matter.
The effort required to turn an idea into working software has fallen dramatically. This Designer Fund article about Anthropic's design team describes prototypes being built before the team has even decided whether an idea is worth pursuing.
I've been experimenting with "agent-driven" development workflows for quite awhile, and I'm still surprised by how much can happen without me writing or reviewing every line of code (don't worry, the projects that are meant for others are still thoroughly reviewed!).
Where I spend my time has changed dramatically. I spend less energy explaining exactly how something should be implemented and more energy collecting context, defining outcomes, establishing constraints, reviewing decisions, and evaluating results.
It seems obvious this shift will continue across product, design, and engineering, but we need to keep in mind, delegating implementation isn't the same as delegating responsibility for the outcome.
Which brings us to the third question...
3. Does it work as intended?
This is what it feels like we're underestimating, and in some instances completely ignoring.
We've spent decades building increasingly sophisticated ways to verify software: unit tests, integration tests, end-to-end tests, code reviews, CI/CD pipelines.
All of which is incredibly valuable, but most of these systems are primarily concerned with technical correctness. They don't necessarily tell us whether the experience actually works.
- A signup form can submit successfully while a confusing experience causes people to abandon the process.
- A dashboard can render perfectly while burying the information its users actually need.
- An agent can follow every component in a design system and still create an interface that's completely wrong for the task.
The code works perfectly, the product not so much.
Even when the implementation perfectly reflects the original design, the design can still be wrong. More than ever we need to evaluate whether what's been built is what we intended, and understand whether what we intended actually works.
If you've read this far, you've probably guessed, I've been experimenting with ways to validate against actual user behaviour and design intent.
In one recent experiment, a validation system identified a release that broke signup and pinpointed where the journey failed. In others, the correct result was that there simply wasn't enough evidence to make a reliable judgment. That distinction matters, because if we're going to rely on agents to evaluate what other agents produce (and we are), we need something more substantial than an LLM confidently declaring that "everything is good, and that's the load-bearing insight" 🙄.
We need evidence.
At Lenny's Summit, Katie Dill from Stripe talked about scaling intent and quality as AI accelerates production. For me, that's meant evaluating more than whether the output matches the instructions, it's knowing whether it also achieves the intended outcome.
Did users accomplish what they came to do? Did the interface communicate what it was supposed to? Did a change introduce friction? Does the implementation still reflect the reasoning behind the design? Is the value obvious to the person using it?
When we can't answer those questions reliably, can the system tell us that? This is where I think a new generation of product tooling will emerge: not just generating software, but checking whether it delivers the intended experience and outcomes.
It's a loop, not a pipeline
I've always thought of product development as a continuous loop, but it's even more obvious today. We need to understand what deserves to exist, build it, evaluate what happens, and use what we learn to make better decisions about what comes next.
I don't believe humans have to perform all the work at either end. AI can help identify opportunities, challenge assumptions, evaluate alternatives, and measure outcomes. And, of course, over time, more of those decisions will become automated, but only if we give those systems the context, constraints, and evidence to make good decisions.
Karri Saarinen from Linear made this point at Lenny's Summit this way: the output isn't the product and customers aren't buying lines of code.
There's a temptation to measure the success of AI-assisted development by how much faster we can ship. More pull requests. More features. Shorter development cycles. More agents working in parallel. But those are simply measures of production, not progress, and let's not even mention revenue.
The important thing isn't which discipline owns each part of the process, it's that collectively the team owns the result. Our users don't care how quickly the software was written or how many agents we had running. They care whether the product solves their problem.
When building software becomes the easy part, knowing what deserves to be built, and whether it actually works, IS the advantage.