Bottlenecks, AI, and Where Product Adds Value
I'm still somewhat of a skeptic about "replace our developers and product managers with AI" discussions (see my AI-washing from 2023), but want to take the other side of the argument for a moment. And to start by thinking about bottlenecks (aka system constraints, longest tent poles, or rate-limiting steps). Then imagine what happens if software development got 10x or 30x faster and cheaper.
Every system has a bottleneck. A century ago, automobiles were built by hand, one at a time, so the bottleneck for potential car owners was finding local craftspeople with expert knowledge and custom kits. The shift to assembly lines removed that bottleneck… but managing teams of assemblers became the next bottleneck. Replacing some humans with robots (to rivet car panels and spray-paint exteriors) partially eased the staffing bottleneck… making materials management and shipping logistics the next bottlenecks. Said another way: when we speed up one step in our larger process, another step becomes the slowest one – the new constraint – the bottleneck.
I've spent the last 40 years seeing the designing and writing and testing of software as the bottleneck in building commercial software products. The most expensive, complex, hard-to-estimate, mysterious, frustrating, opaque, exciting part of the product game. See my 2015 observation that "your development team will never, ever, ever be big enough." I’m not yet ready to accept the avalanche of stories that AI will throw most software developers out into the street. But when Henrik Kniberg posts that he's getting a 32x productivity gain in developing software, I'm really listening.
Very few of us are Knibergs, but let's imagine that AI development tools actually improve overall software development productivity by 15x. Where are the next bottlenecks, and how would that change what product managers do?
Next Bottleneck: Our Backlogs
Over the years, I've done lots of order-of-magnitude estimates of R&D organizations' backlog-to-throughput ratios, framed as "how many years would it take us to build everything in the current backlog if we don't add anything new?" or "what fraction of incoming requests can we reasonably expect to address?" In the old days (pre-2023), there were typically 20x or 50x or 80x more requests and tickets and demands than we could possibly build. This provided a handy heuristic for product managers: if we can't ever address 90%+ of incoming requests, we need to mostly ignore the bottom 90% so that we can focus on the top few dozen items that might just make it into the queue.
Doing business cases or user discovery on that bottom 90% was a complete waste of time (but wasted time that made our requestors slightly less unhappy). Tools that could group thousands of items into clumps were useful, since we could then see which of those clumps were similar, or voted up, or escalated to the CEO.
With a 30x faster development cycle, it might seem useful to open the spigot on our backlog, and finally give all the people everything they want. Start at the top and turn every ticket into code! But an honest look your actual backlog suggests that most of it lacks context, is incomplete, proposes (wrong) solutions without clearly identifying problems, is generic ("make it more user friendly" or "go faster"), misunderstands how your product works (RTFM), or is a single customer’s request for an ultra-obscure feature of zero value to the overall installed base. Mostly junk.
For a concrete example, imagine that your company sells small business accounting software for non-accountant business owners. We'd see in the backlog:
- Lots of requests for more specific starter charts of accounts ("I'm a veterinarian, and your starter set of medical categories only fits human patients" and then "your starter set of veterinarian categories is US-specific, but I need defaults for New Zealand") and inevitably "you have so many starter charts of accounts that I can't find the right one"
- Frustration that users can't edit reconciled transactions, when maintaining an audit trail may be a legal requirement
- "dots should be commas, and vice versa"
- Proposed fixes that would break the product ("create a new accounting category every time we enter a new payee" or "skip all approvals/reviews and automatically pay all invoices")
- Lack of multi-currency/multi-country payments with real-time exchange rates (wrong audience, data and regulatory challenges)
- Users wanting a one-button migration tool to move from our product to a competitor (against our interest)
- Hundreds of (conflicting) tickets to add or re-arrange menus, which in aggregate would horribly degrade every user's experience
- Requests for a personal finance/home budgeting solution (not our space, already over-served by others in the market)
- A few hundred heartfelt appeals for features we added years ago
- Irrelevant or incoherent or just-plain-wrong stuff
And so on. Backlogs are not thoughtful requirements, and can't be taken literally.
Implementing all of it would lead to crappy, confusing, unfocused bloatware. (See Scott Sehlhorst’s “Happy User Peak” from 2006.)
As a product/design/engineering team, we need to have a primary audience and major use case in mind, then thoughtfully curate our products to deliver real value for our core audience. (Not chase every shiny object.) We have to deeply understand the context, be experts in relevant processes, understand what our core users need rather than what they ask for, and fit that into a viable business model. We have to pan for gold in tons of mud. We have to win and retain users by thinking ahead for them. We have to manage our products and customers, not have them manage us.
Product teams add value by deeply understanding our (target) customers' real needs and actual context and underlying economic model. By anticipating future needs and reflecting those (if we like) in backlogs. By shaping a product over time to serve our segment and continue to impress our buyers, rather than shipping a little something for everyone. We apply judgment, context, real-world experience, taste, clear-headed economics and pricing theory, market savvy, and lessons from failures past.
We get paid to sort wheat from chaff… to focus the company's R&D energy on the few things that will truly matter… to push choices that are good for the company and our customers. (We don't earn our keep by bulk-implementing tons of contradictory, surface-level asks.)
Which takes us to…
Next Bottleneck: Useful, Fresh User Insights and Surprises
I've talked with dozens of startups that want to automate the pipeline from customer-submitted tickets and CRM sales comments directly to live code. To build exactly what we're being asked for as quickly as possible. The underlying assumption is that this is a data conversion and AI pattern-matching and code generation problem. (You might guess that I don’t buy this approach.) GIGO.
Even worse, I see lots of teams outsourcing their actual user/customer interviews to AI bots and LLMs and quick-n-dirty surveys. Here's why I think that leads to sucky applications:
During learning and discovery calls, we should be listening in real time for interesting outliers, surprises, not-yet-obvious changes, and things we didn't already know. As Teresa Torres has taught us, these should be live interaction – with real (human) users and customers telling how they really use our stuff and/or why our stuff isn't doing the job. We listen, humbly, rather than argue or teach or sell or criticize or pitch. Note that discovery calls are fundamentally different from sales or support calls. When I'm listening to (learning from) a real user or buyer of my product, I hope to be surprised… to learn something new… to trip over the unanticipated. And immediately set aside my next planned questions.
- "We never run your app on Tuesdays." "That's interesting! Tell me more about that."
- "Here's where we export the data to Tableau." "Why?" "Your analytics take too long for big reports. Faster for us to dump the data and play with it in our BI tool."
- "Before we log in, let me get my cheat sheet." "What's on it?" "Your sign-on has lots of steps, which I can’t remember, so I wrote it all down."
- "Our CFO wouldn't sign off on the full license because we didn't have a business case." "Wow! Did you ask us for one?"
And these calls should be with product trios – product plus design plus engineering – since we each bring our special viewpoint (and blind spots) into the interview. Many times, my designer or developer has asked precisely the right follow-up question (in real time) or spotted an issue that I missed. We make space for spontaneity, for serendipity, when we're in the moment with our users – and can redirect the conversation – versus skimming/scanning AI summaries.
IMO, the more we rely on AI summaries, the less likely we are to spot interesting anomalies. (I'm all for automating outreach, qualification, scheduling, reminders, post-interview gifts/thank-yous, and discovery-related Product Ops investments.)
Said another way, we may be awash in automated or third-party conversations but still miss those aha! moments that drive great product thinking. Accelerating the software development cycle will make those insights even more valuable – so that we're using our 15x throughput improvements for what matters. Increasingly, we'll earn our salaries through real insights and anticipating the future, rather than overseeing R&D pipelines.
And we'll be able create scores of prototypes 15x faster, on the way to launching new features 15x faster. Which takes us to…
Next Bottlenecks: Prototype Reviewers and User-Paced Change
I've seen some brilliant work using AI tools to spin up prototypes in hours rather than weeks. A tremendous opportunity for us to create lots of variations. To clarify our thinking. To convert vague words into visuals. To get outsider input. But in the B2B space, there's typically a serious shortage of actual end users with the time and interest to provide solid prototype feedback. (Large B2C players may have millions of individual consumers, plus the tools to solicit thousands of lightly compensated participants. But B2B/enterprise companies usually have only a few handfuls of enthusiasts willing to weigh in.)
Most importantly, we will probably run into a pace-of-change issue with our production (paying) customers. The ability to build hundreds of new features and improvements each month will naturally tempt us to ship hundreds of new features and improvements each month. But our end users need some stability in the software they use – swapping out menus and shifting data inputs and reconfiguring agents can confuse or frustrate them. There's a limit to how fast we can evolve our fundamental Jobs To Be Done. (Xero did this right by giving me a year's notice of upcoming changes to its invoicing module. Kudos!)
So product (and design and engineering) folks will need to set some reasonable tempo for enhancements. How fast can our installed base absorb change, even if we can move much faster internally? Deeply understanding our real users' real needs – and their ability to absorb our changes – become increasingly important as we speed up the build cycle.
Sound Byte
An order-of-magnitude acceleration of development will shift product attention (and value) further away from pipeline coordination and internal process models… toward deep insights, aggressive discovery, market/subject expertise, thoughtful anticipation, curation and "taste." After all, the competition will have the same generic data and AI tools that we have.
And then…
20x faster development will undercut a lot of the economic assumptions that we've operated under:
- New entrants can (effectively) clone my existing product at very little cost. Likewise, thousands of new products will flood every market. With some finite limit to the world's software spending, prices will drop precipitously.
- We’ll see lots of abandoned-ware, as the cost of maintaining and supporting products swamp initial investments.
- Marketing, selling, and buyer access will dominate software company spending. Software could resemble the snack food market – where manufacturing costs are much smaller than spending on brands, advertising, channels, endorsements, tie-ins, and waiving away the health effects of salted carbs.
- Data ownership – both for personalization and AI training – may let the biggest players accrue more power.
- Customer loyalty may become more important than delivery speed.