An AI-native product carries a line through it that older software never did. The last essay was about the human touch while you build; this one is about the line you build into the product.

Because the model fails confidently, fluently, invisibly, a product built on it needs to know where nothing should vary. On one side the machine is allowed to fluctuate: generate, suggest, improvise. On the other it is locked, deterministic, the same on every run. This is the deterministic-probabilistic boundary, and where it sits is the product.

Older software did not have it: it was predictable, so a human checkpoint was optional, a feature you could add or skip. AI is not predictable, so the product now holds a place where someone has to judge whether this output, this time, can be trusted. The boundary is not a feature you add. It is native: there whether you designed it or not. And it does not hold still.

Native, moving, and quietly hidden.

"Keep a human in the loop" says nothing on its own; the loop is large and the human can stand almost anywhere in it. The question is always where the human stands, doing what, answerable to whom. Waymo keeps the line at the exception: the car drives, and a remote operator is called only when it is stuck. Meta's Ray-Ban glasses, sold with privacy at their core, pushed the line out of sight: footage, including people undressing and using the toilet, reached overseas annotators, themselves the hidden humans the word smart conceals. The pattern is older than the glasses. Meta's "automated" moderation ran for years on contractors in Nairobi and Manila reviewing the worst of the internet for a few dollars an hour. The line was not removed. It was moved to where the customer did not have to look at it. And when reporters looked, Meta's response was not to move the line back: it ended the contract, and over a thousand of the annotators lost their jobs.

Google's AI search draws it mid-page: the overview above improvises; the links below never do. For some queries it pulls the line back entirely, removing the AI answer altogether. All are placements of the same boundary. Meta's does something quieter: it dresses a system that still needs people as one that does not. Where you draw the line is also a decision about how honest the product is willing to be about its own limits. Two things turn on that decision: whether the experience is any good, and whether it can be trusted.

Experience.

I worked on some of the early visions for agentic shopping at Amazon, and a fully probabilistic shopping experience turned out to delight no one: not knowing whether the checkout will go through, whether it is the right product and size, or exactly when it will arrive. For discovery, probabilistic scale allows better recommendations, styling, inspiring visuals, personalisation: AI lets you reimagine the top of the funnel. At checkout it is a disaster. An agent improvising through variant selection, cart edge cases, and an unexpected pop-up while the customer waits is not magic; it is a lost sale, and lost trust.

You can watch the industry draw the line in its own architecture. The emerging commerce stack splits exactly here. Anthropic's Model Context Protocol feeds the roaming agent live product, inventory, and cart context: the discovery side, where variance is welcome. Stripe and OpenAI's Agentic Commerce Protocol, the rails under ChatGPT's Instant Checkout, locks the transaction onto a deterministic path: a payment token authorised for one amount and one merchant, the customer confirming each step, the merchant still the merchant of record. It launched deliberately narrow: single items from a controlled set of sellers - because the free-form part is precisely what you keep away from someone's money. You can pipe product data through any protocol, but the moment of yes, I'll pay is earned, not transferred.

The future of agentic commerce is not an agent that does everything. It is an agent that knows where to stop and hand off. Draw that line well and the product feels magical and safe at once; draw it wrong and you have a beautiful demo that quietly loses the sale.

Trust, accountability, and the law.

Place the line so the model decides what it should not, and the cost returns as liability and broken trust. Air Canada's chatbot invented a bereavement refund policy for a passenger flying to a funeral, and the airline was held liable for it. Moving the human out did not move the responsibility out. And as products hand agents authority to act on the open web, the exposure widens: a 2026 taxonomy of adversarial 'agent traps' from DeepMind found web content built to hijack visiting agents, some classes succeeding more than eighty percent of the time. The same paper names the deeper problem an 'accountability gap': when a hijacked agent does harm, no one can yet say who answers.

Moving the line too fast is its own failure. Klarna replaced the work of some seven hundred service agents with an AI that, on the volume metrics, looked triumphant. Millions of chats, two-thirds of queries handled. The quality told the other story: satisfaction fell on the hard, human cases; the CEO conceded the company had gone too far; it began rehiring. The lesson is not that automation failed. It is that the boundary was drawn by the cheap metric instead of the true one.

The strategic edge.

There is a strong case against drawing lines at all. Every boundary, the argument runs, is training wheels: a patch on what the model cannot yet do. Waymo drove only surface streets until it didn't: freeways came in late 2025; tasks that needed a person last year don't this year. Design your product around today's line and you will be out-designed by whoever bets on the line moving. And the line has always moved.

The case is half right, and knowing which half is the job. There are two kinds of lines, and the industry's failure mode is treating them as one. Capability lines are drawn by what the model cannot yet do, and the critics own them: they retreat every quarter, and defending one is defending a snapshot. Redrawing them is not defeat; it is maintenance. Accountability lines are drawn by something else: who answers when it goes wrong, what a person is owed when a system acts on them, what law and trust require. No benchmark moves them, because someone specific lives on the other side of each one: the grieving passenger, the customer whose hard case the bot fumbled, the stranger in the labelling queue. It is why Instant Checkout keeps the customer confirming every step: not because a model cannot click a button, but because the yes has to belong to someone. Klarna's mistake now has a name: it read an accountability line as a capability line. Capability tells you where the line can go. It never tells you where it should.

This is also where the edge is. As models commoditise, so does automation: everyone accesses the same primitives and can remove the same people at the same cost. What cannot be copied is the trust a well-placed accountability line earns: not a feature but a sequence of judgements, made by someone who understood the work.

What endures.

Underneath it is an old instinct. We have always wanted, when a system acts on us, a findable human inside it. Not because the person is more accurate, but because a person can be asked, can answer, can change their mind. Every gain in capability leaves that instinct exactly where it was.

So here is the part worth holding onto. The model is rented; your competitor runs the same one. What makes a product itself is its character: what it will and will not do, who it is for, how honest it is about its own limits. This is the set of lines drawn through it: where the machine is free, where it is fenced, where a human stands and answers. Those lines are not the plumbing. In an AI-native product, they are the heart.