The most seductive promise in artificial intelligence is that we will no longer have to struggle with difficult work. The most useful promise is narrower: we can struggle with better things.
There is cognitive work that deserves to disappear. Reformatting the same information, searching through a long document for one clause, generating routine variations, or turning rough notes into a clean outline can consume energy without producing much judgment. Assistance here gives time back.
But difficulty is not always waste. The attempt to frame a problem, choose between competing values, remember why an exception exists, or explain an idea in your own language is often the work itself. Remove that effort indiscriminately and the output may improve while the person becomes less able to produce it.
A good AI product must know the difference between relieving toil and displacing agency.
Capability is the product
The obvious measure of an assistant is the quality of what appears in the chat. A stronger measure is what the person can do after the chat is closed.
Did they acquire a mental model? Can they recognize the pattern next time? Do they understand the trade-off behind the recommendation? Could they defend the conclusion to someone who disagrees? If the answer vanishes when the assistant is unavailable, the system delivered an artifact but may not have created capability.
This does not mean every interaction must become a lesson. Sometimes people need a translation, a draft, or a calculation and nothing more. The point is that the product should not make dependence its default growth strategy. It should be willing to explain, expose structure, and hand the work back.
The best outcome is occasionally that the user does not need to ask the same question again.
Fluency is not judgment
Language models are extraordinarily good at producing the shape of an answer. That makes them useful, and it creates their most persistent illusion. A complete paragraph feels resolved. A confident recommendation feels considered. A numbered plan feels as though someone has tested the order.
Fluency can carry judgment, but it is not evidence that judgment occurred. The model may not know which constraint is politically sensitive, which “minor” exception contains the history of a failure, or which elegant idea will collapse when it meets a particular customer. Those facts often live outside the prompt—in experience, institutions, and consequences borne by real people.
A trustworthy assistant makes room for that missing context. It asks when a hidden choice would materially change the answer. It separates observation from inference. It identifies assumptions that deserve inspection. It does not use caveats as a ritual, but it does not smooth uncertainty away merely to sound helpful.
Intelligence is not the removal of uncertainty. It is the ability to work honestly inside it.
Four useful roles for a thinking partner
“Thinking partner” can become an empty phrase unless it changes how the product behaves. We find four roles especially useful.
Framing
Before solving a problem, the assistant can help define it. What outcome actually matters? Which constraint is fixed and which is inherited habit? What would count as a good decision? A precise frame often saves more time than a clever answer.
Divergence
AI can cheaply produce alternatives. The value is not the volume of suggestions but the expansion of the search space: a different metaphor, a counter-position, an option from another discipline, or a question the team has avoided. Divergence is most useful before the user becomes attached to the first plausible path.
Compression
Long material can be turned into a map: major claims, evidence, disagreements, decisions, and open questions. Good compression preserves the distinctions that matter. Bad compression makes everything equally smooth and loses the one inconvenient detail that should drive the decision.
Rehearsal
An assistant can challenge a plan before reality does. It can play the skeptical customer, the security reviewer, the impatient reader, or the colleague who must maintain the system in a year. This is not prediction. It is an inexpensive way to expose weak reasoning while revision is still cheap.
These roles keep the human in a meaningful position. The assistant does not simply complete the task; it improves the conditions under which the person completes it.
Friction can be a form of respect
AI interfaces often compete to feel instantaneous and effortless. Speed is valuable, but a perfectly frictionless system can slide past moments that deserve consent or reflection.
Before sending private context, applying a consequential change, publishing under someone's name, or making a decision that affects other people, a pause is not a failure of design. It is a boundary. The right confirmation can remind the user that assistance has become action.
The same principle applies to memory. Remembering preferences may make an assistant more useful. Remembering everything by default makes convenience difficult to distinguish from surveillance. People should understand what is retained, why it helps, where it travels, and how to remove it.
The Busted Minds approach—one account with product activity kept separate—is part of this philosophy. Identity can make access simpler without turning every interaction across AI, Search, Chess, and the Journal into one behavioural portrait.
The problem with invisible authorship
When AI helps produce a piece of work, the important question is not whether every sentence was typed by a human. It is whether a person remains accountable for the claims, choices, and consequences.
Invisible assistance becomes corrosive when nobody can explain the result. A team ships code it cannot maintain. A manager presents a strategy whose assumptions they never examined. A student submits an argument they would not recognize in different words. The surface quality rises while the underlying ownership disappears.
A healthier workflow keeps authorship active. Ask the assistant for options, then choose and revise. Request an explanation, then restate the reasoning. Use critique to expose gaps, then decide which gaps matter. Verification is not a ceremonial final step; it is the point where generated material becomes someone's work.
The exit test
BM AI is being built around a demanding product test: does the interaction leave the person with more agency than they brought into it?
That can mean a finished draft, but it can also mean a better question. It can mean discovering that the original plan was based on a false constraint. It can mean enough understanding to continue alone. It can even mean the assistant clearly indicating that the next step belongs with a human expert, a primary source, or a decision-maker who bears the risk.
An AI that always keeps the conversation going may be successful by the logic of engagement. A thinking partner sometimes points toward the door: here is the source, here is the structure, here is what remains uncertain, and here is what you can do next.
The ambition of AI should not be to become impossible to live without. It should be to help people become more capable of living, deciding, and creating on their own terms.
Loading the conversation…