Where AI belongs in a product, and where it doesn't

There is a lot of pressure right now to add AI to products, and most of it is untethered from any specific problem. "We should have AI in this" is a positioning statement, not a product decision.

The framework I actually use has two axes: what it costs when the output is wrong, and who is accountable for the result.

The cost-of-being-wrong axis

Some tasks tolerate error gracefully. If a model suggests five tags for a document and two are irrelevant, the user ignores two and moves on. The cost is a second of attention. The task degrades smoothly.

Other tasks fail catastrophically at the edges. If a model assigns a final grade, or approves an expense, or decides which of two people is contacted first — being wrong is not a minor annoyance, it is the whole failure. The task does not degrade smoothly; it breaks.

AI belongs comfortably in the first category. In the second, it belongs only in a specific supporting role, described below.

The accountability axis

The second question is whether a human is answerable for the output. Not "does a human review it" — whether someone's name, job, or professional standing is attached to the result.

Where accountability is real, the model can prepare but must not decide. This is not a hedge; it is the only structure that actually works, because an accountable person will not accept responsibility for a decision they did not make and cannot inspect.

The four quadrants

Low cost, no accountability → automate freely.

Categorization, tagging, extracting structure from unstructured text, generating first-draft copy, semantic search, summarizing a long thread. Ship it, let the user correct it, don't over-engineer the safeguards.

Low cost, accountable → assist and attribute.

Drafting a client email that a person will send under their own name. The model writes; the human reads, edits, and owns. Make editing easy and make the AI-generated state visible.

High cost, no accountability → be careful about silent failure.

Anomaly detection, monitoring, flagging outliers. The risk here isn't a wrong answer, it's a missed one, which nobody notices. Design for recall over precision and make the flagging behaviour legible.

High cost, accountable → prepare, never decide.

This is the interesting quadrant, and it is where the good work is.

GradeSteward sits squarely here. Rubric-based academic assessment has high stakes and unambiguous accountability — the instructor's name is on the mark, and they must be able to defend it to a student, a department, and an appeal process. An AI that assigns grades is unusable in that environment regardless of accuracy, because the instructor cannot stand behind a judgment they did not make.

So the model does something different. It gathers the evidence — locating where in a submission each rubric criterion is addressed. It proposes a provisional mark with the reasoning attached. It drafts feedback in the instructor's register. And then it stops. Every academic decision stays with the instructor, who now starts from a prepared position instead of a blank page.

The value isn't automation. It's that the expensive, tedious part — assembling evidence — is done, and the part requiring judgment is where the human's time actually goes.

Three tests before adding an AI feature

Can you describe the failure mode to a user in one sentence?

"It sometimes suggests a tag that doesn't fit" is fine. If you cannot articulate what going wrong looks like, you do not understand the feature well enough to ship it.

Is the output inspectable?

Can the user see why? Not model internals — the evidence. "This is flagged because these three transactions exceed the threshold" is inspectable. A number with no provenance is not, and users correctly refuse to trust it.

Is correction cheaper than doing it manually?

If fixing a bad AI output takes longer than doing the task from scratch, the feature is net negative even at 80% accuracy. This is the test most AI features quietly fail.

The other place AI belongs

Worth separating: AI as a development multiplier is a different thing entirely, and it has fewer caveats. Using AI throughout research, design, implementation, and testing shortens the path from problem to working product enough that focused custom software becomes economically viable for problems that previously could not justify a build.

That is a genuine change in what is worth building — and it doesn't require putting a single AI feature in front of a user. The two uses are independent, and conflating them leads to products with AI bolted on because AI was used to build them.

Adding AI is a product decision like any other. It earns its place by making a specific task better, or it doesn't go in.