Back to blog
6 min read

Practical AI Systems Start with Narrow Jobs and Clear Failure Modes

AIsystemsproduct

The useful shape of an AI feature

A practical AI system is usually not a model plus a prompt. It is a small decision pipeline with a few carefully chosen responsibilities:

  • accept a bounded input
  • produce a bounded output
  • know when it is uncertain
  • route uncertainty somewhere safe
  • preserve enough context to debug what happened

That sounds obvious, but many AI features still fail because they try to do too much at once. They ask a model to infer intent, invent structure, choose policy, and write the final user-facing answer in one shot. The result may look fluent, but it is hard to operate. When the output is wrong, there is no clean way to tell whether the issue came from the input, the prompt, the retrieval layer, the model, or the product policy.

The practical alternative is to treat AI as one component in a larger system, not the system itself.

Narrow the job before you widen the model

The first design question is not “Which model should this use?” It is “What exact job is the system doing?”

Good candidates are narrow tasks with clear success criteria:

  • classifying a support ticket into a fixed set of buckets
  • extracting fields from a document
  • summarizing a long thread into a few decision-relevant bullets
  • drafting a response that a human will review before sending
  • ranking a shortlist of possible matches

These tasks share a useful property: the system can be wrong without causing the whole product to become ambiguous. If a classifier is uncertain, it can choose “needs review.” If extraction fails, the system can return incomplete data rather than fabricating a value. If a summary is weak, the original source is still available.

This is especially important in product work, where the temptation is to turn every AI feature into a conversation. Conversation is flexible, but flexibility is expensive. It blurs the line between user intent and system behavior. For many workflows, a form, a button, or a constrained action is more useful than an open-ended chat box.

Design for uncertainty as a normal state

AI systems are probabilistic. That does not mean they are unreliable by default; it means uncertainty should be a first-class state, not an exception.

A well-designed system should answer three questions every time it runs:

  1. What should happen when confidence is high?
  2. What should happen when confidence is low?
  3. What should happen when the input is malformed or incomplete?

If those paths are not designed up front, the system will improvise. In practice, improvisation often means hidden failure: a model guesses, a downstream service accepts the guess, and the product silently absorbs the mistake.

A better pattern is to make uncertainty visible in the interface and in the logs.

For example, a document extraction flow can return:

  • structured data fields
  • a confidence score or quality signal
  • a list of missing or ambiguous fields
  • a fallback state such as “review required”

That allows the product to choose the right next step. A finance workflow may require human review. A content workflow may accept a draft. A search workflow may simply rank the result lower.

A concrete example: AI-assisted invoice intake

Consider an invoice intake workflow.

The goal is not to let a model “understand invoices” in the abstract. The goal is to move invoice data into a downstream accounting process with minimal manual correction.

A practical design could look like this:

1. Ingest and normalize

The system accepts a PDF or image, then converts it into a consistent internal representation. This step is non-AI in the strict sense, but it matters because models perform better when the input shape is stable.

2. Extract only the required fields

Instead of asking for every possible detail, the system requests a small schema:

  • vendor name
  • invoice number
  • invoice date
  • total amount
  • currency
  • due date

The narrower the schema, the easier it is to validate.

3. Validate against rules

The output is checked against deterministic rules:

  • dates must parse
  • totals must be numeric
  • currency must be recognized
  • duplicates must be flagged by invoice number and vendor

If the model returns a plausible but invalid value, the system should reject it rather than patch it silently.

4. Route uncertainty

If the model cannot confidently read the invoice number, the system does not guess. It marks that field as missing and routes the document to human review.

5. Keep provenance

The downstream user needs to know where each field came from. That means storing references to the source document and, where possible, the exact regions or text spans used in extraction.

This design is boring in the best way. It does not depend on perfect model behavior. It turns a fuzzy task into a controlled workflow with explicit handoffs.

Contracts matter more than clever prompts

In practical AI systems, the real leverage often comes from contracts:

  • input schemas
  • output schemas
  • validation rules
  • retry policies
  • escalation paths
  • access boundaries
  • audit trails

Prompts still matter, but they are not the whole system. A clever prompt can improve output quality; it cannot by itself define what should happen when the output is incomplete, inconsistent, or unsafe for the next step.

This is where product thinking and systems thinking meet. The product question is not “Can the model do it?” The systems question is “What happens when it doesn’t?”

That shift changes architecture. It encourages smaller steps, narrower scopes, and more explicit failure handling. It also makes the system easier to evolve. If the extraction schema changes, the validation layer makes the change visible. If a better model arrives later, it can be swapped in without redesigning the workflow from scratch.

Where Bitcoin thinking helps

Bitcoin infrastructure is often designed around explicit state, deterministic transitions, and conservative assumptions about failure. Those habits translate well to AI systems.

A practical AI workflow benefits from the same discipline:

  • keep the state machine visible
  • minimize hidden dependencies
  • make transitions explicit
  • assume inputs can be adversarial, malformed, or incomplete
  • require clear rules for retry, rejection, and finalization

That does not mean AI systems should be rigid. It means they should be legible. When the state is explicit, operators can reason about the system. When the system is legible, product teams can improve it incrementally instead of relying on vague prompt edits.

The practical takeaway

If an AI feature is hard to test, hard to explain, and hard to recover when it fails, it is probably too broad.

A better starting point is to define:

  • one narrow job
  • one output contract
  • one uncertainty path
  • one human fallback
  • one source of truth for state

That structure will not make the system magical. It will make it usable. And for most product work, usable systems matter more than impressive demos.