Build a Decision Model Before You Automate the Decision

TL;DR: The useful move is not “let AI decide,” it is writing down the decision model first, then using AI only where it improves inputs, speed, or review.

What does “build your own decision model” actually mean?

The primary source here is the Hacker News (AI) item titled “Build your own decision model.” The provided material is thin, just the title and listing. No author, no method, no claims, no benchmark. So I’m treating it as a prompt, not as evidence for a specific tool or paper.

Still, the phrase is worth sitting with.

Most teams skip the decision model and jump straight to automation. They ask an LLM to rank leads, approve content, triage support tickets, pick vendors, qualify candidates, or summarize risk. Then they are surprised when the model behaves inconsistently.

But the missing part is usually not intelligence. It is policy.

A decision model is the explicit shape of judgment. What matters? What disqualifies an option? Which signals are trusted? Which are weak proxies? Where do you need a human override? What level of uncertainty is acceptable? What should be logged so the decision can be reviewed later?

That is different from training a machine learning model. You can build a decision model in a spreadsheet, a rules file, a scoring rubric, a checklist, a small app, or a workflow engine. AI may sit inside it. It should not silently become it.

Where should AI sit in the decision loop?

I like AI most in the parts of decisions that are high-volume, text-heavy, and annoying, but not final.

For example, AI can extract facts from long documents, group similar support tickets, summarize objections in sales calls, compare contract language against a playbook, or draft a first-pass rationale. These are input and explanation jobs. They speed up the work around a decision.

The actual decision logic should be much more boring.

If a customer refund over a certain threshold requires review, write that down. If a lead is disqualified when the company is outside the service area, make that a hard rule. If a content workflow rejects claims without a cited source, enforce it outside the model. If the AI says “high priority,” require the system to show which criteria triggered that label.

This is where a lot of AI workflow demos get slippery. They show the output, not the governance. They show the agent doing the work, not the line between suggestion and authority.

That line matters. Especially when the decision touches money, access, hiring, healthcare, legal exposure, or user trust.

What would I build first?

Start with one decision you already make often. Not the hardest one. Not the most political one. Pick something repetitive enough to measure and bounded enough to inspect.

Write the current decision in plain English. Then turn it into a small rubric with three parts: required inputs, decision criteria, and review triggers. Add a place for “unknown” instead of forcing every case into yes or no. That one field prevents a lot of fake confidence.

Then run historical cases through it. Do not start with AI. First ask whether humans agree with the rubric. If they do not, the problem is not automation yet. It is alignment.

Once the rubric survives that pass, add AI carefully. Use it to pre-fill fields, extract evidence, suggest a score, or explain tradeoffs. Compare its work against known outcomes and human review. Track boring errors: missed facts, overconfident summaries, bad edge cases, inconsistent reasoning, and input sensitivity.

The point is not to worship determinism. Real decisions often involve judgment. The point is to make the judgment visible enough that AI can assist without becoming an unreviewable black box.

A builder should try this on one workflow this week: take a recurring decision, write the rubric, add explicit stop conditions, then let an LLM fill in the evidence while a human keeps authority. The catch most readers miss is that the decision model is the product. The AI call is just one component inside it.