How AI Decides: Temperature, Thresholds, and Compounding Error

Part three of the AI foundations series. We’ve covered what AI is and how models take in information. Now: how does a model get from patterns to an actual output?

Let me kill a misconception first. When we say AI “decides,” it isn’t deciding the way you do. It’s calculating probabilities from patterns.

Weather prediction is a good mental model. The system sees pressure, temperature, humidity. It doesn’t understand that low pressure causes storms. It knows those patterns correlated with rain in the training data. That’s how AI decides: correlation without causation.

And critically, different model types decide in fundamentally different ways.

Generative models: temperature and sampling

For generative models, output selection is controlled by temperature plus a few sampling parameters.

• Temperature 0: near-deterministic. Ask the same question ten times, get nearly identical answers.
• Temperature 1: variety. Different phrasings, different openings. It reads as creativity; mechanically it’s sampling from a broader range of probable tokens.

Two more parameters shape the candidate pool. Suppose the model sees 1,000 possible next tokens:

• Top-k: only consider the k most likely. Top-k of 50 means the other 950 are discarded.
• Top-p: only consider tokens until their cumulative probability hits p. Top-p of 0.9 keeps adding candidates until they collectively account for 90% of the probability mass.

I had been changing these more or less by feel. The mapping turns out to be straightforward once you know what they control:

Use case	Temperature	Why
Customer service bot	Low	Consistency and reliability matter more than variety
Creative writing assistant	Higher	Variety and surprise are the product
Structured data extraction	0	You want the same parse every time

Same model, different parameters, completely different behavior. Temperature 0 also improves cost predictability, which is a nice side effect.

Classifiers: you set the threshold, not the model

For classification, fraud detection, credit scoring, medical triage, the model hands you a probability. This transaction is 75% likely to be fraud.

Here’s the part that gets missed: the model does not decide whether to block the transaction. You do. You set the threshold. Block everything over 70%? Over 85%? That threshold is the decision. The model just produced a number.

And choosing it means trading false positives against false negatives:

• Block too aggressively and you reject legitimate customers, real people trying to buy groceries or pay bills.
• Let too much through and you lose money, but the victims lose more than you do.

The model has no concept of these human costs. It gives you a probability. The ethical weighting is yours, and pretending otherwise is how teams end up shipping decisions nobody owns.

Confidence is not accuracy

This one deserves its own section because it breaks things quietly.

A model says it is 99% confident a transaction is fraud. Does that mean it’s right 99% of the time? No. It means the input strongly matches patterns it learned as fraud. If it learned the wrong patterns, it can be extremely confident and extremely wrong.

The courses gave a concrete example: a credit risk model reporting 90% confidence that someone will default, while the actual outcomes show it’s right about 70% of the time. That gap breaks everything built on top of it, risk assessment, pricing, automated decisioning.

Test model confidence against actual outcomes before you trust the numbers in production. Calibration is a separate property from accuracy, and it doesn’t come for free.

Explainability kills more projects than performance

I suspect this ends more AI initiatives than any technical limitation.

Why did the AI reject this application? Legal wants to know. Compliance needs documentation. The customer demands an answer. And most models are black boxes, they produce a decision, not a reason.

Tools exist that approximate explanations after the fact: the model weighted income heavily, credit history was the main factor. These are approximations, not true explanations of the computation.

Sometimes the requirements are strict enough that you simply cannot use a model. So before you start, ask whether the decision must be explainable. If it must be, that constraint should shape your architecture from day one, possibly toward explicit rules with a model handling only the leftovers, keeping the decision path auditable.

A pattern I liked for structured decisions: start with rules you can explain and trace, log the cases the rules miss, then add a small model only for those. Record the rule reason when a rule fires and the model reason when the model is used, and keep the threshold tunable without a redeploy.

Feedback loops build self-fulfilling prophecies

Your model makes decisions. Those decisions generate new data. That data shapes future behavior. Sounds like a virtuous cycle. It often isn’t.

A recommendation system shows users what it predicts they want. Users click those recommendations. The system learns they like those things and shows more of the same. Before long everyone is watching the same narrow slice, and the system only ever sees confirmation of its own guess.

Credit models have a nastier version. Reject applicants with a certain profile and those people go elsewhere. You never observe whether they would have been good customers. The model keeps rejecting similar profiles, and you’ve built a wall around entire customer segments, invisibly, from inside the data.

Models don’t merely degrade over time. They can actively make things worse by reinforcing their own biases.

The compounding error problem

This is the most important arithmetic in the series, and it’s the thing I’d put in front of anyone excited about chaining models or building agents.

Suppose you have a three-step pipeline. Each step is 90% accurate. Ninety percent sounds solid.

0.9 × 0.9 × 0.9 = 0.729

73%. Three decisions, a 27% error rate.

Go five steps deep and you’re at 59%. The intuition is that these systems get smarter by working together. What actually happens is that uncertainty compounds. Each step adds noise, and far enough down the chain the output approaches noise.

This doesn’t mean don’t build pipelines. It means:

• Know your per-step accuracy, and multiply it out honestly.
• Put validation between steps rather than trusting the chain.
• Keep chains short where you can.
• Insert human checkpoints where the cost of a wrong final answer is high.

The deterministic-but-unpredictable paradox

These systems are simultaneously deterministic and unpredictable. At temperature 0, the same input yields the same output, deterministic. But we don’t know which patterns the model learned, so we can’t predict what that output will be, unpredictable.

This is essentially why prompt engineering exists. We’re reverse-engineering the learned patterns to trigger the behavior we want. As one course put it, it’s like navigating a maze blindfolded by memorizing which turns worked before.

Boundaries are not optional

Given all of the above, probabilistic decisions, miscalibrated confidence, feedback loops, compounding error, you cannot deploy AI without boundaries:

• Business rules that take over when the model suggests something outside acceptable ranges.
• Human review for high-stakes decisions.
• Monitoring to catch drift before your users do.

The through line in the case studies was always the same: the teams that got AI into production understood and respected the limitations. They used models where models excel and built guardrails everywhere else.

That’s not timidity. Just because a model can decide doesn’t mean it should. In security you threat-model before you build, you ask how this could go wrong and start there. The same discipline applies here, and the question “what happens when this is confidently wrong?” belongs in the design review, not the incident retro.

Next: the thing all of this depends on, data. Quality, representation, bias, and the drift that starts the moment you ship.