Confidence

Every answer carries two numbers from 0 to 1: confidence and answer_confidence. They are on the same scale only for a Noul.

  • confidence on a Choice or a Score is one minus the normalised entropy of probabilities, the full distribution: it says how concentrated that distribution is over every option or level asked (an option a memory rule excluded is not counted). Concentrated on one outcome means a confident answer; spread out means an uncertain one.
  • confidence on a Noul is the larger of noul and 1 − noul: the probability of the answer given, yes or no. It is not an entropy, so it is on another scale. A noul at 0.9 has a confidence of 0.9; a choice between two options at 0.9 and 0.1 has 0.53 (arithmetic: 1 − 0.325 / ln 2). Do not compare a choice's and a noul's confidence against one threshold.
  • answer_confidence is the probability of the answer given, for every type: a choice's chosen option, a score's most likely level, and for a noul the larger of noul and 1 − noul (there it equals confidence). It is always the top value of the probabilities in the same answer.

When Jepela combines answers (option orders, two rounds, windows), matches a choice, applies a tenant calibration, or has the second tier answer (quality balanced or best), it recomputes both from the final probabilities, on the same scales. Whether a confidence of 0.9 means "right nine times in ten" is a calibration property. TypeSafe claims it for Jev. For Jepela it was checked on one constructed set of 60 support messages: calibration error 0.17 (jepela-english) and 0.34 (jepela-typed-decisions) on held-out folds, 0.13 and 0.10 after a fitted temperature (measured through the gateway, 2026-09-23). Fit it on your own labels (Quality); until then, treat confidence as a ranking signal: higher is more certain, and set thresholds from your own observed error rates.

Three tiers

answer = r.answers["intent"]
if answer.confidence >= 0.9:
    act(answer.choice)                      # automatic
elif answer.confidence >= 0.6:
    act_after_confirmation(answer.choice)   # ask the user, or log for review
else:
    route_to_human()

The 0.9 and 0.6 are a starting point, not a measurement. Scale the threshold with the consequence: a read-only action can run at 0.6, a destructive one wants 0.9 or more. Start conservative, then set the thresholds from GET /v1/quality, which gives per question a probability threshold for 90% accuracy chosen on your labels, and whether thresholds chosen that way reached 90% on labels they were not chosen on. That threshold is a probability of the answer given, so compare it with answer_confidence (answer.answer_confidence in both SDKs), not with a choice's or a score's confidence.

What the memory does to confidence

Memory lines change the distribution, and so the confidence, in both directions. In the escalation cookbook (2026-09-23) two lines about a priority account moved the escalate answer from 0.20 to 0.82. memory.compare shows both distributions so you can see it. Lines can also move answers just by being there, in either direction: five neutral lines moved an escalation answer from 0.46 to 0.28 (measured 2026-09-23). Keep memories short and check with compare and placebo.