AI Confidence and Uncertainty
Show when an AI result is well supported and when it needs a human to look.
What it adds
An uncertainty signal on AI outputs, derived from evidence quality, that changes what the workflow does next.
What your agent is told to do
5
What your agent is told to do
5-
1
Derive the signal from things the app can observe: how much supporting evidence was found, whether sources conflict, whether required fields came back empty, and whether the output matched the expected structure. Do not derive it from the model's own self-assessment, which is unreliable and tends to be confident about fabrications.
-
2
Express the result as a small number of named bands, such as well supported, partial, conflicting, and needs review. Bands are honest about the granularity available; a percentage implies a calibration nobody has measured.
-
3
Make the band change behaviour, not just appearance. Low-confidence results stay as drafts requiring approval, route into a review queue, or block downstream automation, while well-supported ones can flow through under the app's existing rules.
-
4
Show the reason for the band on the surface itself, in words, and let the user open the supporting evidence. Provenance and the links themselves are owned by AI Source Citations; this feature owns the judgement about how well that evidence holds up.
-
5
Do not bury the signal in a tooltip or a hover state. Uncertainty that only appears when someone goes looking for it will be missed by exactly the users who most needed the warning.
Edge cases it handles
8
Edge cases it handles
8- A raw probability presented as a calibrated confidence score is misleading, because nothing in the pipeline has been calibrated against outcomes. Use bands with stated meanings instead of an invented number.
- The band must be tied to concrete causes such as missing evidence, sources that disagree with each other, or an ambiguous mapping onto the expected output shape, and the interface must name the cause rather than showing a bare label.
- Uncertainty hidden behind a tooltip, an icon, or a hover is functionally invisible on touch devices and to screen reader users, and must be part of the visible text.
- If the band does not change what the workflow does, it is decoration. A result marked as needing review must actually be prevented from being applied automatically.
- Users must be able to open the evidence behind any band, including the case where the evidence is thin, because the absence of support is itself the finding.
- A model refusal, a timeout, or a truncated response is not low confidence, it is no result. Show it as a failure with a retry, not as a hedged answer.
- Structured output that fails validation must land in the lowest band and never be applied, even when the prose around it reads convincingly.
- The band must be stored with the result, so a record reviewed later shows the uncertainty it carried at the time rather than being re-scored against today's data.
Definition of done
9
Definition of done
9- Every AI result carries one of a small set of named uncertainty bands.
- The band is computed from observable evidence quality, not from the model's self-report.
- The stated reason for the band is visible in text, without hovering.
- Low-confidence results are held as drafts or routed for review and cannot trigger downstream automation.
- Supporting evidence is inspectable from every result, including results with little support.
- Refusals, timeouts, and invalid structured output are presented as failures rather than uncertain answers.
- The band is persisted alongside the result and shown on later review.
- The feature matches the existing design system.
- No existing functionality is broken.
Related features
Prompt Injection Defense
Prompt Injection Defense
Stop instructions hidden in documents, pages, and tool output from steering the AI.
What it does
A boundary between the app's own instructions and untrusted content, backed by enforced tool permissions and confirmation for consequential actions.
How it works
- 1 Enumerate every route by which content the app did not author reaches a model: uploaded files, retrieved passages, fetched pages, records from connected accounts, and the results of tool calls.
- 2 Keep the app's own rules in the trusted portion of the request and enclose untrusted content in a clearly delimited region labelled as material to be analysed rather than instructions to be obeyed.
- 3 Enforce what the model may reach outside the model. Tool permissions, tenant scoping, and rate limits must hold even when the model has been completely persuaded — an instruction telling the model not to delete things is not a control.
Copy the prompt
No account needed
Add this feature to my app:
https://addthisfeature.com/x/prompt-injection-defense
AI Import Column Mapping
AI Import Column Mapping
Guess how an uploaded file's columns line up with the app's fields, then ask.
What it does
A proposed mapping from uploaded columns to destination fields, shown with samples and confirmed before the import runs.
How it works
- 1 Take the uploaded headers and a small sample of rows and propose a mapping to the destination fields. Match on exact and near-exact header names first and only consult the model for the columns that remain unresolved.
- 2 Constrain the candidate set to fields the current user and the importer are actually permitted to write. A mapping that targets a read-only, computed, or privileged field must never be offered, whatever the header says.
- 3 Show the proposal as a table: source column, sample values, destination field, confidence, and the transformation that will be applied. A user cannot approve a mapping they cannot see the effect of.
Copy the prompt
No account needed
Add this feature to my app:
https://addthisfeature.com/x/ai-import-column-mapping
AI Audit Trail
AI Audit Trail
Keep a reviewable record of what the AI saw, produced, recommended, and actually did.
What it does
A per-run record of inputs, sources, configuration, output, approvals, and executed actions, restricted and retained by purpose.
How it works
- 1 Write one record per run covering the feature invoked, the configuration and prompt version in force, the sources retrieved, the tools called, the output produced, who approved it, and what was carried out.
- 2 Distinguish a draft from an executed action explicitly in the record. A trail that shows only what text was produced cannot answer the question that matters in a review, which is whether anything happened.
- 3 Redact at the moment of writing. Strip credentials, tokens, and payment details before the record is stored, and keep only the personal content genuinely needed to understand the decision.
Copy the prompt
No account needed
Add this feature to my app:
https://addthisfeature.com/x/ai-audit-trail
How it works
-
1
Copy the link
Grab the Markdown instruction URL for this feature.
-
2
Give it to your AI
Paste it into Claude Code, Cursor, v0, Lovable — whatever you build with.
-
3
It inspects, then implements
Your agent reads your existing app first, then adds the feature to fit it.
Works with your stack
These instructions are written to adapt. They tell the agent to detect your framework, match your existing design system, and reuse what you already have — rather than assuming a particular stack.
Need it tighter than that? Customize the feature and tell it exactly what you're running.