# AI Confidence and Uncertainty

## Objective

Show when an AI result is well supported and when it needs a human to look.

An uncertainty signal on AI outputs, derived from evidence quality, that changes what the workflow does next.

## Before You Begin

This feature is being added to an application that already exists and already
works. Do not scaffold a new project, and do not assume a blank slate.

Inspect the codebase first and establish:

- The existing application structure and where code of this kind already lives.
- The framework and version in use.
- The existing design system — colours, spacing, typography, and component conventions.
- Existing UI components you can reuse instead of writing new ones.
- The existing database structure, if this feature needs to persist anything.
- The existing authentication and authorization system, if this feature is user-scoped.
- Dependencies already installed, so you don't add a library that duplicates one.
- The existing test setup and conventions.

Only start writing code once you understand the above. If the application
already implements part of this feature, extend it rather than replacing it.

## Implementation Instructions

1. Derive the signal from things the app can observe: how much supporting evidence was found, whether sources conflict, whether required fields came back empty, and whether the output matched the expected structure. Do not derive it from the model's own self-assessment, which is unreliable and tends to be confident about fabrications.
2. Express the result as a small number of named bands, such as well supported, partial, conflicting, and needs review. Bands are honest about the granularity available; a percentage implies a calibration nobody has measured.
3. Make the band change behaviour, not just appearance. Low-confidence results stay as drafts requiring approval, route into a review queue, or block downstream automation, while well-supported ones can flow through under the app's existing rules.
4. Show the reason for the band on the surface itself, in words, and let the user open the supporting evidence. Provenance and the links themselves are owned by AI Source Citations; this feature owns the judgement about how well that evidence holds up.
5. Do not bury the signal in a tooltip or a hover state. Uncertainty that only appears when someone goes looking for it will be missed by exactly the users who most needed the warning.

## UI and UX Requirements

Match the application's existing design system exactly. Reuse its components,
spacing, and typography. This feature should look like it was always there.

## Responsive Requirements

Works on mobile, tablet, and desktop. Touch targets are large enough to hit on a
phone, and nothing overflows horizontally at 320px.

## Accessibility Requirements

- Fully keyboard navigable.
- Correct semantic elements and ARIA roles.
- Visible focus states.
- Meets WCAG AA contrast.
- Dynamic changes are announced to screen readers.
- Respects prefers-reduced-motion.

## Edge Cases

- A raw probability presented as a calibrated confidence score is misleading, because nothing in the pipeline has been calibrated against outcomes. Use bands with stated meanings instead of an invented number.
- The band must be tied to concrete causes such as missing evidence, sources that disagree with each other, or an ambiguous mapping onto the expected output shape, and the interface must name the cause rather than showing a bare label.
- Uncertainty hidden behind a tooltip, an icon, or a hover is functionally invisible on touch devices and to screen reader users, and must be part of the visible text.
- If the band does not change what the workflow does, it is decoration. A result marked as needing review must actually be prevented from being applied automatically.
- Users must be able to open the evidence behind any band, including the case where the evidence is thin, because the absence of support is itself the finding.
- A model refusal, a timeout, or a truncated response is not low confidence, it is no result. Show it as a failure with a retry, not as a hedged answer.
- Structured output that fails validation must land in the lowest band and never be applied, even when the prose around it reads convincingly.
- The band must be stored with the result, so a record reviewed later shows the uncertainty it carried at the time rather than being re-scored against today's data.

## Testing

Exercise the feature end to end in the running application. Cover every edge case
above, then run the existing test suite and confirm nothing regressed.

## Acceptance Criteria

- [ ] Every AI result carries one of a small set of named uncertainty bands.
- [ ] The band is computed from observable evidence quality, not from the model's self-report.
- [ ] The stated reason for the band is visible in text, without hovering.
- [ ] Low-confidence results are held as drafts or routed for review and cannot trigger downstream automation.
- [ ] Supporting evidence is inspectable from every result, including results with little support.
- [ ] Refusals, timeouts, and invalid structured output are presented as failures rather than uncertain answers.
- [ ] The band is persisted alongside the result and shown on later review.
- [ ] The feature matches the existing design system.
- [ ] No existing functionality is broken.

## Adaptation Rules

- Match the existing design system. Do not introduce a new colour palette,
  spacing scale, or component library.
- Reuse existing components and utilities wherever they fit.
- Follow the naming, file layout, and code style already present.
- Do not upgrade, replace, or remove existing dependencies to make this
  feature fit. Adapt the feature to the app, not the app to the feature.
- Do not break existing functionality. If a change is genuinely required in
  existing code, make the smallest one that works and say so.
- If something in these instructions conflicts with how the application is
  built, follow the application and explain the deviation.

## Final Verification

Before you report the work as done:

1. Re-read the acceptance criteria above and check each one against what you
   actually built.
2. Run the application and exercise the feature end to end.
3. Run the existing test suite and confirm you have broken nothing.
4. Check the feature on mobile, tablet, and desktop widths.
5. Check keyboard navigation and focus handling.
6. Summarize what changed: files added, files modified, and anything you
   deliberately did differently because of how this application is built.

If any acceptance criterion is unmet, fix it before reporting completion.