# AI Structured Data Extraction

## Objective

Turn documents and pasted text into a validated record a person approves before saving.

A schema-driven extraction pipeline that converts unstructured input into a validated draft record with field-level evidence.

## Before You Begin

This feature is being added to an application that already exists and already
works. Do not scaffold a new project, and do not assume a blank slate.

Inspect the codebase first and establish:

- The existing application structure and where code of this kind already lives.
- The framework and version in use.
- The existing design system — colours, spacing, typography, and component conventions.
- Existing UI components you can reuse instead of writing new ones.
- The existing database structure, if this feature needs to persist anything.
- The existing authentication and authorization system, if this feature is user-scoped.
- Dependencies already installed, so you don't add a library that duplicates one.
- The existing test setup and conventions.

Only start writing code once you understand the above. If the application
already implements part of this feature, extend it rather than replacing it.

## Implementation Instructions

1. Define each target schema in the app with types, required fields, allowed values, and constraints, and validate every response against it before anything is stored. Treat the schema as the contract and the model output as an untrusted candidate.
2. Present the result as a draft the user reviews field by field, with the extracted value beside the passage it came from. Approval writes the record; nothing writes itself.
3. Distinguish three states per field: found in the source, absent from the source, and inferred. Collapsing absent into empty hides the difference between a document that omitted a value and one the extractor failed on.
4. On invalid output, attempt one bounded repair pass constrained to the failing fields, then give up and surface the document for manual entry. Do not loop retries against a token budget the workspace has to pay for.
5. Field-level extraction of names, dates, and amounts belongs to AI Entity Extraction; call it for candidate values and keep this feature responsible for schema assembly, validation, and the review flow.

## UI and UX Requirements

Match the application's existing design system exactly. Reuse its components,
spacing, and typography. This feature should look like it was always there.

## Responsive Requirements

Works on mobile, tablet, and desktop. Touch targets are large enough to hit on a
phone, and nothing overflows horizontally at 320px.

## Accessibility Requirements

- Fully keyboard navigable.
- Correct semantic elements and ARIA roles.
- Visible focus states.
- Meets WCAG AA contrast.
- Dynamic changes are announced to screen readers.
- Respects prefers-reduced-motion.

## Edge Cases

- Output that does not satisfy the schema must be rejected, not patched into shape. Silently coercing a string into a number or dropping an unexpected key produces records that look valid and are not.
- A field the source never mentioned must be recorded as absent, and a field the model reasoned toward must be marked inferred and flagged in the review interface. The user needs to know which values were read and which were guessed.
- Every field that drives money, identity, scheduling, or access needs a stored pointer to its supporting passage, so a disputed record can be checked against its source months later.
- High-impact writes — creating an invoice, updating a customer record, scheduling something billable — must pass through explicit human approval regardless of confidence.
- Documents longer than the input ceiling must be chunked with overlap and merged deterministically, with conflicts between chunks surfaced for review rather than resolved by last-write-wins.
- Scanned and image-based documents may yield unusable text. Detect that before spending a run, and tell the user the document could not be read rather than returning an empty record.
- A refusal on sensitive content is a legitimate outcome. Show it as a refusal with a manual entry path, not as an extraction failure the user will retry indefinitely.
- Exclude payment credentials, government identifiers, and health details from anything sent to the provider unless the workspace has explicitly opted in, and record what was sent for each run.

## Testing

Exercise the feature end to end in the running application. Cover every edge case
above, then run the existing test suite and confirm nothing regressed.

## Acceptance Criteria

- [ ] Every extraction is validated against a stored schema before any write occurs.
- [ ] Each field is labelled found, absent, or inferred, and the labels are visible in review.
- [ ] Fields with material consequences carry a stored reference to their source passage.
- [ ] No record is created or updated without explicit user approval of the draft.
- [ ] Invalid output triggers at most one bounded repair pass before falling back to manual entry.
- [ ] Unreadable documents and provider refusals produce distinct, actionable messages.
- [ ] Token and cost ceilings are enforced per run and per workspace.
- [ ] The feature matches the existing design system.
- [ ] No existing functionality is broken.

## Adaptation Rules

- Match the existing design system. Do not introduce a new colour palette,
  spacing scale, or component library.
- Reuse existing components and utilities wherever they fit.
- Follow the naming, file layout, and code style already present.
- Do not upgrade, replace, or remove existing dependencies to make this
  feature fit. Adapt the feature to the app, not the app to the feature.
- Do not break existing functionality. If a change is genuinely required in
  existing code, make the smallest one that works and say so.
- If something in these instructions conflicts with how the application is
  built, follow the application and explain the deviation.

## Final Verification

Before you report the work as done:

1. Re-read the acceptance criteria above and check each one against what you
   actually built.
2. Run the application and exercise the feature end to end.
3. Run the existing test suite and confirm you have broken nothing.
4. Check the feature on mobile, tablet, and desktop widths.
5. Check keyboard navigation and focus handling.
6. Summarize what changed: files added, files modified, and anything you
   deliberately did differently because of how this application is built.

If any acceptance criterion is unmet, fix it before reporting completion.