# Vector Index Health

## Objective

Show operators whether each knowledge source is complete, delayed, stale, or failing.

An operator view of index state per source: expected versus indexed counts, embedding age, queue depth, and failures.

## Before You Begin

This feature is being added to an application that already exists and already
works. Do not scaffold a new project, and do not assume a blank slate.

Inspect the codebase first and establish:

- The existing application structure and where code of this kind already lives.
- The framework and version in use.
- The existing design system — colours, spacing, typography, and component conventions.
- Existing UI components you can reuse instead of writing new ones.
- The existing database structure, if this feature needs to persist anything.
- The existing authentication and authorization system, if this feature is user-scoped.
- Dependencies already installed, so you don't add a library that duplicates one.
- The existing test setup and conventions.

Only start writing code once you understand the above. If the application
already implements part of this feature, extend it rather than replacing it.

## Implementation Instructions

1. Record the expected document and chunk counts for every source alongside what is actually indexed, so a shortfall shows as a number on a screen rather than being discovered through a missing answer.
2. Stamp every stored embedding with the content version and the embedding configuration that produced it, and mark it stale when either changes. Without that stamp there is no way to tell current vectors from ones generated under an older setup.
3. Present queue delay and permanent failure as different states with different actions. A backlog clears itself and needs patience; a failing source needs a person, and merging the two hides real breakage behind a spinner.
4. Offer a re-index that builds the new vectors and swaps them in for a source, retiring the old set as part of the same operation. Do not re-index by writing fresh vectors and leaving the previous ones in place, because duplicates skew retrieval scoring.
5. Restrict this view to operators. It exposes document titles, counts, and raw error text across tenants. Per-answer inspection of which chunks produced a specific result belongs to Retrieval Debugger.

## UI and UX Requirements

Match the application's existing design system exactly. Reuse its components,
spacing, and typography. This feature should look like it was always there.

## Responsive Requirements

Works on mobile, tablet, and desktop. Touch targets are large enough to hit on a
phone, and nothing overflows horizontally at 320px.

## Accessibility Requirements

- Fully keyboard navigable.
- Correct semantic elements and ARIA roles.
- Visible focus states.
- Meets WCAG AA contrast.
- Dynamic changes are announced to screen readers.
- Respects prefers-reduced-motion.

## Edge Cases

- Expected and indexed counts must be compared at both the document and the chunk level, since a document can be present but only partly chunked.
- Changing the embedding configuration invalidates every existing vector at once. Flag the whole affected set as stale and show the scale of the re-index before anyone starts it.
- A queue that is merely deep looks identical to one that is stuck. Show the age of the oldest waiting item and the rate of progress so the difference is visible.
- Re-indexing must be safe to run twice. A second run started because the first looked slow must converge on one set of vectors, not two.
- Error text from an embedding provider or an external source can contain customer content, so the whole surface must be behind operator permission and excluded from general logging.
- A full re-index costs money and consumes provider rate limit. Show an estimate before it runs, make it interruptible, and keep the existing index serving queries throughout.
- A source deleted while a sync is running will leave counts that never reconcile. Reconcile against the source's current state rather than the state at the time the run started.
- A source with genuinely zero documents is healthy, not broken, and must read differently from a source whose ingestion produced nothing.

## Testing

Exercise the feature end to end in the running application. Cover every edge case
above, then run the existing test suite and confirm nothing regressed.

## Acceptance Criteria

- [ ] Every source shows expected versus indexed document and chunk counts.
- [ ] Embeddings carry the content version and configuration they were produced under, and stale ones are flagged.
- [ ] Queue delay, in-progress work, and permanent failure are distinct states with distinct actions.
- [ ] Re-indexing a source replaces its vectors without leaving duplicates, and is safe to run more than once.
- [ ] Queries keep being served from the existing index while a re-index runs.
- [ ] The view and its error text are restricted to authorized operators.
- [ ] The feature matches the existing design system.
- [ ] No existing functionality is broken.

## Adaptation Rules

- Match the existing design system. Do not introduce a new colour palette,
  spacing scale, or component library.
- Reuse existing components and utilities wherever they fit.
- Follow the naming, file layout, and code style already present.
- Do not upgrade, replace, or remove existing dependencies to make this
  feature fit. Adapt the feature to the app, not the app to the feature.
- Do not break existing functionality. If a change is genuinely required in
  existing code, make the smallest one that works and say so.
- If something in these instructions conflicts with how the application is
  built, follow the application and explain the deviation.

## Final Verification

Before you report the work as done:

1. Re-read the acceptance criteria above and check each one against what you
   actually built.
2. Run the application and exercise the feature end to end.
3. Run the existing test suite and confirm you have broken nothing.
4. Check the feature on mobile, tablet, and desktop widths.
5. Check keyboard navigation and focus handling.
6. Summarize what changed: files added, files modified, and anything you
   deliberately did differently because of how this application is built.

If any acceptance criterion is unmet, fix it before reporting completion.