AddThisFeature

Vector Index Health

Show operators whether each knowledge source is complete, delayed, stale, or failing.

involved Developer Experience

What it adds

An operator view of index state per source: expected versus indexed counts, embedding age, queue depth, and failures.

What your agent is told to do

5
  1. 1

    Record the expected document and chunk counts for every source alongside what is actually indexed, so a shortfall shows as a number on a screen rather than being discovered through a missing answer.

  2. 2

    Stamp every stored embedding with the content version and the embedding configuration that produced it, and mark it stale when either changes. Without that stamp there is no way to tell current vectors from ones generated under an older setup.

  3. 3

    Present queue delay and permanent failure as different states with different actions. A backlog clears itself and needs patience; a failing source needs a person, and merging the two hides real breakage behind a spinner.

  4. 4

    Offer a re-index that builds the new vectors and swaps them in for a source, retiring the old set as part of the same operation. Do not re-index by writing fresh vectors and leaving the previous ones in place, because duplicates skew retrieval scoring.

  5. 5

    Restrict this view to operators. It exposes document titles, counts, and raw error text across tenants. Per-answer inspection of which chunks produced a specific result belongs to Retrieval Debugger.

Edge cases it handles

8
  • Expected and indexed counts must be compared at both the document and the chunk level, since a document can be present but only partly chunked.
  • Changing the embedding configuration invalidates every existing vector at once. Flag the whole affected set as stale and show the scale of the re-index before anyone starts it.
  • A queue that is merely deep looks identical to one that is stuck. Show the age of the oldest waiting item and the rate of progress so the difference is visible.
  • Re-indexing must be safe to run twice. A second run started because the first looked slow must converge on one set of vectors, not two.
  • Error text from an embedding provider or an external source can contain customer content, so the whole surface must be behind operator permission and excluded from general logging.
  • A full re-index costs money and consumes provider rate limit. Show an estimate before it runs, make it interruptible, and keep the existing index serving queries throughout.
  • A source deleted while a sync is running will leave counts that never reconcile. Reconcile against the source's current state rather than the state at the time the run started.
  • A source with genuinely zero documents is healthy, not broken, and must read differently from a source whose ingestion produced nothing.

Definition of done

8
  • Every source shows expected versus indexed document and chunk counts.
  • Embeddings carry the content version and configuration they were produced under, and stale ones are flagged.
  • Queue delay, in-progress work, and permanent failure are distinct states with distinct actions.
  • Re-indexing a source replaces its vectors without leaving duplicates, and is safe to run more than once.
  • Queries keep being served from the existing index while a re-index runs.
  • The view and its error text are restricted to authorized operators.
  • The feature matches the existing design system.
  • No existing functionality is broken.

Related features

How it works

  1. 1

    Copy the link

    Grab the Markdown instruction URL for this feature.

  2. 2

    Give it to your AI

    Paste it into Claude Code, Cursor, v0, Lovable — whatever you build with.

  3. 3

    It inspects, then implements

    Your agent reads your existing app first, then adds the feature to fit it.

Works with your stack

These instructions are written to adapt. They tell the agent to detect your framework, match your existing design system, and reuse what you already have — rather than assuming a particular stack.

Need it tighter than that? Customize the feature and tell it exactly what you're running.