Webhook Delivery Log and Retry
Show exactly what you sent, what came back, and how to send it again safely.
What it adds
An outbound webhook pipeline with signed payloads, automatic retries with backoff, a per-attempt log, and manual replay.
What your agent is told to do
5
What your agent is told to do
5-
1
Record every delivery attempt: event ID, endpoint, request body, response status, response body excerpt, duration, and attempt number.
-
2
Sign each payload with a per-endpoint secret and include a timestamp in the signed material so receivers can reject replays.
-
3
Retry failed deliveries with exponential backoff and jitter, cap the attempts, then move the delivery to a dead-letter state the user can see and act on.
-
4
Include a stable event ID on every attempt so receivers can deduplicate, and let users replay any past event on demand.
-
5
Do NOT retry on a 4xx that indicates the receiver rejected the payload outright — retrying a 400 forever just burns the endpoint. Retry timeouts, connection failures, 429, and 5xx.
Edge cases it handles
6
Edge cases it handles
6- Support two active signing secrets during rotation so receivers can cut over without dropped deliveries.
- Redact tokens, keys, and personal fields from the stored request and response bodies; the log is a support tool, not a secret store.
- Retries arrive out of order by definition — the payload must carry enough state, or a sequence number, for the receiver to resolve that.
- A consistently failing endpoint should be auto-disabled after a threshold, with the owner notified, rather than retried indefinitely.
- Cap and time out the response body you store; a receiver returning a 10MB HTML error page must not fill the log table.
- A manual replay of a very old event must be marked as a replay, not presented as a fresh delivery.
Definition of done
8
Definition of done
8- Every attempt is logged with status, timing, and attempt number.
- Payloads are signed with a timestamped signature and rotation is supported.
- Failures retry with exponential backoff and land in a dead-letter state after a cap.
- The same event ID is preserved across all attempts of a delivery.
- Users can replay any logged event, and replays are marked as such.
- Stored request and response bodies are redacted and size-capped.
- The feature matches the existing design system.
- No existing functionality is broken.
Related features
AI Cost Budgets
AI Cost Budgets
Cap what AI features are allowed to spend before the bill arrives.
What it does
Monetary spending limits on AI work, scoped by workspace, feature, and time period, enforced before a run starts.
How it works
- 1 Find every place the app calls a model and route all of them through one accounting point that records estimated and actual spend against a scope. A budget that only covers the chat feature is not a budget.
- 2 Estimate the cost of a run from the size of its input before dispatching it, and refuse anything that would exceed the remaining budget on its own.
- 3 Reserve the estimate against the budget when the run starts, then reconcile to the real usage figures when it finishes, releasing whatever was over-reserved.
Copy the prompt
No account needed
Add this feature to my app:
https://addthisfeature.com/x/ai-cost-budgets
Prompt Versioning
Prompt Versioning
Tie every AI output to the exact prompt version that produced it.
What it does
Immutable, numbered versions of each prompt, with the run configuration recorded and every output stamped with the version used.
How it works
- 1 Make every publish create a new immutable version rather than overwriting the previous text. Editing history in place destroys the only record of what produced last month's outputs.
- 2 Capture the whole run configuration with each version, not just the wording: which model tier and parameters were used, which tools were available, and the expected output shape. A prompt that behaves differently under different settings is not one prompt.
- 3 Stamp every generated output with the version identifier that produced it, and keep that stamp with the record so an output found later can be traced back to its exact instructions.
Copy the prompt
No account needed
Add this feature to my app:
https://addthisfeature.com/x/prompt-versioning
Retrieval Debugger
Retrieval Debugger
Show exactly which sources, chunks, and scores produced a given AI answer.
What it does
A per-answer inspector showing the query as issued, the filters applied, the candidate chunks with their scores, and what reached the model.
How it works
- 1 Capture for each answer the query as it was issued, the filters applied, the candidates returned with their scores, and which of those actually made it into the request after the context ceiling was applied.
- 2 Show results after permission filtering, with a count of how many candidates were excluded and why. Displaying the pre-filter set turns the debugger into a way to read content the viewer cannot open.
- 3 Present each scoring stage separately — keyword, semantic, and any reranking — because a chunk that ends up first overall may have been rescued by one stage after being buried by another, and a single blended number hides that.
Copy the prompt
No account needed
Add this feature to my app:
https://addthisfeature.com/x/retrieval-debugger
How it works
-
1
Copy the link
Grab the Markdown instruction URL for this feature.
-
2
Give it to your AI
Paste it into Claude Code, Cursor, v0, Lovable — whatever you build with.
-
3
It inspects, then implements
Your agent reads your existing app first, then adds the feature to fit it.
Works with your stack
These instructions are written to adapt. They tell the agent to detect your framework, match your existing design system, and reuse what you already have — rather than assuming a particular stack.
Need it tighter than that? Customize the feature and tell it exactly what you're running.