Datadog Event Forwarding
Ship the app's metrics, logs, and deploy events to Datadog with tags that stay consistent.
What it adds
A single outbound telemetry path to Datadog, with an agreed tag set, batching, sampling, and redaction applied before send.
What your agent is told to do
5
What your agent is told to do
5-
1
Decide first what is worth sending — the handful of metrics, log streams, and deployment markers someone would actually look at during an incident — and route all of them through one outbound path rather than scattering calls across the codebase.
-
2
Fix a small, mandatory tag set applied to everything: service, environment, version, and where the app is multi-tenant, an opaque tenant identifier. Set them from the same configuration the deploy uses, so a dashboard filter works the same across metrics, logs, and traces.
-
3
Treat every tag value as a cardinality decision. User identifiers, request identifiers, URLs with embedded values, and raw error messages must not become tag values; put them in the log body or trace attributes where they belong.
-
4
Buffer and batch outbound telemetry through the app's existing background processing, with a bounded queue that drops oldest-first when it fills. Telemetry must never be sent inline on the request path or hold a user request open.
-
5
Redact before the payload is constructed, not after: credentials, tokens, authorisation headers, payment details, and personal data must never reach the buffer. Keep the API credential in server-side configuration and never in a browser bundle.
Edge cases it handles
8
Edge cases it handles
8- High-cardinality tags are the standard way this integration becomes expensive overnight, because each distinct value creates a new time series. Enforce an allowed tag key list and reject values that look unbounded.
- Payloads also have size limits. Truncate long log bodies and stack traces at a defined length with a marker, rather than letting an oversized payload be rejected wholesale and the event lost.
- Inconsistent service, environment, version, or tenant metadata makes correlation impossible — a metric tagged one way and a log tagged another cannot be joined. Derive all four from one place and assert their presence before send.
- Secrets and personal data leak in through log lines and exception messages more often than through deliberate fields. Redact at the source and default to excluding unknown structured fields.
- A noisy loop or a failing endpoint will flood the pipeline and the bill. Sample repetitive events at a defined rate and aggregate counters locally before shipping them rather than sending one call per occurrence.
- When Datadog is unreachable, the app must degrade to local logging and continue serving users. Do not retry synchronously, do not grow the buffer without limit, and do not let a telemetry failure surface as a user-facing error.
- Backoff must be applied on rate limiting and on server errors, with jitter, or every instance will retry in lockstep and prolong the outage.
- Losing telemetry is acceptable; losing user requests is not. Make the dropped-event count itself an observable number so silent loss is visible.
Definition of done
8
Definition of done
8- All outbound telemetry leaves through a single path with a mandatory service, environment, version, and tenant tag set.
- Tag keys are restricted to an allowed list and unbounded values are rejected before send.
- No credential or personal data is present in forwarded metrics, logs, or events, verified against a captured payload.
- Sending is batched off the request path and never delays or fails a user request.
- Noisy events are sampled or aggregated, and oversized payloads are truncated rather than dropped whole.
- A provider outage falls back to local logging with a bounded buffer, backoff with jitter, and a visible dropped-event count.
- The feature matches the existing design system.
- No existing functionality is broken.
Related features
AI Cost Budgets
AI Cost Budgets
Cap what AI features are allowed to spend before the bill arrives.
What it does
Monetary spending limits on AI work, scoped by workspace, feature, and time period, enforced before a run starts.
How it works
- 1 Find every place the app calls a model and route all of them through one accounting point that records estimated and actual spend against a scope. A budget that only covers the chat feature is not a budget.
- 2 Estimate the cost of a run from the size of its input before dispatching it, and refuse anything that would exceed the remaining budget on its own.
- 3 Reserve the estimate against the budget when the run starts, then reconcile to the real usage figures when it finishes, releasing whatever was over-reserved.
Copy the prompt
No account needed
Add this feature to my app:
https://addthisfeature.com/x/ai-cost-budgets
Prompt Versioning
Prompt Versioning
Tie every AI output to the exact prompt version that produced it.
What it does
Immutable, numbered versions of each prompt, with the run configuration recorded and every output stamped with the version used.
How it works
- 1 Make every publish create a new immutable version rather than overwriting the previous text. Editing history in place destroys the only record of what produced last month's outputs.
- 2 Capture the whole run configuration with each version, not just the wording: which model tier and parameters were used, which tools were available, and the expected output shape. A prompt that behaves differently under different settings is not one prompt.
- 3 Stamp every generated output with the version identifier that produced it, and keep that stamp with the record so an output found later can be traced back to its exact instructions.
Copy the prompt
No account needed
Add this feature to my app:
https://addthisfeature.com/x/prompt-versioning
Retrieval Debugger
Retrieval Debugger
Show exactly which sources, chunks, and scores produced a given AI answer.
What it does
A per-answer inspector showing the query as issued, the filters applied, the candidate chunks with their scores, and what reached the model.
How it works
- 1 Capture for each answer the query as it was issued, the filters applied, the candidates returned with their scores, and which of those actually made it into the request after the context ceiling was applied.
- 2 Show results after permission filtering, with a count of how many candidates were excluded and why. Displaying the pre-filter set turns the debugger into a way to read content the viewer cannot open.
- 3 Present each scoring stage separately — keyword, semantic, and any reranking — because a chunk that ends up first overall may have been rescued by one stage after being buried by another, and a single blended number hides that.
Copy the prompt
No account needed
Add this feature to my app:
https://addthisfeature.com/x/retrieval-debugger
How it works
-
1
Copy the link
Grab the Markdown instruction URL for this feature.
-
2
Give it to your AI
Paste it into Claude Code, Cursor, v0, Lovable — whatever you build with.
-
3
It inspects, then implements
Your agent reads your existing app first, then adds the feature to fit it.
Works with your stack
These instructions are written to adapt. They tell the agent to detect your framework, match your existing design system, and reuse what you already have — rather than assuming a particular stack.
Need it tighter than that? Customize the feature and tell it exactly what you're running.