Health, releases and trends

Three views of the same question — is production healthy right now, was this deploy worse than the last one, and is anything drifting over months.

#Production health

A card at the top of the dashboard covering a fixed trailing 24 hours.

Window

The last 24 hours, always

Refreshes

Every 30 seconds, while the tab is visible

Resolution

One minute

Healthy minutes is the first number, and it is deliberately not called uptime. Of the minutes in which your agent actually ran something, it is the share where every run succeeded. The card shows the denominator — "1,203 of 1,210" — because a project doing ten runs a day has about ten qualifying minutes, and a percentage without its denominator would read as confidence it has not earned. When nothing ran at all the card says so, rather than reporting 100%.

Beside it are the success rate over the window, throughput as runs per minute averaged across the whole 24 hours including the idle ones, and p95 duration from the same merged histograms the dashboard uses. Below is the requests bar chart, and the count of alerts firing or acknowledged right now — the same number the alerts strip above it shows, read the same way, so the two can never disagree.

There is no range control. The minute-level aggregates behind it are kept for 48 hours, and an hourly bucket cannot answer "how many minutes had traffic".

#Releases

A release is whatever your code put in the release field — a version, a git SHA, a deploy id. Set it once when you create the client and every run carries it; see what gets recorded.

The Releases page lists every release the project has seen, newest first, with when it was first and last seen, its traces, failure rate, p95 and cost per run. Each row offers Compare with previous against the release it replaced in the same environment.

Two limits worth knowing before you plan around them:

• Windows are whole hours. A release first seen at 10:45 is measured from 10:00. The page shows the window it actually read, and a release seen in a single instant still measures a full hour, so its figures can include other runs of the same release in that hour.
• There is no per-release model, tool or provider breakdown, and there will not be one without a change to how data is stored. The release is recorded on the run, not on each model call inside it, so "which model got slower in v42" is a question the aggregates cannot answer. Compare p95 between releases to see that something got slower, then open the traces to see what.

#Trends

On the same page, one metric — failure rate, p95, cost per run or traces — across 30 or 90 days, with a marker wherever a release was first seen. That is the view for "did the last deploy make it worse", and for drift slow enough that no alert would ever fire on it.

You can compare two releases or two date ranges; both show the same measures side by side with the change on each. Lower failure rate and lower p95 are improvements; more traces is neither, just more traffic.

#Through the API

All of these need metrics:read. Health is part of the dashboard family; releases and trends are their own.

Releases live at https://api.tracehatch.com/api/v1/projects/{projectId}/releases.

GET /projects/{projectId}/health/summary for the card, /projects/{projectId}/releases/compare?base=&target= for two releases, /projects/{projectId}/trends?metric=&range=30d for the chart and its markers, and /projects/{projectId}/trends/compare for two ranges. Shapes are in the HTTP API reference.

#Not built yet

Comparing releases across environments, per-release breakdowns by model or tool, and anomaly detection that finds a regression without being asked. A release is also the one dimension with no cardinality limit: put a value that changes per request in that field and you will multiply the stored aggregates for that project, so keep it to something that changes per deploy.