Models, Tools, Spans and PromptsPro

Four pages that compare things across runs instead of inside one. They share the header's environment and time filters, and every row ends in the exact trace and span behind it.

All four need the Pro or Enterprise plan. Free plans get an explanation of what the page would show.

#Models

Model and provider pairs compared by calls, failures, reported tokens, recorded cost, latency and time to first token.

• Sort to find the expensive one, the failing one or the slow one.
• Open a model for its summary and zero-filled trends.
• Retained provider-error groups lead straight into the exact trace and span that produced them.

Unknown timing and pricing, approximate percentiles, Other grouping and the raw history limit are stated on the page rather than hidden.

#Tools

Tool reliability, sorted by failure rate by default — the order that matches what you came to find out.

• Calls, failures, p95 and average reported attempts per tool.
• Retained top errors and recent calls, each opening the trace step with its input and output.
• Tool trends over the range.

"Average reported attempts" counts the attempts recorded inside one tool span, so a tool with retries: 2 that usually succeeds first time reads close to 1. See tool().

#Spans

The explorer for individual steps across every run, with two tabs.

#Explorer

URL-backed filters: span kind, status, name, model/provider, tool and duration bounds. It returns matching span and error counts, approximate latency percentiles, a duration histogram and paginated rows that open the exact trace and span.

Rows and statistics use the same retained population, so the histogram always describes the rows you can see.

#Latency

Completed-run percentile comparisons and trends from aggregate histograms, plus span-kind, model/provider and tool groups compared by operation duration. The slowest retained runs open on their longest operation rather than the root wrapper.

#Prompts

A prompt is the system prompt a model call carried, grouped by its opening line; each distinct text below that line is a version. Nothing is registered — identity travels on every generation span, so prompts, versions, usage and cost exist whether or not bodies were captured.

The list gives the 20 most-called prompts with failure rate, cost, p95 and average score. Opening one gives you:

• Every stored version, with the selected version's text.
• A line diff between any two versions.
• The recent calls behind them, opening the exact trace and span.

Details that matter when reading it:

• Interpolated dates and ids are masked before hashing, so a timestamp in the prompt does not make every call its own version.
• A prompt that still changes on every call is marked dynamic at 50 versions.
• Lifetime counters survive raw expiry and are shown apart from the retained 24-hour window.
• With bodies off there is no text to show or diff — the version identity is still there.
• Nightly retention clears a version's text once its last call leaves the plan window.

SDK prompt registration and prompt management are not built yet.

#Getting back where you were

Comparison ordering and investigation filters are preserved when you drill into a trace and return, and they live in the URL — so an investigation is something you can bookmark or paste into an incident channel, not something you have to reconstruct.