Usage, cost and memory analytics¶

Open Observability → Usage & cost (/traces/usage). The page opens on the last 30 days. Choose 24 hours, 7, 30 or 90 days, all recorded usage, or a custom start and end in UTC. From top to bottom it shows:

Known spend, tokens, model calls, spend per run, latency p95 and prompt-cache reuse. Each figure shows its change from the same length of time just before, when that period has calls.
The monthly budget (see Monthly budget).
Tokens, known spend or model calls over time. Hover over or focus a bar for its figures, and click it to open that period.
Usage by agent and by model. Click a name to filter the page to it. A model without a price links to its rate card.
Runs by spend and tokens, each linking to its trace.
Memory supplied to prompts and retrieval time, then the provider prompt cache and how the figures are counted.

Filter by agent and model. Under More, choose the timezone for day boundaries and how many events to read. Trace, thread and turn filters stay on as you change the range. Export downloads CSV by day, agent, model or run, or the full JSON. CSV cells that a spreadsheet would run as a formula start with a quote. /traces/usage.json and /traces/usage.csv cover all recorded usage unless given window (24h, 7d, 30d, 90d, all or custom with start and end) or the older start_time and end_time. Selected traces also have a collapsible Tokens, charges & memory panel for exactly that trace window. Evalground shows recorded quality, latency and API-cost estimates alongside run history.

Missing dates use an explicit Unknown date bucket; missing measurements are not inferred from response length.

What the numbers mean¶

Metric	Source and limitations
Input/output tokens	Provider response usage, including usage-bearing streaming events. Missing usage is unknown.
Calculated charge	Measured token counts multiplied by a versioned rate card, calculated with `Decimal`. Not a provider invoice.
Memory prompt tokens	Approximate `ceil(rendered_characters / 4)`, per memory type and model call. Not exact tokenizer output or billed-token allocation.
Retrieval latency	Measured wall time around instrumented memory reads, including failures. Not causal attribution of model generation latency.
Model latency	Recorded call duration; streaming follows the existing provider-stream timing contract.
Evalground cost	Benchmark-reported estimate, not invoice reconciliation. Local hardware cost is not included.

Memory types include history, episodic recall, knowledge base, entity memory, summary references, learned skills, personalization, persona style, tool-log digests and mixed evidence packs. Measurements count rendered memory text after selection, history trimming and skill/workflow filtering. System instructions (apart from the separately measured persona style), tool schemas and ephemeral request context are not classified as stored memory. Common retrieval-section headers and message framing are not allocated to individual memory types. Multimodal/non-text history is not assigned a text-token estimate.

Supplying the same memory in two model calls counts twice; retrieving it once counts as one retrieval. Cache discounts cannot reliably be allocated to each memory type from provider totals, so the UI does not invent per-type dollar charges. Mixed personalization/evidence-pack rendering stays in a mixed bucket. Host-provided personalization, explicit memory tools, compression and other uninstrumented operations may have no per-type retrieval samples. Add host instrumentation where needed; historical measurements cannot be reconstructed.

Pricing coverage¶

The offline default catalog covers standard direct OpenAI text-model pricing, verified September 7, 2026. It includes input, output, cache reads, GPT-5.6/GPT-6 cache writes and applicable long-context multipliers. Rates come from official OpenAI pricing, prompt-caching usage semantics, and the corresponding model pages, such as GPT-5.6 Terra.

Unknown providers/models/tiers or incomplete token details yield cost_usd: null and a reason. This includes local models: unconfigured pricing does not mean free hardware. OpenAI-compatible custom endpoints are identified as openai_compatible and never inherit OpenAI list pricing automatically.

Actual response service_tier is retained. If absent, the default standard tier is explicitly assumed in the quote's cost_reason. Configure separate cards for priority/flex/batch, regional premiums, other providers or contract pricing. The built-in catalog does not discover an account's region, discounts or billing contract. Tool fees, image/audio modalities, embeddings, taxes, unobserved provider retries, credits and billing reconciliation are outside these totals.

New model results persist a pricing version, date, scheme-free public source reference and calculated charge. These snapshots are not repriced when the catalog changes. Older results without a snapshot can be calculated using the currently configured catalog; the API/UI labels that basis current_catalog, not historical invoiced spend. Known charges exclude unpriced calls, with the priced/total call count shown beside every subtotal.

Developer APIs¶

No UI or network access is required:

from memorizz.observability import aggregate_usage, query_usage

# Inspect the latest turn in the same execution context as the completed run.
latest = agent.last_usage_analytics(timezone_name="Europe/London")

# Aggregate normalized events from your existing recorder/export pipeline.
summary = aggregate_usage(events, timezone_name="UTC", max_events=10_000)
print(summary["totals"]["cost_usd"])  # Decimal string, or None
print(summary["totals"]["unpriced_calls"])
print(summary["memory"])

# Query historical records. The host must authorize these identity filters.
summary = query_usage(
    provider,
    filters={
        "application_id": "workspace-7",
        "user_id": "user-42",
        "agent_ids": ["researcher"],
        "start_time": "2026-09-01T00:00:00Z",
        "end_time": "2026-09-07T23:59:59Z",
    },
    max_events=10_000,
    timezone_name="Europe/London",
)

SDK aggregation is a data utility, not an authorization boundary. Scope inputs to the authorized tenant first. The UI performs this enforcement using the existing trace principal and audit controls and serves analytics with Cache-Control: no-store.

Custom rate cards¶

from memorizz.observability import DEFAULT_PRICING, PricingRegistry, RateCard
from memorizz.ui.app import create_app

# Illustrative contract rates; replace with your actual provider agreement.
custom = RateCard(
    provider="private_provider",
    model="reader-v1",
    input_per_million="1.00",
    cached_input_per_million="0.10",
    output_per_million="3.00",
    source_url="https://example.com/public-pricing",
    as_of="2026-09-07",
    version="reader-contract-v1",
    service_tier="default",
)
registry = PricingRegistry([*DEFAULT_PRICING.cards.values(), custom])
agent.usage_pricing = registry  # Set at application setup, before concurrent runs.
app = create_app(usage_pricing=registry)

Registries and rate cards are immutable. Pass pricing=registry to either SDK aggregation function. Configure each service-tier card separately. A custom max_input_tokens can refuse unsupported context sizes; alternatively use long_context_threshold, long_input_multiplier and long_output_multiplier. Source URLs must be public HTTPS URLs without credentials, queries or fragments. Do not put sensitive contract identifiers in versions or source references.

The pricing contract expects input_tokens to include cached and cache-write tokens; output tokens include billable reasoning tokens already reported by the provider. Normalize provider-specific usage before quoting: some APIs report uncached input and cache reads/writes separately. Never double-count reasoning tokens or add provider totals to input/output totals.

Rate cards page¶

You can also price a model without code. Open Usage & cost, then Rate cards. A "calls without a price" item on the Overview links there with the provider, model and tier filled in. Enter input, cached input and output prices per million tokens and a public HTTPS page that lists them. For a model you run yourself, such as one on Ollama, tick Local model, no charge; the card cites this documentation page as its source. Cards are saved to rate_cards.json in MEMORIZZ_HOME (~/.memorizz by default) and apply at once to calls without a recorded charge. A card replaces a built-in card with the same provider, model and tier. Only an unscoped admin account can change cards. An app created with create_app(usage_pricing=...) shows its cards read-only; change them in code.

Monthly budget¶

Set a monthly budget in USD on the Usage & cost page, with an alert percentage (80 by default). The panel shows known spend since the first of the month in UTC, a progress bar and a straight-line projection to month end. When spend reaches the alert, the Overview lists it under Needs attention; past the budget, the item turns red. Spend counts only priced calls. The panel says how many calls have no price and links to Rate cards. The budget is saved to usage_budget.json in MEMORIZZ_HOME. Figures are cached for five minutes, and saving a budget or a rate card refreshes them. A scoped account sees spend for its own scope only. Only an unscoped admin account can change the budget.

Host memory instrumentation¶

The public recorder accepts content-free memory metrics:

recorder.record_event(
    "knowledge.read",
    kind="memory_retrieval",
    duration_ms=elapsed_ms,
    attributes={"memory_type": "knowledge_base"},
)
recorder.record_event(
    "knowledge.supplied",
    kind="memory_supply",
    attributes={
        "memory_type": "knowledge_base",
        "memory_chars": rendered_chars,
        "memory_tokens_estimate": (rendered_chars + 3) // 4,
        "token_estimation_method": "rendered_chars_div_4",
    },
)

Use the same tenant/agent/turn context and distinct event IDs. Emit a supply measurement per model call, but emit retrieval duration once per operation. Do not put prompts, memories, account addresses or credentials in attributes. With the model filter selected, only supply events carrying that model identity are included; pre-model retrieval timings cannot be attributed to a model and are omitted rather than guessed.

Efficiency and completeness¶

Agent instrumentation counts text already rendered for the prompt, uses perf_counter around existing reads and emits bounded scalar metadata. It does not make extra memory reads, tokenize the full prompt, or fetch price pages. Memory counters are isolated per execution context and reset every turn.

The aggregator processes a bounded iterable once, deduplicates scoped result spans, then sorts bounded groups/latency samples for tables and percentiles. It does not sum start events or parent rollups. Defaults cap aggregation at 10,000 events; the SDK hard ceiling is 100,000. The UI defaults to 5,000 and caps at 10,000. Native provider event-index queries are preferred; legacy conversation and tool stores are queried through existing bounded pagination adapters. Synchronous analytics routes run reads in FastAPI's worker pool, not its event loop. Charts are server-rendered SVG/HTML with text/table equivalents, no CDN, browser chart library, polling or new runtime dependency.

Inspect coverage.read_complete, per-store read results and coverage.aggregation_truncated before treating a daily total as a complete recorded window. A bounded page is not complete historical accounting; filter by agent/time or aggregate your authorized event-export pipeline for larger workloads. Even a complete read cannot prove uninstrumented calls never occurred. The latest-turn and legacy task/thread-memory paths explicitly leave overall read completeness unestablished.

Local verification¶

python -m pytest tests/unit/test_usage_analytics.py tests/unit/test_observability_operator_security.py
python tests/browser/usage_app.py
# In another terminal, with Node 18+ and Playwright available:
node tests/browser/usage.cjs

Browser fixtures use temporary synthetic data and require no paid provider calls. Set MEMORIZZ_PLAYWRIGHT_MODULE to a local Playwright module path if necessary.