Model Stability

Query real-time model stability metrics including success rate, latency, and throughput.

Overview

OpenModel tracks stability metrics for all models based on real production traffic. These metrics help you evaluate model reliability when choosing which model to use.

Data is derived from actual billed API requests — no synthetic probes or artificial tests. Metrics are aggregated hourly via ClickHouse materialized views with zero overhead on the request path.

The stability data is publicly accessible without authentication, rate-limited by IP address.

API Endpoints

All Models Summary

GET https://api.openmodel.ai/web/v1/model-stability/summary

Returns a 24-hour stability summary for all models. Each model appears as a single aggregated row — no time series.

Query parameters:

Parameter	Type	Default	Description
`hours`	integer	24	Time window in hours (currently only `24` is supported)

Response:

{
  "success": true,
  "data": [
    {
      "model_name": "claude-sonnet-4-6",
      "success_rate": 99.85,
      "avg_latency_ms": 1234,
      "avg_tps": 87.3,
      "confidence": "high"
    },
    {
      "model_name": "gpt-5.4",
      "success_rate": 99.92,
      "avg_latency_ms": 890,
      "avg_tps": 102.5,
      "confidence": "high"
    }
  ]
}

Single Model Detail

GET https://api.openmodel.ai/web/v1/model-stability/:modelKey

Returns time-series stability data and an aggregated summary for a specific model.

Path parameters:

Parameter	Type	Description
`modelKey`	string	Model identifier (e.g., `claude-sonnet-4-6`, `gpt-5.4`)

Query parameters:

Parameter	Type	Default	Description
`hours`	integer	24	Time window in hours (currently only `24` is supported)

Response:

{
  "success": true,
  "data": {
    "model_name": "claude-sonnet-4-6",
    "confidence": "high",
    "summary": {
      "success_rate": 99.85,
      "avg_latency_ms": 1234,
      "avg_ttft_ms": 380,
      "avg_tps": 87.3
    },
    "series": [
      {
        "ts": 1717459200,
        "success_rate": 100.0,
        "avg_latency_ms": 1100,
        "avg_ttft_ms": 350,
        "avg_tps": 90.0,
        "confidence": "high"
      }
    ],
    "updated_at": 1717462800
  }
}

The series array contains one data point per hour within the requested time window. Each data point has its own confidence level since traffic volume can vary significantly between hours.

Metrics

Metric	Description	Calculation
Success Rate	Percentage of requests returning 2xx status codes	`success_count / request_count × 100`
Avg Latency	Mean total request duration (including full streaming transfer)	`total_latency_ms / request_count`
Avg TTFT	Mean time to first token (streaming requests only)	`ttft_sum_ms / ttft_count`
TPS	Average output tokens per second (throughput)	`output_tokens / (generation_ms / 1000)`

Confidence Levels

Confidence indicates how reliable the metrics are based on the sample size within the time window:

Level	Condition	Meaning
`high`	≥ 100 requests	Metrics are statistically reliable
`medium`	≥ 10 requests	Metrics are directionally useful but may fluctuate
`low`	< 10 requests	Insufficient data — metrics may not be representative

When confidence is low, health status labels are not displayed.

Health Status

Models are assigned a health status based on their success rate:

Status	Success Rate	Description
Operational	≥ 99.9%	Fully healthy, no issues detected
Healthy	≥ 99%	Minor issues, generally reliable
Degraded	≥ 95%	Noticeable degradation, some requests failing
Unstable	< 95%	Significant issues, consider alternative models
No Data	0 requests	No traffic recorded in the time window

Data Source & Limitations

Metrics are based on billed requests only. Unbilled failures (typically client-side errors like invalid API keys or malformed requests) are not included, so success rates may be slightly higher than the true value.
Upstream stability-related failures (5xx errors, timeouts, rate limits) are generally billed and thus reflected in the metrics.
The raw request count is not exposed in the API response for privacy reasons — only rates, averages, and confidence levels are returned.
Data is retained for 180 days, matching the underlying consumption log retention period.

Model Stability

On this page