Prompt Caching
Cut input costs with automatic prefix caching
Prompt caching discounts repeated input tokens automatically. Long, stable prefixes — system prompts, retrieved documents, conversation history, few-shot examples — are billed at the cache-read rate instead of the full input rate, with savings up to 90% depending on the model.
How it works
| Step | What happens |
|---|---|
| 1. First request | Input tokens are billed at the normal input rate; reusable prefixes are cached upstream. |
| 2. Repeat request with the same prefix | The cached span is billed at the cache-read rate (a fraction of input price). Only the new suffix is billed at full rate. |
| 3. Usage reporting | The response usage separates cached tokens so you can verify savings. |
TipNo opt-in and no parameters — caching is automatic and works best when the prefix is byte-identical across requests. Even a changed whitespace or timestamp in the system prompt breaks the cacheable span.
Pricing example
For a model charging $1.75/1M input with an 81% cache discount, the cache-read rate is roughly $0.33/1M. A 20K-token stable prefix reused across 1,000 requests drops from ~$35.00 of input cost to ~$6.64 — before counting any output.
Checking a model's cache discount
Each model page shows the cache-read price and discount in the pricing table; the catalog API exposes it directly:
bashcurl -s https://tokenhub.coality.io/api/public/models/glm-5-3 | jq '{price, cache_discount_pct}'
Designing for cache hits
| Do | Avoid |
|---|---|
| Put static content first (system prompt, docs), dynamic content last | Timestamps, request ids, or random seeds inside the prefix |
| Keep one canonical system prompt per workload | Re-formatting the same instructions per request |
| Reuse long few-shot examples verbatim | Paraphrasing examples between calls |