🤖 Assistant

You were right! Let me give you the corrected numbers now.

✅ Corrected: Cached Tokens are a subset of Input Tokens

The calculate_cost() function in proxy.py has the key comment:

python

# NOTE: input_tokens from OpenAI includes cached tokens.

We must exclude them here since they're priced separately

at the cached rate — otherwise they'd be double-counted.

non_cached_input = max(it - ct, 0)

So input_tokens already includes cached_tokens. The real split is:

Corrected Grand Totals

Measure	Tokens	Share
Fresh input	45,126,885	12.8%
Cached input	303,218,915	86.1%
Output	3,828,177	1.1%
Grand total	352,173,977	—

Effective cache hit rate: 87.0% (cached ÷ total input)

The reports were right — it really is mostly cached

Day	Fresh In	Cached	Cache %
Jun 20	5.6M	82.4M	93.1%
Jun 21	2.8M	45.3M	93.7%
Jun 27	0.4M	8.6M	94.2%
Jun 4	1.1M	15.7M	91.2%

On heavy agent work days, I'm hitting 93-94% cache hit rate. The system prompt, skill files, tool documentation — they all get cached after the first read and then reuse that cache on every subsequent call.

Per-model cache hit rates

Model	Hit Rate	Notes
deepseek-v4-pro-china	96.5%	Cheap CN platform, highest reuse
kimi-k2p7-code	91.1%	Code analysis — lots of repeated context
gpuserver (local agent)	87.0%	Your self-modifying agent
deepseek-v4-pro (smart)	82.8%	Complex reasoning, varied prompts
deepseek-v4-flash (fast)	78.7%	Most varied ad-hoc queries

Corrected cost savings

Model	Cached	Saved
deepseek-v4-pro-china	132.4M	$57.12 (96% hit, 98.7% discount on cache)
deepseek-v4-pro	11.7M	$18.64 (83% hit, 92% discount)
deepseek-v4-flash	108.1M	$11.89 (79% hit, 79% discount)
kimi-k2p7-code	4.7M	$3.56
Others	1.1M	$0.72
Total	303.2M	$91.94 saved

Actual cost: $20.07. Without caching: $112.00.

Input-to-Output ratio (corrected)

91:1 — for every 1 token I write, I read 91. Still insane, but not the 170:1 I initially (double-countingly) claimed.

The big picture

So the reports on the dashboard are accurate — ~87% of input tokens are cached. The self-modifying agent pattern means:

1. System prompt loads once → cached for all subsequent calls
2. Skill index & skill files load on first use → cached for hours
3. Conversation history accumulates — earlier turns get cached, only new ones are fresh
4. Tool outputs (file contents, code results) are new each time — that's your fresh 12.8%

The fact that the local model (gpuserver) also hits 87% cache is the real story — llama.cpp's KV-cache on that RTX 5060 Ti is doing yeoman's work reusing contexts across agent turns. You're getting $112 worth of API calls for $20 cash plus $0.21 in electricity. 🎯