It’s very tempting. You install a few add-ons from GitHub, Claude Code costs half as much, and you forget all about the five-hour subscription limit. The authors promise savings from 20 to 98%, and social media is full of setups that “never hit the limit”. So how come the vendor hasn’t built this miracle in? Let me introduce the best-known “prompt savers” and, above all, show how they held up in my tests…
A few days ago a typical post popped up in my feed: Claude Code, plus Ponytail, Caveman, Headroom, Context Mode, Opus for planning, Sonnet for the work, and the punchline “I completely forget about the five-hour limit”. Underneath, comments with more add-ons that “add even more”. A miracle: for 20 USD a month you can program anything. So why isn’t it in Claude Code out of the box? Are they just milking us, or is it AI slop? True to my motto “a claim without a benchmark is just an opinion”, I decided to map the situation and test the prompt compressors.
I built a standalone test on a real project (3gpp-explorer.com, a TypeScript monorepo), ran the five most frequently recommended add-ons against plain Claude Code, and had every run checked by hidden tests and an independent review. The whole harness is public, so you can repeat it with Claude Code on your own repository.
So, let’s talk about saving tokens…
What you pay for in Claude Code
Most of the bill is the context the agent carries along. On every turn, Claude Code sends the model the entire conversation so far: the system prompt, tool definitions, your CLAUDE.md, files it has read, command output. To keep this affordable, the beginning is stored in a cache on Anthropic’s side. The first cache write costs 4 USD per million tokens with Sonnet 5.5, every subsequent read only 0.10 USD. The model’s own output costs 10 USD per million.
Think of it as moving house. Writing to the cache is loading the van; reading from the cache is every further trip with the van already loaded. If your trips are short, you mostly pay for the loading.
In my measurements, writing context to the cache made up 45 to 76% of the cost of a run, and reading another 10 to 29%. Model output was the rest, and of that, the reply text you read in the terminal was only about 1% of the cost (my estimate from character counts). Most of the output is code written to files and the model’s thinking.
This is the key to the whole article. An add-on that shortens replies is touching one percent of the bill. An add-on that shortens command output is touching a part of the context the agent often filters on its own anyway. And every add-on adds something to the start of every session, which is exactly the most expensive item. That’s the whole problem…
Which add-ons I tested
The five add-ons that show up most often in recommendations:
- Caveman is a skill (a set of instructions for the agent) that makes Claude answer telegraphically, like a caveman. It promises around 68% fewer output tokens.
- Ponytail is a skill that makes the agent write as little code as possible: it first looks for a ready-made solution in the standard library or the platform. It promises −26% cost.
- RTK (Rust Token Killer) is a small program that intercepts shell commands and shortens their output before the agent reads it. It promises up to 90% less shell output.
- Context Mode is an MCP server (an extension with its own tools) that processes large outputs in an isolated process and lets only a summary into the conversation. It promises 98% context savings.
- Headroom is a local proxy between Claude Code and the API that compresses the whole context. On a subscription it may only use lossless compression. It promises 21–57% depending on the scenario.
Four tasks from the project’s real history: a short API bug fix, a refactoring across ten files, a new feature with a page, and a fix to data extraction over 40 MB of HTML specifications and JSON output. I added the last task later to give RTK, Context Mode and Headroom the playing field they are built for.
Every task and add-on combination ran three times, in random order, each time in a clean container, with Sonnet 5.5 at medium thinking effort. Results were verified by hidden tests the agent never saw and by a double review from Codex, which didn’t know which add-on had run. In total, 72 runs in the main test and 12 runs in the CLAUDE.md experiment, worth 35 USD at list prices.
Promises versus measurements
One caveat: each author’s promise measures something different. Caveman counts output tokens, RTK shell output, Context Mode the amount of data in the context. I measure cost, because that’s what you pay (and subscription limits are calculated from the same tokens).
Only Ponytail saved clearly and repeatedly. By 18% on the refactoring, 21% on the new feature and 38% on the big-data task. On the short bug fix practically nothing; there’s nothing to shorten there. The mechanism is visible in the data: the agent wrote between a fifth and almost two thirds less code, needed fewer turns and read less. This matches an independent JetBrains test, which measured −10% cost on 80 tasks.
The other four add-ons didn’t change the cost on average. Caveman +5%, RTK −4%, Context Mode −1%, Headroom −2%. Individual tasks diverge in both directions by single digits to tens of percent, but with three runs per cell I can’t tell that apart from noise. I only consider a direction reliable if it repeats across tasks and has an explainable cause. Only Ponytail meets that condition.
What each add-on actually did
Caveman shortened the text. The bill stayed the same. Replies were 3 to 26% shorter, but total output tokens were not (on three tasks out of four there were more of them). Reply text is about one percent of the cost, and Caveman’s rules add 3,800 tokens to the start of every session. On the data task it came out 13% more expensive. JetBrains measured −8.5% output tokens for it in their test and an 11.6% higher cost over the whole run. That fits.
RTK reports savings that don’t show up on the bill. On the data task it intercepted commands and credited itself with savings of 5,600 to 11,000 tokens per run. Yet the cost came out 5% higher and the volume of output in the context didn’t shrink. There are two reasons. The agent filters test output itself (with tail and grep), and a large part of the context is file reads through the built-in tool, which RTK doesn’t touch at all.
The agent never used Context Mode once. Zero calls to its tools in twelve runs, including the task with 40 MB of data. The installation was complete, and in the first three tasks the add-on nudged the agent 8 to 24 times per run to use its tools. The agent preferred the built-in tools, most likely because that’s what it is trained on. In exchange, Context Mode added 14,500 tokens to the start of every session, the most of all. On the short bug fix it came out 24% more expensive; on the other three tasks, by contrast, 4 to 17% cheaper. Its embedded guide offers an explanation: it seems to have steered the agent toward more economical work even without its own tools. With three runs, though, I can’t tell that apart from noise.
Headroom does its own accounting. According to its own statistics, it compressed 1.7 to 4% of request tokens and credited itself with savings of 6.5 to 26% of the cost. The measured change averages −2%. On a subscription it may only use lossless compression, and that doesn’t save much in agentic work.
Context Mode, Caveman and Ponytail share a property you won’t find in their READMEs: an entry tax. Every session with them starts 1,600 to 14,500 tokens longer, and those tokens are written to the cache at the write rate, forty times more expensive than a read. If the add-on doesn’t earn back that tax, it makes things more expensive. RTK adds only a few hundred tokens, and Headroom actually shortens the start of the session slightly.
And one more practical tip: don’t trust the add-on’s counter. Both RTK and Headroom reported savings that didn’t show up in the cost. Measure the cost of the whole session.
40 MB of data, and the agent still didn’t need to read it
Task number 4, with the big data, was meant to give RTK, Context Mode and Headroom a chance. The agent was fixing the extraction of a glossary (a dictionary of terms in new 3GPP mobile network specifications) that was silently losing definitions, and it had 303 HTML specifications, a 1.5 MB JSON output from the faulty run and its log at its disposal. The cause couldn’t be found without working with the data.
In every configuration the agent read the data in small pieces: grep, line ranges, short scripts printing counts. At most 60,000 characters out of the 40 MB made it into the context. The largest single tool output was reading the program’s source code. No run hit the context limit.
So the output-shortening add-ons had nothing to shorten; in my test the agent filtered the bulky data itself. (By the way, the agents found a third real cause of the bug in the data, one the project’s original fix didn’t address.)
What I didn’t test: output the agent can’t or won’t filter. A browser page snapshot, a crash dump, a long test log it needs in full. That’s where RTK and Context Mode may help, and it’s exactly the kind of case their authors talk about.
The savings sitting in your own CLAUDE.md
While preparing the test, I noticed that the project had slimmed down its own CLAUDE.md between the starting commits of the tasks. The starting state of the first task had 67 KB, the others only 7 KB. So I ran the same tasks with the file swapped.
With a 7 KB CLAUDE.md, the short bug fix got a third cheaper, from 0.36 to 0.24 USD, and all three runs came out almost identical. That’s more than any add-on achieved on the same task. And it can be calculated to the cent: the smaller file shortened the start of the session by 29,500 tokens, which at a cache write price of 4 USD per million is exactly 0.12 USD. Back to the moving analogy: that’s what it cost to load boxes the agent didn’t need for the fix.
For the refactoring and the new feature I tried the opposite direction, the 67 KB file instead of the 7 KB one. It came out +9% and +1%, but you can’t read anything from those numbers. The big file made both cache writes and reads more expensive. But the older CLAUDE.md didn’t ask the agent for documentation, so the agent did less work and wrote a quarter less output. Two opposing effects nearly cancelled each other out.
What follows with certainty: you pay for CLAUDE.md at the start of every session, roughly 0.12 USD per 30,000 tokens with Sonnet 5.5. If you run many short tasks or many parallel agents, you pay for that loading many times over. Go through it and throw out what the agent doesn’t need for every task. Details belong in separate files that CLAUDE.md just links to.
Quality: a cheaper solution can also be a worse one
The hidden tests passed in 71 of 72 runs of the main test. The first three tasks were easy for Sonnet 5.5 and didn’t distinguish quality between the add-ons. Only the data task revealed differences.
With Ponytail, the review found a real bug in one run that the hidden tests don’t cover: in certain cases the fix skipped adjacent definition sections. The tests passed, but according to the review the result would lose data in production (I didn’t verify this by running it). A smaller diff also means fewer edge cases handled. Ponytail will save you money, but you MUST read its code.
One Headroom run failed the tests (it let noise into the output), and for RTK the review repeatedly rated correctness lower. With three runs these are examples, though; I don’t draw any ranking from them.
How to measure so it means something
During the analysis I ran into a trap that anyone testing savings on their own will fall into. The cache is shared between sessions on the same account and lasts an hour. If another run with the same prompt prefix ran before yours, the first turn reads cheaply instead of writing expensively. The same task with the same add-on thus cost 0.15 USD once and 0.52 USD the next time. Without correcting for this, my data would have shown Context Mode making the short fix 207% more expensive.
So I compare on the “cold price”, as if every run started with an empty cache. That corresponds to the first session of the day, or to work with longer breaks. If you’re testing an add-on yourself, run the same task several times with breaks longer than an hour; otherwise you’re mostly measuring whatever ran before you.
The main limitations of the test: three runs per combination show a direction. You can’t read exact percentages from them. One project, one model, non-interactive mode. The results apply to the add-on versions as of October 2026, and those change weekly.
One useful yardstick for subscribers: 54 overnight runs worth 18 USD at list prices moved my five-hour limit by about 8 percentage points and my weekly limit by one.
The takeaway
| Situation | What makes sense |
|---|---|
| Your agent writes a lot of new code | Ponytail, around 20% savings, but check the result |
| Many short tasks or parallel agents | trim CLAUDE.md; in the test, a third of the cost of a short fix |
| You’re on a subscription and hitting limits | only add-ons that shorten the agent’s work; the rest tend to eat into limits |
| You work with large files and logs | measure first; the agent filters data itself |
| Output the agent can’t filter (browser, crashes) | RTK or Context Mode are worth your own test |
| You want shorter replies | Caveman, for readability; the bill won’t go down |
I’d keep two things from this test. Ponytail, but only for tasks that produce a lot of new code, and with closer reading of its diff. And a cleanup of your own CLAUDE.md files, because that’s the cheapest saving of all and you don’t have to install anything for it.
I’d uninstall the rest of the setup from the opening post. You pay for Caveman and Context Mode in every session, and on regular work you won’t get it back. Headroom does shorten the start of a session slightly, but it didn’t save anything measurable, and it puts another proxy in the path to the API that can fail. The author of the post probably forgets about the limit for other reasons (my guess would be a big plan).
Next time: agent orchestration and multiple sessions. And a warning up front: ten parallel agents mean ten loadings of the whole context into the cache. It certainly won’t be a token saving; the question is how much speed you get for it.
Sources and validity: measurements were taken on October 19–20, 2026 in Claude Code 2.1.303 with Sonnet 5.5. Versions tested: Caveman 4.0.1, Ponytail 5.3.2, RTK 1.0.1, Context Mode 1.0.169, Headroom 0.41.0. Prices per Anthropic’s official pricing. Independent measurements: JetBrains, tests of Ponytail and Caveman. Harness, data and scripts: cc-token-bench. Add-ons change fast; treat the numbers as the state of October 2026.
