A One-Line Read, 4,745 Tokens Saved

I asked RTK for the first line of a 1,000-line text file. It returned [1000 more lines]. Its accounting recorded 4,745 tokens saved.

The requested line contained 19 bytes. The file contained 19,000. RTK compared its output against the whole file, then credited the difference to compression. It also dropped the line I asked to read. A savings counter can look excellent while the agent gets less useful evidence.

That is the engineering problem with putting compression between a tool and a model: you have to preserve the requested observation and measure the complete task. A smaller payload proves neither by itself.

Measured bytes from a local fixture. Token counts in this figure use RTK's own byte-length estimate.

The Reproduction

I used Windows, Python 3.12 and Rust 1.96.1. The RTK checkout was version 0.48.0, commit fc19d7a77845131de51b2d11d8e7525798ccc7e2. I built the binary and ran the tests selected by head in their names:

cargo +1.96.1 build --locked
cargo +1.96.1 test --locked --bin rtk head -- --nocapture

test result: ok. 53 passed; 0 failed; 0 ignored;
0 measured; 3373 filtered out; finished in 40.73s

That selection covers several command-rewrite cases and other names containing head. It is a narrow check, not the full test suite.

The reproduction script creates a private fixture and SQLite database. It sets RTK_DB_PATH for its child processes, runs the real binary, reads the accounting row, and closes the database before cleanup. From the RTK checkout:

python ../rtk-probe.py target/debug/rtk.exe

The fixture contains line 0000 value 42\n through line 0999 value 42\n. The script uses the first newline-terminated byte sequence as the equivalent one-line baseline; it does not invoke a native Unix head executable.

rewrite exit: 3
rewrite: rtk read fixture.txt --max-lines 1
rtk stdout: "[1000 more lines]"
bytes: full_file=19000, one_line_baseline=19, rtk_stdout=17

Exit 3 belongs to RTK's permission protocol: return the rewritten command while leaving approval to the calling hook. The help text describes exits 0 and 1, while the implementation also documents 2 and 3. I inspected the rewrite and invoked the read command on my generated fixture. I did not install a hook or change permission settings.

Here is the database row, with fields unrelated to the comparison omitted:

{
  "original_cmd": "cat fixture.txt",
  "rtk_cmd": "rtk read",
  "input_tokens": 4750,
  "output_tokens": 5,
  "saved_tokens": 4745
}

rtk gain --format json reported avg_savings_pct: 99.89473684210526. On the requested 19-byte baseline, both outputs round up to five estimated tokens. The local token reduction is zero under the same estimator. The observation is worse: the first line disappeared.

Two Decisions in the Read Path

In read.rs, the implementation loads the file, applies a filter, then applies the line window. It retains the complete input as raw and logs the original command as cat, even when the caller supplies --max-lines.

The tracking.rs estimator uses ceil(byte_length / 4). Rust's string length counts bytes here. This is a rough payload estimate, with no model tokenizer or provider invoice involved.

The missing first line comes from smart_truncate: it admits ordinary lines while the kept count is below max_lines / 2, prioritizes some code structures, and reserves an omission marker. Integer division gives a budget of zero ordinary lines for --max-lines 1. My plain-text fixture ends up with the marker alone. A rewrite from head -1 gives that display heuristic a different contract. The caller asked for data at a position.

RTK also has a never_worse guard. It selects the original text if the filtered text has a larger token estimate. That protects output size. It cannot establish whether the output still answers the command.

A Structural Gate Has Limits Too

OmniRoute has substantial compression code: command detection, JSON filter definitions, deduplication, renderers, raw-output pointers and request-body adapters. Its TypeScript RTK engine is a gateway implementation; I did not treat it as the Rust executable above.

I inspected version 3.8.51, commit 152d95108c9c3d557562311ffed63240a511eb31. The engine entry point distinguishes shell tools from file-reading tools before matching command filters. It preserves content blocks carrying explicit cache markers. Those are useful details: rewriting a source file as if it were compiler output would damage the context before inference even starts.

I ran three focused test files through Node 24.18.1's TypeScript support, without installing or starting the full gateway:

node --test --test-reporter=spec `
  tests/unit/compression/fidelityGate.test.ts `
  tests/unit/compression/cachingAware.test.ts `
  tests/unit/compression/eval-cost-meter.test.ts

tests 29
pass 29
fail 0
skipped 0

The fidelity checker checks protected strings, numeric literals, JSON keys and diff headers. I passed hand-written before/after pairs to the real function using this probe:

node ../fidelity-probe.mjs .

Input	Candidate output	Gate result
14 passed, 1 failed	4 passed, 1 failed	Reject: numeric
14 passed, 1 failed	1 passed, 14 failed	Pass
{"allowed":true,"blocked":false}	{"allowed":false,"blocked":true}	Pass
Do not delete the backup.	Delete the backup.	Pass
port 80	port 8080	Pass

I supplied the changed outputs. These are counterexamples to the checker's guarantee, not evidence that OmniRoute's compressor generated those changes.

The implementation uses substring membership. A number can survive while moving to a different field, and 80 appears inside 8080. JSON key survival says nothing about boolean values. The checker also returns success on an internal exception. Its pipeline integration is opt-in and skips engines marked as sampling. A green gate therefore carries a narrower meaning than semantic equivalence.

The local probes cover bytes and structural checks. They do not measure model behavior or paid task cost.

The Bill Has a Different Denominator

Quesma's Terminal-Bench experiment compared 1,740 attempts with and without RTK 0.45.0. Their aggregate bill fell about 5% for Fable and rose about 5% for DeepSeek. After including failed attempts and dividing spend by passes, the changes were roughly -3% and +7%. Equal weighting per task gave another view: about +1% and +17%.

Those are the authors' results, not runs I reproduced. They used an older RTK version. The disagreement between aggregate and per-task results is useful because expensive outliers can dominate a total. Their reported head accounting problem motivated my smaller reproduction; my local output above comes from 0.48.0.

For an agent task, the cost expression has more terms than the compressed input:

task cost = sum over attempts and turns of:
  uncached_input * input_rate
  + cache_read * cache_read_rate
  + cache_write * cache_write_rate
  + output * output_rate
  + tool_cost

cost per successful task = total spend / successful tasks

Use the provider's disjoint usage buckets so cached input does not get counted twice. Keep failures in total spend. Record turns, retries and whether the agent had to retrieve omitted output.

OmniRoute's computeSavings labels its optional dollar calculation as an input-side estimate and excludes output cost. That scope is reasonable. Promoting it to a claim about completed tasks would require a separate experiment.

The first fix belongs at the command boundary: compare against what the requested command would have returned. Preserve the requested line. Then check whether the agent finishes the same work with fewer paid tokens. Until those measurements agree, I would read a large savings percentage as a debugging counter, with its baseline attached.