One Cache Counter, Two Different Bills

I passed input_tokens=1000 and cache_read_tokens=400 into a cost calculator. With one flag set, it returned $0.020000. With that flag cleared, it returned $0.032000. Adding a pricing threshold raised the second result to $0.064000 while the first stayed unchanged.

All those prices are synthetic. The counts describe different workloads under the two supported conventions: cached tokens can be a subset of the input counter, or an additional bucket beside it. A gateway must preserve that distinction before it selects a price tier. A precise decimal calculation cannot recover a missing denominator.

I examined Otari revision ea6f0567e2bfb0938f4b00db5ba6be76194c628d, using Python 3.13.5 on Linux. The results below execute its unchanged pricing core against synthetic counts and rate objects. They establish arithmetic behavior, including normalization and tier selection. They do not reproduce a provider invoice or run a gateway deployment.

A Usage Record Carries a Convention

Otari is a self-hosted model gateway with authentication, provider credential resolution, routing, budget reservations, and usage recording. Its tree contains database migrations, API routes, repository and service layers, a dashboard, and unit and integration tests. The dependency manifest requires Python 3.13 or newer and delegates provider calls to any-llm.

The substantive part for this note is a shared cost-math module. Settlement uses its normalized meters and pricing functions; the reservation code calls its estimation function. Inspection of the usage-pricing service and reservation service confirms those call sites. I did not execute their database paths.

The internal GatewayUsage carrier stores explicit cache-read and cache-write counts alongside cache_tokens_in_prompt. The cost core's loose-count interface calls this concept cache_tokens_included and requires callers to pass it explicitly.

For cache reads alone, normalization follows these rules:

Convention	Meaning of input counter	Normalized total input	Fresh input when cache reads have a separate rate
Inclusive	Whole prompt, including cached tokens	Reported input	Reported input minus cached reads
Additive	Fresh input, excluding cached tokens	Reported input plus cached reads	Reported input

The flag is semantic metadata. The integers alone cannot tell the calculator which row applies.

A Small Rate Card Makes the Difference Visible

The fixture uses $30 per million fresh-input tokens, $5 per million cache-read tokens, and zero output tokens. These are chosen test values, not a model's advertised tariff. I passed ordinary SimpleNamespace rate objects, which the module supports through structural attribute access.

The core invocation is:

usage = core.billable_usage(
    input_tokens=1000,
    output_tokens=0,
    cache_read_tokens=400,
    cache_tokens_included=True,
)
cost, meters, lines = core.price_billable_usage(pricing, usage)

I then repeated it with the flag set to False. A second rate card adds a threshold at 1,400 normalized input tokens, with fresh input at $60/M and cache reads at $10/M.

Rate card	Convention	Normalized input	Fresh input	Settled USD
Flat	Inclusive	1,000	600	0.020000
Flat	Additive	1,400	1,000	0.032000
Threshold at 1,400	Inclusive	1,000	600	0.020000
Threshold at 1,400	Additive	1,400	1,000	0.064000

The threshold applies to the whole request. It is not a marginal price charged only on tokens above the boundary. In the additive threshold case, the returned charge lines contain 1,000 fresh tokens at $60/M and 400 cached tokens at $10/M, totaling $0.064000.

To describe the same workload under both conventions, the input counters must differ. Inclusive input 1000 with cache reads 400 matches additive input 600 with cache reads 400. Both normalize to 1,000 total tokens and settle at $0.020000 under the flat card. The probe asserts that equivalence, then repeats it for 105 count pairs with a separate thresholded synthetic card.

What Actually Ran

The downloadable probe checks SHA-256 hashes for metered_pricing.py and usage.py before loading them. With those pinned files under /tmp/otari-note/src/gateway/core, I ran:

python3.13 --version
python3.13 public/blog/cache-counter-two-bills/pricing-probe.py /tmp/otari-note

Both commands exited zero. Selected terminal output was:

Python 3.13.5
{"case":"flat-inclusive","total_input":1000,"fresh_input":600,"cost":"0.020000"}
{"case":"flat-additive","total_input":1400,"fresh_input":1000,"cost":"0.032000"}
{"case":"tier-inclusive","total_input":1000,"fresh_input":600,"cost":"0.020000"}
{"case":"tier-additive","total_input":1400,"fresh_input":1000,"cost":"0.064000"}
PASS: 11 named scenarios and 105 equivalent-encoding pairs

The execution has an explicit isolation boundary. The provider SDK's usage base class is replaced by an import-only placeholder that raises if instantiated. Standard Python logging replaces application logging. The actual usage-helper and arithmetic source is unchanged; the probe uses loose counters and never constructs an SDK carrier. This does not verify provider-response parsing or the carrier's integration with the request pipeline.

I attempted to install the real SDK with:

uv --cache-dir /tmp/otari-uv-cache pip install \
  --python /usr/bin/python3.13 --target /tmp/otari-note-deps \
  'any-llm-sdk==1.38.0'

That command exited two after three retries because the package request could not establish its network tunnel. I inspected the project's focused pricing tests, but did not run its test suite. The custom probe's successful result is confined to the isolated cost core.

Missing Prices and Contradictory Counters

Two defensive behaviors deserve attention because they change the resulting bill.

First, an absent cache-read price means cached tokens remain in the fresh-input bucket. Under the flat fixture, a missing rate produces $0.030000. An explicit zero cache-read rate produces $0.018000: only the 600 fresh tokens are charged. None and zero therefore encode different pricing policies. Treating both as falsy would erase a free meter.

Second, an inclusive payload with 100 input tokens and 400 cache-read tokens is contradictory. The current normalizer logs a warning, clears cache attribution, and prices all 100 tokens as fresh input. The probe captured that warning and a cost of $0.003000. It did not receive an exception. Operators inspecting only settled costs would need another signal to notice that the input attribution was discarded.

The probe also confirmed that omitting the convention raises TypeError, and an unusable base rate such as NaN raises ValueError. These checks keep missing semantics and corrupt base pricing from silently becoming zero-cost usage.

Estimates Depend on the Same Normalized Input

For 1,400 estimated input tokens under the thresholded fixture, the estimator reserves $0.084000: every input token at the dearest applicable input-side rate, $60/M. The tested settlement was $0.064000 because some tokens received the lower cache-read rate.

That comparison holds for this supplied count and rate card. A wrong token estimate can still select the wrong tier; the arithmetic alone cannot prove a budget ceiling survives concurrent requests or failed settlement. Those belong to the reservation and database lifecycle.

Otari's useful choice is to normalize meters before selecting rates and to share that cost core across callers. The constraint for any adapter feeding it is equally concrete: preserve whether cache counts are included in the input counter. Renaming fields into one common usage object is insufficient unless their accounting meaning travels with them.