Three Choice Calls to Write Two Letters

A model that scores options can write text, even if it has no token-generation endpoint. Give it the question, the reply so far, and a set of possible next strings. Take the winning option, append its symbol, and ask again. This works as a decoder. It also moves the cost of generation into repeated choice questions.

I checked Jevchat at commit 0eefd0cfa667805d25b7a87bcc51780aeab6962c. Its generation loop builds the next state, asks a scorer, shapes the returned probabilities, and appends one symbol. STOP is a separate outcome. The repository contains a real HTTP client, four scoring strategies, alphabets, and offline tests. It is an experiment with an awkward cost profile, not an empty chatbot wrapper.

A Two-Letter HTTP Replay

I used an offline probe with httpx.MockTransport. The mock replies with a, then b, then STOP. It runs the actual JevClient, Scorer, and generate() code; it never contacts the Jev service. The alphabet contains only a, b, and STOP, with greedy sampling and no minimum length.

From the pinned checkout on Python 3.12.6:

$env:PYTHONPATH = (Get-Location).Path
python -X utf8 path\to\jevchat-probe.py

The first request goes to POST /v1/systemone inside the mock transport. I trimmed the request to its changing fields here; the actual body also carries model, task, and instructions.

{"answer_so_far":"","criteria":["","a","b"]}

The empty label represents the unchanged reply, which means STOP under the hypothesis presentation. The probe's first scripted HTTP response is:

{"answers":{"next0":{"type":"choice","choice":"a","confidence":1.0,"probabilities":{"":0.0,"a":1.0,"b":0.0}}},"usage":{"input_tokens":7,"output_tokens":2}}

The terminal summary was:

{"answer_so_far": "", "criteria": ["", "a", "b"]}
{"answer_so_far": "a", "criteria": ["a", "aa", "ab"]}
{"answer_so_far": "ab", "criteria": ["ab", "aba", "abb"]}
{"step_keys": ["a", "b", "STOP"], "text": "ab", "reason": "stop", "http_calls": 3, "input_tokens": 21}

The 21 input tokens are values returned by my mock, seven per call. They are not a measured bill. The useful observation is structural: this code made three HTTP calls to emit two characters and decide when to stop. The HTTP client packages state and questions into each request; it is not streaming model tokens.

The Alphabet Changes the Request Count

One Jev choice question accepts at most 255 options. Jevchat therefore has to split a larger alphabet. Its bucket strategy adds OTHER to each bucket and asks a separate STOP question. The code drops OTHER, merges the remaining bucket scores, and scales them to leave the score assigned to STOP.

I loaded three checked-in alphabets with UTF-8 mode and called buckets.build(alphabet, 127, 48). These are plan counts, with no model call or latency measurement:

Alphabet	Options including STOP	Questions per output symbol	HTTP requests planned per symbol
ascii	89	2	1
words1k	1,122	10	1
bpe50k	49,862	394	9

The default configuration uses words1k, bucket size 127, and up to 48 questions per HTTP request. A larger vocabulary can reduce the number of generated symbols, but it increases the number of choice questions inside each symbol step. The nine planned requests for bpe50k are a consequence of the batch cap, before accounting for context overflow or retries. No endpoint benchmark follows from that table.

There is another contract in the presentation layer. The options are tails of complete candidate replies, such as a, aa, and ab, rather than bare symbols. That lets a decision model rank text it can see. The code stores the full prefix in state.answer_so_far, while each option carries only a bounded tail. If two symbols render to the same visible tail, the builder keeps the first; the alphabet is no longer one-to-one at that step.

What the Offline Suite Says

The CLI listed the built-in alphabets without an API key:

$env:PYTHONUTF8 = '1'
python -m jevchat.cli alphabets

I ran the repository test suite with PYTHONUTF8=1 and a writable temporary directory. Its result was 157 passed, 1 failed. The failed test_stats_are_reported expected done.stats.elapsed > 0; this run reported 0.0. The generation text and step-count tests passed. The earlier run without UTF-8 mode also hit CP950 decoding errors while loading bundled JSON alphabets, so encoding is a real Windows setup requirement for this revision.

I did not send a paid request, test the model's language quality, or reproduce the project's benchmark claims. The local replay shows that the architecture executes and where its calls multiply. A decision endpoint can act as a decoder, but its choice limit and score presentation become part of the language model's effective interface.