A Zero-Length Tail That Keeps the Whole Prompt

I passed tail=0 to a compression-based text generator, expecting it to hide the recent output. It kept the whole prompt. After the first generated byte, it kept that byte too. The setting meant the opposite of the boundary I expected.

The project is GziPT, a small Python decoder that ranks byte sequences by their compressed size. Nathan Barry's original write-up describes the experiment. My narrower question was whether its context controls and scoring helper behave as their interfaces suggest. In this decoder, those details determine which text the compressor can copy and which candidate wins.

I checked version 0.3.0, revision 3734bf67f07fd0ba848bfc1b6e585224684c3ee2, using Python 3.12.6 and zlib 1.3.1 on Windows. The implementation has no runtime dependencies. All results below come from a synthetic corpus and the actual decoder source, with no neural model or external inference endpoint.

Two Slices Define the Visible Text

Inside generate(), the code prepares the context with:

corpus_window = corpus[:window]
recent = (bytes(prompt) + bytes(out))[-tail:]
ctx = corpus_window + recent

The first slice takes the start of the corpus. It does not rotate through a large training file. The second slice keeps a suffix of prompt plus output. For a positive tail, this matches the intended recent-history limit.

Python evaluates -0 as 0. A slice starting at zero retains everything. I wrapped candidate_lengths() to record its input, then delegated every score to the original function. With corpus abc, prompt PROMPT, two generation steps, and alphabet a,b, the probe printed:

{"tail": 0, "first_context": "abcPROMPT", "second_context": "abcPROMPTa"}
{"tail": 1, "first_context": "abcT", "second_context": "abca"}
{"tail": 4, "first_context": "abcOMPT", "second_context": "abcMPTa"}

This is a context-construction result. The compressor still has its own finite match window; passing the entire history does not make every old byte available for a back-reference. But zero fails to remove recent text, and the input grows with the generated history. A caller cannot use this setting to run a clean corpus-only ablation.

The CLI parses --tail as an integer without a positive-range check. The help text also describes --window as at most 32,768, but the parser does not enforce that bound. I left the implementation unchanged for these tests. A fix would need to choose an explicit zero policy and test it, rather than leave Python slicing to decide.

The Score Includes Compressor Mechanics

For each search depth, the decoder extends every surviving beam by every allowed byte. It sorts candidates by compressed length and keeps beam_width entries. At temperature zero it commits the cheapest complete span; otherwise it samples among the finalists. This temperature operates on compressed-byte differences, not neural token logits.

The scoring helper avoids recompressing the shared context for every candidate. It feeds the context into zlib.compressobj(), copies that state, appends one candidate to each copy, then finishes each stream. The function's docstring claims equivalence with compressing the concatenation in one call.

I compared those paths across three contexts, four candidate lengths, and compression levels 0, 1, 6, and 9: 48 comparisons. Four differed. All four used a 40,000-byte seeded random context at level 0.

Candidate bytes	Copied streaming state	One-shot compression	Difference
0	40,016	40,011	+5
1	40,017	40,012	+5
48	40,064	40,059	+5
97	40,113	40,108	+5

Level 0 stores data without compression, and this fixture exposes a difference in stream/block overhead. The constant offset did not change the ranking of these four candidates. The remaining 44 comparisons matched, including every level-9 case, which is the decoder default. This counterexample narrows the helper's equivalence claim; it does not establish a default decoding failure.

A Small CLI Replay

The downloadable probe creates its corpus by repeating the cat sat on the mat. the dog sat on the rug. plus a newline 80 times. It also runs the actual CLI as a subprocess. From a checkout of the pinned revision:

python -X utf8 path\to\gzipt-probe.py .

The equivalent generation command, with that fixture saved as corpus.txt, is:

python gzipt.py --corpus corpus.txt --prompt "the " --length 32 --horizon 4 --beam-width 16 --temperature 0 --workers 1

The process exited with code zero and printed:

the cat sat on the mat. the dog sat

The output retains a trailing space before its newline. At the first search step, the 16-byte alphabet produced just two scores: one candidate at 72 compressed bytes and 15 at 73. The unique winner was c. Generation with horizons 1, 4, and 8 produced the same 32-byte continuation on this fixture. It gives no evidence that a deeper beam improved text quality here.

There are two separate limits on interpretation. First, repeated toy sentences make copying easy; this run says nothing about general language competence. Second, this is byte generation. The CLI decodes the final bytes as UTF-8 with replacement, so its --length is not a character count or a guarantee of valid multilingual output.

The useful engineering result is the small surface area for inspection. I could observe every context, compare both compression paths, and replay the CLI without downloading weights. For a decoder built around compressed size, those checks belong beside any sample paragraph: the context policy and stream boundaries are part of the model behavior.