Tiled Inference Without Full-Image Masks

Tiling an image can make a segmentation model fit while leaving a large allocation after inference. The model returns masks in tile coordinates. A postprocessor can then expand each mask onto the original image canvas before joining the results.

For 32 detections on a 1024 × 1024 image, that means 32 MiB of boolean masks. The input RGB image occupies 3 MiB. Shrinking the model's working image does not settle the cost of its output representation.

I followed this path in Supervision 0.31.0.dev0, at commit 3169ae56b122096b7fc69465ca94f7d060286cd6. The useful implementation detail is its optional CompactMask: a run-length encoding of each crop, with an offset into the full image. The slicer can move those offsets and suppress duplicate detections without constructing the full mask stack.

A Callback Without a Model

My probe script uses a synthetic callback that returns two filled rectangles for each tile. I used a 1024 × 1024 image, 256 × 256 tiles, no overlap, and one worker. Sixteen callback calls produce 32 detections.

This fixture isolates the representation and coordinate transformations. It does not measure segmentation accuracy, GPU memory, or inference throughput.

From the pinned source checkout, I prepared an isolated environment:

uv venv --python C:\Python312\python.exe .venv
uv pip install --python .venv\Scripts\python.exe -e . pytest
.venv\Scripts\python.exe -X utf8 path\to\mask-probe.py

The run used Python 3.12.6 and NumPy 2.5.3 on Windows. Supervision warned that OpenCV was absent and that it would use its NumPy fallback. I kept that configuration and make no claim about OpenCV-backed speed.

The callback returns dense tile masks in both cases. The two slicers differ in one argument:

sv.InferenceSlicer(
    callback=segment,
    slice_wh=256,
    overlap_wh=0,
    thread_workers=1,
    overlap_filter=sv.OverlapFilter.NON_MAX_SUPPRESSION,
    iou_threshold=0.3,
    compact_masks=True,
)

The script runs the dense path first. During the compact run, it replaces CompactMask.to_dense and CompactMask.__array__ with functions that raise. After the run finishes, it restores those methods and compares the results in a consistent box order.

{
  "fixture": "1024x1024; 16 tiles; two rectangles per tile",
  "detections": 32,
  "dense_mask_bytes": 33554432,
  "compact_buffer_bytes": 5760,
  "repacked_buffer_bytes": 768,
  "pixel_equal": true,
  "compact_pipeline_dense_guard": "passed"
}

I counted the backing buffers for the run lengths, offsets, and crop shapes. These numbers exclude Python object headers, the RGB image, transient tile arrays, and model memory. They are storage measurements for this fixture, not peak process memory. The guard also has a narrow meaning: the tested compact path did not call either of those full-stack conversion methods.

Keep the Tile Until You Know the Mask

In InferenceSlicer._run_callback, the conversion uses the entire tile as each mask's initial crop. It does not trust the detector's box to contain every true mask pixel.

That choice prevents a quiet data-loss bug. I added a second fixture with true pixels at (0, 0) and (99, 99), but a detector box of [0, 0, 10, 10]. Cropping to that box would discard the far corner.

{"outside_detector_box_survives": true, "outside_detector_box_survives_repack": true}

The initial encoding preserves both pixels. Calling repack() preserves them too. In the CompactMask implementation, repack() finds the smallest rectangle containing the actual true pixels; it does not crop back to the detector's reported box.

There is a documentation wrinkle here. The slicer docstring describes repacking in terms of detection bounding boxes. The implementation derives bounds from the mask pixels. The corner test makes that difference concrete.

For the two solid rectangles per tile, repacking reduced the counted buffers from 5,760 to 768 bytes. It decodes crop-sized masks to find those tighter bounds, so it adds work. I did not time that tradeoff.

Suppression and Merging Take Different Paths

The compact option defaults to False. You must opt in, and the operation after slicing still matters.

I supplied two identical compact masks, then instrumented calls to to_dense() while running suppression and merging. Each operation returned one detection:

{
  "duplicate_masks": 2,
  "nms_output": 1,
  "nms_to_dense_calls": 0,
  "nmm_output": 1,
  "nmm_to_dense_calls": 4
}

Non-maximum suppression selects a surviving detection. Non-maximum merging has to construct a combined result. In this revision, the mask overlap and merge code includes dense conversion while updating a compact merge candidate. Passing in a CompactMask therefore does not guarantee that an operation stays compact inside.

The four calls count method invocations in this two-mask fixture. They do not measure four equal allocations, and they are not a bound for a larger merge. For a memory-sensitive pipeline, I would audit the exact overlap strategy before treating the compact flag as sufficient.

NumPy interop is another explicit boundary. np.asarray(compact_mask) constructs the dense stack. I used conversion after the guarded section to verify pixel equality; leaving it inside production postprocessing would bring back the allocation that the compact path avoids.

A Checkerboard Reverses the Saving

Run-length encoding works well when adjacent pixels have the same value. To check the opposite case, I encoded a 128 × 128 checkerboard as one full-crop mask:

Fixture	Dense mask buffers	Compact buffers	Observation
32 rectangles on the full canvas	33,554,432 bytes	5,760 bytes	Tile RLE avoids full-canvas expansion
Same rectangles after repacking	33,554,432 bytes	768 bytes	Tight pixel bounds remove background runs
One checkerboard	16,384 bytes	65,048 bytes	Frequent transitions make RLE larger

The checkerboard round-tripped without a pixel difference. Its compact storage still cost almost four times the dense boolean buffer. A mask with many short runs can spend more space on integer run lengths than on individual boolean pixels. Compression follows mask structure, not the package name.

I also ran the upstream compact-mask, overlap, and slicer integration tests:

.venv\Scripts\python.exe -X utf8 -m pytest -q `
  tests/detection/test_inference_slicer_compact.py `
  tests/detection/test_compact_mask_iou.py `
  tests/detection/test_compact_mask.py

364 passed, 1 warning in 2.56s

The warning was the missing OpenCV backend. These tests and the probe provide evidence for mask representation and geometry on this environment. They do not establish an accuracy or latency improvement for a trained model.

Supervision has enough implementation here to study: tile scheduling, coordinate translation, crop encoding, overlap calculation, and conversion tests. The main design choice I would carry into another segmentation pipeline is keeping the mask's local storage separate from its position on the canvas. After that, each consumer needs an explicit answer to whether it can operate on the compact representation or must allocate pixels again.