Deterministic Gates Before AI Code Review

An LLM review starts with a mundane question: which files, which rules, and how much context should it see? Leaving those choices inside a chat loop turns them into another source of variation. A review result can change because the model skipped a file, mixed two unrelated changes, or received a different rule paragraph.

OpenCodeReview puts a conventional program ahead of the model. It selects files, rejects unsafe paths, groups work, resolves language and project rules, then creates a bounded task for the review loop. That boundary is the valuable part of the design. It gives a team something it can test without asking a model whether it followed instructions.

A model can reason about code after the selector has made the review scope explicit.

The Local Run

I inspected OpenCodeReview at commit a694be568d9b9a935b2ba11a867d5a91d7ffd833 with Go 1.26.0 on Windows. The checkout contains a Go CLI, embedded language-rule documents, diff and path filters, an LLM loop, provider adapters, and a separate delegation interface. This is a real implementation with executable boundaries, not a prompt repository.

I started with the no-provider path. It should expose file rules without an API key or model call:

go run ./cmd/opencodereview delegate rule `
  internal/config/rules/system_rules.go `
  internal/delegate/format.go

### Rule Group 1: system / **/*.go

Applies to:
- internal/config/rules/system_rules.go
- internal/delegate/format.go

#### Go Review Principles
Favor precision over recall: report only defects that are likely real...

The command grouped both files because they resolved to the same embedded **/*.go rule. The expanded output contains checks for ignored errors, typed nils, cancellation, goroutine ownership, locks, timer lifecycle, integer boundaries, and untrusted input. That output is a useful contract for a host agent: it knows the scoped files and the applicable constraints before it begins interpretation.

The CLI describes the split with no mystery:

ocr delegate preview   # select reviewable files and exclusions
ocr delegate rule      # resolve and group the rule text
ocr review             # run the configured provider loop

I did not configure a provider, send code to a remote endpoint, or claim a defect-finding score. The local result tests the deterministic half of the architecture.

The Boundary Is Code, Not Prompt Discipline

internal/config/rules/system_rules.go loads embedded rules and resolves a path using ordered glob matches. First match wins. It then composes optional custom, project, global, and system layers in that priority order. The code retains the source and matching pattern through RuleDetail, so a caller can tell whether src/payments/** won because of a repository rule or a built-in language rule.

That detail matters when a team is debugging a review. “The model missed it” is too broad. You can first ask whether the selector admitted the file, whether an exclusion won, which rule text entered the prompt, and which run identity recorded those inputs.

The agent layer also calculates a stable SHA-256 identity from the effective rules, filter configuration, model settings, and selected input. Its comments say why rule order remains significant: changing an earlier match changes the applicable contract. A resumed review can reuse completed work only after that identity still matches. This is a sound boundary between a changing working tree and a cached model output.

The file selector is stricter than a convenience filter. Tests cover binary files, default test-file exclusions, explicitly included tests, user excludes, generated content, secret paths, and secret renames. The secret-path test has an important policy: an explicit include cannot re-admit a real secret. A review tool must not turn a .env file into model context because a broad glob matched it.

What the Test Suite Actually Established

I ran the focused deterministic packages, then the larger execution package:

go test ./internal/delegate
go test ./internal/config/rules
go test -v ./internal/agent

ok   github.com/alibaba/open-code-review/internal/delegate
ok   github.com/alibaba/open-code-review/internal/config/rules
ok   github.com/alibaba/open-code-review/internal/agent  19.983s

The verbose agent run exercised specific failure paths instead of only green nominal cases. TestGroupDiffs_LLMError_Fallback falls back to per-file dispatch after a refused grouping call. TestDispatchSubtasks_TokenBudgetStopsDispatch stops once a projected group crosses the configured budget. TestManifestFlowMixedFailureIsIsolatedToPartial keeps a completed item while a separate provider item times out or panics. TestPreviewMatchesRunSelectedCoverage checked that preview and execution agree on excluded Markdown, an oversized Go file, and a selected source file.

Those are the properties a wrapper needs. A model may still produce weak comments, but it should not silently turn a partial batch into a successful full review.

Local evidence	What it supports	What it does not support
Delegation CLI output	Paths receive stable rule groups before a provider call.	A model follows every rule.
Rule and agent tests	Ordered rules, exclusions, grouping, budgets, resume states, and partial failure handling.	A remote provider returns useful findings.
Source inspection	Input identity includes resolved configuration and review state.	Cross-model precision, recall, or token cost.

A Few Hard Edges Remain

The repository requires Go 1.25.5 in go.mod. That is a meaningful installation cost for a review CLI, especially when many CI images pin older Go versions. My Go 1.26.0 environment satisfied it; that only proves the checkout builds and tests there.

The default rules exclude test files. The project lets an explicit include override that policy, and its tests enforce the behavior. That default makes sense for noisy review queues, but it can miss a broken test harness or a test-only security bypass. A team should decide this rule per repository instead of accepting the global default by habit.

The built-in Go rule tells the model not to repeat findings from go vet, Staticcheck, the compiler, or gofmt without a concrete user impact. Good constraint. It also means the setup assumes those tools already run elsewhere. If they do not, a developer can mistake a review assistant for a general static-analysis pipeline and leave simple defects uncovered.

The README publishes benchmark figures for review quality and token use. I did not reproduce them. A valid replication needs the named benchmark corpus, exact model configuration, prompt templates, provider behavior, and scoring policy. The local unit suite proves input control and failure handling. It does not prove a precision number.

The Useful Split

Code review has two different jobs. An ordinary program should define the input set, keep secrets out, attach repository rules, account for budgets, and record partial completion. A model should inspect the selected code, fetch the surrounding context, and explain a defect when static checks cannot settle it.

OpenCodeReview makes that split visible in the CLI and source. That is enough reason to study it. Teams building a review loop should borrow the deterministic gates first, then measure the model layer with their own repositories and false-positive tolerance. The gate does not make a model correct. It makes the review run auditable when the model is wrong.