How reviews work

A Warpway review answers seven questions for every eligible pull request: what changed, what risks were investigated, what evidence was gathered, what problems were found, what could not be resolved, who should resolve each open question, and whether your review policy is satisfied.

What starts a review

Pull requests can also be left out by configuration: authors listed in review.ignore_authors, base branches outside review.base_branches when that list is set, and pull requests whose changed files all match review.ignore_paths.

One review per commit

A review run is bound to one repository, one pull request, one exact head commit and one policy version (the resolved configuration, identified by its hash), and it records the model, prompt, lens and tool versions it used. A run's commit and policy version never change. A new commit creates a new run, and the previous run is marked superseded.

The steps of a review

1. Receive the event. The webhook signature is verified and the delivery ID recorded, so a repeated delivery does nothing.
2. Fetch context. Pull request metadata, changed files and diff, commits, existing reviews and comments, linked issues, the CI checks on the head commit, and CODEOWNERS and .warpway.yml from the base commit.
3. Resolve the policy. Defaults, organization settings, repository settings and the base-branch configuration file are combined, organization locks and plan limits applied, and the result stored as a policy version. See Configuration as code.
4. Start the check. Warpway / Review is created on the head commit with status in_progress.
5. Analyze the change. A summary of what changed, affected subsystems, risk areas, semantic categories (such as auth, billing or migration) and files worth reading.
6. Plan the review. Which lenses apply and whether each is required, which rules and organization knowledge are relevant, which tools each lens may use, which runtime profiles to run, and who is likely to answer questions. The planner can add lenses but cannot drop a required one.
7. Run the lenses. Each lens investigates independently with read-only tools on a snapshot of the head commit: reading files and line ranges, searching, finding symbols and references, history and blame, related tests, CI status, linked issues, previous reviews and organization knowledge. Every tool result becomes evidence with its provenance.
8. Arbitrate. Findings and questions from all lenses are deduplicated, each claim is checked against the evidence it cites, vague questions are rejected or rewritten, overlapping questions are merged, contradictions between lenses are exposed rather than hidden, and each lens gets its final state. An Incomplete lens is never turned into Cleared.
9. Route human tasks. At trust level 1 and above, each open question is assigned to the person most likely to know, in Slack or GitHub. See Routing.
10. Evaluate the policy. Deterministic code decides whether the policy is satisfied and lists every blocker.
11. Publish. The check run and the summary comment are updated, but only if this run is still the current one for the pull request's head commit.

The four lens states

Every lens ends in exactly one state:

Waiting for people

Nothing waits in memory. When a lens needs a human answer, the review records its state and the check reports what it is waiting for. When the answer arrives, Warpway checks that it actually settles the question, records it as evidence, re-runs only the affected lenses plus a global sanity pass over the whole change, evaluates the policy again and updates GitHub. If a worker restarts in the middle, the review resumes from its stored state.

New commits

When new commits arrive:

1. a new review run starts for the new head commit and the old run is superseded. The old run may finish its work, but it can no longer update the check, the summary or anything else on the pull request;
2. Warpway compares the new head with the last reviewed head to find what changed;
3. a lens whose files did not change keeps its previous result instead of investigating again, unless that result was Incomplete or the lens or policy changed;
4. earlier human answers are reused as they are if the code the question depended on has not changed; otherwise each answer is checked against the new code, and its task reopens if the answer no longer holds;
5. findings that the new commit fixed are recognized and closed;
6. affected lenses run again, followed by a global sanity pass.

Nobody is asked the same unchanged question twice.

When something fails

Warpway fails closed. If a model call fails, required context cannot be loaded, the repository cannot be fetched, a runtime verification run fails or times out, an answer is ambiguous, or the configuration is invalid, the affected work is not marked Cleared: a lens that could not finish reports Incomplete with the reason, an ambiguous answer leaves its task open, and an invalid configuration is reported as a configuration error. The review policy is then not satisfied, so the check never reports success. Transient failures are retried with backoff first; work that still cannot complete stays visible as a blocker.

Usage limits

If your plan's review limit is reached, Warpway does not review the pull request and the check says Usage limit reached. It never reports success for a review it did not do. See Billing.

What you see on the pull request

The check run. One check, Warpway / Review, bound to the head commit, listing each lens, open human tasks, the policy status and the trust level:

✓ Correctness
✓ Security
✓ Testing / Reliability
✓ Backward Compatibility
⚠ Product Semantics — waiting on Priya

Human tasks: 2 open / 1 resolved

Review policy not satisfied: 2 human tasks remain.

Trust level 2 (Gate). Open in Warpway

A lens that is not required is marked (optional), and an Incomplete lens shows its reason. The check's title sums up the result, for example Waiting on 2 human decisions; Merge gating lists every conclusion. The check's details list the open tasks with why each person was asked, the findings, every policy blocker, and the model, prompt and policy versions the review used.

Concrete findings appear as line annotations. Uncertainty never does.

The summary comment. One comment, edited in place as the review progresses:

## Warpway Review

Commit `3f9c2e1` · Trust level 2 (Gate) · [Details](…)

### No issues found by AI
- Testing / Reliability
- Backward Compatibility
- Product Semantics

### Human decisions
- ✅ Product Semantics: Unused credits survive cancellation. — answered by Priya in Slack
- ⏳ Security: Must revoking an administrator role end active sessions immediately? — assigned to @marcus `T4F2A9C`
  Answer here with `/warpway answer T4F2A9C <your answer>` or [in Warpway](…).

### Findings
- 🔴 **Retry loop can enqueue the same invoice twice.** — `src/billing/retry.ts:42-58`
  A failed send re-queues the invoice without checking whether it is already queued.

### Status
Review policy not satisfied: 1 finding and 1 human task remain.

Below trust level 3 the list of lenses without problems is headed No issues found by AI; at Clear and above it reads Cleared (no human review needed for these areas). When they apply, the comment also lists configuration errors, Incomplete lenses with their reasons and lenses that disagree, and it notes when the pull request changes Warpway's configuration, which only applies after merge. When the review recorded them, it ends with the human review surface of the commit: changed lines analyzed, lines surfaced as relevant human-review context and human decisions requested. These are counts, not an estimate of time saved.

Answering on GitHub. Each open task has a short reference such as T4F2A9C. In a pull request comment, /warpway answer <reference> <answer> answers a task (use the choice id when choices are listed), /warpway wrong-person <reference> says you are not the right person, and /warpway rerun reviews the current commit again.

Reviewer requests. When a question is best answered by an engineer, Warpway can request them as a reviewer, at most two per pull request by default. People reached only in Slack are not. See Routing.

The dashboard. The pull request page shows the change summary, head commit, review plan, every lens with its evidence, findings, human tasks and their conversations, the policy state, an audit timeline and a rerun button.