Validation and Evidence

The engine in son-of-anton-review (feat/cloudflare-native) can run a repository's declared JS validation scripts and include their results in a review. Validation is conditional, not guaranteed on every PR, and a finding is not automatically backed by a persisted artifact.

Executor selection and isolation

runValidation in src/validation-executor.js is the dispatch seam. GILF_VALIDATION_EXECUTOR takes precedence, then the merged validation.provider, then GILF_VALIDATION_PROVIDER, then managed-crabbox.

Aliases include crabbox, e2b, cf-sandbox, cloudflare-sandbox, cfsandbox, selfhost and self_host. Unknown executors or unavailable sandboxes do not silently fall back to executing PR scripts on the orchestration host. They produce a refusal or missing-validation result.

There is an explicit exception: legacy local executes on the worker when GILF_ALLOW_LOCAL_VALIDATION=1. It is refused otherwise. This opt-in does not verify that the repository is trusted. Do not use it for untrusted PRs. Legacy shadow means the older Crabbox/E2B comparison path, not the separate shadow-executor feature below.

For a directly launched engine process, an E2B selection is:

export GILF_VALIDATION_EXECUTOR=managed-e2b

Provision its API key separately. See Validation executors.

What runs

The runner reads root package.json, detects the package manager and selects non-empty scripts named test, typecheck, lint and build, in that order. It does not discover arbitrary project validation commands.

No manifest or no standard scripts produces a gap rather than a pass. Some scripts can be classified as inconclusive before execution. An install failure stops the remote shell before subsequent scripts.

Remote shell steps emit __GILF_VALIDATION_START__:<tag> and __GILF_VALIDATION_END__:<tag>:<exit> markers. The tag can include a per-run nonce. The parser uses the nonce it receives, but markers are an output-parsing mechanism, not proof that untrusted code cannot forge output or that every executor has identical nonce wiring.

A failed step with severity can produce a validation finding. Its path may be empty; the failure output is in the body. These findings have no structured line position and do not create inline GitHub comments.

Skip conditions and precedence

1. Execution policy runs first. execution.mode: never or unmatched execution filters disables validation. always permits execution but does not override diff classification or supply a working executor.
2. With codeChangesOnly enabled, docs-only and generated-only diffs skip repo-native validation. See Docs-only fast path.
3. Otherwise the selected executor runs the available plan.

validationConfigFromEnv sets codeChangesOnly to true unless GILF_VALIDATION_CODE_CHANGES_ONLY is exactly false. mergeValidationConfig spreads the worker defaults over the run config at the top level, so a supplied worker default wins for provider and codeChangesOnly. Nested Crabbox and E2B settings merge in the opposite direction: run-specific values win.

GILF_SKIP_CODEX_ON_VALIDATION_FAILURE=1 is a separate, default-off semantic-review shortcut. When prepared review-memory text is empty, any blocker or high validation finding enables the shortcut, even if it cannot be tied to the diff. For lower-severity findings, an extracted failure path must match a changed path. Prior-review history alone is not this memory-text gate. A validation failure does not normally disable the model when this flag is off.

Shadow validation and cost

GILF_VALIDATION_SHADOW_EXECUTOR selects a second, supported, different executor when its collector is wired. Unset means no shadow lane. The lane starts alongside the primary dispatch and its result is reduced to telemetry, not merged into findings or validation gaps. The runner waits for it after result selection using the remaining GILF_VALIDATION_SHADOW_TIMEOUT_MS budget, default 600000 milliseconds. Exceptions and timeouts are recorded as error telemetry.

Costs are estimates, not bills: estimate:wall-clock multiplies elapsed seconds by a rate card. Built-in cards cover Cloudflare Sandbox and E2B; absent inputs yield null. Override cards with GILF_VALIDATION_COST_RATES, a JSON object. For example, in a shell:

export GILF_VALIDATION_COST_RATES='{"managed-e2b":{"vcpuUsdPerSec":0.000014,"memoryGibUsdPerSec":0.0000045}}'

Native dispatch uses GILF_E2B_BUDGET_USD, default 50, to withhold the exact managed-e2b shadow selector when recorded validation.e2b span cost reaches the ceiling. This is not a reservation or hard billing cap: in-flight spend is not reserved, and a D1 read failure logs an error and forwards the lane unchanged. The budget setting is consumed by the Worker, not forwarded into the container.

Optional artifact evidence

GILF_EVIDENCE_ARTIFACTS=1 enables artifact construction and attachment. It is off by default. GILF_EVIDENCE_REQUIRE_ARTIFACTS=1 requests an additional annotation step inside that feature; setting the require flag alone does not run it.

The artifact builder accepts validation steps/raw output and worker evidence. Kinds include test-output, typecheck-output, lint-output, build-output, log, trace and screenshot. If a collector returns only validation summaries, the builder can store a summary log instead of full step output. Do not promise a captured execution transcript for every executor.

Bodies have a default 256 KiB budget: text retains head and tail, binaries retain only the head. Keys follow review-artifacts/<encoded-repo>/<encoded-sha>/evidence/<sanitized-review>/<kind>/<name>. A configured graph artifact store must accept the upload before new references attach to matching findings. Failed uploads are recorded, not treated as persisted evidence. See Context graph for the storage boundary.

The require step marks execution-claim findings without a reference as evidenceStatus: unverified and adds downgradedReason. It does not remove them, lower their severity or independently verify their content. The predicate accepts an existing reference with a key, URL or ID; the normal attachment path filters failed uploads. Exceptions in the overall artifact hook record an error and return the original findings.

The review body renders artifact URLs or storage keys when present. A storage key is not necessarily a public download URL.

Self-host command contract

The command receives its configured arguments followed by <archivePath> <shellPath> <timeoutMs>. It must start isolated compute, extract the archive, execute the shell and return output. The command string is split on whitespace, not evaluated as a shell program; avoid quoting-dependent command strings.

The child environment is built from a safe allowlist. The engine removes its packaging directory after execution, while the command must tear down its own compute. GILF_SELF_HOST_NETWORK only conveys network-policy intent; the executor must enforce egress. GILF_SELF_HOST_TIMEOUT_MS defaults to 2700000 milliseconds (45 minutes).

Native container forwarding

cloudflare-native/src/container-env.js sets GILF_VALIDATION_EXECUTOR to managed-cf-sandbox when absent and sets GILF_CF_SANDBOX_ENABLED=1. It forwards the E2B key, shadow selector/timeout, E2B timeout/CPU/memory/template and validation cost-rate override.

It does not forward GILF_EVIDENCE_ARTIFACTS, GILF_EVIDENCE_REQUIRE_ARTIFACTS, graph-storage URL/token settings, GILF_VALIDATION_CODE_CHANGES_ONLY, GILF_SKIP_CODEX_ON_VALIDATION_FAILURE, the self-host settings, GILF_ALLOW_LOCAL_VALIDATION, Crabbox overrides or E2B egress settings. Setting these only as Worker vars cannot configure the container paths. Native integration changes are required where those capabilities are needed.

Source evidence

• son-of-anton-review/src/validation-executor.js:35-135,146-163,251-257,304-362,378-581.
• son-of-anton-review/src/codex-review-runner.js:465-472,494-530,941-1097,1254-1385,1529-1544,2528-2627,2711-2714,3517-3570.
• son-of-anton-review/src/validation-config.js:11-31; src/evidence-artifacts.js:235-306,319-559; src/review-format.js:164-182.
• son-of-anton-review/cloudflare-native/src/container-env.js:15-85,100-141; cloudflare-native/container/entrypoint.mjs:325-354.