Test Lab

Test Lab is a read-only aggregation of recorded trace spans, not an experiment launcher. Its API is in son-of-anton-operator-parity on unmerged feat/greptile-operator-parity. Its UI is in anton-ui-trace-parity. The deployed-engine lineage, son-of-anton-review on feat/cloudflare-native, can emit the validation spans but does not expose the Test Lab routes through native ingress.

The API's REVIEW_STATE_DB and the engine's DB are separately configured bindings. Metrics only describe the data visible to the API; matching schema names do not establish that both services read the same deployment's database.

Sources: parity cloudflare/src/worker.js:154-155, cloudflare/src/test-lab-api.js:314-342; engine cloudflare-native/src/ingress.js:264-273, cloudflare-native/wrangler.jsonc:73-80.

Experiment catalog

The API declares its catalog in EXPERIMENTS. running is a catalog label, not a live probe of enabled lanes or successful dispatches.

All entries declare duration_ms as their metric. Planned entries have no span prefix or variants and return zero runs. They are not proof that planner, model-comparison or hypothesis experiments are active. The engine's V5 activation flags are absent from native container environment forwarding.

Sources: parity cloudflare/src/test-lab-api.js:12-55,269-282; engine cloudflare-native/src/container-env.js:15-70.

Sampling and lane interpretation

The API selects tool spans named under validation.. It first chooses recent trace IDs, then fetches all matching validation spans for those traces; the limit counts runs/traces, not spans. A second-lane row is not dropped just because the page boundary falls between two spans.

Each run contains identity fields, prUrl, nullable agree, and a lanes map keyed by executor. Executor and role come from attributes, with name fallbacks for validation.cf-sandbox and validation.e2b and a default role of primary. Multiple spans for one executor in one trace overwrite the same map entry; this is not a multi-sample-per-executor aggregation.

The engine's passed attribute means a plan or labeled steps existed, the lane was available, and its findings array was empty. It is not an independent replay of exit codes or a claim that all desired tests ran. An unavailable lane or no executed plan yields passed: null. Legacy spans without passed are treated as passed only when both stored findings and missing-validation counts are zero; otherwise their verdict is unknown.

Raw step output is not included by validationSpanAttributes; step metadata contains labels and exit codes. This narrower rule is not universal trace redaction.

Sources: parity cloudflare/src/test-lab-api.js:84-152,171-215; engine src/validation-executor.js:205-245.

Metrics

Agreement says the stored verdicts match, not that either verdict is correct. Null agreement is not disagreement. Rates and percentiles are over the bounded selection, not all historical reviews.

Lane cost prefers the span's cost_usd, then the costUsd attribute. The engine's wall-clock estimates and numeric-coercion limits are described in Observability and cost; these are not invoice totals.

Sources: parity cloudflare/src/test-lab-api.js:120-159,193-265; engine src/validation-executor.js:189-245.

Shadow validation configuration

The checked-in native Worker configuration selects:

GILF_VALIDATION_EXECUTOR=managed-cf-sandbox
GILF_VALIDATION_SHADOW_EXECUTOR=managed-e2b
GILF_VALIDATION_SHADOW_TIMEOUT_MS=600000
GILF_E2B_TIMEOUT_MS=600000
GILF_E2B_BUDGET_USD=50

These are repository configuration values, not a live status report. The forwarding allowlist also includes E2B_API_KEY, E2B shape/template settings and GILF_VALIDATION_COST_RATES. Secret provisioning belongs in Validation executors, not an unauthenticated Test Lab request.

The shadow selector must resolve to a supported, different executor with a wired collector. Legacy local and shadow routes are not valid comparison lanes. An unwired shadow collector does not run and emits no comparison sample.

Sources: engine cloudflare-native/wrangler.jsonc:136-150; cloudflare-native/src/container-env.js:55-69; src/validation-executor.js:251-267,389-396.

Shadow affects timing and spend, not findings

runValidation starts primary and shadow collectors concurrently. The shadow result is converted to telemetry and is not merged into primary findings, missing validations or validation summary. Exceptions and timeout results become error-shaped lane attributes.

However, the runner can wait for the shadow lane after its model call, using the remaining timeout measured from validation start. The default is ten minutes total, not ten extra minutes after the model finishes. The wait can delay publication. A promise timeout does not itself cancel the underlying collector; collector timeouts and cleanup remain important.

Do not call shadow validation free, harmless or incapable of delaying a review. Its verdict is observational, but its latency and compute costs are real.

Sources: engine src/validation-executor.js:378-473; src/codex-review-runner.js:1112-1149.

Budget is an estimate gate

The native dispatcher reads all-time SUM(cost_usd) for spans named validation.e2b. At or above GILF_E2B_BUDGET_USD (default 50), it withholds only the E2B shadow selector. A missing database binding or failed spend read leaves the selector unchanged.

The SQL has no repository, date or role filter. It includes any matching E2B span, not just the rows selected on screen. It does not reserve spend before dispatch, account for unflushed runs, or charge missing telemetry. Concurrent work and provider billing can exceed the displayed ceiling.

Test Lab's usedUsd uses the same query shape; limitUsd is read from the API Worker's own GILF_E2B_BUDGET_USD. Agreement with the native gate requires the same underlying data and limit configuration. Changing only the UI/API limit does not change the native dispatcher.

Sources: engine cloudflare-native/src/container-env.js:100-141; parity cloudflare/src/test-lab-api.js:304-340.

API and UI

Parameters: days defaults to 30, clamped to 1–365; limit defaults to 200, clamped to 1–1000; repo is an optional exact filter. Numeric inputs are truncated to integers; invalid numbers fall back. An authenticated request without REVIEW_STATE_DB returns 503 backend_not_configured; an unknown experiment ID returns 404 after the data-loading step succeeds.

For a parity API base URL in OPERATOR_URL:

curl --fail-with-body -sS \
  -H "Authorization: Bearer $GILF_OPERATOR_READ_TOKEN" \
  "$OPERATOR_URL/operator/api/test-lab/validation-executor?days=14&limit=200&repo=owner/name"

The API requires the read token or an allowed read API key. The parity browser uses BFF session authentication instead, with /api/test-lab and /api/test-lab/:id; those client calls use a 30-second timeout. UI routes are /test-lab and /test-lab/:id.

Sources: parity cloudflare/src/test-lab-api.js:63-68,314-342, cloudflare/src/worker.js:80,117-133; UI src/lib/test-lab-api.ts:82-102, src/App.tsx:24-25.

No experiment or runtime was launched during this source audit. Cost and traces explains how to inspect the underlying observations.