Context Regression

IMPLEMENTED: choose Build A, Build B and their retained test corpus in Deploy → Regression. Load test suite… accepts a JSON Context Test array for an exploratory comparison while retaining baseline obligations. The headless interface also accepts --suite file.json for an explicit corpus. The report connects Build diff, assertion differences, per-layer contributions and sample resolutions down to material/source revision. Build changes share one recorded evaluation instant unless a case pins its own historical instant.

The default corpus unions baseline and candidate test definitions. Removing or weakening a test in a candidate cannot delete the previous release's obligation. Distinct changed tests retain both definitions. An explicit test environment is preserved on both Builds. Ordinary tests without an environment inherit the default of the Build that contributed them to the corpus, so a candidate cannot silently change the coordinate of a baseline obligation. Historical tests without an environment fail closed. Explicit suites in exploratory reports do not override the Registry's promotion gate corpus.

Metrics include labelled required-context inclusion and forbidden-context leakage (with actual denominators), unknown applicability, stale rendered context, budget failures, average estimated context tokens, selected material count, tests pass/unknown, and per-layer material/token/trim contributions. Missing labels yield null/unknown, never invented accuracy or precision. Source revisions, rules and rendered layers are compared. These are deterministic checks, not an LLM quality score.

Default policy blocks required test failures, required misses, forbidden context inclusion, stale required context and budget violations. Token growth over 20% and increased unknown applicability warn. Configure blockRequired, blockForbidden, blockStaleRequired, blockBudget, requireTests, warnUnknownIncrease and warnGrowthPercent through environment policy. Additional gates include requireApproval and requireSignature.

A deliberate engineering-notes leak is exercised by the commercial journey: registration succeeds as a test-failing candidate, promotion is BLOCKED, a corrected Build passes and the exact same signed artifact reaches production. Reports provide JSON, Markdown and JUnit. Warnings do not mean causal quality loss; PASS means the configured checks passed, not universal agent correctness.

Live providers (0.4.0)

Builds that require Runtime Context Providers are compared deterministically: each Context Test supplies synthetic provider fixtures, and the report states evaluation.providers = FIXTURE and evaluation.determinism = DETERMINISTIC. Tests whose provider has no fixture evaluate that layer as UNKNOWN and add the PROVIDER_FIXTURE_MISSING warning — they never pass by default. A live evaluation must be explicit (compareRegression(..., { providers }) with a binding's clients) and is labelled LIVE / NONDETERMINISTIC_INPUT with the NONDETERMINISTIC_INPUT warning; promotion gates always use fixtures. The semantic Build diff includes provider requirement changes.

Context CI runs the same comparison in pull requests against the target environment's current Build — see Context CI.

Incident → Regression Guard

A historical test carries a trace reference and observation digest through the versioned project, immutable Build, CI report and promotion gate. Confirm its expectation in Test, correct the design, then validate and Build. The same guard remains an obligation in future comparisons even if the candidate deletes or weakens it. A reviewed version can supersede an identical draft; it cannot supersede different input, coordinates or expectations. Under blockRequired, the candidate artifact must also carry each previously confirmed required incident guard. Otherwise REQUIRED_INCIDENT_GUARD_NOT_CARRIED_FORWARD blocks promotion and missingGuards identifies what to restore, even when the inherited test passes today or appears in an exploratory suite. This prevents a harmless-looking deletion from erasing protection in the next generation. Renaming preserves the scenario digest; changing its obligations requires retaining the original guard alongside the new one. Existing explicit policy overrides remain separate governance decisions.

JSON and Markdown reports expose incident guards, and Deploy → Regression displays their trace and evaluation coordinate:

A BLOCK is a policy decision. It can reflect an unmet expectation, missing coordinate or pending review without proving regression. Drift comparisons likewise describe differences without assigning a quality verdict. Existing JSON/Markdown/JUnit exports and CI/promotion enforcement use the same assertions.