Context reliability

Open Deploy → Reliability after registering a Build. Each environment has an explicit drift baseline, optional scheduled checkpoints, internal alerts and a Flight Recorder policy. The Registry stores these settings; closing the Workstation does not stop a separately hosted Registry.

Governed drift checkpoints

Choose a baseline Build, interval (60–86400 seconds; default 900) and whether to enable scheduling, then Save monitor. Saving establishes a new baseline evaluation time and retires the previous monitor's active alert. Check now works with scheduling disabled. Enabling a schedule makes its first check due immediately; the scheduler polls every 30 seconds and processes at most ten due environments per tick. A successful check advances the persisted slot. A durable five-minute lease fences competing controllers; after a crash the overdue slot is recoverable when the lease expires. Stale completions cannot replace a newer lease. Missed historical intervals are not replayed. The process deployment adapter permits only one Registry process per data directory; a competing process fails before resetting active deployments. This is single-process safety, not HA.

Each checkpoint compares the retained baseline against the environment's last deployed Build, or its promoted Build when there has been no deployment. The report labels this distinction. It does not probe whether the deployment is currently healthy or query an upstream source for newer revisions.

The corpus retains both Builds' Context Tests and inherited incident obligations. Each case keeps its resolver environment. Baseline cases run at the monitor's baseline time; candidate cases run at checkpoint time, including tests normally pinned to a historical date. evaluation.clock=CHECKPOINT_TIME makes this deliberate freshness probe explicit. Ordinary regression and incident replay retain their original time semantics. Saving the monitor again resets the baseline clock.

Reports separate:

• Drift: DETECTED, NOT_DETECTED or UNKNOWN, based on semantic configuration, rendering, decisions and expectation changes.
• Expectations: REGRESSION for a previously passing Context Test that now fails; EXPECTATION_UNMET for failures already present; UNKNOWN for unknown tests; PASS when the explicit tests pass. This evaluates structural context expectations, not model answer quality.
• Environment policy: the current environment's regression policy can block stale required context, missing incident guards or other release obligations even when individual test expectations pass. A checkpoint observes this decision; it does not promote or roll back a release.
• Coverage: PARTIAL when tests are unknown or provider fixtures are missing. A passing assertion does not imply complete dependency coverage.

Execution uses immutable Build materials and synthetic provider fixtures, without live provider calls. Missing/corrupt artifacts, unpinned live sources or external ranking yield UNKNOWN with a reason. Configure a baseline with 1–200 tests; the combined baseline/candidate corpus is bounded to 200 distinct cases. A larger candidate yields an explicit capacity blocker, never a silently sampled success. Changes to the target, policy or monitor during evaluation invalidate the result.

Alerts open on detected drift, unknown coverage or unmet expectations/policy. Repeated equivalent findings share an alert and preserve acknowledgement; recovery or a changed finding resolves it. An acknowledgement does not approve a release or establish a new baseline. Alerts appear in the Workstation and API. The operational feed and separate webhook worker integrate external notifications; the Registry itself makes no notification requests. Keep the Registry online for scheduled checks.

The Registry retains the latest 100 checkpoints and 100 alert records per environment; history returns the latest 50. Stored reports contain up to 100 configuration rows, 20 changed cases and 30 change rows per case, with truncation labelled. Full Build comparisons remain available through Regression. Checkpoint reports may contain test expectations and configuration values; they require Build and environment read access.

Flight Recorder

The incident profile normally decides at capture/ingestion. Optional incidentWindowSeconds (0–3600, default zero) temporarily keeps eligible successful traces inside all existing lifetime/budget/privacy limits; a failure reported while that content is still present qualifies it as an incident. A nonincident trace becomes metadata when the window ends. A successful trace discarded then cannot be recovered when the application reports a failure later. Use replay when late incident discovery is important. Neither profile guarantees exact reproduction: the original input, Build, compatible Runtime identity and required dependency content must still be available.

Content lifetime is 1–604800 seconds (default one day). Content budget is 0–536870912 bytes (512 MiB; default 64 MiB for content profiles). Zero retains no content. Accounting counts the complete compact UTF-8 JSON envelopes that still retain content. It bounds retained logical trace payloads separately for each Registry environment and each Runtime deployment. It does not bound total disk use, metadata, immutable Build archives, WAL files, backups or SQLite free pages. Trace count remains independently bounded by maxTraces (1–100000).

When the content budget is exhausted, oldest content becomes metadata with reason RETENTION_BUDGET_EXCEEDED. A trace larger than the entire budget immediately becomes metadata. Deadlines can be shortened, never extended; discarded input/messages/provider results never reappear on redelivery, restart or policy relaxation. Content with no provable original expiry is downgraded instead of receiving an invented future deadline. Provider permissions, sensitive material flags and existing sanitization limits continue to restrict capture.

Saving a stricter policy immediately reduces existing Registry content and applies to subsequent ingestion. A running Runtime keeps its deployment policy until redeployed; the UI displays a pending capture-policy change. Each deployment and trace records its capture policy and digest. Runtime/Registry reads enforce lifetime, with the existing background sweeps handling idle records. Offline stores and backups need the operator's retention process. Metadata is not anonymous; see privacy.

API

All routes are authenticated, organization/project/environment scoped. Configuration requires environment.configure (Admin). Manual checkpoints and acknowledgement require regression.run; checkpoint/history/status reads also enforce environment.view and build.view as applicable. Application credentials cannot access these routes.

The Runtime accepts --flight-recorder-policy '<JSON>' or flightRecorder in its configuration file. Existing traceRetention/--retention and contentRetentionSeconds remain compatible when no recorder policy is supplied. Changing legacy environment retention switches profile only if the retention mode changes, preserving an incident profile when editing release policy. SQLite schema v3 adds content accounting and due-time indexes; existing v1/v2 stores migrate on open.

This closes a practical loop: investigate an incident, confirm an expectation, carry its guard into subsequent Builds, observe drift at governed checkpoints and retain enough permitted evidence to investigate the next incident. Cross-source live monitoring and binary/runtime archive management remain future work. Reliability Impact explains retained evidence, external evaluations and storage diagnostics.