SEIF Docs

Historical investigation and reproduction

IMPLEMENTED in 0.6.0. Open Observe, select a trace, and choose Historical investigation. The ordinary trace and question-based Investigation remain available. This workspace answers what the trace records about context; it does not establish why a model produced an answer.

  • At the time identifies the exact original Build, full artifact digest, runtime execution identity, source revisions and material availability. Inspect retained source material to read verified bytes from that Build. Rendered context is a separate historical observation. Application-reported consumption is never independent model-processing proof.
  • Now & compare evaluates the latest verified current Build with the retained historical input in the recorded resolution environment and at the historical instant. Save and Build a candidate first. This isolates configuration changes; it does not claim what a live application receives today. When input or dependencies are unavailable, the comparison is explicitly configuration-only.
  • Reproduce runs the original resolution and records a separate receipt. It never substitutes the latest Build or fetches today's live source.
  • Improve opens the current responsible rule/source or creates a Context Test draft. Supply the expected included/excluded layers yourself, review the copied input, then Confirm expected behavior in Test → Save → Test → correct → Build → Regression. Confirmation binds the scenario and expectations; edits require confirmation again. Deployment approval remains separate.

The investigation reference binds the trace id and observed-envelope digest. Later consumer reports or policy expiration produce a different observation snapshot of the same execution. Reproduction receipts reference that snapshot and never replace the original trace. A trace is an observation, not governed Evidence or authority.

Exactness

EXACT_REPRODUCTION requires all of: the verified full original artifact; matching runtime implementation, executable, Node version, OS and architecture; intact retained input with its digest; recorded resolution environment and original evaluation instant; deterministic embedded or verified retained dependencies; identical semantic decisions, including budgets, selection, projection and unknowns; and identical rendered digest. It proves reproducibility of the recorded context resolution, not the original execution, an entire incident, application effects or model processing. Runtime fingerprints are producer observations, not a hardware attestation.

State / categoryMeaning
EXACT_REPRODUCTIONEvery condition above is verified
EQUIVALENT_RENDERED_CONTEXTText matches, but at least one execution or decision condition does not
DETERMINISTIC_MISMATCHSame runtime, but semantic or rendered behavior differs
BUILD_MISSING, BUILD_INTEGRITY_FAILED, BUILD_CONTRACT_INCOMPATIBLE, BUILD_PROVENANCE_INCOMPLETEThe responsible artifact cannot be verified and used
METADATA_ONLY / MISSING_HISTORICAL_INPUTOriginal input missing, redacted, oversized or changed
MISSING_HISTORICAL_ENVIRONMENTA required historical environment coordinate is absent or malformed
HISTORICAL_ENVIRONMENT_UNAVAILABLEThe recorded environment cannot be resolved from the historical Build, including ambiguous legacy deployments
RETENTION_RESTRICTEDRetained content expired
LIVE_DEPENDENCY_UNAVAILABLE / MISSING_HISTORICAL_MATERIALSuccessful provider result lacks intact retained content
LIVE_DEPENDENCY_UNAVAILABLEMutable live source bytes were never pinned
NONDETERMINISTIC_EXTERNAL_DEPENDENCYExternal retrieval ranking is not retained as a deterministic dependency
RUNTIME_CONTRACT_UNAVAILABLERuntime contract unsupported, or output differs under another runtime identity
UNKNOWNRequired historical decisions or evaluation coordinates are incomplete

Compatibility states METADATA_ONLY and LIVE_DEPENDENCY_UNAVAILABLE retain the existing API; category, reason and blockers explain the narrower failure. Legacy traces without the full artifact/runtime fingerprints can achieve equivalent text, never exact reproduction. They are not backfilled from current state.

Environment contract

A deployment can serve production using a Build compiled with development as its policy basis. The deployed server deliberately resolves against that compiled basis (ADR 003). The resolver itself also accepts an explicit environment declared in its supplied design. These are different coordinates, so replacing the resolver input with either the deployment label or the current Build default is incorrect.

New context.request.v1 records preserve environment (deployment/request) and resolutionEnvironment (the actual resolver result). Investigation, reproduction receipts and comparisons expose both. Reproduction and candidate comparison use the recorded resolutionEnvironment; incident tests retain it as their explicit environment, with the operational environment in provenance. An unsupported candidate environment produces CANDIDATE_REQUEST_INCOMPATIBLE / UNKNOWN_ENVIRONMENT, never a fallback.

Legacy traces use the recorded request environment only when it is supported by the historical artifact and does not conflict with a recorded deployment's compiled basis. Missing coordinates or an ambiguous legacy production label block reproduction and test drafting; configuration comparison remains available. No legacy trace is rewritten.

Comparison reports label context drift as DETECTED, NOT_DETECTED or UNKNOWN. They separately expose semantic, rendering and configuration differences. regression: NOT_EVALUATED is intentional: a historical observation supplies no quality expectation. Use a confirmed Context Test and the regression report to evaluate one.

Materials and source capabilities

Pinned local, Git, HTTP or MCP material embedded in a verified Build is available regardless of whether the original connector still works. Its retained bytes, source revision and connector metadata are inspectable. A revision id alone cannot prove material bytes. Missing Builds leave trace provenance inspectable with REVISION_KNOWN_CONTENT_UNAVAILABLE or UNKNOWN.

Live provider results replay only when content retention was explicitly enabled and both item content digests and the complete retained-item metadata digest verify. A recorded provider failure can be replayed as an isolated fixture. A current HTTP endpoint, mutable database or Git branch is never used to invent yesterday's data. Automatic recovery from a missing artifact via remote immutable Git is FUTURE; this version uses the existing immutable Build store. Live ranking and mutable sources without retained bytes remain unreproducible.

Registry API and local operation

All routes retain organization/project scope. GET /v1/traces/:id/investigation requires trace.view; without trace.content, input/messages/provider content are masked. Loading historical Build bytes also requires build.view. POST /v1/traces/:id/reproduce additionally requires trace.reproduce. Its body must be empty: no supplied input, connector, endpoint, command or effect override is accepted.

POST /v1/traces/:id/compare accepts only a registered candidate id; POST /v1/traces/:id/material accepts source and optional material digest; both require historical content and Build access. POST /v1/traces/:id/test accepts a name and explicit expectations and additionally requires regression.run. It returns a draft, never changes a production environment. Local desktop operations use the local owner's filesystem authority. Do not share that account with metadata-only viewers.

Reproduction receipts use existing Registry records (bounded by environment trace count), or the project's .workstation/reproductions directory (100 receipts). They retain result/criteria/diffs, not request or rendered bytes. An investigation is a projection over existing stores, not a redundant history database. Privacy defines retention and Context Tests explains expectation provenance.

Proof and limits

tests/engine/historical.test.mjs exercises real runtime HTTP resolution, two Builds, replay and adversarial cases. scripts/prove-historical.sh drives the native Workstation with an independent Python application. The release pipeline repeats that journey from the DMG under an OS sandbox denying repository/toolchain access, and attests its report and screenshot digests to the final source commit.

FUTURE: replay using archived older runtime binaries; historical remote artifact recovery; copying confidential provider fixtures into tests; full temporal database reconstruction; live current-provider comparison; independent model-processing evidence. No automatically inferred expectation is presented as ground truth.