Logs, Derived History, Replay, and Forking
From raw chunks to reconstructable model input and UI state
Conclusion: there is no single “Replay,” but six reconstruction contracts that cannot substitute for one another
DeepSeek Harness builds raw streams, semantic messages, the model Surface, physical persistence prefixes, browser windows, and Fork lineage on the same seq-numbered log, but it does not collapse them into one universal replay API. A model request can be reconstructed from the log; a failed attempt remains auditable; a crash can be closed deterministically; a disconnect can converge through history plus the live stream; and a Fork copies a stable prefix rather than pointing at a dynamically changing parent branch.
The most important property is not merely that “chat history is saved.” Different consumers must declare which truth they need. The costs follow from that separation: logical history may be more complete than the physical on-disk log, the UI may deliberately hide a failed attempt, and a published Fork does not imply that parent and child were atomically persisted.
1. Six operations commonly conflated as Replay
| Contract | Input | Output | What it does not guarantee |
|---|---|---|---|
| Model Surface replay | The complete event log | Current deriveMessages() | It does not retain old context shadowed by a replacement |
| Request reconstruction | The log prefix before request dispatch | Messages plus the latest complete request header | It does not reproduce network latency, thrown failures, or hangs |
| Stream replay | assistant/chunk | Streaming blocks, usage, finish, and partial UI | A failed retry's partial content need not remain visible |
| Crash recovery | A valid but unclosed durable prefix | Deterministic synthetic results, step end, and turn end | It cannot infer the real external outcome of a started side effect |
| Client convergence | A raw History window, subscribed baseline, and live frames | A contiguous browser event window and UI projection | It is not exact stream-offset resume |
| Fork | A stable completed-Turn prefix | A frozen copy with its own Session ID | Later parent events never flow into the child |
“Replayable from the log” always needs an object. Reconstructing the next Provider request does not replay an external tool side effect; recovering the UI does not mean the UI displays every raw attempt; creating a child does not provide an atomic durability transaction across two Session IDs.
2. Every model attempt records the raw stream before committing a semantic message
Read the current model view
Every attempt calls session.deriveMessages() again and combines it with the system prompt, tools, and request configuration already fixed for the Step.
Record every chunk
Each item from the Provider iterator is first appended as assistant/chunk, and its seq enters this attempt's chunkSeqs.
Assemble canonically
BlockAssembler folds interleaved blocks, deltas, usage, finish, and adapter-private replay state.
Decide the attempt's fate
An error or aborted finish enters the request-error waterfall; a retry action returns directly to the loop without appending assistant/message.
Commit the successful anchor
Success appends an append-origin assistant/message whose sourceEventSeqs cite every chunk in this attempt.
Raw observation precedes semantic commit. A failed attempt therefore leaves chunk evidence without contaminating the model Surface. The successful message carries provider, model, optional replay state, and usage before the loop decides whether to execute tool calls.LOG-ATTEMPT-ASSEMBLY
A fixture producing abc requires all seven protocol chunks to remain in order and be reassembled. Dependencies are absent, so this study reviewed but did not execute the test.LOG-RAW-CHUNK-TEST
3. Retry preserves both audit truth and current user-visible truth
| After a failed attempt | Raw log | Model Surface | Browser assistant node |
|---|---|---|---|
| Chunks already received | All remain, including the terminal failure chunk | They do not enter because there is no final assistant message | llm/retry clears and hides the partial blocks |
| First-token time | Event timestamps remain | Irrelevant | Preserved across retry because the user really waited for and may have seen a response |
| Successful attempt | Appends its own chunks and final message | Projects only the final semantic message | The final message replaces partial stream state and settles |
The Assistant Definition folds chunks under Turn and Step identity. A retry reset never deletes log events; it only resets presentation state. If the Step or Turn closes with visible chunk evidence but no final message, it can create a view-only interrupted node.LOG-CLIENT-ASSISTANT
The user does not see two conflicting answers after an automatic retry, while an auditor can still inspect the failed attempt. These are two intentional projections of one fact source, not lost UI data.
4. “Every request is reconstructable” requires both the Surface and a complete Header
Request at dispatch boundary
= fresh Session(log prefix).deriveMessages()
+ foldRequestHeader(log prefix)
├─ provider / model / reasoning effort
├─ system prompt
├─ ordered tool schemas
└─ temperature / maxTokens / stopThe test runs multiple Steps, a tool call, and prompt and configuration changes, then finds the first response chunk for each dispatch. It creates a fresh Session from the events before that chunk and requires its messages to deep-equal the real request; it folds the latest request header and compares model, reasoning, system, tools, temperature, maxTokens, and stop field by field.LOG-REQUEST-RECONSTRUCTION
Exactness comes from two complementary logs: the Surface fold determines Message history, while the last-wins fold over complete header snapshots determines non-message request parameters. Raw chunks prove what the Provider returned; they do not participate in reconstructing the input of their own request.
5. An empty completion can still be a durable success anchor
When a Provider stops normally without visible content, or max-token truncation makes the assembler discard an unsafe partial tool call, the Agent still appends an assistant/message with empty content and cites that attempt's chunks. Chapter 09 established that an empty assistant does not enter deriveMessages(). Here its purpose is to record that a request completed successfully, together with its usage and replay boundary, rather than adding an empty model turn.
If only a raw terminal chunk remained, a generic consumer would need to understand every adapter stream protocol to recognize the success boundary. One uniform semantic anchor lets request replay, usage, and Turn completion remain Provider-neutral while keeping the next request's Message list clean.LOG-ATTEMPT-ASSEMBLYSESSION-SURFACE-PROJECTION
6. Persistence exposes four distinct read faces
| API | Cold source behavior | Commits recovery? | Publishes a live Session? | Main use |
|---|---|---|---|---|
prepare(id) | Read, repair, validate, and reserve the exact Session object | Yes | The caller later publishes or rolls back | Resume ownership transaction |
load(id) | Return a balanced logical inspection | Yes | No | Readers needing a durably recovered view |
inspect(id) | Return a balanced logical inspection | No | No | Cold history, query, and Fork sources |
readFrom(id, seq) | Read the stored physical suffix directly | No, and adds no closers | No | Incremental storage, replication, and raw inspection |
The four paths share per-session serialization and format validation but deliberately choose different recovery and ownership semantics. With a seek-capable backend, readFrom reads only the suffix; otherwise it reads the complete prefix and slices it, without borrowing the prepared cache.LOG-PERSISTENCE-READ-FACES
For a live Session, inspect may return an in-memory snapshot whose Turn is still open. load flushes first and refuses to package an open live Turn as a “complete durable inspection.” It is not enough to ask whether a function returns events; its promises about logical balance and ownership matter.
7. Crash repair never invents success; it deterministically closes unknown state
| Crash-tail fact | Synthetic result | Guidance to the later model |
|---|---|---|
The assistant requested a tool but no tool/call exists | TOOL_NOT_STARTED error result with no source seq | Retry if the operation is still needed |
A tool/call exists but no tool/result does | TOOL_OUTCOME_UNKNOWN citing the call-event seq | Check external state or ask first; retry only read-only or idempotent work |
| The Step remains open | Add step/end | Restore structural invariants without claiming completion |
| The Turn remains open | Add turn/end {kind:'interrupted'} | Expose an explicit interruption to the next Turn |
The scanner resets pending calls at each Turn boundary and repairs only the final open Turn. Synthetic seqs continue after the last real seq, timestamps reuse the last real event, and Map insertion order preserves call order. A balanced or empty log produces no events.LOG-REPAIR-CLOSERS
A backend may return a valid committed prefix plus a torn marker. Recovery truncates the incomplete physical fragment, retains every committed open-Turn fact, and appends closers instead of rolling back the whole Turn.LOG-TORN-TAIL-CONTRACT
8. Cold history can be logically complete while disk is still physically open
An inspect() result uses storedEvents + synthetic closers but does not call commitRepair. Only load() or prepare() commits recovery after the revision still matches. Contract tests explicitly require the backend revision to remain unchanged across inspect and to change after load.LOG-REPAIR-CONTRACTLOG-PREPARED-COMMIT
physical artifact: turn/start → tool/call ───────────────┐
│ inspect only
logical history: turn/start → tool/call → unknown result → step/end → interrupted turn/end
load / prepare: compare revision → commit repair → reload exact committed graphA cold browser History can therefore show a safely closed interruption while readFrom or the raw artifact still exposes an open physical tail. The former answers, “What is this conversation logically under the recovery rules?” The latter answers, “What bytes and events did the backend actually store?” Forcing them into one view would remove diagnostic information.
9. Write-behind separates hot-path commit from disk durability
The persistence listener clones each committed event into per-Session write-behind. A fixed deadline forms a stable batch. If a write fails, the batch returns to the front in original order and automatic retry pauses, avoiding a tight background loop. Explicit flush() cancels the timer, joins a shared barrier, and retries and drains until both the active write and pending queue are quiescent.LOG-WRITE-BEHIND
The standard policy flushes before the model stream is actually constructed, before a top-level tool body, and before every pre-step. Failure prevents dispatch of the adapter or side-effect body; nested tool calls reuse the already durable outer call.LOG-CHECKPOINT-POLICY
| Boundary | Facts that must already be durable | Ambiguity prevented |
|---|---|---|
| Before a Provider request | User input, request header, Step start, and dynamic context | The model was called but disk cannot reconstruct its input |
| Before a top-level tool body | Assistant tool call and tool/call | A side effect occurred but the log does not even record its start |
| Before the next Step | The previous Step's response and tool results | The next request depends on history that is not durable |
10. Core Fork is an inclusive prefix copy from a live source
SessionStore.fork accepts only the exact live instance currently in the store. An omitted boundary means the final event; an explicit value must be an existing contiguous seq, and the inclusive slice may not end inside an open Turn. The child header inherits cwd and records parentSession plus the exact copied-prefix seedLength.LOG-CORE-FORK
- An empty source can produce an empty child with lineage.
- If the source now has an open Turn, an older stable boundary before that tail remains selectable.
- Title events, plugin markers, and other log-only events after a Turn can enter the prefix.
- Turn brackets are the hard stability boundary; a plugin bracket crossing the prefix can be inherited as seed, with the new end-seed marker separating ownership.
- The seed is detached, snapshotted, and frozen; parent and child do not share a mutable event array.
The tests cover empty forks, frozen seeds, an earlier cut before an open parent tail, every Turn-end reason, trailing log-only events, and rejection of missing or stale instances, invalid boundaries, open Turns, and duplicate child IDs.LOG-FORK-CONTRACT
11. Host Fork lifts a message anchor into “include the whole Turn” product semantics
| Input case | Selected completed boundary |
|---|---|
atSeq lies in a completed Turn | The first turn/end.seq ≥ atSeq, so a message's Fork button does not clip the latter half of its Turn |
atSeq lies in the current open Turn | Reject; never silently fall back to an earlier Turn |
atSeq is omitted or beyond the log tail | The last completed Turn |
| Standalone events follow the boundary | Include them through the event before the next turn/start |
The Host can read attached live state or cold inspect state without resuming an Agent. It copies the prefix, inherits cwd, lineage, seedLength, and the source's current composition, creates the new Agent, and then attempts to inherit the Workspace. If Workspace attachment fails, the response explicitly carries the already created child ID; the child is not implicitly rolled back because peripheral grouping failed.LOG-HOST-FORKLOG-HISTORY-CONTRACT
This is a snapshot copy, not a Git-like shared object database or an event-branch pointer. Parent and child each own a complete prefix beginning at seq zero, then append independently; only the Header links their lineage.
12. A clear crash window separates Fork publication from cross-Session durability
The Host does not flush a live parent before reading it. After child creation, the session/created listener starts seed persistence through asynchronous initFor(). Parent and child writes serialize only on their respective Session-ID chains; no cross-ID transaction exists. A successful “forked” response therefore proves that the child was published in the live registry, not that the parent prefix and child seed are atomically durable together.LOG-PERSISTENCE-ADMISSIONLOG-HOST-FORK
This does not corrupt child content during a healthy process: the child already copied the complete in-memory prefix. The risk exists only when the process crashes or a backend write fails at the relevant moment. The child may not yet be materialized, or the child may be stored while the corresponding parent tail remains in write-behind. Explicit flush is the authoritative durability primitive, and the Fork API does not run a two-sided barrier before returning.
13. History pagination preserves a contiguous raw range instead of returning Message DTOs
The Host counts append-origin user and assistant messages backward from the window tail. Model-only replacements consume no maxMessages quota. An assistant message's sourceEventSeqs moves the cut to the earliest chunk in that attempt, so a page never begins halfway through a stream. The result remains a contiguous raw event range containing all event types.LOG-HISTORY-PAGINATION
The tail page naturally contains partial chunks that do not yet have a final message; older pages are cut only by beforeSeq. Each entry may carry a tool view that the Host computes from the current presenter registry, but that view is never persisted and is not part of the event envelope. If call arguments live on an older page, the current-page backscan can miss them and intentionally fall back to generic presentation; the fact itself remains intact.LOG-HISTORY-VIEW
After all asynchronous source and composition preparation, the Host synchronously copies events and reads the projection watermark so one History response cannot combine events at N with a baseline at N+1. A cold source uses logical inspect, and the entire read never publishes an Agent.LOG-HISTORY-CUTLOG-HISTORY-CONTRACT
14. The browser converges through “fetch tail + buffer live + repair by seq”
subscribe starts ────────────────┐
├─ liveBuffer
history tail request ──────────┘
↓ install raw window
drop seq overlap → append contiguous live tail
↓
if next.seq > tail.seq + 1: buffer + refetch tail + restitchOpen first fetches History. If session/subscribed.lastSeq is already beyond the page tail, it immediately fetches again. After installing the window, it stitches frames received in the meantime by seq: overlap is dropped, while a hole is buffered and triggers a tail repair instead of being appended directly.LOG-CLIENT-STITCH
ConversationNodeAssembler rebuilds Contexts on full replacement, matches only one tail event on live append, and preserves existing Context and View identity while locally replaying dependencies when older pages prepend. Business Definitions decide how assistant, tool, Turn, and other events become visible nodes.LOG-CONVERSATION-ASSEMBLER
Reconnect therefore does not mean “continue from a server stream offset.” It establishes a durable raw-History baseline, then uses seq to deduplicate and repair live frames. Seq simultaneously provides ordering, deduplication, gap detection, and UI correlation.
15. Query trace and raw export answer different audit questions
| Capability | What it does | Evidence boundary |
|---|---|---|
| Event records | After a canonical fold, label every event current, shadowed, or log-only | Status belongs to this complete-log observation |
| Event trace | Return the direct replacement chain, removed nodes, direct sources, and later direct citers | Except for replacement chains, it does not compute full transitive provenance |
| Session trace | Build ancestors and recursive descendants from Header parent links | A missing parent returns complete:false rather than inventing a root |
| Raw ZIP export | After flushing live logs, export backend artifacts byte-for-byte with optional descendants and referenced media | It never rewrites UI views or the current Surface into the artifact |
Query is an explainable computation over the logical event graph; Export preserves physical evidence. The former answers why a node was shadowed and where it came from, while the latter lets independent tooling inspect the original files again.LOG-QUERY-TRACELOG-RAW-EXPORT
16. Confirmed boundaries, gaps, and scale risks
The wire schema requires maxMessages only to be a positive integer and sets no maximum. Pagination also first copies or filters the complete event log, while one message group can itself contain many raw chunks. A caller can therefore request a very large page, and one huge attempt can make a response much larger than its “message count” suggests. This is a source-visible resource boundary; practical exploitability still depends on upstream log and transport limits.LOG-HISTORY-SCHEMALOG-HISTORY-PAGINATION
The client catch path only logs the error and clears stitching; buffered live events remain. If no later frame triggers the gap again and no reconnect or resync occurs, they can remain invisible indefinitely. A later relevant event or reconnect converges again, but the current implementation has no backoff timer.LOG-CLIENT-STITCH
This is the crash-window inference from the prior section. If product language treats the response as “permanently created,” it needs another parent-and-child flush or backend transaction. If it promises only live child admission, the API documentation should state that boundary clearly.
- Intentional tradeoff: failed-retry content is hidden in the UI and retained in the raw log.
- Intentional tradeoff: a tool render view is a current-code projection and may use a generic fallback when call arguments sit on another page.
- Intentional tradeoff: logical inspection can expose deterministic recovery without forcing a read-only page to mutate disk.
- Unknown: this study did not execute fault injection, oversized-History, or real disconnect experiments; source deductions must not be presented as reproduced incidents.
17. Design assessment and verification status
| Choice | Benefit | Cost |
|---|---|---|
| Record raw attempts and semantic messages separately | Failures remain auditable without contaminating model context | Consumers must know which layer is current truth |
| Repair expresses only unknown and interrupted states | It never guesses that a side effect succeeded or failed | The recovered model still has to inspect the external world |
| Separate logical inspection from physical reads | Cold reads stay side-effect-free while presenting balanced UI | “Visible in history” does not always mean “committed to disk” |
| Fork copies the complete prefix | The child is self-contained, independently recoverable, and exportable | Storage is duplicated and cross-ID atomicity requires another mechanism |
| History convergence instead of stream-offset resume | Disconnects, overlap, and missing frames return to durable truth | Pages may be fetched again, and repair retry policy remains incomplete |
This chapter cross-checked production control flow in the Agent loop, repair, SessionStore Fork, Host History and Fork, persistence coordinator and write-behind, client Session and Conversation assembler, query, and export. It also reviewed request-reconstruction, Fork, repair, torn-tail, and persistence-contract test source. Upstream dependencies are not installed, so it does not claim those tests passed locally. The next chapter dissects backends, database schemas, and data ownership.
My Learning Notes
Autosaved only in this browser. Nothing is uploaded or committed. Export Markdown whenever you want to keep a copy.