DSHarness Systems Dissection Pinned baseline 47f943859b · 36 verified / 0 in progress / 36 chapters
中文
Sessions and Persistence·Chapter 12

Projections, Checkpoints, and Cold Start

Which disposable acceleration state exists beyond the complete event log

VerifiedUpstream 47f943859bScope: Distinguish semantic checkpoints, projection caches, watermarks, write-behind, tail replay, and recovery thresholds.

Conclusion: the event log owns truth; checkpoints and projections own timing and cost

DeepSeek Harness does not persist a second conversational truth beside the Session log. It keeps the append-only SessionEvent stream and SessionHeader authoritative, then layers three different mechanisms over them: semantic flush barriers before irreversible work, in-memory projection cells for fast current views, and durable projection checkpoints that shorten some folds. A browser receives yet another derived layer—watermarked baselines and non-durable push frames—and converges them back toward the log.

The architecture is strongest when those layers remain distinct. A semantic checkpoint proves that earlier facts reached storage; a projection checkpoint merely remembers a fold; a watermark says how far a value has observed; a baseline is a transport cut; and a client row is disposable convergence state. Treating any one of them as the log itself creates exactly the recovery bugs the design is trying to avoid.

1. Six names for six different contracts

LayerRepresentationAuthorityWhat it proves
Durable factSessionHeader + append-only SessionEventSource of truthWhich metadata and committed events the backend actually stores
Semantic checkpointAwaited sessions.flush(session)Durability barrier, not dataAll participating flush listeners settled for the admitted prefix
Live projection cell{state, observedSeq} per unit and live SessionDisposable derived stateThe last event passed through that unit's fold
Projection checkpoint{ver, seq, val} per keyDurable cache onlyA versioned fold shortcut bound to one log lifecycle
Host baseline{asOfSeq, values}Derived wire snapshotOne consistent cut suitable for seeding a client
Client rowkey -> {value, seq}Disposable convergence stateThe newest accepted host value for that key
Persistence contract

The persisted unit is the existing Session event, not a parallel persisted-message model. Non-replayable metadata remains in the Header. Projection checkpoint types explicitly say that a row is never authoritative and must be discarded when its version or claimed log endpoint is unusable.ARCH-LOGSESSION-FORMAT-HEADERPROJ-AUTHORITY-LAYERS

2. A semantic checkpoint is an awaited barrier, not a stored checkpoint object

append facts to the live Session
        ↓
sessions.flush(session)
        ├─ persistence listener drains write-behind
        ├─ any other scoped durability listener settles
        └─ first failure is thrown only after all listeners settle
        ↓
construct model stream or enter top-level tool body

SessionStore.flush() resolves the exact live Session's scoped listeners, invokes all of them, waits with Promise.allSettled, and then throws the first rejection if any. Its return value only says whether a durability listener participated. There is no durable “checkpoint record” produced by this API.

Semantic policy

The standard policy places this barrier before the downstream model stream is constructed, before a top-level tool body can run, and at every agent/pre-step boundary. Model and tool boundaries fail closed; nested tool dispatches reuse the already-durable outer call.LOG-CHECKPOINT-POLICY

3. Write-behind separates hot-path admission from durable quiescence

1

Clone on admission

Every committed event enters a persistence-owned queue through structuredClone.

2

Batch on a fixed deadline

The first pending event arms one bounded timer; the hot append path does not wait for disk.

3

Retain failures

A failed batch is prepended back in original order, and automatic retry pauses.

4

Join one barrier

Concurrent explicit flush callers share the same drain to a quiescent queue.

Production controller

An explicit flush cancels the batching timer, waits for overlapping work, retries retained events, and drains until no active or pending write remains. Per-session-ID serialization prevents durable appends for one log from interleaving.LOG-WRITE-BEHIND

Persistence and checkpoint scheduling are separate plugins. Loading a backend without the semantic policy is valid, but a crash may then lose events still inside the batching window or an outstanding write. The shipped base composition mounts both the JSONL persistence provider and the checkpoint policy explicitly.PROJ-SHIPPED-COMPOSITION

4. Projection units are synchronous folds over every committed event

ProjectionDefinition<K, S> = {
  key,
  schema,             // validates the wire-facing view
  init(): S,
  apply(S, event): S, // pure and synchronous
  view(S): Value,
  stateVersion
}
Registry drive

The registry subscribes once to session/event. Every event passes every registered unit's apply; an uninterested unit must return the same state reference. Only a changed reference causes schema validation and a change notification. Missing cells lazily fold the full in-memory log, while observedSeq advances even when the state reference does not.PROJ-REGISTRY-DRIVE

The synchronous requirement is load-bearing. A carrier can copy a Session's event snapshot and immediately call snapshot() in the same JavaScript turn, obtaining values whose shared asOfSeq is exactly session.seq - 1. An async unit would tear that cut.

5. Thirteen production keys share one framework but encode different lifecycles

Key / versionFold semanticsProduction source
todos v2Latest whole todo/write; cleared by the next turn/startpackages/todo/tool-todo/src/index.ts:122-147
plan v1command/run(plan) records intent; plan/mode commits active state and clears pendingpackages/plan/plan-mode/src/index.ts:235-265
goal v4Last-wins durable goal/change projectionpackages/goal/goal/src/index.ts:201-212
title v1Latest session/titlepackages/session/session-title/src/index.ts:304-316
sessionStats v1Whole-log turn, Step, model, tool, TTFT, and decode totalspackages/session/session-stats/src/projection.ts:1-21,88-182
tokenUsage v1Usage samples replace the prior sample for the same Turn and Steppackages/llm/token-meter/src/usage-projection.ts:97-140
contextPressure v4Provider prompt sample plus signed Surface movement estimates the next requestpackages/llm/token-meter/src/usage-projection.ts:142-205
contextBreakdown v2Latest request envelope plus message-surface shadow pricingpackages/llm/token-meter/src/breakdown-projection.ts:31-69
subagentTiming v2Completed and currently active post-descriptor Turn durationpackages/subagent/subagent/src/projection.ts:36-85
subagent v2Last valid durable mode/label descriptor, including its seqpackages/subagent/subagent/src/projection.ts:131-155
permissions v1Three whole-value knob events, viewed against composition defaultspackages/interaction/permission-presets/src/index.ts:227-251
sessionListMetadata v1Blank-to-nonblank state and latest human-prompt time for listingpackages/host/apiproxy/src/api-proxy.ts:1289-1300
imageLimits v1Per-boot attachment limits; constant state, live-service view, baseline onlypackages/host/apiproxy/src/api-proxy.ts:1302-1323

This is a repository-wide inventory of production registrations at the pinned commit, not a promise that all thirteen keys are active in every deployment. Registration is composition-dependent, and the process-wide table also means a key mounted by one preset can appear in snapshots for Sessions whose own preset never produces a meaningful value.

6. A watermark advances on observation; a push occurs only on change

event seq N enters every unit
  ├─ apply returns same reference
  │    observedSeq = N
  │    no session/projection frame
  └─ apply returns new reference
       observedSeq = N
       schema.parse(view(state))
       emit { key, value, seq: N }

This distinction explains why imageLimits never emits a push yet still appears in a tail baseline, and why a quiet value in a full snapshot can legitimately be stamped at the log's current asOfSeq. The watermark means “observed through,” not “last changed at.”

Host carrier

The Host translates registry changes into non-persistent session/projection mux frames. The history tail—and only the tail—carries a full baseline. An attached Session copies events and snapshots projections with no await between them; a detached Session folds its inspected events.PROJ-HOST-CARRIERLOG-HISTORY-CUT

7. The durable projection cache is a versioned shortcut, never a read authority

GuardWhy it existsFailure behavior
stateVersionState shape or fold semantics may changeDiscard that key's row and refold
createdAt + cwdA Session ID names a slot, not a unique lifecycle foreverDiscard the whole unrelated record
Anchored floorA cached row may claim events beyond a shrunken or repaired stored logDetect overreach and reread from seq 0
Lossless JSON boundaryInternal unit state must be durably serializableFail that cache write; never mutate the Session log
Stored shape

The session_projcache domain is version 3. Each Session record binds one complete key-to-row checkpoint to a log identity. Domain-version incompatibility can discard the cache medium because the event log remains sufficient for reconstruction.PROJ-CACHE-SCHEMA

Restore mathematics

restoreFloor() chooses one event below the lowest usable watermark. restore() accepts a row only when its version matches and its seq lies within the supplied suffix's provable bounds. An unsafe partial restore throws so the caller can reread from zero instead of blending incompatible state.PROJ-REGISTRY-RESTORE

8. The implemented cold-cache ladder is not wired into shipped cold History

implemented SessionProjectionCache.coldSnapshot(id)
  cache rows → restoreFloor → persistence.readFrom(tail)
  → registry.restore → fail-soft refreshed write-back

shipped ordinary cold History
  persistence.inspect(full logical log)
  → registry.restore({}, all events, 0)

shipped Session list
  cachedSnapshot(header) only, zero log I/O
Implemented ladder

coldSnapshot() validates log identity, tail-folds from an anchored floor, falls back to one full read after identity or endpoint failure, and writes refreshed rows back without failing the read.PROJ-CACHE-LADDER

Repository-wide call-site audit

At the pinned commit, production search finds the coldSnapshot method definition and generated API catalog, but no shipped caller. session.list uses only cachedSnapshot; ordinary and subagent cold History already load an inspected full log and fold from zero; subagent listing uses a cached row when conclusive and otherwise performs a full inspection. The cache is therefore a shipped list/metadata accelerator, not an active cold-History tail-replay accelerator.PROJ-CACHE-INTEGRATION

9. Inspect, prepare, and resume form a cold-start protocol, not three aliases

OperationLogical recoveryDurable mutationOwnership / publication
inspect(id)Validates and adds deterministic closers in memoryNo repair commitNo Agent; may retain an exact prepared graph for reuse
readFrom(id, seq)None; returns valid physical stored suffixNoneNo preparation cache or publication
prepare(id)Builds the balanced exact SessionCommits repair if needed, then reloadsExclusively reserves the unpublished Session
Agent resumeConsumes the preparationPersists later live suffixes normallyRuns setup, publishes exact Session + Agent, or rolls back
Persistence read faces

Cold inspections can present a balanced logical turn while leaving a torn or interrupted physical tail untouched. Preparation uses phases loading, ready, committing, and reserved; same-ID inspectors share a load, while only the exact reserved Session may publish.LOG-PERSISTENCE-READ-FACESLOG-PREPARED-COMMIT

Agent transaction

The Agent factory fuses caller, owner-fiber, and factory cancellation around persistence.prepare. Setup runs before publication; failure disposes the prepared Agent and releases the reservation. SessionStore.enter is the final same-ID collision boundary.PROJ-COLD-RESUME

10. Browser cold open is deliberately not Agent resume

1

Restore selection

The browser remembers its current Session address and selects it after the list arrives.

2

Open History

Session.open() requests the tail page; it does not request an Agent.

3

Inspect cold storage

The Host uses an attached Session when available, otherwise persistence inspection.

4

Resume only on demand

Prompt, rename, model, command, and similar Agent operations enter the shared resolver.

A cold Session must be listed by persistence and carry a project cwd to be Web-servable. History can still degrade to generic tool cards when the recorded preset no longer supplies a standing presenter scope. A real resume is stricter: it must pass format, continuity, ownership, preset composition, setup, and final publication checks.

No silent fresh fallback

The shared resolver reuses a live Agent, deduplicates same-ID resumes, inspects ownership and composition, then calls agents.resume. Not-found and subagent ownership receive typed errors; every other genuine resume failure remains an error. A fresh Session is created only on explicit create/adopt paths after persistence confirms the ID is absent.PROJ-HOST-COLD-OPENPROJ-COLD-RESUME

11. Client convergence combines a full baseline, sparse pushes, and generation repair

history tail baseline { asOfSeq: N, values: all registered keys }
                       ↓ seed
resident ProjectionValueStore
                       ↑ apply only if frame.seq is greater
session/projection { key, value, seq }

new mux generation:
session/subscribed { lastSeq: D }
→ remove client rows with seq > D
→ refetch list and resync opened History windows
Value store

The manager owns one projection store per Session even before a Session object exists. Both baselines and push frames use strict higher-seq-wins. A full History baseline also clears omitted keys at or below its cut; a partial session.list block applies only carried keys and deliberately clears nothing.PROJ-CLIENT-STORE

Restart recovery

session/subscribed.lastSeq removes rows that claim state beyond the Host's durable endpoint, allowing lower-seq truth reconstructed after a crash to land. History installation then drains buffered live events, drops overlap, and refetches the tail when it detects a seq hole.PROJ-CLIENT-MANAGERLOG-CLIENT-STITCH

The mux and host feeds are independent streams with no shared total-order claim. events.mux({since}) also ignores since in v1. The supported convergence story is reopen the streams, refresh list state, refetch History, and repair each domain—not resume one globally linear stream offset.

12. Rebuild and invalidation operate at four different scopes

ChangeInvalidated stateHow truth returns
Another event appendsLive cell advances eagerlyChanged units push; full baseline later stamps every value through the new cut
Unit state or fold semantics changePersisted row with old stateVersionDiscard row and fold from a provable log prefix
Session ID is recreated or storage root changesWhole checkpoint record with mismatched identityDiscard all rows for that unrelated lifecycle
Last registrant unloadsRegistration and every live WeakMap cell for the keyA later snapshot omits the key; re-registration lazily folds the in-memory log
Projection-cache domain format changesEntire derived mediumDrop it; the Session log remains sufficient

No live registry API hydrates its WeakMap cells from a persisted checkpoint. The cache's detached restore() path returns a snapshot and refreshed checkpoint, but a resumed live Session still lazily folds its in-memory event array on first registry touch. There is also no cache eviction or retention surface; records accumulate until out-of-band maintenance removes them.

13. Four documentation claims have drifted from current production control flow

Documented claimCurrent implementationEvidence
Duplicate projection keys throwSame-version registrations share one ref-counted unit; only version mismatch throwssession-projection/README.md:11 vs src/index.ts:194-220
Titles do not join session.listList rows carry projection blocks; cold cached titles seed the manager before openhost/apiproxy/README.md:33 vs api-proxy.ts:1725-1764
History resumes an unattached Session and has no persistence-only read pathCold History uses inspect() and explicitly does not publish an Agentclient/connection/README.md:23-26 vs api/sessions.ts:264-283
skill.list resolves a cold Session from its HeaderThe handler first requires ctx.sessions.get(sessionId); unattached Sessions return not foundhost/apiproxy/README.md:59 vs api-proxy.ts:3205-3245

14. Source-level risk: detach can briefly let the projection cache lead the log

Session detach
  1. remove Session from SessionStore
  2. emit contained, non-awaited session/disposed observers
       ├─ persistence starts asynchronous retirement flush
       └─ projection cache snapshots and writes
            live-entry check now fails → no sessions.flush()
High-confidence static deduction; no crash injection performed

The cache documentation says the log always leads the cache. That is enforced for an ordinary live write, but not strictly at detach: the Session has already left the store, so the cache skips its explicit flush and relies on the persistence listener's separate retirement drain. The cache source itself acknowledges possible residual overreach. A process crash between the two durable writes can therefore leave a checkpoint whose seq is ahead of the stored log.PROJ-CACHE-DETACH-WINDOW

coldSnapshot() could detect that overreach through its anchored floor, but no shipped caller uses it. The zero-I/O list read does not validate the stored log endpoint. A phantom list projection can consequently reach the client; a later lower-seq cold-History baseline loses under higher-seq-wins until a true resume emits session/subscribed and truncates the row, or the log advances beyond it.

15. Source-level risk: equal-seq replacement cannot cross a Host generation

Deterministic client-rule deduction; scenario not executed

The client rejects every update whose seq is lower than or equal to the resident row, while generation truncation removes only rows strictly beyond the new Host's durable lastSeq. If a Host restarts with the same log endpoint but a different value computed at that same endpoint, the old row survives and the new equal-seq baseline cannot replace it.PROJ-CLIENT-EQUAL-SEQ

imageLimits makes the scenario concrete. Its state is always null, its view reads the current boot's attachment configuration, and it intentionally never pushes. A browser that remains alive across a Host restart can retain old limits at seq N; subscribed keeps the row because it is not beyond N, and the fresh baseline at N loses as equal. The value converges only after another event advances the baseline, a full client reset, or a protocol that carries a generation/version dimension.

The same edge applies to a unit's view semantics changing at an unchanged log watermark, and to unload/reload of the same key. stateVersion protects the Host's persisted cache but is absent from the client wire row, so it cannot resolve this client-generation ambiguity.PROJ-IMAGE-LIMITS

16. Source-level risk: one optional unit can make ordinary History unavailable

Carrier inconsistency

A detached History baseline calls registry.restore({}, events, 0), which folds every registered unit. If any unit's apply, view, or schema rejects the log, ordinary session.history returns internal instead of serving the transcript without projections. Session listing explicitly contains projection failure per row, and subagent History also degrades without the block; ordinary History does not.PROJ-HISTORY-FAILURE

This is not merely corrupt state inside the failing domain: the registry's process-wide all-unit fold lets an unrelated projection block the base transcript read. Whether fail-loud is desired needs an explicit product decision. Current carrier behavior is inconsistent.

Capability invalidation lag

Unregistering the final unit deletes the registration and cells but emits no invalidation frame. Already connected clients retain that key until a later full tail baseline omits it. If the key is reloaded at the same log seq, the strict equal-seq rule can retain the previous value even after that baseline arrives.PROJ-REGISTRY-REGISTRATIONPROJ-CLIENT-STORE

17. Design assessment and verification status

ChoiceBenefitCost
One authoritative event logEvery derived layer can be discarded and rebuiltCold full folds remain expensive when acceleration is not connected
Synchronous, whole-value projection unitsAtomic cuts and simple client replacement semanticsEvery event touches every unit; large values ride every tail baseline
Versioned persisted fold shortcutsNo cache migrations are required for semantic changesVersion discipline is manual, and cache retention is unbounded
Inspect separately from resumeCold browsing does not activate agents or mutate durable recovery stateRead and execution paths expose different logical/physical moments
Higher-seq-wins on the clientStale baselines and replayed frames cannot regress current stateSeq alone cannot distinguish an equal-watermark Host generation
Chapter verification scope

This chapter traced the fixed upstream commit through Session flush dispatch, checkpoint policy, persistence write-behind, projection registry and all production registrations, projection-cache schema and read/write paths, Host History/list carriers, cold inspection and exact preparation, Agent resume, and browser projection/gap-repair stores. It also reviewed relevant contract tests in source. Dependencies were not installed and no tests, crash injection, or live reconnect experiment ran locally; every risk section above is therefore labeled as static deduction rather than reproduced behavior.

My Learning Notes

Autosaved only in this browser. Nothing is uploaded or committed. Export Markdown whenever you want to keep a copy.