Tool Parallelism and Result Ordering
Barriers, rolling windows, reclassification, and model-order settlement
Conclusion: Scheduler-Controlled Dispatch and Bodies May Overlap; Native Authoritative Commit Stays Ordered
Parallel tool calling in DeepSeek Harness does not hand the model's array directly to Promise.all. The visible definition first classifies each call through a fail-closed contract. The Agent Loop then builds a bounded rolling window in model order and turns every exclusive call into a barrier. Pre-policy, approval, and guards run in order; around-dispatch and bodies may overlap; Native top-level post/finalize, tool/result, and additional context commit in model order. Asynchronous tools/result observer continuations and Code Mode nested dispatch-log side work sit outside that commit guarantee. The default window is 10, and 1 restores serial scheduler-controlled dispatch.
classify → ordered prepare → concurrent dispatch → model-order finalize → durable result → next-step context. The complete Assistant message is persisted before the Loop extracts all tool-call blocks and enters this machine. PARALLEL-LOOP-ENTRYPARALLEL-GROUPING
The design protects replay, policy order, and parent Session state ahead of maximum throughput. Its costs are equally explicit: a fast result can wait behind a slow earlier sibling, the classifier cannot say that writes to two different paths are compatible, and a per-Agent, per-Step cap is not a process-wide resource budget.
1. Four Different Orders Must Be Kept Separate
| Order | Guarantee | What may be out of order |
|---|---|---|
| Model order | The order of tool-call blocks in Assistant content is the index and commit baseline | Nothing |
| Start order | Calls enter prepare and dispatch in model order; a later call does not start first | Body completion |
| Completion order | Parallel bodies may complete in any order | This is where concurrency actually exists |
| Commit order | Post-policy, tool/result, contexts, and concludes-turn follow model order | Completion of asynchronous read-only observers is outside this guarantee |
The implementation places completed dispatches into slots indexed by model order. The committed cursor can advance only across a contiguous ready prefix. A later call that returns first therefore cannot enter post, Session history, or next-step context first. PARALLEL-ROLLING-POOL
2. executionMode() Is Strict Opt-In and Evaluated Per Call
| Input state | Classification | Reason |
|---|---|---|
The visible definition's isConcurrencySafe(args) returns exactly true | parallel | The only opt-in path |
| No classifier, false, or a truthy non-boolean | exclusive | Overlap is denied by default |
| Unknown, hidden by scope, or collapsed as a direct code-only call | exclusive | No executable definition resolves |
| The classifier throws | exclusive | A classification failure cannot elevate authority |
Arguments to a defineTool definition are invalid | exclusive | The helper soft-validates before calling the typed classifier |
ToolExecutionMode has only the tagged parallel and exclusive arms. isConcurrencySafe is host metadata; schema projection sends only name, description, and parameters. PARALLEL-MODE-CONTRACTPARALLEL-CLASSIFIERPARALLEL-TYPED-CLASSIFIER
3. This Is a Unary Promise, Not a Resource-Conflict Graph
The classifier contract supplies only the current call's parsed arguments and expects a synchronous unary judgment; it receives neither siblings nor the current pool. JavaScript does not technically prevent a closure from touching other state or initiating I/O, but doing so violates the classification contract and introduces dependencies the scheduler cannot see. Returning true is a strong promise: this call may overlap with any sibling that also returns true. It can express “read mode is parallel, write mode is exclusive” for one tool, but not “these two writes are safe only when their targets differ.”
A parallel body must not mutate parent-owned state. Races in a shared recorder, provider, or cache must commute, serialize internally, or fail closed. The scheduler does not verify that promise. An overly broad declaration turns a latent race into production behavior; an overly narrow one only costs latency. The upstream design note explicitly chooses this conservative tradeoff. PARALLEL-SAFETY-NOTE
4. Groups Are Discovered Incrementally, Not Pre-Sliced Once
model calls: P1 P2 X3 P4
classify P1 → open a candidate parallel suffix
start P1, reclassify/start P2
reclassify X3 → stop replenishing, drain P1/P2
run X3 alone through post/commit
classify/start P4The outer loop classifies only the current first call. An exclusive first call becomes a singleton group that is fully awaited. A parallel first call passes the remaining suffix to runGroup, whose pool classifies each later call again immediately before it starts. When one becomes exclusive, the current group returns its actual consumed count and leaves the barrier for the next outer iteration. PARALLEL-GROUPING
Exclusive does not merely mean “one body at a time.” The whole singleton group completes prepare, body, post, finalization, result append, and context acceptance before the next group starts. Tests cover parallel → exclusive → parallel grouping and reclassification after registry replacement. PARALLEL-GROUP-TESTS
5. The Rolling Pool Keeps Its Cap Without Fixed-Window Idling
inFlight holds dispatch Promises that have launched but have not yet been reaped by the driver; a settled Promise may remain briefly between settlement and deletion. The pool fills in order up to maxParallelToolCalls. Any settlement wakes the driver through Promise.race; the driver removes that slot, commits the contiguous ready prefix if possible, and then admits another call. It does not wait for an entire fixed window, so capacity beside one slow call remains useful.
cap = 2
start C1, C2
C2 settles first → slot[1] ready, cannot commit, one capacity opens
start C3
C1 settles → commit C1, C2; continue rolling
C3 settles → commit C3The cap limits concurrently unsettled parallel dispatches in one Agent Step. It does not count settled slots waiting for an earlier commit and does not constrain another Agent. Tests prove cap-2 replenishment and fully serial behavior at cap 1. PARALLEL-ROLLING-POOLPARALLEL-CAP-TESTS
6. Why Only the Middle of the Pipeline Is Concurrent
| Stage | Concurrency | Reason |
|---|---|---|
Append tool/call | Ordered | Establish durable identity and the original argument fact first |
| Create execution, pre-execute, approval, guards | Ordered | Policy may carry order-sensitive state |
tools/execute wrappers and body | May overlap | This is where external I/O and compute latency live; wrappers must be reentrant |
| Post-execute, rendering/finalization, result notification | Invoked in order | Promises returned by observers are not awaited, so their continuations may overlap |
Append tool/result, accept context | Ordered | Authoritative model-history and next-Step order |
ToolRuntime's symbol-keyed private scheduler view splits the same execution pipeline into prepare, dispatch, and finalize/finish. Ordinary callers still use the complete execute(). PARALLEL-PIPELINE-SPLITPARALLEL-PREPAREPARALLEL-DISPATCHPARALLEL-FINALIZE
7. What “Ordered Results” Means in the Session Log
Every started call first appends tool/call with the provider's raw argument string and retains its seq. At commit, the Loop creates tool/result from final content, isError, error information, and metadata, then points sourceEventSeqs:[callSeq] back to the exact call. Only then does it accept each additionalContexts item from that result. PARALLEL-DURABLE-PAIRING
The production test settles C2 first and confirms that no result exists before C1 is released. The final deriveMessages() output still contains C1 then C2. PARALLEL-ORDER-TESTS
8. Head-of-Line Blocking Deliberately Buys Consistency
The next provider request, Session replay, post hooks, and next-step contexts all agree with the model's original call order. A later network response cannot reorder policy state merely because it was faster.
One slow early call blocks post/finalization and model visibility for every completed later result, whose slots remain in memory. The rolling pool may continue starting safe later work, but the model cannot consume it early. This is an explicit latency-versus-determinism tradeoff, not a missed scheduler wakeup.
9. Live Reclassification Closes Most Queue Drift but Does Not Snapshot a Definition
The scheduler reclassifies the first call after every barrier and rereads the current visible definition before each parallel-pool replenishment. If a synchronous result observer replaces a tool, an unstarted call sees the new classifier. A test proves the current pool drains before the replacement's new exclusive definition starts. PARALLEL-RECLASS-TESTS
Classification does not pin ToolDefinition into the execution. After classification, startCall awaits ordered prepare; dispatch then awaits the tools/execute waterfall; only the body finally resolves the live definition again by name. A replacement from parallel to mutating exclusive anywhere in that interval can occupy the old parallel slot. The execution has also captured the old definition's finalizer, potentially producing “old finalizer + new body.” A complete repair must snapshot/bind one definition across classification and execution, or atomically validate a registry generation and schedule at actual body resolution. Reclassifying after prepare closes only one example window. PARALLEL-GROUPINGPARALLEL-PREPAREPARALLEL-DISPATCH
10. Result Observers Are Invoked in Order but Are Not Awaitable Scheduler Gates
finishScheduledExecution synchronously publishes the frozen result. notifyResult invokes callbacks in sequence but sends each returned Promise to an independent rejection handler instead of awaiting it. A registry change made before an observer's first await can therefore affect the next classification; a change made afterward may lose that race.
Policy that must block later calls belongs in an awaited pre/post waterfall or must finish synchronously. Asynchronous tools/result work is appropriate for telemetry, not barriers. The current “result observer makes next call exclusive” test uses a synchronous callback and proves only synchronous mutation. PARALLEL-RESULT-OBSERVERPARALLEL-RECLASS-TESTS
11. Cancellation Stops Replenishment, Drains Started Calls, and Synthesizes Results for Skipped Calls
| Cancellation point | Body | Durable result | Later behavior |
|---|---|---|---|
| Before a group | No body starts | Every model call gets a synthetic call/result with ABORTED_BEFORE_DISPATCH | No next LLM request |
| During pre/approval | The current body does not start; siblings are no longer prepared | The current and unstarted siblings receive before-dispatch errors | The Step ends |
| During parallel bodies | The signal propagates; the scheduler waits for all started Promises to quiesce | A success normally becomes ABORTED; a tool-owned error may survive; all commit in model order | No replenishment and no later exclusive barrier runs |
| During result/post commit | Started work is still drained | A final cancellation check may replace an outcome that remains successful | Contexts remain in the inbox |
The Loop's synthetic pairs ensure every tool-call in this Assistant message is closed for replay. This is an important change from the earlier design note. PARALLEL-ABORTPARALLEL-ABORT-RESULTPARALLEL-ABORT-TESTS
12. Cancellation Does Not Discard Accepted Additional Context
Post-policy still runs in order for started calls, and their additionalContexts enter the Agent's next-step inbox after the corresponding results commit. If cancellation ends the Turn, those items are not yet appended as user/message; they enter a new Step on the next wake. Tests pin C1, C2 context order and their parked inbox state after cancellation. PARALLEL-CONTEXT-TEST
concludesTurn is likewise aggregated only when a successful result commits in order. It does not cancel siblings already submitted in the same Assistant message. The Agent treats the Step as completed only after the scheduler finishes the batch. PARALLEL-AGENT-STEP
13. Ordinary Tool Failures and Scheduler-Invariant Failures Use Different Channels
| Failure class | Carrier | Does the group continue? |
|---|---|---|
| Arguments, policy denial, approval, guard, body, output, post, or finalizer failure | ToolRuntime normalizes a ToolExecutionResult.isError | Yes; it commits in model order and the model may self-correct next Step |
| An internal staged scheduler Promise unexpectedly rejects | Thrown to the Agent Turn error boundary | New dispatch stops and allSettled drains started work |
The second class is not an ordinary extension result. The Native scheduler keeps already appended tool/call events and does not fabricate outcomes it cannot attribute, so the failed Step may contain calls without results. The Turn ultimately closes with an error reason. PARALLEL-SCHEDULER-FAILUREPARALLEL-FAILURE-TEST
14. Two Independent Tens: Native Steps and Code Runs Have Separate Pools
| Configuration | Default | Scope | Dynamism |
|---|---|---|---|
agent-loop.maxParallelToolCalls | 10 | Each top-level Step of each Agent | Settings may update it; each group reads once, without disturbing an in-flight group |
tools.maxParallelSubCalls | 10 | The nested binding pool of each run_code | Fixed when ToolRuntime is constructed |
Both require a positive integer, and 1 means serial. They are not one shared semaphore. Multiple Agents, concurrent runs, and provider concurrency may multiply, so API quota, process pools, and connection pools still require provider-level admission control. PARALLEL-CAP-CONFIGPARALLEL-CODE-DRIVER
15. Code Mode Repeats the Principle Without Reusing the Same Loop
run_code is one top-level exclusive tool from the model's perspective. Inside the program, every tools.name(args) call enters a separate per-run driver. That driver owns a pending queue, in-flight set, commit queue, and exclusiveActive. One ordered lane performs start/prepare and commit/finalize, while only nested dispatch bodies overlap.
program submissions
→ normalize lossless JSON + assign <outer>:code:N
→ pendingQueue (submission order)
→ classify at start
→ bounded inFlight bodies / exclusive barrier
→ commitQueue head cursor
→ program promise value or ToolCallError
→ defer contexts onto outer run_code resultA nested call does not create top-level tool/call/result. It writes tool/code-dispatch-start and tool/code-dispatch for UI and trajectory reconstruction. Only the outer run_code call's curated logs and result enter model history. PARALLEL-CODE-DRIVERPARALLEL-CODE-BINDING
16. Code Mode Barriers, Context, and Settlement
The driver always tries to commit a ready commitQueue[0] before starting the pending head. Exclusive waits for inFlight.size===0 and holds exclusiveActive until its own post/commit completes. A started binding resolves or rejects from its tool outcome only in ordered-commit settle(), so the program observes the policy-final canonical value or one ToolCallError message rather than raw body completion. A queued binding cancelled before start is the exception: abandon() rejects it.
Nested contexts are passed to exec.deferContext in commit order on the outer execution. They can survive even if the program later fails. Once the outer run settles, its run-scoped controller cancels in-flight calls, rejects queued-unstarted calls, and drains both the driver and durable log work before the outer tool returns. PARALLEL-CODE-BINDINGPARALLEL-CODE-DRAINPARALLEL-CODE-TESTS
17. Durable Nested-Event Order Is Weaker Than Binding Order in Code Mode
Within the ordered commit lane, settle() first returns the result to the program and then launches asynchronous shapeDispatchLog → session.append(tool/code-dispatch) work. That work does not block the binding and is only drained collectively when the run ends. If an earlier log-content listener is slower than a later one, settlement events may append out of submission order. Start events remain ordered and the UI correlates by subCallId, so pairing is intact, but event seq/time must not be treated as canonical body-completion order. PARALLEL-CODE-BINDINGPARALLEL-CODE-DRAIN
The current concurrency test compares the set of settlement ids rather than their order. There is no assertion for settlement-event order under asynchronous log shaping. This must remain distinct from strict append order for Native top-level tool/result. PARALLEL-CODE-TESTS
18. Test Coverage Is Strong but Several Precise Timing Cases Remain
| Proven behavior | Evidence style |
|---|---|
| Native parallel overlap, three-group barriers, dynamic replacement, rolling cap, and cap 1 | Deterministic gated tools |
| Native out-of-order body completion with ordered result, history, and context | Release C2, assert no result, then release C1 |
| Synthetic pairing and quiescence when cancelled before, during pre, during bodies, or before a barrier | Session-event and inbox assertions |
| Native internal scheduler failure drains and closes the Turn as error | Injected private-scheduler rejection |
| Code Mode overlap, cap, exclusive barrier through post, and run-settlement drain | Per-run gated tools |
The Code Mode test named “out-of-order completion” actually uses FIFO release(); its own comment acknowledges that reverse release is impossible, after which it calls releaseAll(). It does not make B complete before A. The commit cursor is clear in source, but the interleave claimed by the test name is not proven by that fixture. PARALLEL-CODE-ORDER-TEST
Neither Native nor Code Mode directly tests replacement of a same-name definition after a parallel classification but before actual body resolution. There is also no characterization test for asynchronous result-observer continuations or asynchronous dispatch-log reordering. These are independent cases, not one fixed test count.
19. Documentation Audit: Both READMEs Are Current; Their Linked Implemented Note Is Not
packages/core/agent-loop/README and packages/core/tools/README accurately describe the rolling pool, exclusive barriers, ordered policy/results/context, default cap 10, and Code Mode's separate parallel bridge. PARALLEL-AGENT-READMEPARALLEL-README
The linked 2026-07-10 implemented Agent Note still says that unstarted calls have “no audit event” on cancellation, that Code Mode's internal queue remains serial and outside this scheduler, and that verification covers only a serial Code Mode boundary. Current source and tests use synthetic call/result pairs and a parallel per-run pool. The note should no longer be treated as the current “full safety contract”; it needs an update or an explicit supersession marker. PARALLEL-NOTE-DRIFT
20. Static Findings and Final Assessment
| Finding | Classification | Impact / recommendation |
|---|---|---|
| A live definition can change after classification but before actual body resolution | Conditional concurrency-safety risk | Add tests; bind the definition or validate a registry generation at body resolution |
| Async result observers are not awaited by the scheduler | Extension timing boundary | Put gating work in an awaited waterfall |
| Code nested settlement appends may reorder under asynchronous log shaping | Observability boundary | Pair by subCallId; do not infer body completion order from event seq |
| The Code “out-of-order” test never reverses completion | Regression-coverage gap | Use per-id gates and release the second call first |
| The implemented design note conflicts with current cancellation and Code scheduling | Documentation drift | Update its decisions and verification claims |
| Both caps are local windows, not global admission control | Operational capacity boundary | Keep global quota in providers and hosts |
| The unary classifier cannot see resource relationships | Deliberately conservative | Keep relationally safe operations exclusive |
The core algorithm is compact and disciplined: scheduler-controlled concurrency is concentrated in dispatch/bodies, intermediate results live in slots, and Native top-level post/result/context returns to one lane; cancellation and ordinary tool errors have explicit convergence paths. Async observers and Code nested logs are explicit exceptions. The highest-value next work is not a scheduler rewrite; it is closing the live-definition classify/body-resolution TOCTOU, separately covering definition replacement, observer continuations, nested-log reordering, and genuine reverse completion, and bringing the design note back in line with source.
This chapter traced the ToolRuntime classifier and staged pipeline, Agent Loop scheduler and Session appends, Code Mode bridge, configuration, READMEs, implemented note, and corresponding unit tests line by line at the pinned commit. Targeted Vitest then passed all 120 tests across three files. That run proves the existing suite is green; the uncovered timing cases listed above remain static findings, not claimed regression proof.
My Learning Notes
Autosaved only in this browser. Nothing is uploaded or committed. Export Markdown whenever you want to keep a copy.