docs: close invocation comparison experiment

Problem: The completed matrix, verification evidence, bounded conclusions, and workflow learning needed a durable closeout record.

Risk: Results prove scripted contract parity only; real-provider latency, cost, quality, and production adapter reliability remain unmeasured.

docs/plans/2026-07-12-chat-event-agent-production-events-design.md

@@ -1,6 +1,6 @@
 # ChatAgent / EventAgent Production Events Design
 
-**Status:** implemented in Agent Lab by Todos 45-50; Todo 51 closeout in progress
+**Status:** implemented and closed in Agent Lab by Todos 45-51
 
 **Date:** 2026-07-12
 

docs/plans/todo-51-comparison-matrix-closeout.md

@@ -1,6 +1,6 @@
 # Todo 51 Comparison Matrix Closeout Plan
 
-**Status:** in_progress
+**Status:** done
 
 ## Goal
 
@@ -61,3 +61,50 @@ The matrix compares shared user-visible semantics and normalized business result
 - `node --check src/agent_lab/presentation/static/app.js`
 - `uv run python -c "from agent_lab.main import app; print(app.title)"`
 - `git diff --check`
+
+## Observe
+
+- The shared matrix runs 6 scenarios in both modes (`12 passed`): ordinary chat, volume, schedule, web search, termination, and parallel volume plus schedule.
+- Review required Todo 51.1 because the first matrix did not enforce fast first replies or inspect the actual mode-specific model request surface.
+- After Todo 51.1, each event scenario proves the first ChatAgent delta precedes tool completion; web search has exactly two model answers; schedule/parallel add one deterministic template; silent/terminate flows add no unintended second model answer.
+- `dual_agent` proves compact generated event catalog plus no provider tools; `chat_agent_tools` proves exact enabled schemas plus no generated catalog. Audit source is mode-correct.
+- Parallel cases require both handlers to start, release in reverse completion order, and still emit tool results in accepted input order.
+
+## Verification Evidence
+
+- Matrix: `12 passed in 0.22s`.
+- Focused runtime/plugin/storage/WebSocket regression: `231 passed`, one existing Starlette deprecation warning.
+- Full suite: `417 passed`, one existing Starlette deprecation warning.
+- `node --check src/agent_lab/presentation/static/app.js`: passed.
+- App import printed `Agent Lab`.
+- `git diff --check`: passed.
+
+## Experiment Conclusions
+
+Confirmed by this implementation and matrix:
+
+- one generic event kernel and flat built-in plugin registry serve both invocation modes;
+- both modes preserve the same normalized event execution and user-visible result-policy contract;
+- fast first replies, web-search two-answer ordering, multi-event parallelism, terminal behavior, session restoration, and comparable metric accounting have repeatable regression coverage;
+- token totals, logical tool counts, fallback rows, and turn wall time are separated so the two architectures can be measured without known double-counting.
+
+Not established by this closeout:
+
+- which mode has lower real-provider TTFT, turn latency, token cost, or monetary cost;
+- which mode produces better natural-language quality or event accuracy on live stochastic models;
+- real device, calendar, dialogue termination, and internet-search reliability.
+
+The current matrix uses scripted model streams and injected/default in-memory adapters. A production decision still requires paired live-provider and service-adapter benchmarks using fresh databases and the same comparison controls.
+
+## Remaining Risks
+
+- Some providers may suppress visible content when emitting native tool calls, weakening direct mode's first-reply behavior.
+- Fallback frequency and schema/prompt growth depend on the real capability set and user distribution.
+- Model randomness, provider usage reporting, and real search latency can dominate local deterministic results.
+- Reusing historical SQLite sessions can contaminate paired comparisons; benchmark runs should use isolated databases.
+- Monetary cost is not stored because it depends on provider/model pricing; estimate it from model call tokens outside the runtime.
+
+## Workflow Evolution Update
+
+- The requirement/todo loop, isolated worktree, per-todo commits, TDD, and follow-up `x.x` rows successfully contained review findings without rewriting completed todos.
+- Subagent model overrides and the default user-level uv cache were environment-specific friction. The main session used inherited subagent routing, explicit verification, and controlled Git commits. No repository workflow rule was added because these are not stable project requirements.

docs/plans/todo-51.1-first-reply-and-protocol-matrix.md

@@ -1,6 +1,6 @@
 # Todo 51.1 First Reply and Protocol Matrix Plan
 
-**Status:** in_progress
+**Status:** done
 
 ## Goal
 
@@ -25,3 +25,10 @@ Make the closeout matrix enforce the experiment's primary latency contract and t
 - repeat parallel cases
 - `uv run pytest`
 - `git diff --check`
+
+## Observe / Update
+
+- Side-effect scenarios now stream a natural ChatAgent acknowledgement/farewell before event completion.
+- Matrix request capture proves dual mode sends no provider tools and injects exactly the enabled compact event catalog; direct mode sends exactly the enabled schemas and no generated catalog.
+- Detected event audits report `text_event` for dual mode and `provider_resolved` for direct mode.
+- Matrix passed (`12 passed`), parallel scenarios passed 20 repeated runs, and the full suite passed (`417 passed`).

docs/plans/todos.md

@@ -98,5 +98,5 @@
 | 49.3 | done | `docs/plans/todo-49.3-stale-replay-token-order.md` | Prevent a stale replay invocation from invalidating an already-running replay for the current session. | A late stale invocation exits before advancing the replay token; the current session still restores normally. |
 | 50 | done | `docs/plans/todo-50-comparable-model-tool-metrics.md` | Record mode-aware model/tool spans and include EventAgent usage so both architectures can be compared fairly. | Persistence/API tests report agent, call kind, tokens, TTFT, tool latency, fallback usage, and total turn wall time without double counting. |
 | 50.1 | done | `docs/plans/todo-50.1-metric-key-migration-dedup.md` | Deduplicate legacy metric keys before creating the idempotency index so existing experiment databases remain openable. | Migration keeps the earliest row for each duplicate non-null session/metric key, creates the unique index, and remains safe under concurrent initialization. |
-| 51 | in_progress | `docs/plans/todo-51-comparison-matrix-closeout.md` | Add repeatable comparison fixtures, update project documentation, and close the experimental implementation. | Both modes pass the same scenario matrix; full tests, JS syntax, app import, and documentation checks pass. |
-| 51.1 | in_progress | `docs/plans/todo-51.1-first-reply-and-protocol-matrix.md` | Require fast visible first replies for event scenarios and prove each mode uses its intended capability protocol surface. | Matrix asserts first reply ordering, no unintended second reply, dual compact event catalog/no provider tools, direct enabled schemas/no generated event catalog, and correct event source. |
+| 51 | done | `docs/plans/todo-51-comparison-matrix-closeout.md` | Add repeatable comparison fixtures, update project documentation, and close the experimental implementation. | Both modes pass the same 12-run scenario matrix; focused (`231 passed`) and full (`417 passed`) tests, JS syntax, app import, and documentation checks pass. |
+| 51.1 | done | `docs/plans/todo-51.1-first-reply-and-protocol-matrix.md` | Require fast visible first replies for event scenarios and prove each mode uses its intended capability protocol surface. | Matrix asserts first reply ordering, no unintended second reply, dual compact event catalog/no provider tools, direct enabled schemas/no generated event catalog, and correct event source. |