docs: close benchmark runner todo

Problem: Todo 52 needed final evidence, completed follow-up status, operator-facing limitations, and workflow-evolution findings after implementation and repeated reviews.

Risk: No live provider benchmark was executed; the closeout explicitly limits conclusions to deterministic mock/runtime behavior until the operator supplies a real target and credential.

docs/plans/2026-07-13-agent-benchmark-design.md

@@ -1,6 +1,6 @@
 # Agent Invocation Benchmark Design
 
-**Status:** approved for implementation by the user's "feasible means proceed" instruction
+**Status:** implemented and verified on 2026-07-14 (Asia/Shanghai)
 
 **Date:** 2026-07-13
 

docs/plans/todo-52-agent-benchmark-runner.md

@@ -1,5 +1,7 @@
 # Agent Benchmark Runner Implementation Plan
 
+**Status:** done
+
 > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
 
 **Goal:** Build a packaged benchmark CLI that runs deterministic mock or live OpenAI-compatible comparisons for both invocation modes and writes secret-safe JSON/Markdown reports.
@@ -65,9 +67,62 @@
 - Modify: `docs/plans/todo-52-agent-benchmark-runner.md`
 - Modify: `docs/plans/todos.md`
 
-- [ ] Add a secret-free example covering both modes and all six cases; ignore `outputs/benchmarks/`.
-- [ ] Document `.env`, config copy/edit, mock command, live command, output fields, exit codes, fairness controls, and the limits of scripted/mock adapters.
-- [ ] Run `agent-lab-benchmark --config benchmarks/single-target.example.json --mock --output-dir /tmp/agent-lab-benchmark` and inspect both reports.
-- [ ] Run focused benchmark tests, `uv run pytest`, CLI help, app import, `node --check`, and `git diff --check`.
-- [ ] Perform specification review then code-quality review; add Todo 52.x rows for post-execution fixes.
-- [ ] Record evidence and Workflow Evolution findings, mark Todo 52 done, and commit with `docs: close benchmark runner todo`.
+- [x] Add a secret-free example covering both modes and all six cases; ignore `outputs/benchmarks/`.
+- [x] Document `.env`, config copy/edit, mock command, live command, output fields, exit codes, fairness controls, and the limits of scripted/mock adapters.
+- [x] Run `agent-lab-benchmark --config benchmarks/single-target.example.json --mock --output-dir /tmp/agent-lab-benchmark` and inspect both reports.
+- [x] Run focused benchmark tests, `uv run pytest`, CLI help, app import, `node --check`, and `git diff --check`.
+- [x] Perform specification review then code-quality review; add Todo 52.x rows for post-execution fixes.
+- [x] Record evidence and Workflow Evolution findings, mark Todo 52 done, and commit with `docs: close benchmark runner todo`.
+
+## Final Evidence
+
+Verified on 2026-07-14 (Asia/Shanghai):
+
+- benchmark suite: `196 passed in 1.19s`;
+- full suite: `601 passed in 3.67s`, with one existing Starlette/httpx deprecation warning;
+- `agent-lab-benchmark --help`: exit `0`;
+- application import prints `Agent Lab`;
+- `node --check src/agent_lab/presentation/static/app.js`: exit `0`;
+- `git diff --check`: exit `0`;
+- zero-network mock CLI: `12 attempted / 12 passed / 0 failed`;
+- final JSON: `/tmp/agent-lab-benchmark-final-20260714/20260713T194850Z/report.json`;
+- final Markdown: `/tmp/agent-lab-benchmark-final-20260714/20260713T194850Z/report.md`;
+- both parallel-mode runs report `parallel_events_overlapped=true`;
+- both web-search runs report two model calls and one tool execution;
+- no mock run has missing token usage.
+
+Specification and code-quality reviews found no remaining Critical or Important
+issues after Todos 52.1-52.13. The remaining non-blocking risks are documented
+below.
+
+## Residual Risks and Limits
+
+- No live provider request was run because no real benchmark target or credential
+  was supplied; live latency, usage support, semantic adherence, and provider
+  tool-stream compatibility still require operator verification.
+- The example uses one run per case, so its 60 p95 warnings are expected and the
+  mock percentile values are not suitable for ranking architectures.
+- Atomic bundle publication uses a relative directory symlink and is verified on
+  macOS; Windows symlink behavior is not verified.
+- An interruption inside `mkdtemp()` may leave an empty hidden bundle directory.
+  A close failure during a narrow consumer-pause cancellation window may be
+  logged as a secondary async-generator exception. Neither case changes the
+  primary cancellation result or exposes a mixed public report.
+
+## Workflow Evolution Update
+
+The requirement loop exposed three durable workflow lessons:
+
+- paired filesystem artifacts need one explicit atomic publication boundary;
+  repeated two-file rollback fixes were replaced by a single report bundle;
+- benchmark plans must define clock boundaries, missing-data semantics, and
+  observable concurrency proof before implementation, not only metric names and
+  event counts;
+- parallel subagents need shared-contract integration checks when one task changes
+  an output layout consumed by another task's tests.
+
+These lessons are recorded in this task report and the benchmark design. No
+`AGENTS.md` change was made because the repository's existing PAOU, todo-first,
+TDD, per-todo commit, and post-review follow-up rules already caused the issues to
+be found and corrected; adding more global workflow rules would duplicate the
+current core process.

docs/plans/todo-52.12-benchmark-metric-integrity-and-parallel-proof.md

@@ -1,6 +1,6 @@
 # Todo 52.12 Benchmark Metric Integrity and Parallel Proof Plan
 
-**Status:** in_progress
+**Status:** done
 
 ## Goal
 
@@ -36,4 +36,3 @@ Ensure benchmark latency, token-cost, and multi-event concurrency results measur
 - zero-network mock CLI still reports 12 attempted runs and validates parallel overlap
 - `uv run pytest`
 - `git diff --check`
-

docs/plans/todo-52.13-provider-cleanup-exception-priority.md

@@ -1,6 +1,6 @@
 # Todo 52.13 Provider Cleanup Exception Priority Plan
 
-**Status:** in_progress
+**Status:** done
 
 ## Goal
 
@@ -28,4 +28,3 @@ Preserve the original provider failure, cancellation, or generator-close signal
 - `uv run pytest tests/test_benchmark_runner.py tests/test_benchmark_reporting.py -q`
 - `uv run pytest`
 - `git diff --check`
-

docs/plans/todos.md

@@ -100,7 +100,7 @@
 | 50.1 | done | `docs/plans/todo-50.1-metric-key-migration-dedup.md` | Deduplicate legacy metric keys before creating the idempotency index so existing experiment databases remain openable. | Migration keeps the earliest row for each duplicate non-null session/metric key, creates the unique index, and remains safe under concurrent initialization. |
 | 51 | done | `docs/plans/todo-51-comparison-matrix-closeout.md` | Add repeatable comparison fixtures, update project documentation, and close the experimental implementation. | Both modes pass the same 12-run scenario matrix; focused (`231 passed`) and full (`417 passed`) tests, JS syntax, app import, and documentation checks pass. |
 | 51.1 | done | `docs/plans/todo-51.1-first-reply-and-protocol-matrix.md` | Require fast visible first replies for event scenarios and prove each mode uses its intended capability protocol surface. | Matrix asserts first reply ordering, no unintended second reply, dual compact event catalog/no provider tools, direct enabled schemas/no generated event catalog, and correct event source. |
-| 52 | in_progress | `docs/plans/todo-52-agent-benchmark-runner.md` | Add a configurable in-process benchmark CLI with deterministic mock cases and live OpenAI-compatible execution. | Mock/live runs compare both modes, measure provider/visible TTFT, wall time, tokens and event correctness, write secret-safe JSON/Markdown reports, and use stable exit codes. |
+| 52 | done | `docs/plans/todo-52-agent-benchmark-runner.md` | Add a configurable in-process benchmark CLI with deterministic mock cases and live OpenAI-compatible execution. | Mock/live runs compare both modes, measure provider/visible TTFT, wall time, tokens and event correctness, write secret-safe JSON/Markdown reports, and use stable exit codes. |
 | 52.1 | done | `docs/plans/todo-52.1-benchmark-url-control-characters.md` | Reject non-printable/control characters in benchmark target URLs before constructing the HTTP client. | Strict config tests cover NUL, CR/LF, tabs, and other Unicode/control characters while valid HTTP(S) URLs remain accepted. |
 | 52.2 | done | `docs/plans/todo-52.2-benchmark-url-transport-compatibility.md` | Validate benchmark target hostname/URL compatibility with the production HTTP transport at the config boundary. | Invalid IDNA/port/transport URLs are rejected as config errors while valid Unicode hostnames accepted by the transport remain valid. |
 | 52.3 | done | `docs/plans/todo-52.3-benchmark-case-mode-single-source.md` | Make benchmark case IDs and invocation modes single-source enums shared by config, catalog, runner, and tests. | Adding or removing an enum member cannot leave config literals, exported choices, and catalog keys silently inconsistent. |
@@ -112,5 +112,5 @@
 | 52.9 | done | `docs/plans/todo-52.9-atomic-report-bundles-and-audit-parity.md` | Replace two-path report publication with one atomic bundle boundary and close report/CLI audit parity gaps. | Concurrent writers publish complete bundles without rollback deletion races; Markdown preserves all failure facts and output errors map to exit code 2. |
 | 52.10 | done | `docs/plans/todo-52.10-benchmark-bundle-cli-contract.md` | Align CLI report discovery tests with the atomically published bundle layout. | CLI tests consume the reported bundle paths without restoring root-level compatibility copies or weakening redaction, exit-code, and cardinality checks. |
 | 52.11 | done | `docs/plans/todo-52.11-bundle-interrupt-and-path-safety.md` | Preserve complete bundles across post-symlink interrupts and reject unsafe report timestamp path components. | Interruptions cannot leave dangling public bundles; explicit timestamps stay inside output_dir; fsync tests verify both files and both directories. |
-| 52.12 | in_progress | `docs/plans/todo-52.12-benchmark-metric-integrity-and-parallel-proof.md` | Exclude consumer backpressure from model elapsed, reject missing token usage, and prove parallel handlers overlap. | Model latency measures provider-await time; absent usage cannot appear as zero cost; the parallel case fails unless both handlers start concurrently. |
-| 52.13 | in_progress | `docs/plans/todo-52.13-provider-cleanup-exception-priority.md` | Preserve active provider/cancellation exceptions when iterator cleanup also fails. | Provider failures and cancellation retain priority; cleanup failures propagate only when no earlier exception is active; timing attempts are still recorded. |
+| 52.12 | done | `docs/plans/todo-52.12-benchmark-metric-integrity-and-parallel-proof.md` | Exclude consumer backpressure from model elapsed, reject missing token usage, and prove parallel handlers overlap. | Model latency measures provider-await time; absent usage cannot appear as zero cost; the parallel case fails unless both handlers start concurrently. |
+| 52.13 | done | `docs/plans/todo-52.13-provider-cleanup-exception-priority.md` | Preserve active provider/cancellation exceptions when iterator cleanup also fails. | Provider failures and cancellation retain priority; cleanup failures propagate only when no earlier exception is active; timing attempts are still recorded. |