Explorar o código

docs: plan no-overwrite report publication

Problem: Final report publication still used a non-atomic inode check followed by overwriting rename.

Risk: Hard-link publication relies on temporary and final files sharing the destination filesystem, which the writer guarantees by colocating them.
zhenyu.hu hai 2 semanas
pai
achega
336bfee886

+ 27 - 0
docs/plans/todo-52.8-benchmark-no-overwrite-publication.md

@@ -0,0 +1,27 @@
+# Todo 52.8 Benchmark No-Overwrite Publication Plan
+
+**Status:** in_progress
+
+## Goal
+
+Remove the final check-then-replace race from concurrent benchmark report publication.
+
+## Scope
+
+- Fully write and fsync JSON/Markdown temporary files in the destination directory.
+- Publish each file using `os.link(temp, final)`, whose destination creation is atomic and fails if the path already exists.
+- If either candidate path conflicts, remove only links whose inode matches the current temporary file, choose the next suffix, and retry the pair.
+- Remove temporary files after both links succeed; retain no lock or placeholder files.
+- Test a competitor creating/replacing destinations immediately before each link call.
+
+## Files
+
+- Modify: `src/agent_lab/infrastructure/benchmark_reporting.py`
+- Modify: `tests/test_benchmark_reporting.py`
+
+## Verification
+
+- `uv run pytest tests/test_benchmark_reporting.py -q`
+- repeated concurrent/competitor stress tests
+- `uv run pytest`
+- `git diff --check`

+ 1 - 0
docs/plans/todos.md

@@ -108,3 +108,4 @@
 | 52.5 | done | `docs/plans/todo-52.5-benchmark-ledger-event-identity.md` | Cross-check detected, batch-result, and tool-metric event identities and preserve attempted model counts on failed runs. | A successful result from the wrong event cannot pass; failed streams still report started ChatAgent/fallback attempts from timing records. |
 | 52.6 | in_progress | `docs/plans/todo-52.6-benchmark-report-auditability.md` | Complete report latency coverage, concurrent pair reservation, runner-level error auditability, and byte-bounded errors. | Reports aggregate model/tool latency, concurrent writers cannot mix pairs, catastrophic runner errors appear in report totals/details, zero samples warn, and errors stay within 2 KiB UTF-8. |
 | 52.7 | in_progress | `docs/plans/todo-52.7-benchmark-output-reservation-and-cardinality.md` | Prevent report overwrite/stale reservations, suppress argparse value echoes, and reject missing or extra benchmark results. | Final paths are atomically reserved without lock files, malformed CLI values are not echoed, and actual run cardinality must equal cases × modes × iterations. |
+| 52.8 | in_progress | `docs/plans/todo-52.8-benchmark-no-overwrite-publication.md` | Publish completed report files with atomic no-overwrite hard links instead of check-then-replace ownership logic. | A destination created at any point wins without being overwritten; the writer retries another suffix or fails cleanly with no mixed pair. |