agent-lab/docs/plans/todo-52.7-benchmark-output-reservation-and-cardinality.md

Todo 52.7 Benchmark Output Reservation and Cardinality Plan

Status: done

Goal

Eliminate remaining report publication races and prevent an incomplete benchmark execution from being reported as successful.

Scope

• Reserve both final report paths with exclusive file creation before writing temporary content; never overwrite a path created by another process.
• Remove lock/reservation sidecar files entirely; clean only placeholders/temporary/published files owned by the current attempt after failure.
• Replace argparse error details with a generic safe message so malformed values cannot be echoed to stderr.
• Require len(results) == len(cases) * len(modes) * runs_per_case; mismatches become redacted system errors in the report and exit code 2.

Files

• Modify: src/agent_lab/infrastructure/benchmark_reporting.py
• Modify: src/agent_lab/presentation/benchmark_cli.py
• Modify: tests/test_benchmark_reporting.py
• Modify: tests/test_benchmark_cli.py

Verification

• uv run pytest tests/test_benchmark_reporting.py tests/test_benchmark_cli.py -q
• concurrent reservation stress test
• uv run pytest
• git diff --check