Sfoglia il codice sorgente

docs: plan benchmark bundle safety fixes

Problem: A post-symlink interrupt can remove the backing bundle, and an unsafe explicit timestamp can escape the report output directory.

Risk: Preserving complete bundles on ambiguous publication errors may leave hidden orphan bundles, which is safer than a dangling public report and should remain bounded to exceptional paths.
zhenyu.hu 2 settimane fa
parent
commit
05e1f41d13

+ 33 - 0
docs/plans/todo-52.11-bundle-interrupt-and-path-safety.md

@@ -0,0 +1,33 @@
+# Todo 52.11 Bundle Interrupt and Path Safety Plan
+
+**Status:** in_progress
+
+## Goal
+
+Close the post-symlink interruption window and constrain report bundle names to the configured output directory.
+
+## Observation
+
+- A `KeyboardInterrupt` can arrive after `os.symlink()` creates the public bundle entry but before the implementation records publication. Cleanup then removes the backing bundle and leaves a dangling public symlink.
+- The optional `timestamp` argument is used as a path component without basename validation, so an absolute path or separator-containing value can escape `output_dir`.
+- Existing fsync tests count calls but do not prove that both report files, the bundle directory, and the output directory are the objects being synced.
+
+## Scope
+
+- Add a regression that injects interruption immediately after successful symlink creation and prove the public bundle remains complete rather than dangling.
+- Once public publication is attempted, retain the already-complete private bundle on ambiguous interruption/error paths; cleanup only before publication begins.
+- Validate an explicit timestamp as a bounded safe basename using a conservative ASCII identifier character set; reject absolute paths, separators, dot segments, controls, and empty values before filesystem writes.
+- Strengthen fsync tests to identify the two report files and both directories, not only count invocations.
+
+## Files
+
+- Modify: `src/agent_lab/infrastructure/benchmark_reporting.py`
+- Modify: `tests/test_benchmark_reporting.py`
+
+## Verification
+
+- `uv run pytest tests/test_benchmark_reporting.py -q`
+- `uv run pytest tests/test_benchmark_reporting.py tests/test_benchmark_cli.py -q`
+- `uv run pytest`
+- `git diff --check`
+

+ 1 - 0
docs/plans/todos.md

@@ -111,3 +111,4 @@
 | 52.8 | in_progress | `docs/plans/todo-52.8-benchmark-no-overwrite-publication.md` | Publish completed report files with atomic no-overwrite hard links instead of check-then-replace ownership logic. | A destination created at any point wins without being overwritten; the writer retries another suffix or fails cleanly with no mixed pair. |
 | 52.9 | in_progress | `docs/plans/todo-52.9-atomic-report-bundles-and-audit-parity.md` | Replace two-path report publication with one atomic bundle boundary and close report/CLI audit parity gaps. | Concurrent writers publish complete bundles without rollback deletion races; Markdown preserves all failure facts and output errors map to exit code 2. |
 | 52.10 | in_progress | `docs/plans/todo-52.10-benchmark-bundle-cli-contract.md` | Align CLI report discovery tests with the atomically published bundle layout. | CLI tests consume the reported bundle paths without restoring root-level compatibility copies or weakening redaction, exit-code, and cardinality checks. |
+| 52.11 | in_progress | `docs/plans/todo-52.11-bundle-interrupt-and-path-safety.md` | Preserve complete bundles across post-symlink interrupts and reject unsafe report timestamp path components. | Interruptions cannot leave dangling public bundles; explicit timestamps stay inside output_dir; fsync tests verify both files and both directories. |