Explorar o código

docs: plan provider cleanup exception fix

Problem: Iterator cleanup can replace an active provider error or cancellation, hiding root cause and breaking interruption semantics.

Risk: Cleanup failures become secondary when another exception is active; tests must still prove standalone close failures propagate.
zhenyu.hu hai 2 semanas
pai
achega
16f3e2f9c1

+ 31 - 0
docs/plans/todo-52.13-provider-cleanup-exception-priority.md

@@ -0,0 +1,31 @@
+# Todo 52.13 Provider Cleanup Exception Priority Plan
+
+**Status:** in_progress
+
+## Goal
+
+Preserve the original provider failure, cancellation, or generator-close signal when cleanup of the wrapped provider iterator also fails.
+
+## Observation
+
+`TimingChatClient.stream_chat()` awaits `iterator.aclose()` in `finally`. If a provider exception or `CancelledError` is already propagating and `aclose()` raises another exception, the cleanup exception replaces the original. This can hide the real provider failure and convert cancellation into a normal runtime error instead of preserving interruption semantics.
+
+## Scope
+
+- Add regressions for provider exception plus close failure and cancellation plus close failure.
+- Keep the active provider/cancellation/generator-close exception as the primary propagated exception.
+- Propagate `aclose()` failure when no other exception is active.
+- Always append the timing attempt without changing provider-await elapsed, usage completeness, or parallel-probe behavior.
+
+## Files
+
+- Modify: `src/agent_lab/application/benchmark.py`
+- Modify: `tests/test_benchmark_runner.py`
+
+## Verification
+
+- `uv run pytest tests/test_benchmark_runner.py -q`
+- `uv run pytest tests/test_benchmark_runner.py tests/test_benchmark_reporting.py -q`
+- `uv run pytest`
+- `git diff --check`
+

+ 1 - 0
docs/plans/todos.md

@@ -113,3 +113,4 @@
 | 52.10 | done | `docs/plans/todo-52.10-benchmark-bundle-cli-contract.md` | Align CLI report discovery tests with the atomically published bundle layout. | CLI tests consume the reported bundle paths without restoring root-level compatibility copies or weakening redaction, exit-code, and cardinality checks. |
 | 52.11 | done | `docs/plans/todo-52.11-bundle-interrupt-and-path-safety.md` | Preserve complete bundles across post-symlink interrupts and reject unsafe report timestamp path components. | Interruptions cannot leave dangling public bundles; explicit timestamps stay inside output_dir; fsync tests verify both files and both directories. |
 | 52.12 | in_progress | `docs/plans/todo-52.12-benchmark-metric-integrity-and-parallel-proof.md` | Exclude consumer backpressure from model elapsed, reject missing token usage, and prove parallel handlers overlap. | Model latency measures provider-await time; absent usage cannot appear as zero cost; the parallel case fails unless both handlers start concurrently. |
+| 52.13 | in_progress | `docs/plans/todo-52.13-provider-cleanup-exception-priority.md` | Preserve active provider/cancellation exceptions when iterator cleanup also fails. | Provider failures and cancellation retain priority; cleanup failures propagate only when no earlier exception is active; timing attempts are still recorded. |