Forráskód Böngészése

docs: plan mixed validation fallback fix

Problem: a missing field can still trigger fallback alongside substantive validation failures, and arbitrary handler payload serialization exceptions can escape the event pipeline.

Risk: fallback eligibility must remain true for pure missing-only input while broad exception handling must not swallow cancellation or BaseException signals.
zhenyu.hu 2 hete
szülő
commit
08dc721117

+ 57 - 0
docs/plans/todo-45.6-mixed-validation-and-handler-snapshot.md

@@ -0,0 +1,57 @@
+# Todo 45.6 Mixed Validation and Handler Snapshot Plan
+
+**Status:** in_progress
+
+## Goal
+
+Close the final reviewed cost and error-isolation gaps: mixed validation failures must never trigger LLM fallback, and arbitrary exceptions while snapshotting a handler payload must become normalized handler failures.
+
+## Scope
+
+- Treat validation as fallback-eligible only when every reported top-level failure classifies as missing-only.
+- Cover required fields combined with enum, type, additional-property, and composed-schema substantive errors.
+- Catch any normal exception raised while producing a strict JSON snapshot of a handler payload in async and sync execution.
+- Verify ToolRegistry and EventAgent receive normalized error payloads rather than exceptions.
+
+## Boundaries
+
+- Preserve fallback for pure missing-only input, including the unique viable composed branch from Todo 45.5.
+- Do not catch `BaseException` subclasses such as cancellation or process-exit signals.
+- Keep strict JSON rejection behavior; do not coerce unusual dictionary subclasses or values.
+
+## Files
+
+- Modify: `src/agent_lab/application/events/kernel.py`
+- Modify: `tests/test_event_kernel.py`
+- Modify: `tests/test_event_agent.py`
+
+## Requirement Execution Loop
+
+### Plan
+
+1. Add RED tests for required plus enum/type/additionalProperties and required plus composed substantive errors; fallback count must remain zero.
+2. Add RED tests whose handler returns a dictionary subclass raising `RuntimeError` during JSON serialization.
+3. Require all validation issues to be missing-only before setting `missing_required=True`.
+4. Normalize strict payload snapshot exceptions to `HANDLER_ERROR` in sync and async paths.
+5. Verify the behavior through EventKernel, ToolRegistry, and EventAgent, then run full reviews.
+
+### Act
+
+- Assert mixed invalid input reports `INVALID_ARGUMENTS`, `used_fallback=False`, and zero fallback calls.
+- Assert pure missing input still falls back exactly once.
+- Assert exploding payload snapshots return `event handler returned non-JSON payload` without escaping from kernel, ToolRegistry, or EventAgent.
+
+### Observe
+
+Confirm fallback counts represent only missing information and every ordinary plugin payload failure remains isolated inside the event result pipeline.
+
+### Update
+
+- Re-run final Todo 45 specification and quality reviews.
+- Mark Todos 45 through 45.6 done only after fresh verification.
+
+## Verification
+
+- `UV_CACHE_DIR=.uv-cache uv run pytest tests/test_event_kernel.py tests/test_event_agent.py -q`
+- `UV_CACHE_DIR=.uv-cache uv run pytest`
+- `git diff --check`

+ 1 - 0
docs/plans/todos.md

@@ -80,6 +80,7 @@
 | 45.3 | in_progress | `docs/plans/todo-45.3-sync-resolution-completeness.md` | Make synchronous kernel execution reject explicitly incomplete deterministic resolution instead of invoking the handler. | A structured incomplete result with an optional-only schema returns `INVALID_ARGUMENTS`; the handler is not called and async behavior remains aligned. |
 | 45.4 | in_progress | `docs/plans/todo-45.4-event-kernel-quality-boundaries.md` | Close reviewed configuration, composed-schema fallback, mutation isolation, registry encapsulation, and payload-serialization gaps. | Default EventAgent config survives omitted overrides; nested required errors can fall back; handlers and callers cannot corrupt audit arguments; validators stay private; non-JSON payloads become `HANDLER_ERROR`. |
 | 45.5 | in_progress | `docs/plans/todo-45.5-composed-schema-and-copy-failures.md` | Prevent unrelated composed-schema branches from triggering fallback and normalize argument snapshot failures without exposing validator internals. | Complete-invalid discriminated unions never call fallback; missing viable branches still can; uncopyable/non-JSON resolver and provider arguments return stable results instead of raising or becoming definition errors. |
+| 45.6 | in_progress | `docs/plans/todo-45.6-mixed-validation-and-handler-snapshot.md` | Require every validation failure to be missing-only before fallback and contain arbitrary handler payload serialization exceptions. | Required plus enum/type/additional-property errors never call fallback; sync/async handler snapshot failures become stable `HANDLER_ERROR` results through ToolRegistry and EventAgent. |
 | 46 | pending | `docs/plans/todo-46-configurable-tool-invocation-mode.md` | Add `dual_agent` and `chat_agent_tools` runtime modes on the shared event kernel. | Runtime tests prove mode-specific requests, direct execution, valid provider history, and dual-mode regression safety. |
 | 47 | pending | `docs/plans/todo-47-built-in-event-plugins.md` | Add terminate-session, device-volume, schedule, and web-search as built-in plugins on the generic event kernel. | Plugin contract tests cover valid, ambiguous, invalid, idempotent, and failure results for all four events. |
 | 48 | pending | `docs/plans/todo-48-parallel-event-batches.md` | Execute multiple flat events with bounded parallelism, failure isolation, aggregation, and a terminal barrier. | Runtime tests prove independent concurrency, stable result order, isolated failures, and termination after sibling completion or timeout. |