|
|
@@ -0,0 +1,59 @@
|
|
|
+# Todo 45.2 Event Fallback Semantics Plan
|
|
|
+
|
|
|
+**Status:** in_progress
|
|
|
+
|
|
|
+## Goal
|
|
|
+
|
|
|
+Make fallback and validation semantics suitable for latency/cost experiments: invoke LLM fallback only for incomplete deterministic resolution, compare raw and parsed JSON with type sensitivity, and distinguish plugin-definition failures from user argument errors.
|
|
|
+
|
|
|
+## Scope
|
|
|
+
|
|
|
+- Fall back only when required information is missing or a structured resolver explicitly marks its result incomplete.
|
|
|
+- Complete but invalid arguments return `INVALID_ARGUMENTS` without an LLM call.
|
|
|
+- Compare fallback raw JSON and parsed arguments canonically with JSON type sensitivity and reject non-standard JSON values.
|
|
|
+- Add `DEFINITION_ERROR` for validator/schema infrastructure failures.
|
|
|
+- Deep-copy/freeze registered schemas so post-registration mutation cannot bypass validation.
|
|
|
+- Add focused regression tests for each quality-review finding.
|
|
|
+
|
|
|
+## Files
|
|
|
+
|
|
|
+- Modify: `src/agent_lab/application/events/models.py`
|
|
|
+- Modify: `src/agent_lab/application/events/registry.py`
|
|
|
+- Modify: `src/agent_lab/application/events/kernel.py`
|
|
|
+- Modify: `src/agent_lab/application/tools.py` only for status compatibility mapping
|
|
|
+- Modify: `tests/test_event_kernel.py`
|
|
|
+- Modify: `tests/test_event_agent.py` only if fallback contracts change
|
|
|
+
|
|
|
+## Plan
|
|
|
+
|
|
|
+### Plan
|
|
|
+
|
|
|
+1. Add RED tests for complete-invalid no-fallback, JSON type-sensitive mismatch, broken `$ref`, and schema mutation.
|
|
|
+2. Verify each fails for the reviewed reason.
|
|
|
+3. Implement explicit resolution completeness, canonical strict JSON consistency, definition-error status, and registry schema copying.
|
|
|
+4. Run focused and full regressions.
|
|
|
+
|
|
|
+### Act
|
|
|
+
|
|
|
+- Test resolver output with all required fields present but wrong type/enum/additional property; fallback call count must remain zero.
|
|
|
+- Test missing required fields; fallback remains exactly one call.
|
|
|
+- Test raw `true` vs parsed `1`, raw `1` vs parsed `1.0`, nested mismatches, NaN, and Infinity; reject inconsistent/non-standard data.
|
|
|
+- Test an unresolved or failing `$ref`; return `DEFINITION_ERROR`, not `INVALID_ARGUMENTS`.
|
|
|
+- Register a definition, mutate the caller-owned schema afterward, and prove registry validation behavior is unchanged.
|
|
|
+- If structured deterministic resolution is introduced, preserve plain-dict resolver compatibility and require an explicit incomplete flag only for optional-schema cases.
|
|
|
+- Run RED, minimum implementation, focused/full verification.
|
|
|
+
|
|
|
+### Observe
|
|
|
+
|
|
|
+Confirm fallback counts now reflect missing information rather than arbitrary validation failures, raw/parsed values remain auditable, and plugin configuration failures are distinguishable from user mistakes.
|
|
|
+
|
|
|
+### Update
|
|
|
+
|
|
|
+- Mark 45, 45.1, and 45.2 `done` only after renewed reviews.
|
|
|
+- Commit separately with latency/cost and definition-error risks in the body.
|
|
|
+
|
|
|
+## Verification
|
|
|
+
|
|
|
+- `uv run pytest tests/test_event_kernel.py tests/test_event_agent.py -q`
|
|
|
+- `uv run pytest`
|
|
|
+- `git diff --check`
|