瀏覽代碼

docs: plan parallel event batch execution

Problem: direct and dual modes still use ad hoc unbounded batches and Runtime always performs another ChatAgent round regardless of plugin result policy.

Risk: scheduling must preserve order, isolate failures, scope deduplication, execute terminal work last, and avoid event-name branches while maintaining valid provider transcripts.
zhenyu.hu 2 周之前
父節點
當前提交
c457203ceb
共有 2 個文件被更改,包括 111 次插入1 次删除
  1. 110 0
      docs/plans/todo-48-parallel-event-batches.md
  2. 1 1
      docs/plans/todos.md

+ 110 - 0
docs/plans/todo-48-parallel-event-batches.md

@@ -0,0 +1,110 @@
+# Todo 48 Parallel Event Batches and Result Policies Plan
+
+**Status:** in_progress
+
+## Goal
+
+Execute flat multi-event batches through one shared scheduler in both invocation modes, then apply plugin-declared result policies without any event-name branches.
+
+## Shared Batch Executor
+
+Add a generic `EventBatchExecutor` that owns batch mechanics around an `EventKernel`:
+
+- configurable global concurrency limit;
+- per-event timeout from `EventDefinition.timeout_seconds`;
+- overall batch deadline;
+- stable result ordering matching input order;
+- failure and timeout isolation;
+- conflict-key locking so mutually conflicting events do not overlap;
+- non-terminal events first, then terminal events after all accepted siblings reach a terminal result or timeout;
+- exact duplicate coalescing inside a batch;
+- replay protection scoped by caller-provided session/turn scope plus event ID and canonical request data.
+
+The executor remains generic and never checks built-in event names.
+
+## Result Policy Aggregation
+
+Runtime derives one continuation decision from ordered event results and definition metadata:
+
+- `TERMINATE` success: emit any deterministic sibling confirmations, finish the turn/session, and suppress another ChatAgent call.
+- `SILENT_SUCCESS` success: keep the already streamed first acknowledgement and do not call ChatAgent again.
+- `TEMPLATE_FOLLOW_UP` success: emit a plugin-owned deterministic user message, append it to active history, and do not call ChatAgent again unless another result requires LLM synthesis.
+- `LLM_FOLLOW_UP` success: continue ChatAgent exactly once with compact results.
+- Any non-terminal failure/timeout: request one aggregated corrective ChatAgent follow-up.
+- Mixed batches: deterministic templates first, then at most one LLM follow-up; successful termination suppresses LLM follow-up after siblings finish.
+
+Plugins own deterministic template text through generic definition metadata or a stable payload field; Runtime does not format by event name.
+
+## Runtime and EventAgent Flow
+
+- Replace unbounded `asyncio.gather` in EventAgent and direct Runtime with the same batch executor.
+- Keep the EventAgent worker boundary for `dual_agent`; it resolves arguments with deterministic-first/LLM fallback and returns ordered tool replies.
+- Stop unconditionally enqueueing the legacy EventAgent summary. Runtime builds one compact summary only when the aggregated policy requires an LLM follow-up.
+- Direct mode keeps assistant/tool provider transcript and uses the same ordered batch results.
+- Reuse `EventAgentParams` for shared batch configuration:
+  - `max_parallel_events` default `4`, constrained to `1..16`;
+  - `batch_timeout_seconds` default `15`, positive and bounded.
+- Scope replay protection by session/turn so parser-local short IDs do not collide across turns.
+
+## Built-in Acceptance
+
+- `session.terminate`: farewell first, accepted siblings finish or time out, termination executes last, no second ChatAgent call, reusable session emits terminal completion.
+- `device.volume.adjust`: successful action has no second answer; failure gets one corrective answer.
+- `calendar.schedule.create`: successful action emits a deterministic confirmation without a model call.
+- `knowledge.web.search`: first answer, search, then one distinct ChatAgent follow-up grounded in compact results.
+- A mixed batch can contain schedule + web search + volume + terminate while preserving stable result order and terminal semantics.
+
+## Scope
+
+- Add batch execution/result types and timeout status.
+- Add shared scheduler use in EventAgent and direct Runtime.
+- Add generic tool-reply/result-policy aggregation helpers.
+- Add deterministic schedule confirmation owned by the plugin.
+- Add audit/output evidence for batch start/results, timeouts, deduplication, policy decision, and terminal completion.
+- Add focused executor, EventAgent, Runtime, and built-in behavior tests.
+
+## Boundaries
+
+- No retry framework beyond existing single execution; web retry remains future work.
+- No dynamic DAG, workflow engine, distributed lock, or production idempotency store.
+- Do not implement UI, reconnect transcript persistence, or comparison metrics here.
+- Do not cancel successful siblings because another event fails.
+- Do not catch `BaseException`/cancellation as an ordinary event failure.
+
+## Files
+
+- Add: `src/agent_lab/application/events/batch.py`
+- Modify: `src/agent_lab/application/events/models.py`
+- Modify: `src/agent_lab/application/events/__init__.py`
+- Modify: `src/agent_lab/application/contracts.py`
+- Modify: `src/agent_lab/application/tools.py`
+- Modify: `src/agent_lab/application/event_agent.py`
+- Modify: `src/agent_lab/application/runtime.py`
+- Modify: `src/agent_lab/application/events/builtin_plugins.py`
+- Add: `tests/test_event_batch.py`
+- Modify: `tests/test_event_agent.py`
+- Modify: `tests/test_debug_runtime.py`
+- Modify: `tests/test_builtin_event_plugins.py`
+- Modify: `tests/test_websocket_api.py` for config validation/defaults if needed
+
+## Requirement Execution Loop
+
+### Plan
+
+1. Add RED unit tests for bounded overlap, stable order, conflict serialization, timeout isolation, duplicate coalescing, scoped replay, and terminal-last execution.
+2. Add RED EventAgent/direct parity tests proving both use the same scheduler and produce the same ordered payloads.
+3. Add RED policy aggregation tests for silent, template, LLM, failure, terminate, and mixed batches.
+4. Add RED reusable-session termination and web-search two-answer tests.
+5. Implement the minimum generic scheduler and aggregation helpers.
+6. Run focused/full verification and independent specification/quality reviews.
+
+### Observe / Update
+
+Add `48.x` rows for any review-driven correction. Mark complete only when both modes share the scheduler and all four built-in result policies work without name-specific Runtime logic.
+
+## Verification
+
+- `UV_CACHE_DIR=.uv-cache uv run pytest tests/test_event_batch.py tests/test_event_agent.py tests/test_debug_runtime.py tests/test_builtin_event_plugins.py -q`
+- `UV_CACHE_DIR=.uv-cache uv run pytest`
+- `UV_CACHE_DIR=.uv-cache uv run python -c "from agent_lab.main import app; print(app.title)"`
+- `git diff --check`

+ 1 - 1
docs/plans/todos.md

@@ -88,7 +88,7 @@
 | 47.1 | done | `docs/plans/todo-47.1-plugin-normalization-and-validation.md` | Share plugin normalization across invocation modes and harden negation, numeric, date/timezone, query, and adapter idempotency boundaries. | Direct/dual normalization matches; strict date/timezone/query validation and canonical adapter keys pass final gates. |
 | 47.2 | done | `docs/plans/todo-47.2-plugin-token-and-compatibility-boundaries.md` | Reject remaining negated/non-integer volume tokens, canonicalize integral JSON numbers, preserve fallback state, use standard date-time semantics, and restore positional dataclass compatibility. | Named negation/token cases, integral floats, fallback provenance, RFC3339, and positional compatibility pass. |
 | 47.3 | done | `docs/plans/todo-47.3-bare-volume-numeric-guard.md` | Apply malformed numeric rejection to the existing bare `volume to`, `volume at`, and `volume=` syntaxes. | Bare decimal/grouped tokens are rejected with no side effect; integer forms remain valid; bounded gate APPROVED. |
-| 48 | pending | `docs/plans/todo-48-parallel-event-batches.md` | Execute multiple flat events with bounded parallelism, failure isolation, aggregation, and a terminal barrier. | Runtime tests prove independent concurrency, stable result order, isolated failures, and termination after sibling completion or timeout. |
+| 48 | in_progress | `docs/plans/todo-48-parallel-event-batches.md` | Execute multiple flat events with bounded parallelism, failure isolation, aggregation, and a terminal barrier. | Runtime tests prove independent concurrency, stable result order, isolated failures, and termination after sibling completion or timeout. |
 | 49 | pending | `docs/plans/todo-49-tool-mode-ui-session-restore.md` | Add UI mode selection, workspace/session restoration, and clear EventAgent applicability feedback. | Static/WebSocket tests prove requests, snapshots, loaded sessions, and audit context preserve the selected mode. |
 | 50 | pending | `docs/plans/todo-50-comparable-model-tool-metrics.md` | Record mode-aware model/tool spans and include EventAgent usage so both architectures can be compared fairly. | Persistence/API tests report agent, call kind, tokens, TTFT, tool latency, fallback usage, and total turn wall time without double counting. |
 | 51 | pending | `docs/plans/todo-51-comparison-matrix-closeout.md` | Add repeatable comparison fixtures, update project documentation, and close the experimental implementation. | Both modes pass the same scenario matrix; full tests, JS syntax, app import, and documentation checks pass. |