|
|
@@ -0,0 +1,110 @@
|
|
|
+# Todo 48 Parallel Event Batches and Result Policies Plan
|
|
|
+
|
|
|
+**Status:** in_progress
|
|
|
+
|
|
|
+## Goal
|
|
|
+
|
|
|
+Execute flat multi-event batches through one shared scheduler in both invocation modes, then apply plugin-declared result policies without any event-name branches.
|
|
|
+
|
|
|
+## Shared Batch Executor
|
|
|
+
|
|
|
+Add a generic `EventBatchExecutor` that owns batch mechanics around an `EventKernel`:
|
|
|
+
|
|
|
+- configurable global concurrency limit;
|
|
|
+- per-event timeout from `EventDefinition.timeout_seconds`;
|
|
|
+- overall batch deadline;
|
|
|
+- stable result ordering matching input order;
|
|
|
+- failure and timeout isolation;
|
|
|
+- conflict-key locking so mutually conflicting events do not overlap;
|
|
|
+- non-terminal events first, then terminal events after all accepted siblings reach a terminal result or timeout;
|
|
|
+- exact duplicate coalescing inside a batch;
|
|
|
+- replay protection scoped by caller-provided session/turn scope plus event ID and canonical request data.
|
|
|
+
|
|
|
+The executor remains generic and never checks built-in event names.
|
|
|
+
|
|
|
+## Result Policy Aggregation
|
|
|
+
|
|
|
+Runtime derives one continuation decision from ordered event results and definition metadata:
|
|
|
+
|
|
|
+- `TERMINATE` success: emit any deterministic sibling confirmations, finish the turn/session, and suppress another ChatAgent call.
|
|
|
+- `SILENT_SUCCESS` success: keep the already streamed first acknowledgement and do not call ChatAgent again.
|
|
|
+- `TEMPLATE_FOLLOW_UP` success: emit a plugin-owned deterministic user message, append it to active history, and do not call ChatAgent again unless another result requires LLM synthesis.
|
|
|
+- `LLM_FOLLOW_UP` success: continue ChatAgent exactly once with compact results.
|
|
|
+- Any non-terminal failure/timeout: request one aggregated corrective ChatAgent follow-up.
|
|
|
+- Mixed batches: deterministic templates first, then at most one LLM follow-up; successful termination suppresses LLM follow-up after siblings finish.
|
|
|
+
|
|
|
+Plugins own deterministic template text through generic definition metadata or a stable payload field; Runtime does not format by event name.
|
|
|
+
|
|
|
+## Runtime and EventAgent Flow
|
|
|
+
|
|
|
+- Replace unbounded `asyncio.gather` in EventAgent and direct Runtime with the same batch executor.
|
|
|
+- Keep the EventAgent worker boundary for `dual_agent`; it resolves arguments with deterministic-first/LLM fallback and returns ordered tool replies.
|
|
|
+- Stop unconditionally enqueueing the legacy EventAgent summary. Runtime builds one compact summary only when the aggregated policy requires an LLM follow-up.
|
|
|
+- Direct mode keeps assistant/tool provider transcript and uses the same ordered batch results.
|
|
|
+- Reuse `EventAgentParams` for shared batch configuration:
|
|
|
+ - `max_parallel_events` default `4`, constrained to `1..16`;
|
|
|
+ - `batch_timeout_seconds` default `15`, positive and bounded.
|
|
|
+- Scope replay protection by session/turn so parser-local short IDs do not collide across turns.
|
|
|
+
|
|
|
+## Built-in Acceptance
|
|
|
+
|
|
|
+- `session.terminate`: farewell first, accepted siblings finish or time out, termination executes last, no second ChatAgent call, reusable session emits terminal completion.
|
|
|
+- `device.volume.adjust`: successful action has no second answer; failure gets one corrective answer.
|
|
|
+- `calendar.schedule.create`: successful action emits a deterministic confirmation without a model call.
|
|
|
+- `knowledge.web.search`: first answer, search, then one distinct ChatAgent follow-up grounded in compact results.
|
|
|
+- A mixed batch can contain schedule + web search + volume + terminate while preserving stable result order and terminal semantics.
|
|
|
+
|
|
|
+## Scope
|
|
|
+
|
|
|
+- Add batch execution/result types and timeout status.
|
|
|
+- Add shared scheduler use in EventAgent and direct Runtime.
|
|
|
+- Add generic tool-reply/result-policy aggregation helpers.
|
|
|
+- Add deterministic schedule confirmation owned by the plugin.
|
|
|
+- Add audit/output evidence for batch start/results, timeouts, deduplication, policy decision, and terminal completion.
|
|
|
+- Add focused executor, EventAgent, Runtime, and built-in behavior tests.
|
|
|
+
|
|
|
+## Boundaries
|
|
|
+
|
|
|
+- No retry framework beyond existing single execution; web retry remains future work.
|
|
|
+- No dynamic DAG, workflow engine, distributed lock, or production idempotency store.
|
|
|
+- Do not implement UI, reconnect transcript persistence, or comparison metrics here.
|
|
|
+- Do not cancel successful siblings because another event fails.
|
|
|
+- Do not catch `BaseException`/cancellation as an ordinary event failure.
|
|
|
+
|
|
|
+## Files
|
|
|
+
|
|
|
+- Add: `src/agent_lab/application/events/batch.py`
|
|
|
+- Modify: `src/agent_lab/application/events/models.py`
|
|
|
+- Modify: `src/agent_lab/application/events/__init__.py`
|
|
|
+- Modify: `src/agent_lab/application/contracts.py`
|
|
|
+- Modify: `src/agent_lab/application/tools.py`
|
|
|
+- Modify: `src/agent_lab/application/event_agent.py`
|
|
|
+- Modify: `src/agent_lab/application/runtime.py`
|
|
|
+- Modify: `src/agent_lab/application/events/builtin_plugins.py`
|
|
|
+- Add: `tests/test_event_batch.py`
|
|
|
+- Modify: `tests/test_event_agent.py`
|
|
|
+- Modify: `tests/test_debug_runtime.py`
|
|
|
+- Modify: `tests/test_builtin_event_plugins.py`
|
|
|
+- Modify: `tests/test_websocket_api.py` for config validation/defaults if needed
|
|
|
+
|
|
|
+## Requirement Execution Loop
|
|
|
+
|
|
|
+### Plan
|
|
|
+
|
|
|
+1. Add RED unit tests for bounded overlap, stable order, conflict serialization, timeout isolation, duplicate coalescing, scoped replay, and terminal-last execution.
|
|
|
+2. Add RED EventAgent/direct parity tests proving both use the same scheduler and produce the same ordered payloads.
|
|
|
+3. Add RED policy aggregation tests for silent, template, LLM, failure, terminate, and mixed batches.
|
|
|
+4. Add RED reusable-session termination and web-search two-answer tests.
|
|
|
+5. Implement the minimum generic scheduler and aggregation helpers.
|
|
|
+6. Run focused/full verification and independent specification/quality reviews.
|
|
|
+
|
|
|
+### Observe / Update
|
|
|
+
|
|
|
+Add `48.x` rows for any review-driven correction. Mark complete only when both modes share the scheduler and all four built-in result policies work without name-specific Runtime logic.
|
|
|
+
|
|
|
+## Verification
|
|
|
+
|
|
|
+- `UV_CACHE_DIR=.uv-cache uv run pytest tests/test_event_batch.py tests/test_event_agent.py tests/test_debug_runtime.py tests/test_builtin_event_plugins.py -q`
|
|
|
+- `UV_CACHE_DIR=.uv-cache uv run pytest`
|
|
|
+- `UV_CACHE_DIR=.uv-cache uv run python -c "from agent_lab.main import app; print(app.title)"`
|
|
|
+- `git diff --check`
|