Просмотр исходного кода

docs: adopt generic event kernel architecture

Problem: the design described four concrete events too directly and did not clearly separate reusable orchestration from built-in capability behavior.

Risk: plugin contracts remain flat and intentionally exclude dynamic installation or a workflow engine until repeated requirements justify them.
zhenyu.hu 2 недель назад
Родитель
Сommit
999121dce6

+ 45 - 17
docs/plans/2026-07-12-chat-event-agent-production-events-design.md

@@ -97,22 +97,51 @@ Neither mode is assumed to win before measurement:
 
 The likely trade-off is that direct tools may reduce argument-fallback calls on event-heavy conversations, while dual-agent mode should reduce prompt tokens and preserve first-answer behavior as the enabled capability set grows.
 
-### 2.3 Flat extensibility model
+### 2.3 Generic event kernel and flat built-in plugins
 
-The initial event model stays flat. There is one registry of self-contained event definitions; the first version does not introduce event category inheritance, nested capability trees, plugins, or a general workflow engine.
+The architecture separates a generic event kernel from concrete built-in event plugins. The plugin registry stays flat: the first version does not introduce event category inheritance, nested capability trees, dynamic installation, or a general workflow engine.
+
+The generic kernel owns only reusable orchestration concepts:
+
+- `EventRequest` and ordered `EventBatch` creation from either model protocol;
+- deterministic resolver chains and bounded LLM fallback;
+- schema validation and confirmation decisions;
+- execution-port invocation;
+- deduplication, concurrency, conflicts, cancellation, timeout, and terminal barriers;
+- normalized `EventResult` values and result-policy aggregation;
+- audit and performance spans.
+
+The kernel must not import, compare, or branch on built-in event names.
+
+A built-in plugin owns one concrete capability:
+
+- its namespaced event name and schema version;
+- the concise ChatAgent catalog description and provider tool schema;
+- deterministic resolver and validator;
+- optional LLM fallback policy;
+- execution handler/port adapter;
+- confirmation, risk, idempotency, concurrency, and result policies;
+- focused contract tests.
+
+Initial plugin names are:
+
+- `session.terminate`;
+- `device.volume.adjust`;
+- `calendar.schedule.create`;
+- `knowledge.web.search`.
 
 Adding a capability should normally require only:
 
-1. one new event definition containing its name, concise ChatAgent description, argument schema, resolution strategy, result policy, timeout, concurrency metadata, and terminal flag;
+1. one new built-in plugin definition containing its namespaced name, schema version, concise ChatAgent description, argument schema, resolution strategy, result policy, timeout, concurrency metadata, and terminal flag;
 2. one deterministic resolver/validator, with optional LLM fallback configuration;
 3. one handler or service-owned port adapter;
 4. focused contract, execution, failure, and concurrency tests.
 
-The parsers, ChatAgent protocols, Runtime, and shared event orchestration must not require an event-name-specific branch. They operate on the metadata declared by each definition. The event-specific flow sections below describe initial behavior and acceptance criteria, not hardcoded Runtime architecture.
+The parsers, ChatAgent protocols, Runtime, and generic kernel must not require an event-name-specific branch. They operate on the metadata declared by each plugin definition. The event-specific flow sections below describe built-in plugin behavior and acceptance criteria, not hardcoded Runtime architecture.
 
 Multiple events remain a flat ordered list in one batch. Generic batch orchestration handles deduplication, conflict metadata, bounded concurrency, result aggregation, and terminal barriers.
 
-The flat model should evolve only when real additions reveal repeated structures that cannot be expressed cleanly with definition metadata. Examples include several events sharing one authorization flow, a multi-step transaction/rollback requirement, or recurring dependencies between event outputs. Until then, those abstractions are deliberately out of scope.
+The flat plugin registry should evolve only when real additions reveal repeated structures that cannot be expressed cleanly with definition metadata. Examples include several events sharing one authorization flow, a multi-step transaction/rollback requirement, or recurring dependencies between event outputs. Until then, those abstractions are deliberately out of scope.
 
 ## 3. Event Flows
 
@@ -191,11 +220,11 @@ Conflict policy:
 - an event failure does not cancel unrelated events;
 - follow-up outputs are aggregated by policy: deterministic confirmations first, then at most one ChatAgent LLM follow-up for all results that require natural synthesis.
 
-## 4. Runtime and Registry Changes
+## 4. Generic Kernel, Runtime, and Plugin Registry
 
-The design extends the registry-owned event definition with execution metadata rather than adding event-name conditionals throughout the runtime.
+The design extracts a generic kernel from Runtime and extends registry-owned plugin definitions with execution metadata rather than adding event-name conditionals throughout orchestration.
 
-Conceptually, an event definition contains:
+Conceptually, a built-in plugin definition contains:
 
 - name, description, and JSON argument schema;
 - argument-generation behavior;
@@ -208,18 +237,17 @@ Conceptually, an event definition contains:
 - whether failure is safe to expose to ChatAgent;
 - whether the event makes the session terminal.
 
-The runtime remains responsible for orchestration and ordering:
+Runtime remains responsible for session/turn and model-stream lifecycle. It delegates event work to the generic kernel:
 
 1. stream ChatAgent output;
 2. parse hidden text events or collect native tool calls into the same internal event batch;
-3. resolve event arguments using deterministic parsers first and LLM fallback only when needed;
-4. apply deduplication, conflict, and concurrency rules;
-5. dispatch independent EventAgent work concurrently;
-6. publish audit/tool results;
-7. apply each event's result policy;
-8. optionally run one aggregated follow-up ChatAgent round.
-
-EventAgent remains responsible for argument resolution, validation, and handler invocation. Production side effects remain behind service-owned ports/adapters.
+3. submit the flat batch to the event kernel;
+4. let the kernel resolve, validate, confirm, deduplicate, schedule, execute, and normalize results using plugin metadata;
+5. publish normalized audit/tool results;
+6. apply the kernel's aggregated continuation decision;
+7. optionally run one aggregated follow-up ChatAgent round.
+
+EventAgent supplies optional LLM fallback to the kernel; it does not own built-in event behavior. Production side effects remain behind service-owned ports/adapters implemented by plugins.
 
 The local fast Gate for obvious high-confidence intents remains a later optimization. It should be evaluated only after the two configured model-driven modes establish a baseline and production-like measurements show that event-catalog/tool-schema cost or event-selection errors justify the added routing complexity.
 

+ 9 - 1
docs/plans/todo-44-provider-tool-transcript.md

@@ -1,6 +1,14 @@
 # Todo 44 Provider Tool Transcript Plan
 
-**Status:** in_progress
+**Status:** done
+
+## Result
+
+- Added structured assistant tool calls and provider-compatible tool replies.
+- Added local transcript-state validation and explicit tool-choice behavior.
+- Removed implicit forcing for a single ChatAgent tool while EventAgent explicitly forces its selected function.
+- Todo 44.1 hardened identity, payload precedence, history projection, and public request boundaries.
+- Verified with focused tests (`97 passed`) and the full suite (`108 passed`, one existing warning).
 
 ## Goal
 

+ 69 - 0
docs/plans/todo-44.1-tool-transcript-hardening.md

@@ -0,0 +1,69 @@
+# Todo 44.1 Tool Transcript Hardening Plan
+
+**Status:** done
+
+## Result
+
+- Rejected duplicate call IDs within and across assistant messages.
+- Protected provider request fields from `extra_body` overrides and validated forced choices locally.
+- Projected EventAgent history without unresolved provider protocol state.
+- Kept public pre-messages user-facing by rejecting internal tool protocol messages.
+- Completed role-field and test-double validation; full suite passes (`108 passed`).
+
+## Goal
+
+Close identity, payload-precedence, history-projection, and request-boundary gaps found by Todo 44 code review before direct Runtime tool mode builds on the transcript layer.
+
+## Scope
+
+- Reject duplicate assistant tool-call IDs before serialization.
+- Prevent `extra_body` from overriding reserved provider request fields, especially explicit `tool_choice`.
+- Validate forced tool choices locally against the supplied tools.
+- Project EventAgent conversation history without unresolved assistant tool-call protocol state.
+- Keep public `pre_messages` user-facing only by rejecting both tool replies and assistant tool calls.
+- Complete role-specific field validation and update all affected fakes.
+
+## Files
+
+- Modify: `src/agent_lab/domain/messages.py`
+- Modify: `src/agent_lab/application/contracts.py`
+- Modify: `src/agent_lab/application/event_agent.py`
+- Modify: `src/agent_lab/infrastructure/chat_client.py`
+- Modify: `tests/test_websocket_api.py`
+- Modify: `tests/test_event_agent.py`
+- Modify: `tests/test_debug_runtime.py`
+
+## Plan
+
+### Plan
+
+1. Add failing negative tests for every review finding.
+2. Verify failures are caused by current acceptance/override behavior, not test setup.
+3. Implement the smallest domain/client/history/request-boundary hardening.
+4. Run focused and full regressions.
+
+### Act
+
+- Add tests rejecting duplicate tool-call IDs within one assistant message and across a transcript.
+- Add tests proving `extra_body` cannot override `messages`, `tools`, `tool_choice`, model, streaming, or token/sampling controls.
+- Add tests rejecting forced tool choice with no tools or a function name absent from the supplied list.
+- Add an EventAgent test with a prior complete assistant/tool round and assert its argument prompt contains visible assistant content without unresolved `tool_calls`.
+- Add request-contract tests rejecting `pre_messages` that contain either tool replies or assistant tool calls.
+- Update ChatMessage role-field tests so irrelevant `tool_call_id`/`tool_calls` fields fail instead of being ignored.
+- Update all fake `stream_chat` signatures and assertions for optional `tool_choice`.
+- Run RED, apply minimum fixes, then run focused/full verification.
+
+### Observe
+
+Confirm transcript identity is unique, explicit controlled fields win over provider extras, EventAgent receives a provider-valid projected history, and public pre-messages cannot inject partial internal protocol state.
+
+### Update
+
+- Mark 44 and 44.1 `done` only after spec and quality re-review.
+- Commit separately with the malformed transcript and provider override risks in the body.
+
+## Verification
+
+- `uv run pytest tests/test_websocket_api.py tests/test_event_agent.py tests/test_debug_runtime.py -q`
+- `uv run pytest`
+- `git diff --check`

+ 4 - 3
docs/plans/todos.md

@@ -72,10 +72,11 @@
 | 43 | done | `docs/plans/todo-43-typed-provider-tool-calls.md` | Preserve native provider tool calls as a distinct stream event without changing the existing text-event path. | `uv run pytest tests/test_openai_stream_parser.py -q`; focused typed-event regressions; full suite passes (`82 passed`, one existing warning). |
 | 43.1 | done | `docs/plans/todo-43.1-typed-event-consumer-regression.md` | Repair Runtime and EventAgent consumers after typed stream kinds exposed legacy `event` assumptions. | Runtime consumes `text_event`, EventAgent consumes `provider_tool_call`; focused and full tests pass. |
 | 43.2 | done | `docs/plans/todo-43.2-remove-legacy-event-consumption.md` | Remove legacy `event` consumption from production paths and migrate remaining fixtures to typed stream kinds. | Wrong-source negative tests and real client `[DONE]` integration pass; full suite passes (`82 passed`). |
-| 44 | in_progress | `docs/plans/todo-44-provider-tool-transcript.md` | Add provider-compatible assistant tool-call and tool-result message serialization plus explicit tool-choice behavior. | Client/domain tests prove legal transcripts serialize and malformed tool history is rejected. |
+| 44 | done | `docs/plans/todo-44-provider-tool-transcript.md` | Add provider-compatible assistant tool-call and tool-result message serialization plus explicit tool-choice behavior. | Focused transcript tests pass (`97 passed`); full suite passes (`108 passed`, one existing warning). |
+| 44.1 | done | `docs/plans/todo-44.1-tool-transcript-hardening.md` | Harden transcript identity, reserved payload fields, EventAgent history projection, and public pre-message boundaries. | Duplicate IDs, invalid choices, reserved overrides, projected history, and pre-message boundaries are covered; full suite passes. |
 | 45 | pending | `docs/plans/todo-45-configurable-tool-invocation-mode.md` | Add `dual_agent` and `chat_agent_tools` runtime modes that share one registry and user-visible behavior. | Runtime tests prove mode-specific requests, direct execution, valid provider history, and dual-mode regression safety. |
-| 46 | pending | `docs/plans/todo-46-flat-event-policies-hybrid-resolution.md` | Extend flat event definitions with deterministic-first argument resolution, LLM fallback, and generic result policies. | Tests prove deterministic events skip EventAgent LLM and ambiguous cases use one bounded fallback. |
-| 47 | pending | `docs/plans/todo-47-production-event-definitions.md` | Add experimental terminate-session, device-volume, schedule, and web-search event definitions with normalized contracts. | Focused tests cover valid, ambiguous, invalid, idempotent, and failure results for all four events. |
+| 46 | pending | `docs/plans/todo-46-generic-event-kernel.md` | Build a generic event kernel with definition metadata, deterministic-first resolution, policies, execution ports, and result aggregation. | Kernel contract tests prove Runtime orchestration contains no built-in event-name branches and plugins share one pipeline. |
+| 47 | pending | `docs/plans/todo-47-built-in-event-plugins.md` | Add terminate-session, device-volume, schedule, and web-search as built-in plugins on the generic event kernel. | Plugin contract tests cover valid, ambiguous, invalid, idempotent, and failure results for all four events. |
 | 48 | pending | `docs/plans/todo-48-parallel-event-batches.md` | Execute multiple flat events with bounded parallelism, failure isolation, aggregation, and a terminal barrier. | Runtime tests prove independent concurrency, stable result order, isolated failures, and termination after sibling completion or timeout. |
 | 49 | pending | `docs/plans/todo-49-tool-mode-ui-session-restore.md` | Add UI mode selection, workspace/session restoration, and clear EventAgent applicability feedback. | Static/WebSocket tests prove requests, snapshots, loaded sessions, and audit context preserve the selected mode. |
 | 50 | pending | `docs/plans/todo-50-comparable-model-tool-metrics.md` | Record mode-aware model/tool spans and include EventAgent usage so both architectures can be compared fairly. | Persistence/API tests report agent, call kind, tokens, TTFT, tool latency, fallback usage, and total turn wall time without double counting. |