docs: plan comparable invocation metrics

Problem: Existing usage rows only cover ChatAgent rounds and sum elapsed spans, hiding EventAgent fallback cost and making parallel architectures incomparable.

Risk: Additive metrics must preserve legacy API shape and avoid double-counting replayed events or overlapping latency.

docs/plans/todo-50-comparable-model-tool-metrics.md

@@ -0,0 +1,65 @@
+# Todo 50 Comparable Model and Tool Metrics Plan
+
+**Status:** in_progress
+
+## Goal
+
+Make `dual_agent` and `chat_agent_tools` directly comparable without double-counting model tokens, parallel latency, or replayed events.
+
+## Metric Contract
+
+- Keep the existing `/api/sessions/{id}/usage` endpoint and `calls` / `turns` / `session` shape.
+- Extend the persisted call ledger with:
+  - `agent`: `chat_agent`, `event_agent`, or `event_kernel`;
+  - `call_kind`: `chat_completion`, `argument_fallback`, or `tool_execution`;
+  - `event_id`, `event_name`, `used_fallback`, and `tool_latency_ms`;
+  - deterministic `metric_key` for idempotent persistence.
+- Derive `tool_invocation_mode` from the immutable session config instead of duplicating it per row.
+- Add independent `turns.wall_time_ms`, measured once from the runtime monotonic clock.
+- Token totals include only real model calls. Tool rows carry zero tokens.
+- Turn/session wall time is calculated only from `turns.wall_time_ms`; model elapsed and concurrent tool latencies remain per-call diagnostics and are never summed as wall time.
+- Persist only logical event results (`deduplicated_from is None`) so coalesced/replayed clones do not inflate tool or fallback counts.
+
+## Runtime Scope
+
+- ChatAgent records model TTFT from the first provider stream item while preserving visible-text TTFT in existing round events.
+- EventAgent consumes the complete fallback stream so trailing usage is retained, and exposes one model-call metric per fallback attempt, including failed resolution attempts.
+- Event kernel records handler/tool latency on each result.
+- Direct mode persists ChatAgent model calls plus logical tool executions.
+- Dual mode additionally persists EventAgent fallback model calls.
+
+## Storage and UI Scope
+
+- Migrate legacy SQLite databases with safe additive columns/indexes; legacy rows default to `chat_agent` / `chat_completion`.
+- Extend call, turn, and session summaries with mode, call counts, fallback counts, tool counts, and independent wall time.
+- Keep legacy `elapsed_ms` fields for compatibility; UI labels architecture/call kind and prefers `turn_wall_time_ms` for totals.
+
+## Files
+
+- Modify: `src/agent_lab/domain/messages.py`
+- Modify: `src/agent_lab/application/session_store.py`
+- Modify: `src/agent_lab/application/event_agent.py`
+- Modify: `src/agent_lab/application/events/models.py`
+- Modify: `src/agent_lab/application/events/kernel.py`
+- Modify: `src/agent_lab/application/runtime.py`
+- Modify: `src/agent_lab/infrastructure/sqlite_store.py`
+- Modify: `src/agent_lab/presentation/static/app.js`
+- Modify focused tests for event kernel/agent, runtime, SQLite, and WebSocket/UI.
+
+## Requirement Execution Loop
+
+### Plan
+
+1. Add RED storage/API tests for typed call rows, legacy migration, mode derivation, idempotent metric keys, and wall-time aggregation.
+2. Add RED EventAgent/kernel tests for trailing usage, first-provider-item TTFT, fallback attempts, and tool latency.
+3. Add RED runtime parity tests proving direct and dual persist the same logical tool count while dual includes fallback model cost only when used.
+4. Implement the smallest additive schema, metric capture, and UI formatting changes.
+5. Run focused tests, full suite, JS syntax, migration stress, diff check, then specification and quality review.
+
+## Verification
+
+- `uv run pytest tests/test_event_kernel.py tests/test_event_agent.py tests/test_sqlite_store.py tests/test_debug_runtime.py tests/test_websocket_api.py -q`
+- `node --check src/agent_lab/presentation/static/app.js`
+- `uv run pytest`
+- repeated legacy/concurrent SQLite migration test
+- `git diff --check`

docs/plans/todos.md

@@ -96,5 +96,5 @@
 | 49.1 | done | `docs/plans/todo-49.1-session-mode-and-atomic-transcript-safety.md` | Enforce persisted mode, atomically store direct tool rounds, repair incomplete legacy tails, serialize SQLite migration, and hide empty protocol bubbles. | Mode mismatches are rejected without config overwrite; disconnect cannot leave unresolved tool calls; concurrent migration succeeds; replay shows only visible assistant content. |
 | 49.2 | done | `docs/plans/todo-49.2-session-load-race-and-visible-repair.md` | Prevent stale session loads from overwriting the selected session and preserve visible text while stripping incomplete tool protocol. | Out-of-order loads cannot mutate the active session UI; whitespace-only protocol bubbles stay hidden; incomplete tool rounds retain provider-valid visible content. |
 | 49.3 | done | `docs/plans/todo-49.3-stale-replay-token-order.md` | Prevent a stale replay invocation from invalidating an already-running replay for the current session. | A late stale invocation exits before advancing the replay token; the current session still restores normally. |
-| 50 | pending | `docs/plans/todo-50-comparable-model-tool-metrics.md` | Record mode-aware model/tool spans and include EventAgent usage so both architectures can be compared fairly. | Persistence/API tests report agent, call kind, tokens, TTFT, tool latency, fallback usage, and total turn wall time without double counting. |
+| 50 | in_progress | `docs/plans/todo-50-comparable-model-tool-metrics.md` | Record mode-aware model/tool spans and include EventAgent usage so both architectures can be compared fairly. | Persistence/API tests report agent, call kind, tokens, TTFT, tool latency, fallback usage, and total turn wall time without double counting. |
 | 51 | pending | `docs/plans/todo-51-comparison-matrix-closeout.md` | Add repeatable comparison fixtures, update project documentation, and close the experimental implementation. | Both modes pass the same scenario matrix; full tests, JS syntax, app import, and documentation checks pass. |