# Todo 52.4 Benchmark Visible TTFT and Call Counts Plan

**Status:** done

## Goal

Prevent benchmark runs from passing with hidden extra model cost or an artificially early visible-response timestamp.

## Scope

- Record call-level and run-level visible TTFT only for a `message_delta` containing non-whitespace content.
- Add expected ChatAgent call count and expected fallback count to each benchmark case expectation.
- Compare both persisted ledger counts and timing-wrapper counts against the case expectation.
- Keep the six current cases at zero fallback; web search expects two ChatAgent calls and all other cases expect one.

## Files

- Modify: `src/agent_lab/application/benchmark.py`
- Modify: `tests/test_benchmark_config.py`
- Modify: `tests/test_benchmark_runner.py`

## Verification

- `uv run pytest tests/test_benchmark_runner.py tests/test_benchmark_config.py -q`
- `uv run pytest`
- `git diff --check`

## Observe / Update

- Visible TTFT now ignores blank/whitespace deltas at both call and run boundaries.
- Each case declares expected ChatAgent/fallback counts; ledger and timing attempts are checked against those expectations and each other.
- Focused tests passed (`111 passed`) and the full suite passed at this checkpoint (`516 passed`).