Преглед на файлове

docs: plan strict benchmark url validation

Problem: Task 1 review found control characters could pass config validation and fail later inside httpx client construction.

Risk: Unicode control/format rejection must not reject ordinary internationalized path characters.
zhenyu.hu преди 2 седмици
родител
ревизия
d903ccc054
променени са 2 файла, в които са добавени 25 реда и са изтрити 0 реда
  1. 24 0
      docs/plans/todo-52.1-benchmark-url-control-characters.md
  2. 1 0
      docs/plans/todos.md

+ 24 - 0
docs/plans/todo-52.1-benchmark-url-control-characters.md

@@ -0,0 +1,24 @@
+# Todo 52.1 Benchmark URL Control Characters Plan
+
+**Status:** in_progress
+
+## Goal
+
+Keep malformed benchmark targets inside the configuration error boundary instead of leaking an `httpx.InvalidURL` during live client construction.
+
+## Scope
+
+- Reject ASCII and Unicode characters whose category is control/format, including NUL, CR, LF, and tab.
+- Retain the existing HTTP(S), no-userinfo, no-query, and no-fragment requirements.
+- Add focused valid/invalid URL regression tests.
+
+## Files
+
+- Modify: `src/agent_lab/application/benchmark.py`
+- Modify: `tests/test_benchmark_config.py`
+
+## Verification
+
+- `uv run pytest tests/test_benchmark_config.py tests/test_tool_invocation_comparison.py -q`
+- `uv run pytest`
+- `git diff --check`

+ 1 - 0
docs/plans/todos.md

@@ -101,3 +101,4 @@
 | 51 | done | `docs/plans/todo-51-comparison-matrix-closeout.md` | Add repeatable comparison fixtures, update project documentation, and close the experimental implementation. | Both modes pass the same 12-run scenario matrix; focused (`231 passed`) and full (`417 passed`) tests, JS syntax, app import, and documentation checks pass. |
 | 51.1 | done | `docs/plans/todo-51.1-first-reply-and-protocol-matrix.md` | Require fast visible first replies for event scenarios and prove each mode uses its intended capability protocol surface. | Matrix asserts first reply ordering, no unintended second reply, dual compact event catalog/no provider tools, direct enabled schemas/no generated event catalog, and correct event source. |
 | 52 | in_progress | `docs/plans/todo-52-agent-benchmark-runner.md` | Add a configurable in-process benchmark CLI with deterministic mock cases and live OpenAI-compatible execution. | Mock/live runs compare both modes, measure provider/visible TTFT, wall time, tokens and event correctness, write secret-safe JSON/Markdown reports, and use stable exit codes. |
+| 52.1 | in_progress | `docs/plans/todo-52.1-benchmark-url-control-characters.md` | Reject non-printable/control characters in benchmark target URLs before constructing the HTTP client. | Strict config tests cover NUL, CR/LF, tabs, and other Unicode/control characters while valid HTTP(S) URLs remain accepted. |