agent-lab/docs/plans/todo-4-real-llm-smoke.md

Todo 4 Real LLM Smoke Plan

Status: done

Result

Implemented by subagent and locally verified.

Commands:

uv run pytest
uv run python scripts/ws_smoke.py --help

Results:

• uv run pytest: 9 passed, 1 warning in 0.22s
• ws_smoke.py --help: command help prints successfully.

Evaluation note: this todo made the local real-LLM run path documented and smokeable, but raised a provider-compatibility risk around tool-call history. Todo 5 now covers that before final cleanup.

Goal

Document and lightly verify the real OpenAI Chat Completions compatible run path without requiring a live API key in automated tests.

Scope

This todo covers:

• README run instructions for uv, env vars, and local server startup.
• .env.example with non-secret defaults/placeholders.
• A small manual smoke script that connects to the local WebSocket endpoint and prints streamed messages.
• Tests for smoke-script request construction if practical.

This todo does not cover:

• Running a real paid/network LLM call in CI or automated tests.
• Adding secrets to the repo.
• Adding another frontend framework.
• Changing runtime behavior unless a small bug blocks the documented smoke path.

Files

• Modify: README.md
• Create: .env.example
• Create: scripts/ws_smoke.py
• Create or modify: tests only if needed for the smoke helper.
• Do not modify: docs/plans/todos.md.

Environment Contract

Use these variables:

• AGENT_LAB_OPENAI_API_KEY
• AGENT_LAB_OPENAI_BASE_URL
• AGENT_LAB_OPENAI_DEFAULT_MODEL
• AGENT_LAB_REQUEST_TIMEOUT_SECONDS

The default base URL is already https://api.openai.com/v1. For compatible gateways, users set AGENT_LAB_OPENAI_BASE_URL to the gateway /v1 root.

Steps

1. Update README.md with:
   • uv sync
   • cp .env.example .env
   • required env vars
   • uv run uvicorn agent_lab.main:app --reload
   • browser URL http://127.0.0.1:8000
   • WebSocket smoke command.
2. Add .env.example with placeholder key and clear compatible endpoint defaults.
3. Add scripts/ws_smoke.py:
   • connects to ws://127.0.0.1:8000/ws/debug by default.
   • sends one valid debug request.
   • prints each server message as compact JSON.
   • accepts optional CLI args for URL, message, model, and tool enablement.
4. If adding smoke helper logic beyond straightforward I/O, add a focused test.

5. Run:

   uv run pytest
   uv run python scripts/ws_smoke.py --help
   

Verification

Expected result:

• Tests still pass.
• Smoke helper help output works without starting the server.
• README contains exact env names and local run commands.