feat: auto-discover runners (#12)
This commit is contained in:
banteg
+89
-33
@@ -5,13 +5,13 @@ domain model. Use the existing runners (Codex/Claude) as references.
|
||||
|
||||
## Quick checklist
|
||||
|
||||
1. Implement `Runner` in `src/takopi/runners/<engine>.py`.
|
||||
1. Implement `Runner` in `src/takopi/runners/<engine>.py` (usually via
|
||||
`JsonlSubprocessRunner`).
|
||||
2. Emit Takopi events from `takopi.model` and implement resume helpers
|
||||
(`format_resume`, `extract_resume`, `is_resume_line`).
|
||||
3. Register an `EngineBackend` in `src/takopi/engines.py` with setup checks
|
||||
and runner construction.
|
||||
4. Add CLI subcommand in `src/takopi/cli.py`.
|
||||
5. Extend tests (runner contract + engine-specific translation tests).
|
||||
3. Define `BACKEND = EngineBackend(...)` in the runner module (auto-discovered),
|
||||
including `install_cmd` (and `cli_cmd` only if the binary name differs).
|
||||
4. Extend tests (runner contract + engine-specific translation tests).
|
||||
|
||||
---
|
||||
|
||||
@@ -25,20 +25,41 @@ make it easy to drop in another engine without changing the Takopi domain model.
|
||||
- Engine id: `"pi"` (used in config, resume tokens, and CLI subcommand).
|
||||
- Canonical resume line: the engine’s own CLI resume command, e.g.
|
||||
`` `pi --resume <session_id>` ``.
|
||||
- If your engine uses the standard `"<engine> resume <token>"` format, you can
|
||||
reuse `compile_resume_pattern()`. Otherwise, define a custom regex in the
|
||||
runner (like Claude does).
|
||||
- Pick the resume line format you want to support and define a regex for it in
|
||||
the runner (Claude is a good example). If you choose the
|
||||
`"<engine> resume <token>"` shape, you can use that exact regex.
|
||||
|
||||
### 2) Implement `src/takopi/runners/pi.py`
|
||||
|
||||
Skeleton outline:
|
||||
Recommended: `JsonlSubprocessRunner`
|
||||
|
||||
For JSONL CLIs, this base class centralizes subprocess + JSONL plumbing,
|
||||
lock timing, and completion semantics. Your runner usually only needs:
|
||||
|
||||
- `command()` (binary name)
|
||||
- `build_args(...)`
|
||||
- `translate(...)` (map one JSON object to a list of Takopi events)
|
||||
|
||||
Optional hooks for common variants:
|
||||
|
||||
- `stdin_payload(...)`: return `None` if the prompt is passed via argv
|
||||
- `env(...)`: add or redact environment variables
|
||||
- `invalid_json_events(...)`: customize the warning event
|
||||
- `process_error_events(...)`: customize `rc != 0` handling
|
||||
- `stream_end_events(...)`: customize stream-end fallback (no `CompletedEvent`)
|
||||
- `handle_started_event(...)`: customize session-id validation
|
||||
|
||||
If you call `note_event(...)`, your state object must include `note_seq` or
|
||||
override `next_note_id(...)`.
|
||||
|
||||
Skeleton outline (JSONL CLI):
|
||||
|
||||
```py
|
||||
ENGINE: EngineId = "pi"
|
||||
_RESUME_RE = re.compile(r"(?im)^\s*`?pi\s+--resume\s+(?P<token>[^`\\s]+)`?\\s*$")
|
||||
|
||||
@dataclass
|
||||
class PiRunner(SessionLockMixin, ResumeTokenMixin, Runner):
|
||||
class PiRunner(ResumeTokenMixin, JsonlSubprocessRunner):
|
||||
engine: EngineId = ENGINE
|
||||
resume_re: re.Pattern[str] = _RESUME_RE
|
||||
|
||||
@@ -46,30 +67,55 @@ class PiRunner(SessionLockMixin, ResumeTokenMixin, Runner):
|
||||
model: str | None = None
|
||||
allowed_tools: list[str] | None = None
|
||||
|
||||
def _build_args(self, prompt: str, resume: ResumeToken | None) -> list[str]:
|
||||
args = ["--jsonl"]
|
||||
def command(self) -> str:
|
||||
return self.pi_cmd
|
||||
|
||||
def build_args(
|
||||
self, prompt: str, resume: ResumeToken | None, *, state: Any
|
||||
) -> list[str]:
|
||||
_ = prompt, state
|
||||
args = ["--jsonl", "--verbose"]
|
||||
if resume is not None:
|
||||
args.extend(["--resume", resume.value])
|
||||
if self.model is not None:
|
||||
args.extend(["--model", self.model])
|
||||
if self.allowed_tools:
|
||||
args.extend(["--allowed-tools", ",".join(self.allowed_tools)])
|
||||
args.append("--")
|
||||
args.append(prompt)
|
||||
return args
|
||||
|
||||
async def run(
|
||||
self, prompt: str, resume: ResumeToken | None
|
||||
) -> AsyncIterator[TakopiEvent]:
|
||||
async for evt in self._run_with_resume_lock(prompt, resume, self._run):
|
||||
yield evt
|
||||
def stdin_payload(
|
||||
self, prompt: str, resume: ResumeToken | None, *, state: Any
|
||||
) -> bytes | None:
|
||||
_ = resume, state
|
||||
return prompt.encode()
|
||||
|
||||
def translate(
|
||||
self,
|
||||
data: dict[str, Any],
|
||||
*,
|
||||
state: Any,
|
||||
resume: ResumeToken | None,
|
||||
found_session: ResumeToken | None,
|
||||
) -> list[TakopiEvent]:
|
||||
_ = state, resume, found_session
|
||||
...
|
||||
```
|
||||
|
||||
Key implementation notes:
|
||||
|
||||
- Use `SessionLockMixin` to enforce per-session serialization.
|
||||
- Use `ResumeTokenMixin` for `format_resume` / `extract_resume` / `is_resume_line`.
|
||||
- Use `iter_jsonl(...)` + `drain_stderr(...)` from `takopi.utils.streams`.
|
||||
- Use `BaseRunner` (or `JsonlSubprocessRunner`) for per-session serialization.
|
||||
- Mix in `ResumeTokenMixin` (with a `resume_re`) or override
|
||||
`format_resume` / `extract_resume` / `is_resume_line` so the runner owns
|
||||
resume encoding/decoding.
|
||||
- For JSONL CLIs, prefer `JsonlSubprocessRunner` and implement `command`,
|
||||
`build_args`, and `translate` (override `stdin_payload` if the prompt should
|
||||
be passed via argv instead of stdin).
|
||||
- If you don’t use `JsonlSubprocessRunner`, use `iter_jsonl(...)` +
|
||||
`drain_stderr(...)` from `takopi.utils.streams`.
|
||||
- **Minimal mode is supported:** start with exactly one `StartedEvent` and one
|
||||
`CompletedEvent`. `ActionEvent`s are optional and can be added later. If you
|
||||
do emit actions, you can emit only `phase="completed"` notes without tracking
|
||||
pending state.
|
||||
- **Do not truncate** tool outputs in the runner; pass full strings into events.
|
||||
Truncation belongs in renderers.
|
||||
|
||||
@@ -94,13 +140,28 @@ Mapping guidance:
|
||||
If Pi emits warnings/errors before the final event, surface them as completed
|
||||
`ActionEvent`s (e.g., `kind="warning"`).
|
||||
|
||||
### 4) Register engine in `src/takopi/engines.py`
|
||||
### 4) Expose the backend (auto-discovered)
|
||||
|
||||
Add:
|
||||
Takopi discovers runners by importing modules in `takopi.runners` and looking
|
||||
for a module-level `BACKEND: EngineBackend` (from `takopi.backends`).
|
||||
|
||||
- `_pi_check_setup()` that verifies `pi` exists on PATH
|
||||
- `_pi_build_runner()` that reads `[pi]` config and returns `PiRunner`
|
||||
- A new `EngineBackend(id="pi", display_name="Pi", ...)` entry
|
||||
At the bottom of `src/takopi/runners/pi.py`, define:
|
||||
|
||||
```py
|
||||
BACKEND = EngineBackend(
|
||||
id="pi",
|
||||
build_runner=build_runner,
|
||||
install_cmd="npm install -g @acme/pi-cli",
|
||||
)
|
||||
```
|
||||
|
||||
No changes to `engines.py` or `cli.py` are required.
|
||||
|
||||
Only modules that define `BACKEND` are treated as engines. Internal/testing
|
||||
modules (like `mock.py`) should omit it.
|
||||
|
||||
If the CLI binary name differs from the engine id, set `cli_cmd="pi-cli"` on
|
||||
the backend.
|
||||
|
||||
Example config (minimal):
|
||||
|
||||
@@ -110,12 +171,7 @@ model = "pi-large"
|
||||
allowed_tools = ["Bash", "Read"]
|
||||
```
|
||||
|
||||
### 5) Add CLI subcommand
|
||||
|
||||
Expose `takopi pi` alongside `takopi codex` / `takopi claude` by adding a new
|
||||
`@app.command()` in `src/takopi/cli.py`.
|
||||
|
||||
### 6) Tests + fixtures
|
||||
### 5) Tests + fixtures
|
||||
|
||||
- Add `tests/test_pi_runner.py` for translation behavior.
|
||||
- Reuse `tests/test_runner_contract.py` to ensure lock/resume invariants.
|
||||
|
||||
Reference in New Issue
Block a user