docs: restructure docs into diataxis (#121)
This commit is contained in:
banteg
@@ -0,0 +1,7 @@
|
||||
# For agents
|
||||
|
||||
These pages are **high-signal reference** for LLM agents (and humans acting like one).
|
||||
|
||||
- [Repo map](repo-map.md)
|
||||
- [Invariants](invariants.md)
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
# Invariants
|
||||
|
||||
These are the “don’t break this” rules that keep Takopi reliable.
|
||||
|
||||
## Runner contract
|
||||
|
||||
The runner contract is enforced by `tests/test_runner_contract.py`:
|
||||
|
||||
- Exactly one `StartedEvent`
|
||||
- Exactly one `CompletedEvent`
|
||||
- `CompletedEvent` is last
|
||||
- `CompletedEvent.resume == StartedEvent.resume`
|
||||
|
||||
See also the [Plugin API](../plugin-api.md) runner contract section.
|
||||
|
||||
## Per-thread serialization
|
||||
|
||||
At most one active run may operate on the same thread/session at a time.
|
||||
This is enforced both by scheduling and by per-resume-token runner locks.
|
||||
|
||||
Normative details live in the [Specification](../specification.md) (§5.2).
|
||||
|
||||
## Resume lines
|
||||
|
||||
Resume lines embedded in chat are the engine’s canonical resume command (e.g. `claude --resume <id>`).
|
||||
|
||||
- The runner is authoritative for formatting and extraction.
|
||||
- Transports/rendering must preserve the resume line reliably (even when trimming/splitting).
|
||||
|
||||
Normative details live in the [Specification](../specification.md) (§3).
|
||||
|
||||
## Local contribution hygiene
|
||||
|
||||
- Run `just check` before code commits.
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
# Repo map
|
||||
|
||||
Quick pointers for navigating the Takopi codebase.
|
||||
|
||||
## Where things start
|
||||
|
||||
- CLI entry point: `src/takopi/cli.py`
|
||||
- Telegram backend entry point: `src/takopi/telegram/backend.py`
|
||||
- Telegram bridge loop: `src/takopi/telegram/bridge.py`
|
||||
- Transport-agnostic handler: `src/takopi/runner_bridge.py`
|
||||
|
||||
## Core concepts
|
||||
|
||||
- Domain types (resume tokens, events, actions): `src/takopi/model.py`
|
||||
- Runner protocol: `src/takopi/runner.py`
|
||||
- Router selection and resume polling: `src/takopi/router.py`
|
||||
- Per-thread scheduling: `src/takopi/scheduler.py`
|
||||
- Progress reduction and rendering: `src/takopi/progress.py`, `src/takopi/markdown.py`
|
||||
|
||||
## Engines and streaming
|
||||
|
||||
- Runner implementations: `src/takopi/runners/*`
|
||||
- JSONL decoding schemas: `src/takopi/schemas/*`
|
||||
|
||||
## Plugins
|
||||
|
||||
- Public API boundary (`takopi.api`): `src/takopi/api.py`
|
||||
- Entrypoint discovery + lazy loading: `src/takopi/plugins.py`
|
||||
- Engine/transport/command backend loading: `src/takopi/engines.py`, `src/takopi/transports.py`, `src/takopi/commands.py`
|
||||
|
||||
## Configuration
|
||||
|
||||
- Settings model + TOML/env loading: `src/takopi/settings.py`
|
||||
- Config migrations: `src/takopi/config_migrations.py`
|
||||
|
||||
## Docs and contracts
|
||||
|
||||
- Normative behavior: [Specification](../specification.md)
|
||||
- Runner invariants: `tests/test_runner_contract.py`
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
# Commands & directives
|
||||
|
||||
This page documents Takopi’s user-visible command surface: message directives, in-chat commands, and the CLI.
|
||||
|
||||
## Message directives
|
||||
|
||||
Takopi parses the first non-empty line of a message for a directive prefix.
|
||||
|
||||
| Directive | Example | Effect |
|
||||
|----------|---------|--------|
|
||||
| `/engine` | `/codex fix flaky test` | Select an engine for this message. |
|
||||
| `/project` | `/happy-gadgets add escape-pod` | Select a project alias. |
|
||||
| `@branch` | `@feat/happy-camera rewind to checkpoint` | Run in a worktree for the branch. |
|
||||
| Combined | `/happy-gadgets @feat/flower-pin observe unseen` | Project + branch. |
|
||||
|
||||
Notes:
|
||||
|
||||
- Directives are only parsed at the start of the first non-empty line.
|
||||
- Parsing stops at the first non-directive token.
|
||||
- If a reply contains a `ctx:` line, Takopi ignores new directives and uses the reply context.
|
||||
|
||||
See [Context resolution](context-resolution.md) for the full rules.
|
||||
|
||||
## Context footer (`ctx:`)
|
||||
|
||||
When a run has project context, Takopi appends a footer line rendered as inline code:
|
||||
|
||||
- With branch: `` `ctx: <project> @<branch>` ``
|
||||
- Without branch: `` `ctx: <project>` ``
|
||||
|
||||
This line is parsed from replies and takes precedence over new directives.
|
||||
|
||||
## Telegram in-chat commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `/cancel` | Reply to the progress message to stop the current run. |
|
||||
| `/agent` | Show/set the default agent for the current scope. |
|
||||
| `/file put <path>` | Upload a document into the repo/worktree (requires file transfer enabled). |
|
||||
| `/file get <path>` | Fetch a file or directory back into Telegram. |
|
||||
| `/topic <project> @branch` | Create/bind a topic (topics enabled). |
|
||||
| `/ctx` | Show topic context binding (topics enabled). |
|
||||
| `/ctx set <project> @branch` | Update topic context binding. |
|
||||
| `/ctx clear` | Remove topic context binding. |
|
||||
| `/new` | Clear stored sessions for the current scope (topic/chat). |
|
||||
|
||||
## CLI
|
||||
|
||||
Takopi’s CLI is an auto-router by default; engine subcommands override the default engine.
|
||||
|
||||
### Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `takopi` | Start Takopi (runs onboarding if setup/config is missing and you’re in a TTY). |
|
||||
| `takopi <engine>` | Run with a specific engine (e.g. `takopi codex`). |
|
||||
| `takopi init <alias>` | Register the current repo as a project. |
|
||||
| `takopi chat-id` | Capture the current chat id. |
|
||||
| `takopi chat-id --project <alias>` | Save the captured chat id to a project. |
|
||||
| `takopi plugins` | List discovered plugins without loading them. |
|
||||
| `takopi plugins --load` | Load each plugin to validate types and surface import errors. |
|
||||
|
||||
### Common flags
|
||||
|
||||
| Flag | Description |
|
||||
|------|-------------|
|
||||
| `--onboard` | Force the interactive setup wizard before starting. |
|
||||
| `--transport <id>` | Override the configured transport backend id. |
|
||||
| `--debug` | Write debug logs to `debug.log`. |
|
||||
| `--final-notify/--no-final-notify` | Send the final response as a new message vs an edit. |
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
# Configuration
|
||||
|
||||
Takopi reads configuration from `~/.takopi/takopi.toml`.
|
||||
|
||||
If you expect to edit config while Takopi is running, set:
|
||||
|
||||
```toml
|
||||
watch_config = true
|
||||
```
|
||||
|
||||
## Top-level keys
|
||||
|
||||
| Key | Type | Default | Notes |
|
||||
|-----|------|---------|-------|
|
||||
| `watch_config` | bool | `false` | Hot-reload config changes (transport excluded). |
|
||||
| `default_engine` | string | `"codex"` | Default engine id for new threads. |
|
||||
| `default_project` | string\|null | `null` | Default project alias. |
|
||||
| `transport` | string | `"telegram"` | Transport backend id. |
|
||||
|
||||
## `transports.telegram`
|
||||
|
||||
```toml
|
||||
[transports.telegram]
|
||||
bot_token = "..."
|
||||
chat_id = 123
|
||||
```
|
||||
|
||||
| Key | Type | Default | Notes |
|
||||
|-----|------|---------|-------|
|
||||
| `bot_token` | string | (required) | Telegram bot token from @BotFather. |
|
||||
| `chat_id` | int | (required) | Default chat id. |
|
||||
| `message_overflow` | `"trim"`\|`"split"` | `"trim"` | How to handle long final responses. |
|
||||
| `voice_transcription` | bool | `false` | Enable voice note transcription. |
|
||||
| `voice_max_bytes` | int | `10485760` | Max voice note size (bytes). |
|
||||
| `voice_transcription_model` | string | `"gpt-4o-mini-transcribe"` | OpenAI transcription model name. |
|
||||
| `session_mode` | `"stateless"`\|`"chat"` | `"stateless"` | Auto-resume mode. |
|
||||
| `show_resume_line` | bool | `true` | Show resume line in message footer. |
|
||||
|
||||
### `transports.telegram.topics`
|
||||
|
||||
| Key | Type | Default | Notes |
|
||||
|-----|------|---------|-------|
|
||||
| `enabled` | bool | `false` | Enable forum-topic features. |
|
||||
| `scope` | `"auto"`\|`"main"`\|`"projects"`\|`"all"` | `"auto"` | Where topics are managed. |
|
||||
|
||||
### `transports.telegram.files`
|
||||
|
||||
| Key | Type | Default | Notes |
|
||||
|-----|------|---------|-------|
|
||||
| `enabled` | bool | `false` | Enable `/file put` and `/file get`. |
|
||||
| `auto_put` | bool | `true` | Auto-save uploads. |
|
||||
| `auto_put_mode` | `"upload"`\|`"prompt"` | `"upload"` | Whether uploads also start a run. |
|
||||
| `uploads_dir` | string | `"incoming"` | Relative path inside the repo/worktree. |
|
||||
| `allowed_user_ids` | int[] | `[]` | Allowed senders; empty allows private chats (group usage requires admin). |
|
||||
| `deny_globs` | string[] | (defaults) | Glob denylist (e.g. `.git/**`, `**/*.pem`). |
|
||||
|
||||
File size limits (not configurable):
|
||||
|
||||
- uploads: 20 MiB
|
||||
- downloads: 50 MiB
|
||||
|
||||
## `projects.<alias>`
|
||||
|
||||
```toml
|
||||
[projects.happy-gadgets]
|
||||
path = "~/dev/happy-gadgets"
|
||||
worktrees_dir = ".worktrees"
|
||||
default_engine = "claude"
|
||||
worktree_base = "master"
|
||||
chat_id = -1001234567890
|
||||
```
|
||||
|
||||
| Key | Type | Default | Notes |
|
||||
|-----|------|---------|-------|
|
||||
| `path` | string | (required) | Repo root (expands `~`). Relative paths are resolved against the config directory. |
|
||||
| `worktrees_dir` | string | `".worktrees"` | Worktree root (relative to `path` unless absolute). |
|
||||
| `default_engine` | string\|null | `null` | Per-project default engine. |
|
||||
| `worktree_base` | string\|null | `null` | Base branch for new worktrees. |
|
||||
| `chat_id` | int\|null | `null` | Bind a Telegram chat to this project. |
|
||||
|
||||
Legacy config note: top-level `bot_token` / `chat_id` are auto-migrated into `[transports.telegram]` on startup.
|
||||
|
||||
## Plugins
|
||||
|
||||
### `plugins.enabled`
|
||||
|
||||
```toml
|
||||
[plugins]
|
||||
enabled = ["takopi-transport-slack", "takopi-engine-acme"]
|
||||
```
|
||||
|
||||
- `enabled = []` (default) means “load all installed plugins”.
|
||||
- If non-empty, only distributions with matching names are visible (case-insensitive).
|
||||
|
||||
### `plugins.<id>`
|
||||
|
||||
Plugin-specific configuration lives under `[plugins.<id>]` and is passed to command plugins as `ctx.plugin_config`.
|
||||
|
||||
## Engine-specific config tables
|
||||
|
||||
Engines can have top-level config tables keyed by engine id, for example:
|
||||
|
||||
```toml
|
||||
[codex]
|
||||
model = "..."
|
||||
```
|
||||
|
||||
The shape is engine-defined.
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
# Context resolution
|
||||
|
||||
This page documents how Takopi resolves **run context** (project, worktree/branch, engine) from messages.
|
||||
For step-by-step usage, see [Projects](../how-to/projects.md) and [Worktrees](../how-to/worktrees.md).
|
||||
|
||||
## Overview
|
||||
|
||||
Projects let you give a repo an alias (used as `/alias` in messages) and opt into
|
||||
worktree-based runs via `@branch`.
|
||||
|
||||
- If no projects are configured, Takopi runs in the startup working directory.
|
||||
- If a project is configured, `@branch` resolves/creates a git worktree and runs
|
||||
the task in that worktree.
|
||||
- Progress/final messages include a `ctx:` footer when project context is active.
|
||||
|
||||
## Config schema (relevant subset)
|
||||
|
||||
All config lives in `~/.takopi/takopi.toml`.
|
||||
See [Config](config.md) for the full reference.
|
||||
|
||||
```toml
|
||||
default_engine = "codex" # optional
|
||||
default_project = "z80" # optional
|
||||
transport = "telegram" # optional, defaults to "telegram"
|
||||
|
||||
[transports.telegram]
|
||||
bot_token = "..." # required
|
||||
chat_id = 123 # required
|
||||
|
||||
[projects.z80]
|
||||
path = "~/dev/z80" # required (repo root)
|
||||
worktrees_dir = ".worktrees" # optional, default ".worktrees"
|
||||
default_engine = "codex" # optional, per-project override
|
||||
worktree_base = "master" # optional, base for new branches
|
||||
chat_id = -123 # optional, project chat id
|
||||
```
|
||||
|
||||
Legacy config note: top-level `bot_token` / `chat_id` are auto-migrated into
|
||||
`[transports.telegram]` on startup.
|
||||
|
||||
Note on `worktrees_dir`:
|
||||
|
||||
- The default `.worktrees` lives inside the repo root. You'll see it as an
|
||||
untracked directory (with nested git worktrees) unless you ignore it.
|
||||
- Options:
|
||||
- add `.worktrees/` to your repo `.gitignore`, or
|
||||
- set `worktrees_dir` to a path outside the repo (e.g. `~/.takopi/worktrees/<alias>`).
|
||||
- add it to `.git/info/exclude` if you prefer a local-only ignore.
|
||||
|
||||
Validation rules:
|
||||
|
||||
- `projects` is optional.
|
||||
- Each project entry must include `path` (string, non-empty).
|
||||
- `default_project` must match a configured project alias.
|
||||
- Project aliases cannot collide with engine ids or reserved commands (`/cancel`).
|
||||
- `default_engine` and per-project `default_engine` must be valid engine ids.
|
||||
- `projects.<alias>.chat_id` must be unique and must not match `transports.telegram.chat_id`.
|
||||
- `transport` defaults to `"telegram"` when omitted; override per-run with `--transport`.
|
||||
|
||||
## `takopi init`
|
||||
|
||||
`takopi init <alias>` registers the current repo as a project alias.
|
||||
|
||||
Important behavior:
|
||||
|
||||
- The stored `path` is the **main checkout** of the repo, even if you run
|
||||
`takopi init` inside a worktree. Takopi resolves the repo root via the git
|
||||
common dir and writes that path to `[projects.<alias>].path`.
|
||||
- `worktree_base` is set from the current repo using this resolution order:
|
||||
`origin/HEAD` → current branch → `master` → `main`.
|
||||
|
||||
## Directives and context resolution
|
||||
|
||||
Takopi parses the first non-empty line of a message for a directive prefix.
|
||||
|
||||
Supported directives:
|
||||
|
||||
- `/engine` or `/engine@bot`: chooses the engine
|
||||
- `/project`: chooses a project alias
|
||||
- `@branch`: chooses a git branch/worktree
|
||||
|
||||
Rules:
|
||||
|
||||
- Directives must be a contiguous prefix of the line; parsing stops at the first
|
||||
non-directive token.
|
||||
- At most one engine directive, one project directive, and one `@branch` are
|
||||
allowed (duplicates are errors).
|
||||
- If a reply contains a `ctx:` line, Takopi **ignores new directives** and uses
|
||||
the reply context.
|
||||
|
||||
## Context footer (`ctx:`)
|
||||
|
||||
When a run has project context, Takopi appends a footer line rendered as inline
|
||||
code (backticked):
|
||||
|
||||
- With branch: `` `ctx: <project> @<branch>` ``
|
||||
- Without branch: `` `ctx: <project>` ``
|
||||
|
||||
The `ctx:` line is parsed from replies and takes precedence over new directives.
|
||||
|
||||
When a message arrives in a chat whose `chat_id` matches `projects.<alias>.chat_id`,
|
||||
Takopi defaults the project context to that alias unless a reply `ctx:` or explicit
|
||||
`/project` directive is present.
|
||||
|
||||
## Worktree resolution
|
||||
|
||||
When `@branch` is present:
|
||||
|
||||
```
|
||||
worktrees_root = <project.path> / <worktrees_dir>
|
||||
worktree_path = worktrees_root / <branch>
|
||||
```
|
||||
|
||||
Branch validation:
|
||||
|
||||
- Must be non-empty
|
||||
- Must not start with `/`
|
||||
- Must not contain `..` path segments
|
||||
- May include `/` (nested directories)
|
||||
- The resolved worktree path must stay within `worktrees_root`
|
||||
|
||||
Worktree creation rules:
|
||||
|
||||
1) If `worktree_path` exists:
|
||||
- It must be a git worktree or Takopi errors.
|
||||
2) If it does not exist:
|
||||
- If local branch exists: `git worktree add <path> <branch>`
|
||||
- Else if remote `origin/<branch>` exists:
|
||||
`git worktree add -b <branch> <path> origin/<branch>`
|
||||
- Else:
|
||||
`git worktree add -b <branch> <path> <base>`
|
||||
|
||||
Base branch selection:
|
||||
|
||||
1) `projects.<alias>.worktree_base` (if set)
|
||||
2) `origin/HEAD` (if present)
|
||||
3) current checked out branch
|
||||
4) `master` if it exists
|
||||
5) `main` if it exists
|
||||
6) otherwise error
|
||||
|
||||
When `@branch` is omitted:
|
||||
|
||||
- Takopi runs in `<project.path>` (the main checkout).
|
||||
|
||||
## Examples
|
||||
|
||||
Start a new thread in a worktree:
|
||||
|
||||
```
|
||||
/z80 @feat/streaming fix flaky test
|
||||
```
|
||||
|
||||
Reply to a progress message to continue in the same context:
|
||||
|
||||
```
|
||||
ctx: z80 @feat/streaming
|
||||
```
|
||||
@@ -0,0 +1,26 @@
|
||||
# Environment variables
|
||||
|
||||
Takopi supports a small set of environment variables for logging and runtime behavior.
|
||||
|
||||
## Logging
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `TAKOPI_LOG_LEVEL` | Minimum log level (default `info`; `--debug` forces `debug`). |
|
||||
| `TAKOPI_LOG_FORMAT` | `console` (default) or `json`. |
|
||||
| `TAKOPI_LOG_COLOR` | Force color on/off (`1/true/yes/on` or `0/false/no/off`). |
|
||||
| `TAKOPI_LOG_FILE` | Append JSON lines to a file. `--debug` defaults this to `debug.log`. |
|
||||
| `TAKOPI_TRACE_PIPELINE` | Log pipeline events at `info` instead of `debug`. |
|
||||
|
||||
## CLI behavior
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `TAKOPI_NO_INTERACTIVE` | Disable interactive prompts (useful for CI / non-TTY). |
|
||||
|
||||
## Engine-specific
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `PI_CODING_AGENT_DIR` | Override Pi agent session directory base path. |
|
||||
|
||||
@@ -0,0 +1,65 @@
|
||||
# Reference
|
||||
|
||||
Reference docs are **authoritative and exact**. Use these when you need stable facts, schemas, and contracts.
|
||||
|
||||
If you’re trying to achieve a goal (“enable topics”, “fetch a file”), use **[How-to](../how-to/index.md)**.
|
||||
If you’re trying to understand the *why*, use **[Explanation](../explanation/index.md)**.
|
||||
|
||||
## Most-used reference pages
|
||||
|
||||
- [Commands & directives](commands-and-directives.md)
|
||||
- Message prefixes like `/engine`, `/project`, and `@branch`
|
||||
- In-chat commands like `/cancel`, `/new`, `/ctx`, `/file …`, `/topic …`
|
||||
- [Configuration](config.md)
|
||||
- `takopi.toml` options and defaults
|
||||
- Telegram transport options (sessions, topics, files, voice transcription)
|
||||
|
||||
## Normative behavior
|
||||
|
||||
- [Specification](specification.md)
|
||||
The normative (“MUST/SHOULD/MAY”) contract for:
|
||||
- resume tokens + resume lines
|
||||
- event model
|
||||
- progress/final message semantics
|
||||
- per-thread serialization rules
|
||||
|
||||
## Plugins and extension contracts
|
||||
|
||||
- [Plugin API](plugin-api.md)
|
||||
The **only** supported import surface for plugins: `takopi.api`
|
||||
- [Context resolution](context-resolution.md)
|
||||
How Takopi resolves project + worktree context from directives, replies, and chat ids.
|
||||
|
||||
## Transport reference
|
||||
|
||||
- [Telegram transport](transports/telegram.md)
|
||||
Rate limits, outbox behavior, retries, message editing rules.
|
||||
|
||||
## Runner reference
|
||||
|
||||
These are “engine adapter” implementation details: JSONL formats, mapping rules, and emitted events.
|
||||
|
||||
- [Runners overview](runners/index.md)
|
||||
- Claude:
|
||||
- [runner.md](runners/claude/runner.md)
|
||||
- [stream-json-cheatsheet.md](runners/claude/stream-json-cheatsheet.md)
|
||||
- [takopi-events.md](runners/claude/takopi-events.md)
|
||||
- Codex:
|
||||
- [exec-json-cheatsheet.md](runners/codex/exec-json-cheatsheet.md)
|
||||
- [takopi-events.md](runners/codex/takopi-events.md)
|
||||
- OpenCode:
|
||||
- [runner.md](runners/opencode/runner.md)
|
||||
- [stream-json-cheatsheet.md](runners/opencode/stream-json-cheatsheet.md)
|
||||
- [takopi-events.md](runners/opencode/takopi-events.md)
|
||||
- Pi:
|
||||
- [runner.md](runners/pi/runner.md)
|
||||
- [stream-json-cheatsheet.md](runners/pi/stream-json-cheatsheet.md)
|
||||
- [takopi-events.md](runners/pi/takopi-events.md)
|
||||
|
||||
## For LLM agents
|
||||
|
||||
If you’re an LLM agent contributing to Takopi, start here:
|
||||
|
||||
- [Agent entrypoint](agents/index.md)
|
||||
- [Repo map](agents/repo-map.md)
|
||||
- [Invariants](agents/invariants.md) (runner contract, resume handling, “don’t break this” rules)
|
||||
@@ -0,0 +1,262 @@
|
||||
# Plugin API
|
||||
|
||||
Takopi’s **public plugin API** is exported from:
|
||||
|
||||
```
|
||||
takopi.api
|
||||
```
|
||||
|
||||
Anything not imported from `takopi.api` should be considered **internal** and
|
||||
subject to change. The API version is tracked by `TAKOPI_PLUGIN_API_VERSION`.
|
||||
|
||||
---
|
||||
|
||||
## Versioning
|
||||
|
||||
- Current API version: `TAKOPI_PLUGIN_API_VERSION = 1`
|
||||
- Plugins should pin to a compatible Takopi range, e.g.:
|
||||
|
||||
```toml
|
||||
dependencies = ["takopi>=0.14,<0.15"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Exported symbols
|
||||
|
||||
### Engine backends and runners
|
||||
|
||||
| Symbol | Purpose |
|
||||
|--------|---------|
|
||||
| `EngineBackend` | Declares an engine backend (id + runner builder) |
|
||||
| `EngineConfig` | Dict-based engine config table |
|
||||
| `Runner` | Runner protocol |
|
||||
| `BaseRunner` | Helper base class with resume locking |
|
||||
| `JsonlSubprocessRunner` | Helper for JSONL-streaming CLIs |
|
||||
| `EventFactory` | Helper for building takopi events |
|
||||
|
||||
### Transport backends
|
||||
|
||||
| Symbol | Purpose |
|
||||
|--------|---------|
|
||||
| `TransportBackend` | Transport backend protocol |
|
||||
| `SetupIssue` | Setup issue for onboarding / validation |
|
||||
| `SetupResult` | Setup issues + config path |
|
||||
| `Transport` | Transport protocol (send/edit/delete) |
|
||||
| `Presenter` | Renders progress to `RenderedMessage` |
|
||||
| `RenderedMessage` | Rendered text + transport metadata |
|
||||
| `SendOptions` | Reply/notify/replace flags |
|
||||
| `MessageRef` | Transport-specific message reference |
|
||||
| `TransportRuntime` | Transport runtime facade (routers/projects hidden) |
|
||||
| `ResolvedMessage` | Parsed prompt + resume/context resolution |
|
||||
| `ResolvedRunner` | Runner selection result |
|
||||
|
||||
### Command backends
|
||||
|
||||
| Symbol | Purpose |
|
||||
|--------|---------|
|
||||
| `CommandBackend` | Slash command plugin protocol |
|
||||
| `CommandContext` | Context passed to a command handler |
|
||||
| `CommandExecutor` | Helper to send messages or run engines |
|
||||
| `CommandResult` | Simple response payload for a command |
|
||||
| `RunRequest` | Engine run request used by commands |
|
||||
| `RunResult` | Engine run result (captured output) |
|
||||
| `RunMode` | `"emit"` (send) or `"capture"` (collect) |
|
||||
|
||||
### Core types and helpers
|
||||
|
||||
| Symbol | Purpose |
|
||||
|--------|---------|
|
||||
| `EngineId` | Engine id type alias |
|
||||
| `ResumeToken` | Resume token (engine + value) |
|
||||
| `StartedEvent` / `ActionEvent` / `CompletedEvent` | Core event types |
|
||||
| `Action` | Action metadata for `ActionEvent` |
|
||||
| `RunContext` | Project/branch context |
|
||||
| `ConfigError` | Configuration error type |
|
||||
| `DirectiveError` | Error raised when parsing directives |
|
||||
| `RunnerUnavailableError` | Router error when a runner is unavailable |
|
||||
|
||||
### Bridge helpers (for transport plugins)
|
||||
|
||||
| Symbol | Purpose |
|
||||
|--------|---------|
|
||||
| `ExecBridgeConfig` | Transport + presenter config |
|
||||
| `IncomingMessage` | Normalized incoming message |
|
||||
| `RunningTask` / `RunningTasks` | Per-message run coordination |
|
||||
| `handle_message()` | Core message handler used by transports |
|
||||
|
||||
---
|
||||
|
||||
## Runner contract (engine plugins)
|
||||
|
||||
Runners emit events in a strict sequence (see `tests/test_runner_contract.py`):
|
||||
|
||||
- Exactly **one** `StartedEvent`
|
||||
- Exactly **one** `CompletedEvent`
|
||||
- `CompletedEvent` is **last**
|
||||
- `CompletedEvent.resume == StartedEvent.resume`
|
||||
|
||||
Action events are optional. The minimal valid run is:
|
||||
|
||||
```
|
||||
StartedEvent -> CompletedEvent
|
||||
```
|
||||
|
||||
### Resume tokens
|
||||
|
||||
Runners own the resume format:
|
||||
|
||||
- `format_resume(token)` returns a command line users can paste
|
||||
- `extract_resume(text)` parses resume tokens from user text
|
||||
- `is_resume_line(line)` lets Takopi strip resume lines before running
|
||||
|
||||
---
|
||||
|
||||
## EngineBackend
|
||||
|
||||
```py
|
||||
EngineBackend(
|
||||
id: str,
|
||||
build_runner: Callable[[EngineConfig, Path], Runner],
|
||||
cli_cmd: str | None = None,
|
||||
install_cmd: str | None = None,
|
||||
)
|
||||
```
|
||||
|
||||
- `id` must match the entrypoint name and the ID regex.
|
||||
- `build_runner` should raise `ConfigError` for invalid config.
|
||||
- `cli_cmd` is used to check whether the engine CLI is on `PATH`.
|
||||
- `install_cmd` is surfaced in onboarding output.
|
||||
|
||||
---
|
||||
|
||||
## TransportBackend
|
||||
|
||||
```py
|
||||
class TransportBackend(Protocol):
|
||||
id: str
|
||||
description: str
|
||||
|
||||
def check_setup(...) -> SetupResult: ...
|
||||
def interactive_setup(self, *, force: bool) -> bool: ...
|
||||
def lock_token(
|
||||
self, *, transport_config: dict[str, object], config_path: Path
|
||||
) -> str | None: ...
|
||||
def build_and_run(
|
||||
self,
|
||||
*,
|
||||
transport_config: dict[str, object],
|
||||
config_path: Path,
|
||||
runtime: TransportRuntime,
|
||||
final_notify: bool,
|
||||
default_engine_override: str | None,
|
||||
) -> None: ...
|
||||
```
|
||||
|
||||
Transport backends are responsible for:
|
||||
|
||||
- Validating config and onboarding users (`check_setup`, `interactive_setup`)
|
||||
- Providing a lock token so Takopi can prevent parallel runs
|
||||
- Starting the transport loop in `build_and_run`
|
||||
|
||||
---
|
||||
|
||||
## CommandBackend
|
||||
|
||||
```py
|
||||
class CommandBackend(Protocol):
|
||||
id: str
|
||||
description: str
|
||||
|
||||
async def handle(self, ctx: CommandContext) -> CommandResult | None: ...
|
||||
```
|
||||
|
||||
Command handlers receive a `CommandContext` with:
|
||||
|
||||
- the raw command text and parsed args
|
||||
- the original message + reply metadata
|
||||
- `config_path` for the active `takopi.toml` (when known)
|
||||
- `plugin_config` from `[plugins.<id>]` (dict, defaults to `{}`)
|
||||
- `runtime` (engine/project resolution)
|
||||
- `executor` (send messages or run engines)
|
||||
|
||||
Use `ctx.executor.run_one(...)` or `ctx.executor.run_many(...)` to reuse Takopi's
|
||||
engine pipeline. Use `mode="capture"` to collect results and build a custom reply.
|
||||
|
||||
`ctx.message` and `ctx.reply_to` are `MessageRef` objects with:
|
||||
|
||||
- `channel_id` (`int | str`, chat/channel id)
|
||||
- `message_id` (`int | str`, message id)
|
||||
- `thread_id` (`int | str | None`; set when the transport supports threads, like Telegram topics)
|
||||
- `raw` (transport-specific payload, may be `None`)
|
||||
|
||||
Example: key per-thread state by `(ctx.message.channel_id, ctx.message.thread_id)`.
|
||||
|
||||
---
|
||||
|
||||
## TransportRuntime helpers
|
||||
|
||||
`TransportRuntime` keeps transports away from internal router/project types. Key helpers:
|
||||
|
||||
- `resolve_message(text, reply_text)` → `ResolvedMessage` (prompt, resume token, context)
|
||||
- `resolve_engine(engine_override, context)` → `EngineId`
|
||||
- `resolve_runner(resume_token, engine_override)` → `ResolvedRunner` (runner + availability info)
|
||||
- `resolve_run_cwd(context)` → `Path | None` (raises `ConfigError` for project/worktree issues)
|
||||
- `format_context_line(context)` → `str | None`
|
||||
- `available_engine_ids()` / `missing_engine_ids()` / `engine_ids` / `default_engine`
|
||||
- `project_aliases()`
|
||||
- `config_path` (active config path when available)
|
||||
- `plugin_config(plugin_id)` → `dict` from `[plugins.<id>]`
|
||||
|
||||
---
|
||||
|
||||
## Bridge usage (transport plugins)
|
||||
|
||||
Most transports can delegate message handling to `handle_message()`. Use
|
||||
`TransportRuntime` to resolve messages and select a runner:
|
||||
|
||||
```py
|
||||
from takopi.api import (
|
||||
ExecBridgeConfig,
|
||||
IncomingMessage,
|
||||
RunningTask,
|
||||
RunningTasks,
|
||||
TransportRuntime,
|
||||
handle_message,
|
||||
)
|
||||
|
||||
async def on_message(...):
|
||||
resolved = runtime.resolve_message(text=text, reply_text=reply_text)
|
||||
entry = runtime.resolve_runner(
|
||||
resume_token=resolved.resume_token,
|
||||
engine_override=resolved.engine_override,
|
||||
)
|
||||
context_line = runtime.format_context_line(resolved.context)
|
||||
incoming = IncomingMessage(
|
||||
channel_id=...,
|
||||
message_id=...,
|
||||
text=...,
|
||||
reply_to=...,
|
||||
thread_id=...,
|
||||
)
|
||||
await handle_message(
|
||||
exec_cfg,
|
||||
runner=entry.runner,
|
||||
incoming=incoming,
|
||||
resume_token=resolved.resume_token,
|
||||
context=resolved.context,
|
||||
context_line=context_line,
|
||||
strip_resume_line=runtime.is_resume_line,
|
||||
running_tasks=running_tasks,
|
||||
on_thread_known=on_thread_known,
|
||||
)
|
||||
```
|
||||
|
||||
`handle_message()` implements:
|
||||
|
||||
- Progress updates and throttling
|
||||
- Resume handling
|
||||
- Cancellation propagation
|
||||
- Final rendering
|
||||
|
||||
This keeps transport backends thin and consistent with core behavior.
|
||||
@@ -0,0 +1,382 @@
|
||||
Below is a concrete implementation spec for the **Anthropic Claude Code (“claude” CLI / Agent SDK runtime)** runner shipped in Takopi (v0.3.0).
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### Goal
|
||||
|
||||
Provide the **`claude`** engine backend so Takopi can:
|
||||
|
||||
* Run Claude Code non-interactively via the **Agent SDK CLI** (`claude -p`). ([Claude Code][1])
|
||||
* Stream progress in Telegram by parsing **`--output-format stream-json --verbose`** (newline-delimited JSON). Note: `--output-format` only works with `-p/--print`. ([Claude Code][1])
|
||||
* Support resumable sessions via **`--resume <session_id>`** (Takopi emits a canonical resume line the user can reply with). ([Claude Code][1])
|
||||
|
||||
### Non-goals (v1)
|
||||
|
||||
* Interactive Q&A inside a single run (e.g., answering `AskUserQuestion` prompts mid-flight).
|
||||
* Full “slash commands” integration (Claude Code docs note many slash commands are interactive-only). ([Claude Code][1])
|
||||
* MCP prompt-handling for permissions (use allow rules instead).
|
||||
|
||||
---
|
||||
|
||||
## UX and behavior
|
||||
|
||||
### Engine selection
|
||||
|
||||
* Default: `takopi` (auto-router uses `default_engine` from config)
|
||||
* Override: `takopi claude`
|
||||
|
||||
Takopi runs in auto-router mode by default; `takopi claude` or `/claude` selects
|
||||
Claude for new threads.
|
||||
|
||||
### Resume UX (canonical line)
|
||||
|
||||
Takopi appends a **single backticked** resume line at the end of the message, like:
|
||||
|
||||
```text
|
||||
`claude --resume 8b2d2b30-...`
|
||||
```
|
||||
|
||||
Rationale:
|
||||
|
||||
* Claude Code supports resuming a specific conversation by session ID with `--resume`. ([Claude Code][1])
|
||||
* The CLI reference also documents `--resume/-r` as the resume mechanism.
|
||||
|
||||
Takopi should parse either:
|
||||
|
||||
* `claude --resume <id>`
|
||||
* `claude -r <id>` (short form from docs)
|
||||
|
||||
**Note:** Claude session IDs should be treated as **opaque strings**. Do not assume UUID format.
|
||||
|
||||
### Permissions / non-interactive runs
|
||||
|
||||
In `-p` mode, Claude Code can require tool approvals. Takopi cannot click/answer interactive prompts, so **users must preconfigure permissions** (via Claude Code settings or `--allowedTools`). Claude’s settings system supports allow/deny tool rules. ([Claude Code][2])
|
||||
|
||||
**Safety note:** `-p/--print` skips the workspace trust dialog; only use this flag in trusted directories.
|
||||
|
||||
Takopi should document this clearly: if permissions aren’t configured and Claude tries to use a gated tool, the run may block or fail.
|
||||
|
||||
---
|
||||
|
||||
## Config additions
|
||||
|
||||
Takopi config lives at `~/.takopi/takopi.toml`.
|
||||
|
||||
Add a new optional `[claude]` section.
|
||||
|
||||
Recommended v1 schema:
|
||||
|
||||
```toml
|
||||
# ~/.takopi/takopi.toml
|
||||
|
||||
default_engine = "claude"
|
||||
|
||||
[claude]
|
||||
model = "claude-sonnet-4-5-20250929" # optional (Claude Code supports model override in settings too)
|
||||
allowed_tools = ["Bash", "Read", "Edit", "Write"] # optional but strongly recommended for automation
|
||||
dangerously_skip_permissions = false # optional (high risk; prefer sandbox use only)
|
||||
use_api_billing = false # optional (keep ANTHROPIC_API_KEY for API billing)
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
* `--allowedTools` exists specifically to auto-approve tools in programmatic runs. ([Claude Code][1])
|
||||
* Claude Code tools (Bash/Edit/Write/WebSearch/etc.) and whether permission is required are documented. ([Claude Code][2])
|
||||
* If `allowed_tools` is omitted, Takopi defaults to `["Bash", "Read", "Edit", "Write"]`.
|
||||
* Takopi only reads `model`, `allowed_tools`, `dangerously_skip_permissions`, and `use_api_billing` from `[claude]`.
|
||||
* By default Takopi strips `ANTHROPIC_API_KEY` from the subprocess environment so Claude uses subscription billing. Set `use_api_billing = true` to keep the key.
|
||||
|
||||
---
|
||||
|
||||
## Code changes (by file)
|
||||
|
||||
### 1) New file: `src/takopi/runners/claude.py`
|
||||
|
||||
#### Backend export
|
||||
|
||||
Expose a module-level `BACKEND = EngineBackend(...)` (from `takopi.backends`).
|
||||
Takopi auto-discovers runners by importing `takopi.runners.*` and looking for
|
||||
`BACKEND`.
|
||||
|
||||
`BACKEND` should provide:
|
||||
|
||||
* Engine id: `"claude"`
|
||||
* `install_cmd`:
|
||||
* Install command for `claude` (used by onboarding when missing on PATH).
|
||||
* Error message should include official install options and “run `claude` once to authenticate”.
|
||||
|
||||
* Install methods include install scripts, Homebrew, and npm. ([Claude Code][4])
|
||||
* Agent SDK / CLI can use Claude Code authentication from running `claude`, or API key auth. ([Claude][5])
|
||||
|
||||
* `build_runner()` should parse `[claude]` config and instantiate `ClaudeRunner`.
|
||||
|
||||
#### Runner implementation
|
||||
|
||||
Implement a new `Runner`:
|
||||
|
||||
#### Public API
|
||||
|
||||
* `engine: EngineId = "claude"`
|
||||
* `format_resume(token) -> str`: returns `` `claude --resume {token}` ``
|
||||
* `extract_resume(text) -> ResumeToken | None`: parse last match of `--resume/-r`
|
||||
* `is_resume_line(line) -> bool`: matches the above patterns
|
||||
* `run(prompt, resume)` async generator of `TakopiEvent`
|
||||
|
||||
#### Subprocess invocation
|
||||
|
||||
Use Agent SDK CLI non-interactively:
|
||||
|
||||
Core invocation:
|
||||
|
||||
* `claude -p --output-format stream-json --verbose` ([Claude Code][1])
|
||||
* `--verbose` overrides config and is required for full stream-json output.
|
||||
|
||||
Resume:
|
||||
|
||||
* add `--resume <session_id>` if resuming. ([Claude Code][1])
|
||||
|
||||
Model:
|
||||
|
||||
* add `--model <name>` if configured. ([Claude Code][1])
|
||||
|
||||
Permissions:
|
||||
|
||||
* add `--allowedTools "<rules>"` if configured. ([Claude Code][1])
|
||||
* add `--dangerously-skip-permissions` only if explicitly enabled (high risk; document clearly).
|
||||
|
||||
Prompt passing:
|
||||
|
||||
* Pass the prompt as the final positional argument after `--` (CLI expects `prompt` as an argument). This also protects prompts that begin with `-`. ([Claude Code][1])
|
||||
|
||||
Other flags:
|
||||
|
||||
* Claude exposes more CLI flags, but Takopi does not surface them in config.
|
||||
|
||||
#### Stream parsing
|
||||
|
||||
In stream-json mode, Claude emits newline-delimited JSON objects. ([Claude Code][1])
|
||||
|
||||
Per the official Agent SDK TypeScript reference, message types include:
|
||||
|
||||
* `system` with `subtype: 'init'` and fields like `session_id`, `cwd`, `tools`, `model`, `permissionMode`, `output_style`. ([Claude Code][3])
|
||||
* `assistant` / `user` messages with Anthropic SDK message objects. ([Claude Code][3])
|
||||
* final `result` message with:
|
||||
|
||||
* `subtype: 'success'` or error subtype(s),
|
||||
* `is_error`,
|
||||
* `result` (string on success),
|
||||
* `usage`, `total_cost_usd`, `modelUsage`,
|
||||
* `errors` list on failures,
|
||||
* `permission_denials`. ([Claude Code][3])
|
||||
|
||||
Takopi should:
|
||||
|
||||
* Parse each line as JSON; on decode error emit a warning ActionEvent (like CodexRunner does) and continue.
|
||||
* Prefer stdout for JSON; log stderr separately (do not merge).
|
||||
* Treat unknown top-level fields (e.g., `parent_tool_use_id`) as optional metadata and ignore them unless needed.
|
||||
|
||||
#### Mapping to Takopi events
|
||||
|
||||
**StartedEvent**
|
||||
|
||||
* Emit upon first `system/init` message:
|
||||
|
||||
* `resume = ResumeToken(engine="claude", value=session_id)`
|
||||
(treat `session_id` as opaque; do not validate as UUID)
|
||||
* `title = model` (or user-specified config title; default `"claude"`)
|
||||
* `meta` should include `cwd`, `tools`, `permissionMode`, `output_style` for debugging.
|
||||
|
||||
**Action events (progress)**
|
||||
The core useful progress comes from tool usage.
|
||||
|
||||
Claude Code tools list is documented (Bash/Edit/Write/WebSearch/WebFetch/TodoWrite/Task/etc.). ([Claude Code][2])
|
||||
|
||||
Strategy:
|
||||
|
||||
* When you see an **assistant message** with a content block `type: "tool_use"`:
|
||||
|
||||
* Emit `ActionEvent(phase="started")` with:
|
||||
|
||||
* `action.id = tool_use.id`
|
||||
* `action.kind` based on tool name (complete mapping):
|
||||
|
||||
* `Bash` → `command`
|
||||
* `Edit`/`Write`/`NotebookEdit` → `file_change` (best-effort path extraction)
|
||||
* `Read` → `tool`
|
||||
* `Glob`/`Grep` → `tool`
|
||||
* `WebSearch`/`WebFetch` → `web_search`
|
||||
* `TodoWrite`/`TodoRead` → `note`
|
||||
* `AskUserQuestion` → `note`
|
||||
* `Task`/`Agent` → `tool`
|
||||
* `KillShell` → `command`
|
||||
* otherwise → `tool`
|
||||
* `action.title`:
|
||||
|
||||
* Bash: use `input.command` if present
|
||||
* Read/Write/Edit/NotebookEdit: use file path (best-effort; field may be `file_path` or `path`)
|
||||
* Glob/Grep: use pattern
|
||||
* WebSearch: use query
|
||||
* WebFetch: use URL
|
||||
* TodoWrite/TodoRead: short summary (e.g., “update todos”)
|
||||
* AskUserQuestion: short summary (e.g., “ask user”)
|
||||
* otherwise: tool name
|
||||
* `detail` includes a compacted copy of input (or a safe summary).
|
||||
|
||||
* When you see a **user message** with a content block `type: "tool_result"`:
|
||||
|
||||
* Emit `ActionEvent(phase="completed")` for `tool_use_id`
|
||||
* `ok = not is_error`
|
||||
* `content` may be a string or an array of content blocks; normalize to a string for summaries
|
||||
* `detail` includes a small summary (char count / first line / “(truncated)”)
|
||||
|
||||
This mirrors CodexRunner’s “started → completed” item tracking and renders well in existing `TakopiProgressRenderer`.
|
||||
|
||||
**CompletedEvent**
|
||||
|
||||
* Emit on `result` message:
|
||||
|
||||
* `ok = (is_error == false)` (treat `is_error` as authoritative; `subtype` is informational)
|
||||
* `answer = result` on success; on error, a concise message using `errors` and/or denials
|
||||
* `usage` attach:
|
||||
|
||||
* `total_cost_usd`, `usage`, `modelUsage`, `duration_ms`, `duration_api_ms`, `num_turns` ([Claude Code][3])
|
||||
* Always include `resume` (same session_id).
|
||||
* Emit exactly one completed event per run. After emitting it, ignore any
|
||||
trailing JSON lines (do not emit a second completion).
|
||||
* We do not use an idle-timeout completion; completion is driven by Claude’s
|
||||
`result` event or process exit handling.
|
||||
|
||||
**Permission denials**
|
||||
Because result includes `permission_denials`, optionally emit warning ActionEvent(s) *before* CompletedEvent (CompletedEvent must be final):
|
||||
|
||||
* kind: `warning`
|
||||
* title: “permission denied: <tool_name>”
|
||||
This preserves the “warnings before started/completed” ordering principle Takopi already tests for CodexRunner.
|
||||
|
||||
#### Session serialization / locks
|
||||
|
||||
Must match Takopi runner contract:
|
||||
|
||||
* Lock key: `claude:<session_id>` (string) in a `WeakValueDictionary` of `anyio.Lock`.
|
||||
* When resuming:
|
||||
|
||||
* acquire lock before spawning subprocess.
|
||||
* When starting a new session:
|
||||
|
||||
* you don’t know session_id until `system/init`, so:
|
||||
|
||||
* spawn process,
|
||||
* wait until the **first** `system/init`,
|
||||
* acquire lock for that session id **before** yielding StartedEvent,
|
||||
* then continue yielding.
|
||||
|
||||
This mirrors CodexRunner’s correct behavior and ensures “new run + resume run” serialize once the session is known.
|
||||
Assumption: Claude emits a single `system/init` per run. If multiple `init`
|
||||
events arrive, ignore the subsequent ones (do not attempt to re-lock).
|
||||
|
||||
#### Cancellation / termination
|
||||
|
||||
Reuse the existing subprocess lifecycle pattern (like `CodexRunner.manage_subprocess`):
|
||||
|
||||
* Kill the process group on cancellation
|
||||
* Drain stderr concurrently (log-only)
|
||||
* Ensure locks release in `finally`
|
||||
|
||||
## Documentation updates
|
||||
|
||||
### README
|
||||
|
||||
Add a “Claude Code engine” section that covers:
|
||||
|
||||
* Installation (install script / brew / npm). ([Claude Code][4])
|
||||
* Authentication:
|
||||
|
||||
* run `claude` once and follow prompts, or use API key auth (Agent SDK docs mention `ANTHROPIC_API_KEY`). ([Claude][5])
|
||||
* Non-interactive permission caveat + how to configure:
|
||||
|
||||
* settings allow/deny rules,
|
||||
* or `--allowedTools` / `[claude].allowed_tools`. ([Claude Code][2])
|
||||
* Resume format: `` `claude --resume <id>` ``.
|
||||
|
||||
### `docs/developing.md`
|
||||
|
||||
Extend “Adding a Runner” with:
|
||||
|
||||
* “ClaudeRunner parses Agent SDK stream-json output”
|
||||
* Mention key message types and the init/result messages.
|
||||
|
||||
---
|
||||
|
||||
## Test plan
|
||||
|
||||
Mirror the existing `CodexRunner` tests patterns.
|
||||
|
||||
### New tests: `tests/test_claude_runner.py`
|
||||
|
||||
1. **Contract & locking**
|
||||
|
||||
* `test_run_serializes_same_session` (stub `run_impl` like Codex tests)
|
||||
* `test_run_allows_parallel_new_sessions`
|
||||
* `test_run_serializes_new_session_after_session_is_known`:
|
||||
|
||||
* Provide a fake `claude` executable in tmp_path that:
|
||||
|
||||
* prints system/init with session_id,
|
||||
* then waits on a file gate,
|
||||
* a second invocation with `--resume` writes a marker file and exits,
|
||||
* assert the resume invocation doesn’t run until gate opens.
|
||||
|
||||
2. **Resume parsing**
|
||||
|
||||
* `format_resume` returns `claude --resume <id>`
|
||||
* `extract_resume` handles both `--resume` and `-r`
|
||||
|
||||
3. **Translation / event ordering**
|
||||
|
||||
* Fake `claude` outputs:
|
||||
|
||||
* system/init
|
||||
* assistant tool_use (Bash)
|
||||
* user tool_result
|
||||
* result success with `result: "ok"`
|
||||
* Assert Takopi yields:
|
||||
|
||||
* StartedEvent
|
||||
* ActionEvent started
|
||||
* ActionEvent completed
|
||||
* CompletedEvent(ok=True, answer="ok")
|
||||
|
||||
4. **Failure modes**
|
||||
|
||||
* `result` subtype error with `errors: [...]`:
|
||||
|
||||
* CompletedEvent(ok=False)
|
||||
* permission_denials exist:
|
||||
|
||||
* warning ActionEvent(s) emitted before CompletedEvent
|
||||
|
||||
5. **Cancellation**
|
||||
|
||||
* Stub `claude` that sleeps; ensure cancellation kills it (pattern already used for codex subprocess cancellation tests).
|
||||
|
||||
---
|
||||
|
||||
## Implementation checklist (v0.3.0)
|
||||
|
||||
* [x] Export `BACKEND = EngineBackend(...)` from `src/takopi/runners/claude.py`.
|
||||
* [x] Add `src/takopi/runners/claude.py` implementing the `Runner` protocol.
|
||||
* [x] Add tests + stub executable fixtures.
|
||||
* [x] Update README and developing docs.
|
||||
* [ ] Run full test suite before release.
|
||||
|
||||
---
|
||||
|
||||
If you want, I can also propose the exact **event-to-action mapping table** (tool → kind/title/detail rules) you should start with, based on Claude Code’s documented tool list (Bash/Edit/Write/WebSearch/etc.). ([Claude Code][2])
|
||||
|
||||
[1]: https://code.claude.com/docs/en/headless "Run Claude Code programmatically - Claude Code Docs"
|
||||
[2]: https://code.claude.com/docs/en/settings "Claude Code settings - Claude Code Docs"
|
||||
[3]: https://code.claude.com/docs/en/sdk/sdk-typescript "Agent SDK reference - TypeScript - Claude Docs"
|
||||
[4]: https://code.claude.com/docs/en/quickstart "Quickstart - Claude Code Docs"
|
||||
[5]: https://platform.claude.com/docs/en/agent-sdk/quickstart "Quickstart - Claude Docs"
|
||||
@@ -0,0 +1,108 @@
|
||||
# Claude `stream-json` event cheatsheet
|
||||
|
||||
`claude -p --output-format stream-json --verbose` writes **one JSON object per line**
|
||||
(JSONL) with a required `type` field. (`--output-format` only works with `-p`.)
|
||||
|
||||
This cheatsheet is derived from `humanlayer/claudecode-go/types.go` and
|
||||
`client_test.go`.
|
||||
|
||||
## Top-level event lines
|
||||
|
||||
### `system` (init)
|
||||
|
||||
Fields:
|
||||
- `type`: `"system"`
|
||||
- `subtype`: `"init"`
|
||||
- `session_id`
|
||||
- `tools`: array of tool names
|
||||
- `mcp_servers`: array of `{name, status}`
|
||||
- `cwd`, `model`, `permissionMode`, `apiKeySource` (optional)
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"system","subtype":"init","session_id":"session_01","cwd":"/repo","model":"sonnet","permissionMode":"auto","apiKeySource":"env","tools":["Bash","Read","Write","WebSearch"],"mcp_servers":[{"name":"approvals","status":"connected"}]}
|
||||
```
|
||||
|
||||
### `assistant` / `user`
|
||||
|
||||
Fields:
|
||||
- `type`: `"assistant"` or `"user"`
|
||||
- `session_id`
|
||||
- `message` (see below)
|
||||
|
||||
Example (assistant text):
|
||||
```json
|
||||
{"type":"assistant","session_id":"session_01","message":{"id":"msg_1","type":"message","role":"assistant","content":[{"type":"text","text":"Planning next steps."}],"usage":{"input_tokens":120,"output_tokens":45}}}
|
||||
```
|
||||
|
||||
Example (assistant tool use):
|
||||
```json
|
||||
{"type":"assistant","session_id":"session_01","message":{"id":"msg_2","type":"message","role":"assistant","content":[{"type":"tool_use","id":"toolu_1","name":"Bash","input":{"command":"ls -la"}}]}}
|
||||
```
|
||||
|
||||
Example (user tool result, string content):
|
||||
```json
|
||||
{"type":"user","session_id":"session_01","message":{"id":"msg_3","type":"message","role":"user","content":[{"type":"tool_result","tool_use_id":"toolu_1","content":"total 2\nREADME.md\nsrc\n"}]}}
|
||||
```
|
||||
|
||||
Example (user tool result, array content):
|
||||
```json
|
||||
{"type":"user","session_id":"session_01","message":{"id":"msg_4","type":"message","role":"user","content":[{"type":"tool_result","tool_use_id":"toolu_2","content":[{"type":"text","text":"Task completed"}]}]}}
|
||||
```
|
||||
|
||||
Optional parent field (for nested tool usage):
|
||||
```json
|
||||
{"type":"assistant","parent_tool_use_id":"toolu_parent","session_id":"session_01", ...}
|
||||
```
|
||||
|
||||
### `result`
|
||||
|
||||
Fields (success path):
|
||||
- `type`: `"result"`
|
||||
- `subtype`: `"success"` (or `"completion"`)
|
||||
- `session_id`
|
||||
- `total_cost_usd`, `is_error`, `duration_ms`, `duration_api_ms`, `num_turns`
|
||||
- `result`: final answer string
|
||||
- `usage`: usage object
|
||||
- `modelUsage`: optional per-model usage
|
||||
|
||||
Example (success):
|
||||
```json
|
||||
{"type":"result","subtype":"success","session_id":"session_01","total_cost_usd":0.0123,"is_error":false,"duration_ms":12345,"duration_api_ms":12000,"num_turns":2,"result":"Done.","usage":{"input_tokens":150,"output_tokens":70,"service_tier":"standard","server_tool_use":{"web_search_requests":0}}}
|
||||
```
|
||||
|
||||
Example (error + permission denials):
|
||||
```json
|
||||
{"type":"result","subtype":"error","session_id":"session_02","total_cost_usd":0.001,"is_error":true,"duration_ms":2000,"duration_api_ms":1800,"num_turns":1,"result":"","error":"Permission denied","permission_denials":[{"tool_name":"Bash","tool_use_id":"toolu_9","tool_input":{"command":"git fetch origin main"}}]}
|
||||
```
|
||||
|
||||
## Message object (`message` field)
|
||||
|
||||
Fields:
|
||||
- `id`, `type`, `role`
|
||||
- `model` (optional)
|
||||
- `content`: array of content blocks
|
||||
- `usage` (assistant messages)
|
||||
|
||||
## Content block shapes (in `message.content[]`)
|
||||
|
||||
### Text
|
||||
```json
|
||||
{"type":"text","text":"Hello"}
|
||||
```
|
||||
|
||||
### Tool use
|
||||
```json
|
||||
{"type":"tool_use","id":"toolu_1","name":"Bash","input":{"command":"ls -la"}}
|
||||
```
|
||||
|
||||
### Tool result
|
||||
String content:
|
||||
```json
|
||||
{"type":"tool_result","tool_use_id":"toolu_1","content":"ok"}
|
||||
```
|
||||
|
||||
Array content (Task tool format):
|
||||
```json
|
||||
{"type":"tool_result","tool_use_id":"toolu_2","content":[{"type":"text","text":"Task done"}]}
|
||||
```
|
||||
@@ -0,0 +1,225 @@
|
||||
# Claude Code -> Takopi event mapping (spec)
|
||||
|
||||
This document describes how the Claude Code runner translates Claude CLI JSONL events into Takopi events.
|
||||
|
||||
> **Authoritative source:** The schema definitions are in `src/takopi/schemas/claude.py` and the translation logic is in `src/takopi/runners/claude.py`. When in doubt, refer to the code.
|
||||
|
||||
The goal is to make a Claude runner feel identical to the Codex runner from the bridge/renderer point of view while preserving Takopi invariants (stable action ids, per-session serialization, single completed event).
|
||||
|
||||
---
|
||||
|
||||
## 1. Input stream contract (Claude CLI)
|
||||
|
||||
Claude Code CLI emits **one JSON object per line** (JSONL) when invoked with
|
||||
`--output-format stream-json` (only valid with `-p/--print`).
|
||||
|
||||
Recommended invocation (matches claudecode-go):
|
||||
|
||||
```
|
||||
claude -p --output-format stream-json --verbose -- <query>
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `--verbose` is required for `stream-json` output (clis may otherwise drop events).
|
||||
- `-p/--print` is required for `--output-format` and `--include-partial-messages`.
|
||||
- `-- <query>` is required to safely pass prompts that start with `-`.
|
||||
- Resuming uses `--resume <session_id>` and optional `--fork-session`.
|
||||
- The CLI does **not** read the prompt from stdin in claudecode-go; it passes the
|
||||
prompt as the final positional argument after `--`.
|
||||
|
||||
---
|
||||
|
||||
## 2. Resume tokens and resume lines
|
||||
|
||||
- Engine id: `claude`
|
||||
- Canonical resume line (embedded in chat):
|
||||
|
||||
```
|
||||
`claude --resume <session_id>`
|
||||
```
|
||||
|
||||
Runner must implement its own regex because the resume format is
|
||||
`claude --resume <session_id>`. Suggested regex:
|
||||
|
||||
```
|
||||
(?im)^\s*`?claude\s+(?:--resume|-r)\s+(?P<token>[^`\s]+)`?\s*$
|
||||
```
|
||||
|
||||
**Note:** Claude session IDs should be treated as opaque strings.
|
||||
|
||||
Resume rules:
|
||||
- If a resume token is provided to `run()`, the runner MUST verify that any
|
||||
`session_id` observed in the stream matches it.
|
||||
- If the stream yields a different `session_id`, emit a fatal error and end the run.
|
||||
|
||||
---
|
||||
|
||||
## 3. Session lifecycle + serialization
|
||||
|
||||
Takopi requires **serialization per session id**:
|
||||
|
||||
- For new runs (`resume=None`), do **not** acquire a lock until a `session_id`
|
||||
is observed (usually the first `system.init` event).
|
||||
- Once the session id is known, acquire a lock for `claude:<session_id>` and hold
|
||||
it until the run completes.
|
||||
- For resumed runs, acquire the lock immediately on entry.
|
||||
|
||||
This matches the Codex runner behavior in `takopi/runners/codex.py`.
|
||||
|
||||
---
|
||||
|
||||
## 4. Event translation (Claude JSONL -> Takopi)
|
||||
|
||||
### 4.1 Top-level `system` events
|
||||
|
||||
Claude emits a system init event early in the stream:
|
||||
|
||||
```
|
||||
{"type":"system","subtype":"init","session_id":"...", ...}
|
||||
```
|
||||
|
||||
**Mapping:**
|
||||
- Emit a Takopi `started` event as soon as `session_id` is known.
|
||||
- Assume only one `system.init` per run; if more appear, ignore the subsequent
|
||||
ones to avoid re-locking.
|
||||
- Optional: emit a `note` action summarizing tools/MCP servers (debug-only).
|
||||
|
||||
### 4.2 `assistant` / `user` message events
|
||||
|
||||
Claude messages include a `message` object with a `content[]` array. Each content
|
||||
block can represent text, tool usage, or tool results.
|
||||
|
||||
For each content block:
|
||||
|
||||
#### A) `type = "tool_use"`
|
||||
**Mapping:** emit `action` with `phase="started"`.
|
||||
|
||||
- `action.id` = `content.id`
|
||||
- `action.kind` = map from tool name (see section 5)
|
||||
- `title`:
|
||||
- if kind=`command`: use `input.command` if present
|
||||
- else: tool name or derived label
|
||||
- `detail` should include:
|
||||
- `tool_name`, `tool_input`, `message_id`, `parent_tool_use_id` (if provided)
|
||||
|
||||
#### B) `type = "tool_result"`
|
||||
**Mapping:** emit `action` with `phase="completed"`.
|
||||
|
||||
- `action.id` = `content.tool_use_id`
|
||||
- `ok`:
|
||||
- if `content.is_error` exists and is true -> `ok=False`
|
||||
- else `ok=True`
|
||||
- `detail` should include:
|
||||
- `tool_use_id`, `content` (raw), `message_id`
|
||||
|
||||
The runner SHOULD keep a small in-memory map from `tool_use_id -> tool_name`
|
||||
(learned from `tool_use`) so the completed action title can match the started
|
||||
action title.
|
||||
|
||||
#### C) `type = "text"`
|
||||
**Mapping:**
|
||||
- Default: do **not** emit an action (avoid duplicate rendering).
|
||||
- Store the latest assistant text as a fallback final answer if `result.result`
|
||||
is empty or missing.
|
||||
|
||||
#### D) `type = "thinking"` or other unknown types
|
||||
**Mapping:** optional `note` action (phase completed) with title derived from
|
||||
content; otherwise ignore.
|
||||
|
||||
### 4.3 `result` events
|
||||
|
||||
The terminal event looks like:
|
||||
|
||||
```
|
||||
{"type":"result","subtype":"success", ...}
|
||||
```
|
||||
|
||||
**Mapping:** emit a single Takopi `completed` event:
|
||||
|
||||
- `ok = !event.is_error`
|
||||
- `answer = event.result` (fallback to last assistant text if empty)
|
||||
- `error = event.error` (if present)
|
||||
- `resume = ResumeToken(engine="claude", value=event.session_id)`
|
||||
- `usage = event.usage` (pass through)
|
||||
- Emit exactly one `completed` event; ignore any trailing JSON lines afterward.
|
||||
No idle-timeout completion is used.
|
||||
|
||||
#### Permission denials
|
||||
`result.permission_denials` may contain tool calls that were blocked. Emit a
|
||||
warning action for each denial *before* the final `completed` event:
|
||||
|
||||
- `action.kind = "warning"`
|
||||
- `title = "permission denied: <tool_name>"`
|
||||
- `detail = {tool_name, tool_use_id, tool_input}`
|
||||
- `ok = False`, `level = "warning"`
|
||||
|
||||
### 4.4 Error handling / malformed lines
|
||||
|
||||
- If a JSONL line is invalid JSON: emit a warning action and continue.
|
||||
- If the subprocess exits non-zero or the stream ends without a `result` event:
|
||||
emit `completed` with `ok=False` and `error` explaining the failure.
|
||||
- Emit **exactly one** `completed` event per run.
|
||||
|
||||
---
|
||||
|
||||
## 5. Tool name -> ActionKind mapping heuristics
|
||||
|
||||
Claude tool names can evolve. The runner SHOULD map based on tool name and input
|
||||
shape. Suggested rules:
|
||||
|
||||
| Tool name pattern | ActionKind | Title logic |
|
||||
| --- | --- | --- |
|
||||
| `Bash`, `Shell` | `command` | `input.command` |
|
||||
| `Write`, `Edit`, `MultiEdit`, `NotebookEdit` | `file_change` | `input.path` |
|
||||
| `Read` | `tool` | `Read <path>` |
|
||||
| `WebSearch` | `web_search` | `input.query` |
|
||||
| (default) | `tool` | tool name |
|
||||
|
||||
For `file_change`, emit `detail.changes = [{"path": <path>, "kind": "update"}]`.
|
||||
If input indicates creation (ex: `create: true`), use `kind: "add"`.
|
||||
|
||||
If a tool name is unknown, map to `tool` and include the full input in `detail`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Usage mapping
|
||||
|
||||
Takopi `completed.usage` should mirror the Claude `result.usage` object
|
||||
without transformation. Optionally include `modelUsage` inside `usage` or
|
||||
`detail` if downstream consumers want it (currently unused by renderers).
|
||||
|
||||
---
|
||||
|
||||
## 7. Implementation checklist (v0.3.0)
|
||||
|
||||
Claude runner implementation summary (no Takopi domain model changes):
|
||||
|
||||
1. [x] Create `takopi/runners/claude.py` implementing `Runner` and (custom)
|
||||
resume parsing.
|
||||
2. [x] Define `BACKEND` in `takopi/runners/claude.py`:
|
||||
- `install_cmd`: install command for the `claude` binary
|
||||
- `build_runner`: read `[claude]` config + construct runner
|
||||
3. [x] Add new docs (this file + `stream-json-cheatsheet.md`).
|
||||
4. [x] Add fixtures in `tests/fixtures/` (see below).
|
||||
5. [x] Add unit tests mirroring `tests/test_codex_*` but for Claude translation
|
||||
and resume parsing (recommended, not required for initial handoff).
|
||||
|
||||
---
|
||||
|
||||
## 8. Suggested Takopi config keys
|
||||
|
||||
A minimal TOML config for Claude:
|
||||
|
||||
```toml
|
||||
[claude]
|
||||
# model: opus | sonnet | haiku
|
||||
model = "sonnet"
|
||||
|
||||
allowed_tools = ["Bash", "Read", "Edit", "Write", "WebSearch"]
|
||||
dangerously_skip_permissions = false
|
||||
use_api_billing = false
|
||||
```
|
||||
|
||||
Takopi only maps these keys to Claude CLI flags; other options should be configured in Claude Code settings.
|
||||
If `allowed_tools` is omitted, Takopi defaults to `["Bash", "Read", "Edit", "Write"]`.
|
||||
When `use_api_billing` is false (default), Takopi strips `ANTHROPIC_API_KEY` from the Claude subprocess environment to prefer subscription billing.
|
||||
@@ -0,0 +1,345 @@
|
||||
# Codex `exec --json` event cheatsheet
|
||||
|
||||
`codex exec --json` writes **one JSON object per line** (JSONL) to stdout. Each
|
||||
line is a top-level **thread event** with a `type` field.
|
||||
|
||||
Below: **required + commonly emitted fields** for every line type plus a
|
||||
**full-line example** for each shape that can be emitted. Fields noted as
|
||||
optional may be omitted (or `null`) depending on Codex version and lifecycle.
|
||||
Unknown fields may appear; ignore what you don't use.
|
||||
|
||||
## Top-level event lines (non-item)
|
||||
|
||||
### `thread.started`
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
- `thread_id`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"thread.started","thread_id":"0199a213-81c0-7800-8aa1-bbab2a035a53"}
|
||||
```
|
||||
|
||||
### `turn.started`
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"turn.started"}
|
||||
```
|
||||
|
||||
### `turn.completed`
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
- `usage.input_tokens`
|
||||
- `usage.cached_input_tokens`
|
||||
- `usage.output_tokens`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"turn.completed","usage":{"input_tokens":24763,"cached_input_tokens":24448,"output_tokens":122}}
|
||||
```
|
||||
|
||||
### `turn.failed`
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
- `error.message`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"turn.failed","error":{"message":"model response stream ended unexpectedly"}}
|
||||
```
|
||||
|
||||
### `error`
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
- `message`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"error","message":"stream error: broken pipe"}
|
||||
```
|
||||
|
||||
Note: Codex may emit transient reconnect notices as `type="error"` with messages
|
||||
like `"Reconnecting... 1/5"` while it retries a dropped stream. Treat those as
|
||||
non-fatal progress updates (the turn continues).
|
||||
|
||||
## Item event lines (`item.*`)
|
||||
|
||||
Every item line includes:
|
||||
- `type` (`item.started`, `item.updated`, or `item.completed`)
|
||||
- `item.id`
|
||||
- `item.type`
|
||||
- fields for the specific `item.type` below
|
||||
|
||||
`item.id` is stable for the item; updates/completion reuse the same id.
|
||||
|
||||
### `agent_message` (only `item.completed`)
|
||||
|
||||
Fields:
|
||||
- `item.text`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_3","type":"agent_message","text":"Done. I updated the docs and added examples."}}
|
||||
```
|
||||
|
||||
### `reasoning` (only `item.completed`, if enabled)
|
||||
|
||||
Fields:
|
||||
- `item.text`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_0","type":"reasoning","text":"**Scanning docs for exec JSON schema**"}}
|
||||
```
|
||||
|
||||
### `command_execution` (`item.started` and `item.completed`)
|
||||
|
||||
Fields:
|
||||
- `item.command`
|
||||
- `item.aggregated_output`
|
||||
- `item.exit_code` (null or omitted until completion)
|
||||
- `item.status` (`in_progress`, `completed`, `failed`)
|
||||
|
||||
Example (started):
|
||||
```json
|
||||
{"type":"item.started","item":{"id":"item_1","type":"command_execution","command":"bash -lc ls","aggregated_output":"","exit_code":null,"status":"in_progress"}}
|
||||
```
|
||||
|
||||
Example (completed, success):
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_1","type":"command_execution","command":"bash -lc ls","aggregated_output":"docs\nsrc\n","exit_code":0,"status":"completed"}}
|
||||
```
|
||||
|
||||
Example (completed, failure):
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_2","type":"command_execution","command":"bash -lc false","aggregated_output":"","exit_code":1,"status":"failed"}}
|
||||
```
|
||||
|
||||
Note: `aggregated_output` is truncated to **64 KiB**; truncated output ends with
|
||||
`\n...(truncated)`.
|
||||
|
||||
### `file_change` (only `item.completed`)
|
||||
|
||||
Fields:
|
||||
- `item.changes[].path`
|
||||
- `item.changes[].kind` (`add`, `delete`, `update`)
|
||||
- `item.status` (`completed`, `failed`)
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_4","type":"file_change","changes":[{"path":"docs/exec-json-cheatsheet.md","kind":"add"},{"path":"docs/exec.md","kind":"update"}],"status":"completed"}}
|
||||
```
|
||||
|
||||
### `mcp_tool_call` (`item.started` and `item.completed`)
|
||||
|
||||
Fields:
|
||||
- `item.server`
|
||||
- `item.tool`
|
||||
- `item.arguments` (JSON value; defaults to `null` if absent)
|
||||
- `item.result` (object or `null`; may be omitted)
|
||||
- `item.result.content` (array of MCP content blocks)
|
||||
- `item.result.structured_content` (JSON value or `null`)
|
||||
- `item.error` (object or `null`; may be omitted)
|
||||
- `item.error.message` (if `error` is present)
|
||||
- `item.status` (`in_progress`, `completed`, `failed`)
|
||||
|
||||
Example (started):
|
||||
```json
|
||||
{"type":"item.started","item":{"id":"item_5","type":"mcp_tool_call","server":"docs","tool":"search","arguments":{"q":"exec --json"},"result":null,"error":null,"status":"in_progress"}}
|
||||
```
|
||||
|
||||
Example (completed, success):
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_5","type":"mcp_tool_call","server":"docs","tool":"search","arguments":{"q":"exec --json"},"result":{"content":[{"type":"text","text":"Found 3 matches.","annotations":{"audience":["assistant"],"lastModified":"2025-01-01T00:00:00Z","priority":0.5}}],"structured_content":{"matches":3}},"error":null,"status":"completed"}}
|
||||
```
|
||||
|
||||
Example (completed, failure):
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_6","type":"mcp_tool_call","server":"docs","tool":"search","arguments":{"q":"exec --json"},"result":null,"error":{"message":"tool timeout"},"status":"failed"}}
|
||||
```
|
||||
|
||||
### `web_search` (only `item.completed`)
|
||||
|
||||
Fields:
|
||||
- `item.query`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_7","type":"web_search","query":"codex exec --json schema"}}
|
||||
```
|
||||
|
||||
### `todo_list` (`item.started`, `item.updated`, and `item.completed`)
|
||||
|
||||
Fields:
|
||||
- `item.items[].text`
|
||||
- `item.items[].completed`
|
||||
|
||||
Example (started):
|
||||
```json
|
||||
{"type":"item.started","item":{"id":"item_8","type":"todo_list","items":[{"text":"Scan docs","completed":false},{"text":"Write cheatsheet","completed":false}]}}
|
||||
```
|
||||
|
||||
Example (updated):
|
||||
```json
|
||||
{"type":"item.updated","item":{"id":"item_8","type":"todo_list","items":[{"text":"Scan docs","completed":true},{"text":"Write cheatsheet","completed":false}]}}
|
||||
```
|
||||
|
||||
Example (completed):
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_8","type":"todo_list","items":[{"text":"Scan docs","completed":true},{"text":"Write cheatsheet","completed":true}]}}
|
||||
```
|
||||
|
||||
### `error` (non-fatal warning as an item; only `item.completed`)
|
||||
|
||||
Fields:
|
||||
- `item.message`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_9","type":"error","message":"command output truncated"}}
|
||||
```
|
||||
|
||||
## MCP content block shapes (`mcp_tool_call.result.content`)
|
||||
|
||||
`result.content` is an array of **content blocks**. Each block is one of the
|
||||
types below; all optional fields may appear depending on the server.
|
||||
|
||||
### Text content
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
- `text`
|
||||
- `annotations.audience` (optional)
|
||||
- `annotations.lastModified` (optional)
|
||||
- `annotations.priority` (optional)
|
||||
|
||||
Example block:
|
||||
```json
|
||||
{"type":"text","text":"Hello","annotations":{"audience":["assistant"],"lastModified":"2025-01-01T00:00:00Z","priority":0.5}}
|
||||
```
|
||||
|
||||
### Image content
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
- `data` (base64)
|
||||
- `mimeType`
|
||||
- `annotations.*` (same as above, optional)
|
||||
|
||||
Example block:
|
||||
```json
|
||||
{"type":"image","data":"<base64>","mimeType":"image/png","annotations":{"audience":["assistant"]}}
|
||||
```
|
||||
|
||||
### Audio content
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
- `data` (base64)
|
||||
- `mimeType`
|
||||
- `annotations.*` (optional)
|
||||
|
||||
Example block:
|
||||
```json
|
||||
{"type":"audio","data":"<base64>","mimeType":"audio/wav","annotations":{"audience":["assistant"]}}
|
||||
```
|
||||
|
||||
### Resource link
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
- `name`
|
||||
- `uri`
|
||||
- `description` (optional)
|
||||
- `mimeType` (optional)
|
||||
- `size` (optional)
|
||||
- `title` (optional)
|
||||
- `annotations.*` (optional)
|
||||
|
||||
Example block:
|
||||
```json
|
||||
{"type":"resource_link","name":"docs/exec.md","uri":"file:///repo/docs/exec.md","description":"Exec docs","mimeType":"text/markdown","size":1234,"title":"exec.md","annotations":{"audience":["assistant"]}}
|
||||
```
|
||||
|
||||
### Embedded resource
|
||||
|
||||
Fields:
|
||||
- `type`
|
||||
- `resource` (either text or blob contents)
|
||||
- `annotations.*` (optional)
|
||||
|
||||
Example block (embedded text):
|
||||
```json
|
||||
{"type":"resource","resource":{"uri":"file:///repo/README.md","text":"Hello","mimeType":"text/markdown"},"annotations":{"audience":["assistant"]}}
|
||||
```
|
||||
|
||||
Example block (embedded blob):
|
||||
```json
|
||||
{"type":"resource","resource":{"uri":"file:///repo/image.png","blob":"<base64>","mimeType":"image/png"},"annotations":{"audience":["assistant"]}}
|
||||
```
|
||||
|
||||
## Consumer considerations (rendering + success/failure)
|
||||
|
||||
Use this section to decide what to surface to end users vs. what to treat as
|
||||
machine-only metadata.
|
||||
|
||||
### What to render for users
|
||||
|
||||
- **Final answer:** render `item.completed` where `item.type = "agent_message"` as
|
||||
the main response.
|
||||
- **Progress updates (optional):**
|
||||
- `item.completed` with `item.type = "reasoning"` can be shown as brief
|
||||
activity breadcrumbs (only if you want to expose reasoning summaries).
|
||||
- `item.started` / `item.completed` with `item.type = "command_execution"` can
|
||||
be shown as “running command …” status lines without printing full output.
|
||||
- `item.completed` with `item.type = "file_change"` can be rendered as a list
|
||||
of changed paths and kinds (add/update/delete).
|
||||
- `item.*` with `item.type = "todo_list"` can be shown as a progress checklist.
|
||||
- **Errors:** render `type = "error"` and `item.type = "error"` as user-visible
|
||||
warnings or failures.
|
||||
|
||||
### Fields you can safely skip for UX
|
||||
|
||||
- `command_execution.aggregated_output` is often noisy; many consumers omit or
|
||||
truncate it, and rely on `command_execution.status` + `exit_code` instead.
|
||||
- `mcp_tool_call.result.content` can be large and tool-specific; consider showing
|
||||
only high-level status unless you know the tool’s schema.
|
||||
- `usage` fields (`turn.completed.usage.*`) are typically telemetry-only.
|
||||
|
||||
### Success and failure signals
|
||||
|
||||
- **Turn success:** `type = "turn.completed"` indicates overall success.
|
||||
- **Turn failure:** `type = "turn.failed"` with `error.message` indicates failure.
|
||||
- **Item success/failure:** use `item.status` on the item payload:
|
||||
- `command_execution.status`: `completed` = success, `failed` = failure.
|
||||
- `file_change.status`: `completed` = patch applied, `failed` = patch failed.
|
||||
- `mcp_tool_call.status`: `completed` = tool succeeded, `failed` = tool failed.
|
||||
- **Fatal stream errors:** `type = "error"` means the JSONL stream itself hit an
|
||||
unrecoverable error (except transient `"Reconnecting... X/Y"` notices, which
|
||||
are non-fatal).
|
||||
|
||||
### Suggested minimal rendering
|
||||
|
||||
If you want a compact UI, the following is usually enough:
|
||||
- Thread/turn lifecycle: `thread.started`, `turn.started`, `turn.completed` or
|
||||
`turn.failed`
|
||||
- Final answer: `item.completed` with `item.type = "agent_message"`
|
||||
- Optional progress: `item.started` / `item.completed` for `command_execution`
|
||||
and `file_change`
|
||||
|
||||
### Optional/conditional emission notes
|
||||
|
||||
- `turn.failed` only appears on failure; otherwise `turn.completed` is emitted.
|
||||
- `reasoning` items only appear when reasoning summaries are enabled.
|
||||
- `todo_list` items only appear when the plan tool is active; they are the
|
||||
primary source of `item.updated`.
|
||||
- `file_change` and `web_search` items are emitted only as `item.completed`
|
||||
in the current `codex exec --json` stream.
|
||||
@@ -0,0 +1,432 @@
|
||||
# Codex -> Takopi event mapping
|
||||
|
||||
This document describes how Codex exec --json events are translated to Takopi's normalized event model.
|
||||
|
||||
> **Authoritative source:** The schema definitions are in `src/takopi/schemas/codex.py` and the translation logic is in `src/takopi/runners/codex.py`. When in doubt, refer to the code.
|
||||
|
||||
## The 3-event Takopi schema
|
||||
|
||||
The Takopi event model uses 3 event types. The `action` event includes a `phase` field to represent started/updated/completed lifecycles.
|
||||
|
||||
### 1) `started`
|
||||
|
||||
Emitted once **as soon as you know the resume token** (Codex: `thread.started.thread_id`).
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "started",
|
||||
"engine": "codex",
|
||||
"resume": { "engine": "codex", "value": "0199..." },
|
||||
"title": "Codex", // optional
|
||||
"meta": { "raw": { ... } } // optional: for debugging
|
||||
}
|
||||
```
|
||||
|
||||
### 2) `action`
|
||||
|
||||
Emitted for **everything that is progress / updates / warnings / per-item lifecycle**.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "action",
|
||||
"engine": "codex",
|
||||
"action": {
|
||||
"id": "item_5",
|
||||
"kind": "tool", // command | tool | file_change | web_search | subagent | note | turn | warning | telemetry
|
||||
"title": "docs.search", // short label for renderer
|
||||
"detail": { ... } // structured payload (freeform)
|
||||
},
|
||||
"phase": "started", // started | updated | completed
|
||||
"ok": true, // optional; present when phase=completed (or warnings)
|
||||
"message": "optional text", // optional; logs/warnings can use this
|
||||
"level": "info" // optional: debug|info|warning|error
|
||||
}
|
||||
```
|
||||
|
||||
### 3) `completed`
|
||||
|
||||
Emitted once at end-of-run with the **final answer** (from `agent_message`) and final status.
|
||||
|
||||
```json
|
||||
{
|
||||
"type": "completed",
|
||||
"engine": "codex",
|
||||
"resume": { "engine": "codex", "value": "0199..." }, // if known
|
||||
"ok": true,
|
||||
"answer": "Done. I updated the docs...",
|
||||
"error": null,
|
||||
"usage": { "input_tokens": 24763, "cached_input_tokens": 24448, "output_tokens": 122 } // optional
|
||||
}
|
||||
```
|
||||
|
||||
Why this fits Takopi cleanly:
|
||||
|
||||
* Your `started` corresponds to the old “session.started” concept (runner learns resume token; bridge can now safely serialize per thread).
|
||||
* Your `action` is “everything that would have been action.started/action.completed/log/error” collapsed into one stream.
|
||||
* Your `completed` corresponds to final `RunResult` + status, using Codex’s `agent_message` as the answer source.
|
||||
|
||||
---
|
||||
|
||||
## How everything fits together (end-to-end)
|
||||
|
||||
From the bridge/runner point of view:
|
||||
|
||||
1. **Bridge receives Telegram prompt**
|
||||
2. Bridge tries to extract a resume line (`codex resume <uuid>`) from the message/reply (runner-owned parsing).
|
||||
3. Bridge calls `runner.run(prompt, resumeTokenOrNone)`
|
||||
4. Codex runner spawns `codex exec --json ...` and reads JSONL line-by-line.
|
||||
5. The *first moment the runner can know thread identity* is:
|
||||
|
||||
* `thread.started` → contains `thread_id` (this is your resume value)
|
||||
6. Runner must (per Takopi’s concurrency invariant) **acquire the per-thread lock as soon as the new thread token is known**, before emitting `started`.
|
||||
7. Runner translates subsequent Codex JSONL lines into `action` events for progress rendering.
|
||||
8. Runner captures the final answer from `item.completed` where `item.type="agent_message"`.
|
||||
9. Runner emits exactly one `completed` event when the run ends (`turn.completed` or failure), including the captured final answer.
|
||||
|
||||
---
|
||||
|
||||
## Direct translation: every Codex `exec --json` line → your 3-event schema
|
||||
|
||||
Codex emits two categories: **top-level lines** and **item lines**.
|
||||
|
||||
### A) Top-level lines
|
||||
|
||||
#### `thread.started`
|
||||
|
||||
Codex:
|
||||
|
||||
```json
|
||||
{"type":"thread.started","thread_id":"0199..."}
|
||||
```
|
||||
|
||||
→ Takopi:
|
||||
|
||||
* emit **`started`**:
|
||||
|
||||
* `resume.value = thread_id`
|
||||
|
||||
This is exactly the “learn resume tag” moment you described.
|
||||
|
||||
---
|
||||
|
||||
#### `turn.started`
|
||||
|
||||
Codex:
|
||||
|
||||
```json
|
||||
{"type":"turn.started"}
|
||||
```
|
||||
|
||||
→ Takopi (recommended):
|
||||
|
||||
* emit **`action`** with a synthetic action id, e.g. `"turn_0"`
|
||||
|
||||
* `kind="turn"`, `phase="started"`, `title="turn started"`
|
||||
|
||||
You *can* also drop it if your UI doesn’t care, but if you want “every codex type translates”, this maps cleanly into `action`.
|
||||
|
||||
---
|
||||
|
||||
#### `turn.completed`
|
||||
|
||||
Codex includes usage:
|
||||
|
||||
```json
|
||||
{"type":"turn.completed","usage":{...}}
|
||||
```
|
||||
|
||||
→ Takopi:
|
||||
|
||||
* emit **`completed`**
|
||||
|
||||
* `ok=true`
|
||||
* `answer = last seen agent_message text` (or `""` if none)
|
||||
* `usage = usage` (optional)
|
||||
|
||||
This is your authoritative “run succeeded” boundary.
|
||||
|
||||
---
|
||||
|
||||
#### `turn.failed`
|
||||
|
||||
Codex:
|
||||
|
||||
```json
|
||||
{"type":"turn.failed","error":{"message":"..."}}
|
||||
```
|
||||
|
||||
→ Takopi:
|
||||
|
||||
* emit **`completed`**
|
||||
|
||||
* `ok=false`
|
||||
* `error = error.message`
|
||||
* `answer = last seen agent_message` (if any; usually empty)
|
||||
|
||||
This is “run ended, but failed”.
|
||||
|
||||
---
|
||||
|
||||
#### Top-level `error` (stream error)
|
||||
|
||||
Codex:
|
||||
|
||||
```json
|
||||
{"type":"error","message":"stream error: broken pipe"}
|
||||
```
|
||||
|
||||
Cheatsheet meaning: this is a **fatal stream failure** (not just a tool failure).
|
||||
However, Codex may also emit transient reconnect notices as `type="error"` with
|
||||
messages like `"Reconnecting... 1/5"` while it retries a dropped stream. Treat
|
||||
those as non-fatal progress updates (do **not** end the run).
|
||||
|
||||
→ Takopi:
|
||||
|
||||
* if you haven’t emitted `completed` yet: emit **`completed`** with `ok=false` and `error=message`
|
||||
* if you *already* emitted `completed`, treat it as an extra warning (or ignore; it’s “post-mortem noise”)
|
||||
|
||||
---
|
||||
|
||||
### B) Item lines: `item.started`, `item.updated`, `item.completed`
|
||||
|
||||
All item lines include `item.id` and it is stable across updates/completion.
|
||||
That means your `action.action.id` should just be `item.id` — perfect match to “stable within a run”.
|
||||
|
||||
#### General rule (for any item.* line)
|
||||
|
||||
* `action.action.id = item.id`
|
||||
* `action.phase = started | updated | completed`
|
||||
* `action.action.kind` derived from `item.type`
|
||||
* `action.action.detail` contains the relevant item fields (possibly trimmed)
|
||||
|
||||
Now, map each `item.type`:
|
||||
|
||||
---
|
||||
|
||||
## Item-type mapping: `item.type` → `action.kind/title/detail/ok`
|
||||
|
||||
Below is a “complete coverage” mapping for all item types listed in the cheatsheet.
|
||||
|
||||
### 1) `agent_message` (only `item.completed`)
|
||||
|
||||
Codex:
|
||||
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_3","type":"agent_message","text":"..."}}
|
||||
```
|
||||
|
||||
→ Takopi:
|
||||
|
||||
* **do not emit an `action`** (recommended)
|
||||
* instead: **store** `final_answer = item.text`
|
||||
* final answer will be surfaced by the eventual `completed` event
|
||||
|
||||
Reason: you want `completed` to be “final answer delivery”, and you probably don’t want the answer duplicated in progress rendering.
|
||||
|
||||
(If you *do* want to render it as it arrives, you can emit an `action` too, but then your renderer must avoid showing it twice.)
|
||||
|
||||
---
|
||||
|
||||
### 2) `reasoning` (only `item.completed`, if enabled)
|
||||
|
||||
Codex gives a text breadcrumb.
|
||||
|
||||
→ Takopi `action`:
|
||||
|
||||
* `kind="note"`
|
||||
* `title="reasoning"` (or “thought”)
|
||||
* `phase="completed"`
|
||||
* `message=item.text` (or put it under `detail.text`)
|
||||
|
||||
This is usually safe to show as a short “what it’s doing” line (or ignore if you don’t want to surface it).
|
||||
|
||||
---
|
||||
|
||||
### 3) `command_execution` (`item.started` and `item.completed`)
|
||||
|
||||
Codex fields include `command`, `status`, `aggregated_output` (often noisy), and
|
||||
`exit_code` (null or omitted until completion).
|
||||
|
||||
→ Takopi `action`:
|
||||
|
||||
* `kind="command"`
|
||||
* `title=item.command` (or a shortened version like `pytest`)
|
||||
* `detail={ command, exit_code, status }` (optionally include output tail)
|
||||
* `phase="started"` on `item.started`
|
||||
* `phase="completed"` on `item.completed`
|
||||
* `ok = (item.status == "completed")` (and `exit_code == 0` when present)
|
||||
|
||||
Note: “failed” command becomes `ok=false` but it’s still just an `action` completion — the overall run might still succeed later, depending on agent behavior.
|
||||
|
||||
---
|
||||
|
||||
### 4) `file_change` (only `item.completed`)
|
||||
|
||||
Codex contains `changes[]` and `status`.
|
||||
|
||||
→ Takopi `action`:
|
||||
|
||||
* `kind="file_change"`
|
||||
* `title="file changes"`
|
||||
* `detail={ changes }`
|
||||
* `phase="completed"`
|
||||
* `ok = (item.status == "completed")`
|
||||
|
||||
This is a great progress line for your UI (“updated docs/…, added …”).
|
||||
|
||||
---
|
||||
|
||||
### 5) `mcp_tool_call` (`item.started` and `item.completed`)
|
||||
|
||||
Codex contains server/tool/arguments/status and may include result/error on
|
||||
completion. Result can be large; may include base64 in content blocks.
|
||||
|
||||
→ Takopi `action`:
|
||||
|
||||
* `kind="tool"`
|
||||
* `title=f"{item.server}.{item.tool}"`
|
||||
* `detail={ server, tool, arguments, status }`
|
||||
* on completion, include *summary* of result:
|
||||
|
||||
* e.g. `detail.result_summary = { content_blocks: N, has_structured: bool }`
|
||||
* include `detail.error_message` if failed
|
||||
* `phase="started"` or `"completed"`
|
||||
* `ok = (item.status == "completed")`
|
||||
|
||||
Recommendation: **do not dump** full `result.content` into `detail` if it can contain large blobs; keep a summary and optionally stash full raw elsewhere for debugging.
|
||||
|
||||
---
|
||||
|
||||
### 6) `web_search` (only `item.completed`)
|
||||
|
||||
Codex includes `query`.
|
||||
|
||||
→ Takopi `action`:
|
||||
|
||||
* `kind="web_search"`
|
||||
* `title="web search"`
|
||||
* `detail={ query }`
|
||||
* `phase="completed"`
|
||||
* `ok=true` (this is just “it did a search”; success/failure is typically not expressed here)
|
||||
|
||||
---
|
||||
|
||||
### 7) `todo_list` (`item.started`, `item.updated`, `item.completed`)
|
||||
|
||||
Codex includes checklist items with `completed` booleans.
|
||||
|
||||
→ Takopi `action`:
|
||||
|
||||
* `kind="note"` (or `"todo"`)
|
||||
* `title="plan"`
|
||||
* `detail={ items, done: count_done, total: count_total }`
|
||||
* `phase` maps 1:1 to started/updated/completed
|
||||
* `ok=true` when phase completed (optional)
|
||||
|
||||
This is the one case where `item.updated` is common; your unified `action` event is exactly the right shape for it.
|
||||
|
||||
---
|
||||
|
||||
### 8) Item `error` (non-fatal warning as an item; only `item.completed`)
|
||||
|
||||
Codex:
|
||||
|
||||
```json
|
||||
{"type":"item.completed","item":{"id":"item_9","type":"error","message":"command output truncated"}}
|
||||
```
|
||||
|
||||
Cheatsheet: this is a **non-fatal warning** (different from top-level fatal `error`).
|
||||
|
||||
→ Takopi `action`:
|
||||
|
||||
* `kind="warning"` (or `"note"`)
|
||||
* `title="warning"`
|
||||
* `message=item.message`
|
||||
* `level="warning"`
|
||||
* `phase="completed"`
|
||||
* `ok=true` (because it’s informational) **or** omit `ok`
|
||||
|
||||
---
|
||||
|
||||
## Suggested “single-pass” translator logic (pseudocode)
|
||||
|
||||
This shows how to implement it without needing more than one pass or complicated buffering:
|
||||
|
||||
```python
|
||||
final_answer = None
|
||||
resume = None
|
||||
did_emit_started = False
|
||||
did_emit_completed = False
|
||||
turn_index = 0
|
||||
|
||||
def emit(evt): yield evt # emit to the output event stream
|
||||
|
||||
for line in codex_jsonl_stream:
|
||||
t = line["type"]
|
||||
|
||||
if t == "thread.started":
|
||||
resume = {"engine": "codex", "value": line["thread_id"]}
|
||||
# acquire per-thread lock here (for new sessions) before emitting started
|
||||
emit({"type":"started","engine":"codex","resume":resume,"title":"Codex"})
|
||||
did_emit_started = True
|
||||
continue
|
||||
|
||||
if t == "turn.started":
|
||||
emit({"type":"action","engine":"codex",
|
||||
"action":{"id":f"turn_{turn_index}","kind":"turn","title":"turn started","detail":{}},
|
||||
"phase":"started"})
|
||||
continue
|
||||
|
||||
if t == "item.started" or t == "item.updated" or t == "item.completed":
|
||||
item = line["item"]
|
||||
item_type = item["type"]
|
||||
item_id = item["id"]
|
||||
|
||||
if t == "item.completed" and item_type == "agent_message":
|
||||
final_answer = item.get("text","")
|
||||
continue
|
||||
|
||||
# map item_type -> kind/title/detail/ok
|
||||
action_evt = map_item_to_action(item, phase=t.split(".")[1])
|
||||
emit(action_evt)
|
||||
continue
|
||||
|
||||
if t == "turn.completed":
|
||||
emit({"type":"completed","engine":"codex","resume":resume,
|
||||
"ok":True,"answer":final_answer or "",
|
||||
"error":None,"usage":line.get("usage")})
|
||||
did_emit_completed = True
|
||||
continue
|
||||
|
||||
if t == "turn.failed":
|
||||
emit({"type":"completed","engine":"codex","resume":resume,
|
||||
"ok":False,"answer":final_answer or "",
|
||||
"error":line["error"]["message"]})
|
||||
did_emit_completed = True
|
||||
continue
|
||||
|
||||
if t == "error": # fatal stream error
|
||||
if not did_emit_completed:
|
||||
emit({"type":"completed","engine":"codex","resume":resume,
|
||||
"ok":False,"answer":final_answer or "",
|
||||
"error":line.get("message")})
|
||||
did_emit_completed = True
|
||||
continue
|
||||
|
||||
# Optional: if stream ends without turn.completed/failed,
|
||||
# emit completed with ok=False and error="unexpected EOF"
|
||||
```
|
||||
|
||||
This design preserves the Takopi ordering/serialization principles: `started` happens as soon as resume token is known, actions stream in order, and exactly one `completed` closes the run.
|
||||
|
||||
---
|
||||
|
||||
## One practical note: what “completed” should mean
|
||||
|
||||
Even though you *learn* the final answer at `agent_message`, you generally want `completed` to be emitted at the **turn boundary** (`turn.completed` / `turn.failed`), because:
|
||||
|
||||
* you can attach usage (`turn.completed.usage`) only there,
|
||||
* you guarantee `completed` is truly the last event,
|
||||
* you still use `agent_message` as the authoritative answer payload.
|
||||
|
||||
That still matches your intent (“completed is when we get final answer”) because the answer comes from `agent_message`; you just *publish* it at the terminal boundary.
|
||||
@@ -0,0 +1,9 @@
|
||||
# Runners
|
||||
|
||||
Runner docs describe the **engine-specific** behavior: event shapes, JSON streaming, and integration notes.
|
||||
|
||||
- Claude: [Runner](claude/runner.md), [Stream JSON cheatsheet](claude/stream-json-cheatsheet.md), [Takopi events](claude/takopi-events.md)
|
||||
- Codex: [Exec JSON cheatsheet](codex/exec-json-cheatsheet.md), [Takopi events](codex/takopi-events.md)
|
||||
- OpenCode: [Runner](opencode/runner.md), [Stream JSON cheatsheet](opencode/stream-json-cheatsheet.md), [Takopi events](opencode/takopi-events.md)
|
||||
- Pi: [Runner](pi/runner.md), [Stream JSON cheatsheet](pi/stream-json-cheatsheet.md), [Takopi events](pi/takopi-events.md)
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
# OpenCode Runner
|
||||
|
||||
This runner integrates with the [OpenCode CLI](https://github.com/sst/opencode).
|
||||
Shipped in Takopi v0.5.0.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
npm i -g opencode-ai@latest
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
Add to your `takopi.toml`:
|
||||
|
||||
```toml
|
||||
[opencode]
|
||||
model = "claude-sonnet" # optional
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```bash
|
||||
takopi opencode
|
||||
```
|
||||
|
||||
## Resume Format
|
||||
|
||||
Resume line format: `` `opencode --session ses_XXX` ``
|
||||
|
||||
The runner recognizes both `--session` and `-s` flags (with or without `run`).
|
||||
|
||||
Note: The resume line is meant to reopen the interactive TUI session. `opencode run` is headless and requires a message or command, so it is not the canonical resume command.
|
||||
|
||||
## JSON Event Format
|
||||
|
||||
OpenCode outputs JSON events with the following types:
|
||||
|
||||
| Event Type | Description |
|
||||
|------------|-------------|
|
||||
| `step_start` | Beginning of a processing step |
|
||||
| `tool_use` | Tool invocation with input/output |
|
||||
| `text` | Text output from the model |
|
||||
| `step_finish` | End of a step (reason: "stop" or "tool-calls" when present) |
|
||||
| `error` | Error event |
|
||||
|
||||
See [stream-json-cheatsheet.md](./stream-json-cheatsheet.md) for detailed event format documentation.
|
||||
@@ -0,0 +1,145 @@
|
||||
# OpenCode `run --format json` Event Cheatsheet
|
||||
|
||||
`opencode run --format json` writes one JSON object per line (JSONL) to stdout.
|
||||
Each line has a `type` field indicating the event type.
|
||||
|
||||
## Event Types
|
||||
|
||||
### `step_start`
|
||||
|
||||
Marks the beginning of a processing step.
|
||||
|
||||
Fields:
|
||||
- `type`: `"step_start"`
|
||||
- `timestamp`: Unix timestamp in milliseconds
|
||||
- `sessionID`: Session identifier (format: `ses_XXX`)
|
||||
- `part.id`: Part identifier
|
||||
- `part.sessionID`: Session ID (duplicated)
|
||||
- `part.messageID`: Message ID
|
||||
- `part.type`: `"step-start"`
|
||||
- `part.snapshot`: Git snapshot hash
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"step_start","timestamp":1767036059338,"sessionID":"ses_494719016ffe85dkDMj0FPRbHK","part":{"id":"prt_b6b8e7ec7001qAZUB7eTENxPpI","sessionID":"ses_494719016ffe85dkDMj0FPRbHK","messageID":"msg_b6b8e702b0012XuEC4bGe0XhKa","type":"step-start","snapshot":"71db24a798b347669c0ebadb2dfad238f991753d"}}
|
||||
```
|
||||
|
||||
### `tool_use`
|
||||
|
||||
Tool invocation event. Emitted when a tool finishes (`status == "completed"`).
|
||||
|
||||
Fields:
|
||||
- `type`: `"tool_use"`
|
||||
- `timestamp`: Unix timestamp in milliseconds
|
||||
- `sessionID`: Session identifier
|
||||
- `part.id`: Part identifier
|
||||
- `part.callID`: Unique call ID for this tool invocation
|
||||
- `part.tool`: Tool name (e.g., "bash", "read", "write", "grep")
|
||||
- `part.state.status`: `"completed"` (the CLI JSON output does not emit pending/running tool states)
|
||||
- `part.state.input`: Tool input parameters
|
||||
- `part.state.output`: Tool output (when completed)
|
||||
- `part.state.title`: Human-readable description
|
||||
- `part.state.metadata`: Additional metadata (exit codes, etc.)
|
||||
- `part.state.time.start`: Start timestamp
|
||||
- `part.state.time.end`: End timestamp
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"tool_use","timestamp":1767036061199,"sessionID":"ses_494719016ffe85dkDMj0FPRbHK","part":{"id":"prt_b6b8e85bb001CzBoN2dDlEZJnP","sessionID":"ses_494719016ffe85dkDMj0FPRbHK","messageID":"msg_b6b8e702b0012XuEC4bGe0XhKa","type":"tool","callID":"r9bQWsNLvOrJGIOz","tool":"bash","state":{"status":"completed","input":{"command":"echo hello","description":"Print hello to stdout"},"output":"hello\n","title":"Print hello to stdout","metadata":{"output":"hello\n","exit":0,"description":"Print hello to stdout"},"time":{"start":1767036061123,"end":1767036061173}}}}
|
||||
```
|
||||
|
||||
### `text`
|
||||
|
||||
Text output from the model.
|
||||
|
||||
Fields:
|
||||
- `type`: `"text"`
|
||||
- `timestamp`: Unix timestamp in milliseconds
|
||||
- `sessionID`: Session identifier
|
||||
- `part.id`: Part identifier
|
||||
- `part.type`: `"text"`
|
||||
- `part.text`: The actual text content
|
||||
- `part.time.start`: Start timestamp
|
||||
- `part.time.end`: End timestamp
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"text","timestamp":1767036064268,"sessionID":"ses_494719016ffe85dkDMj0FPRbHK","part":{"id":"prt_b6b8e8ff2002mxSx9LtvAlf8Ng","sessionID":"ses_494719016ffe85dkDMj0FPRbHK","messageID":"msg_b6b8e8627001yM4qKJCXdC7W1L","type":"text","text":"```\nhello\n```","time":{"start":1767036064265,"end":1767036064265}}}
|
||||
```
|
||||
|
||||
### `step_finish`
|
||||
|
||||
Marks the end of a processing step.
|
||||
|
||||
Fields:
|
||||
- `type`: `"step_finish"`
|
||||
- `timestamp`: Unix timestamp in milliseconds
|
||||
- `sessionID`: Session identifier
|
||||
- `part.id`: Part identifier
|
||||
- `part.type`: `"step-finish"`
|
||||
- `part.reason`: Optional. `"stop"` (final) or `"tool-calls"` (continuing) when present.
|
||||
- `part.snapshot`: Git snapshot hash
|
||||
- `part.cost`: Cost in USD
|
||||
- `part.tokens.input`: Input token count
|
||||
- `part.tokens.output`: Output token count
|
||||
- `part.tokens.reasoning`: Reasoning token count
|
||||
- `part.tokens.cache.read`: Cache read tokens
|
||||
- `part.tokens.cache.write`: Cache write tokens
|
||||
|
||||
Example (final step):
|
||||
```json
|
||||
{"type":"step_finish","timestamp":1767036064273,"sessionID":"ses_494719016ffe85dkDMj0FPRbHK","part":{"id":"prt_b6b8e9209001ojZ4ECN1geZISm","sessionID":"ses_494719016ffe85dkDMj0FPRbHK","messageID":"msg_b6b8e8627001yM4qKJCXdC7W1L","type":"step-finish","reason":"stop","snapshot":"09dd05d11a4ac013136c1df10932efc0ad9116e8","cost":0.001,"tokens":{"input":671,"output":8,"reasoning":0,"cache":{"read":21415,"write":0}}}}
|
||||
```
|
||||
|
||||
Example (tool-calls step):
|
||||
```json
|
||||
{"type":"step_finish","timestamp":1767036061205,"sessionID":"ses_494719016ffe85dkDMj0FPRbHK","part":{"id":"prt_b6b8e85fb001L4I3WHMqH6EQNI","sessionID":"ses_494719016ffe85dkDMj0FPRbHK","messageID":"msg_b6b8e702b0012XuEC4bGe0XhKa","type":"step-finish","reason":"tool-calls","snapshot":"ee3406d50c7d9048674bbb1a3e325d82513b74ed","cost":0,"tokens":{"input":21772,"output":110,"reasoning":0,"cache":{"read":0,"write":0}}}}
|
||||
```
|
||||
|
||||
### `error`
|
||||
|
||||
Session error event.
|
||||
|
||||
Fields:
|
||||
- `type`: `"error"`
|
||||
- `timestamp`: Unix timestamp in milliseconds
|
||||
- `sessionID`: Session identifier
|
||||
- `error.name`: Error type
|
||||
- `error.data.message`: Human-readable error (when available)
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"error","timestamp":1767036065000,"sessionID":"ses_494719016ffe85dkDMj0FPRbHK","error":{"name":"APIError","data":{"message":"Rate limit exceeded","statusCode":429,"isRetryable":true}}}
|
||||
```
|
||||
|
||||
## Mapping to Takopi Events
|
||||
|
||||
| OpenCode Event | Takopi Event | Condition |
|
||||
|----------------|--------------|-----------|
|
||||
| `step_start` | `StartedEvent` | First occurrence |
|
||||
| `tool_use` | `ActionEvent(phase="completed")` | `status == "completed"` |
|
||||
| `text` | (accumulate text) | - |
|
||||
| `step_finish` | `CompletedEvent` | `reason == "stop"` |
|
||||
| `step_finish` | (ignored) | `reason == "tool-calls"` |
|
||||
| `error` | `CompletedEvent(ok=False)` | - |
|
||||
|
||||
If `step_finish` omits `reason`, Takopi treats a clean process exit as successful completion and emits `CompletedEvent(ok=True)` with accumulated usage.
|
||||
|
||||
## Session ID Format
|
||||
|
||||
OpenCode uses session IDs in the format: `ses_XXXXXXXXXXXXXXXXXXXX`
|
||||
|
||||
Example: `ses_494719016ffe85dkDMj0FPRbHK`
|
||||
|
||||
## Tool Types
|
||||
|
||||
Common tool names in OpenCode:
|
||||
- `bash`: Shell command execution
|
||||
- `read`: Read file contents
|
||||
- `write`: Write file contents
|
||||
- `edit`: Edit file contents
|
||||
- `glob`: File pattern matching
|
||||
- `grep`: Content search
|
||||
- `webfetch`: Fetch web content
|
||||
- `websearch`: Web search
|
||||
- `task`: Spawn sub-agent tasks
|
||||
@@ -0,0 +1,82 @@
|
||||
# OpenCode to Takopi Event Mapping
|
||||
|
||||
This document describes how OpenCode JSON events are translated to Takopi's normalized event model.
|
||||
|
||||
> **Authoritative source:** The schema definitions are in `src/takopi/schemas/opencode.py` and the translation logic is in `src/takopi/runners/opencode.py`. When in doubt, refer to the code.
|
||||
|
||||
## Event Translation
|
||||
|
||||
### StartedEvent
|
||||
|
||||
Emitted on the first `step_start` event that contains a `sessionID`.
|
||||
|
||||
```
|
||||
OpenCode: {"type":"step_start","sessionID":"ses_XXX",...}
|
||||
Takopi: StartedEvent(engine="opencode", resume=ResumeToken(engine="opencode", value="ses_XXX"))
|
||||
```
|
||||
|
||||
### ActionEvent
|
||||
|
||||
Tool usage is translated to action events. Note: `opencode run --format json` currently only emits `tool_use` events when the tool finishes (`status == "completed"`). Pending/running tool states exist in the schema but are not emitted by the CLI JSON stream.
|
||||
|
||||
**Started phase** (when tool is pending/running, if emitted by the JSON stream):
|
||||
```
|
||||
OpenCode: {"type":"tool_use","part":{"tool":"bash","state":{"status":"pending",...}}}
|
||||
Takopi: ActionEvent(engine="opencode", action=Action(kind="command"), phase="started")
|
||||
```
|
||||
|
||||
**Completed phase** (when tool finishes):
|
||||
```
|
||||
OpenCode: {"type":"tool_use","part":{"tool":"bash","state":{"status":"completed","metadata":{"exit":0}}}}
|
||||
Takopi: ActionEvent(engine="opencode", action=Action(kind="command"), phase="completed", ok=True)
|
||||
```
|
||||
|
||||
### CompletedEvent
|
||||
|
||||
Emitted on `step_finish` with `reason="stop"` or on `error` events.
|
||||
|
||||
**Success**:
|
||||
```
|
||||
OpenCode: {"type":"step_finish","part":{"reason":"stop","tokens":{...},"cost":0.001}}
|
||||
Takopi: CompletedEvent(engine="opencode", ok=True, answer="<accumulated text>", usage={...})
|
||||
```
|
||||
|
||||
If `step_finish` omits `reason`, Takopi treats a clean process exit as successful completion and emits `CompletedEvent(ok=True)` with the accumulated usage.
|
||||
|
||||
**Error**:
|
||||
```
|
||||
OpenCode: {"type":"error","error":{"name":"APIError","data":{"message":"API rate limit exceeded"}}}
|
||||
Takopi: CompletedEvent(engine="opencode", ok=False, error="API rate limit exceeded")
|
||||
```
|
||||
|
||||
## Tool Kind Mapping
|
||||
|
||||
| OpenCode Tool | Takopi ActionKind |
|
||||
|---------------|-------------------|
|
||||
| `bash`, `shell` | `command` |
|
||||
| `edit`, `write`, `multiedit` | `file_change` |
|
||||
| `read` | `tool` |
|
||||
| `glob` | `tool` |
|
||||
| `grep` | `tool` |
|
||||
| `websearch`, `web_search` | `web_search` |
|
||||
| `webfetch`, `web_fetch` | `web_search` |
|
||||
| `todowrite`, `todoread` | `note` |
|
||||
| `task` | `tool` |
|
||||
| (other) | `tool` |
|
||||
|
||||
## Usage Accumulation
|
||||
|
||||
Token usage is accumulated across all `step_finish` events and reported in the final `CompletedEvent.usage`:
|
||||
|
||||
```json
|
||||
{
|
||||
"total_cost_usd": 0.001,
|
||||
"tokens": {
|
||||
"input": 22443,
|
||||
"output": 118,
|
||||
"reasoning": 0,
|
||||
"cache_read": 21415,
|
||||
"cache_write": 0
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -0,0 +1,133 @@
|
||||
Below is a concrete implementation spec for the **Pi (pi-coding-agent CLI)** runner shipped in Takopi (v0.5.0).
|
||||
|
||||
---
|
||||
|
||||
## Scope
|
||||
|
||||
### Goal
|
||||
|
||||
Provide the **`pi`** engine backend so Takopi can:
|
||||
|
||||
* Run Pi non-interactively via the **pi CLI** (`pi --print`).
|
||||
* Stream progress by parsing **`--mode json`** (newline-delimited JSON). Each line is a JSON object.
|
||||
* Support resumable sessions via **`--session <path>`** (Takopi emits a canonical resume line the user can reply with).
|
||||
|
||||
### Non-goals (v1)
|
||||
|
||||
* Interactive TUI flows (session picker, prompts, etc.)
|
||||
* RPC mode (requires a long-running process and JSON commands)
|
||||
|
||||
---
|
||||
|
||||
## UX and behavior
|
||||
|
||||
### Engine selection
|
||||
|
||||
* Default: `takopi` (auto-router uses `default_engine` from config)
|
||||
* Override: `takopi pi`
|
||||
|
||||
### Resume UX (canonical line)
|
||||
|
||||
Takopi appends a **single backticked** resume line at the end of the message, like:
|
||||
|
||||
```text
|
||||
`pi --session ccd569e0`
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
* `pi --resume/-r` opens an interactive session picker, so Takopi uses `--session <path>` instead.
|
||||
* The resume token is the **session id** (short prefix), derived from the first JSON
|
||||
object in the session file. If the id cannot be read, Takopi falls back to the
|
||||
session file path.
|
||||
* If the path contains spaces, the runner will quote it.
|
||||
|
||||
### Non-interactive runs
|
||||
|
||||
Use `--print` and `--mode json` for headless JSONL output.
|
||||
|
||||
Pi does not accept `-- <prompt>` to protect prompts starting with `-`. Takopi prefixes a leading space if the prompt begins with `-` so it is not parsed as a flag.
|
||||
|
||||
---
|
||||
|
||||
## Config additions
|
||||
|
||||
Takopi config lives at `~/.takopi/takopi.toml`.
|
||||
|
||||
Add a new optional `[pi]` section.
|
||||
|
||||
Recommended schema:
|
||||
|
||||
```toml
|
||||
# ~/.takopi/takopi.toml
|
||||
|
||||
default_engine = "pi"
|
||||
|
||||
[pi]
|
||||
model = "..." # optional; passed as --model
|
||||
provider = "..." # optional; passed as --provider
|
||||
extra_args = [] # optional list of strings, appended verbatim
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
* `extra_args` lets you pass new Pi flags without changing Takopi.
|
||||
* Session files are stored under Pi's default session dir:
|
||||
`~/.pi/agent/sessions/--<cwd>--` (with path separators replaced by `-`).
|
||||
|
||||
---
|
||||
|
||||
## Code changes (by file)
|
||||
|
||||
### 1) New file: `src/takopi/runners/pi.py`
|
||||
|
||||
Expose a module-level `BACKEND = EngineBackend(...)`.
|
||||
|
||||
#### Runner invocation
|
||||
|
||||
The runner should launch Pi in headless JSON mode:
|
||||
|
||||
```text
|
||||
pi --print --mode json --session <session.jsonl> <prompt>
|
||||
```
|
||||
|
||||
When resuming, `<session.jsonl>` is the resume token extracted from the chat.
|
||||
|
||||
#### Event translation
|
||||
|
||||
Pi JSONL output is `AgentSessionEvent` (from `@mariozechner/pi-agent-core`).
|
||||
The runner should translate:
|
||||
|
||||
* `tool_execution_start` -> `action` (phase: started)
|
||||
* `tool_execution_end` -> `action` (phase: completed)
|
||||
* `agent_end` -> `completed`
|
||||
|
||||
For the final answer, use the most recent assistant message text (from
|
||||
`message_end` events). For errors, if the assistant stopReason is `error` or
|
||||
`aborted`, emit `completed(ok=false, error=...)`.
|
||||
|
||||
---
|
||||
|
||||
## Installation and auth
|
||||
|
||||
Install the CLI globally:
|
||||
|
||||
```text
|
||||
npm install -g @mariozechner/pi-coding-agent
|
||||
```
|
||||
|
||||
Auth is stored under `~/.pi/agent/auth.json`. Run `pi` once interactively to
|
||||
set up credentials before using Takopi.
|
||||
|
||||
---
|
||||
|
||||
## Known pitfalls
|
||||
|
||||
* `--resume` is interactive; Takopi uses `--session <path>` instead.
|
||||
* Prompts that start with `-` are interpreted as flags by the CLI. Takopi
|
||||
prefixes a space to make them safe.
|
||||
|
||||
---
|
||||
|
||||
If you want, I can also add a sample `takopi.toml` snippet to the README or
|
||||
include a small quickstart section for Pi in the onboarding panel.
|
||||
@@ -0,0 +1,67 @@
|
||||
# Pi `--mode json` event cheatsheet
|
||||
|
||||
`pi --print --mode json` writes **one JSON object per line** (JSONL) with a
|
||||
required `type` field. These are `AgentSessionEvent` objects from
|
||||
`@mariozechner/pi-agent-core`.
|
||||
|
||||
## Top-level event lines
|
||||
|
||||
### `agent_start`
|
||||
|
||||
```json
|
||||
{"type":"agent_start"}
|
||||
```
|
||||
|
||||
### `agent_end`
|
||||
|
||||
```json
|
||||
{"type":"agent_end","messages":[{"role":"assistant","content":[{"type":"text","text":"Done."}],"stopReason":"stop","timestamp":123}]}
|
||||
```
|
||||
|
||||
### `turn_start` / `turn_end`
|
||||
|
||||
```json
|
||||
{"type":"turn_start"}
|
||||
```
|
||||
|
||||
```json
|
||||
{"type":"turn_end","message":{...},"toolResults":[...]}
|
||||
```
|
||||
|
||||
### `message_start` / `message_update` / `message_end`
|
||||
|
||||
```json
|
||||
{"type":"message_start","message":{"role":"assistant","content":[{"type":"text","text":"Working..."}]}}
|
||||
```
|
||||
|
||||
```json
|
||||
{"type":"message_update","message":{...},"assistantMessageEvent":{"type":"text_delta","delta":"...","contentIndex":0}}
|
||||
```
|
||||
|
||||
```json
|
||||
{"type":"message_end","message":{"role":"assistant","content":[{"type":"text","text":"Done."}],"stopReason":"stop"}}
|
||||
```
|
||||
|
||||
### `tool_execution_start`
|
||||
|
||||
```json
|
||||
{"type":"tool_execution_start","toolCallId":"tool_1","toolName":"bash","args":{"command":"ls"}}
|
||||
```
|
||||
|
||||
### `tool_execution_update`
|
||||
|
||||
```json
|
||||
{"type":"tool_execution_update","toolCallId":"tool_1","toolName":"bash","args":{"command":"ls"},"partialResult":{"content":[{"type":"text","text":"..."}]}}
|
||||
```
|
||||
|
||||
### `tool_execution_end`
|
||||
|
||||
```json
|
||||
{"type":"tool_execution_end","toolCallId":"tool_1","toolName":"bash","result":{"content":[{"type":"text","text":"ok"}],"details":{}},"isError":false}
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
* `message_end` with `role = "assistant"` contains the final assistant text.
|
||||
* `assistantMessageEvent` in `message_update` provides streaming deltas.
|
||||
* `tool_execution_*` events map cleanly to Takopi `action` events.
|
||||
@@ -0,0 +1,152 @@
|
||||
# Pi -> Takopi event mapping (spec)
|
||||
|
||||
This document describes how the Pi runner translates Pi CLI `--mode json` JSONL events into Takopi events.
|
||||
|
||||
> **Authoritative source:** The schema definitions are in `src/takopi/schemas/pi.py` and the translation logic is in `src/takopi/runners/pi.py`. When in doubt, refer to the code.
|
||||
|
||||
The goal is to make Pi feel identical to the Codex/Claude runners from the bridge/renderer point of view while preserving Takopi invariants (stable action ids, per-session serialization, single completed event).
|
||||
|
||||
---
|
||||
|
||||
## 1. Input stream contract (Pi CLI)
|
||||
|
||||
Pi CLI emits **one JSON object per line** (JSONL) when invoked with:
|
||||
|
||||
```
|
||||
pi --print --mode json <prompt>
|
||||
```
|
||||
|
||||
Notes:
|
||||
- `--print` is required for non-interactive runs.
|
||||
- `--mode json` outputs all agent events (no TUI banners).
|
||||
- Pi does not support `-- <prompt>`; prompts starting with `-` must be
|
||||
prefixed (Takopi does this automatically).
|
||||
|
||||
---
|
||||
|
||||
## 2. Resume tokens and resume lines
|
||||
|
||||
- Engine id: `pi`
|
||||
- Canonical resume line (embedded in chat):
|
||||
|
||||
```
|
||||
`pi --session <id>`
|
||||
```
|
||||
|
||||
The token is the **short session id**, derived from the first JSON object in the
|
||||
session file. If the id cannot be read, Takopi falls back to the session file path.
|
||||
|
||||
Why not `--resume`?
|
||||
- `--resume/-r` opens an interactive session picker; it does not accept a
|
||||
session token. Takopi must use `--session <path>` instead.
|
||||
|
||||
---
|
||||
|
||||
## 3. Session lifecycle + serialization
|
||||
|
||||
Takopi requires **serialization per session token**:
|
||||
|
||||
- For new runs (`resume=None`), do **not** acquire a lock until a `started`
|
||||
event is emitted (Takopi emits this as soon as the first JSON event arrives).
|
||||
- Once the session is known, acquire a lock for `pi:<session_path>` and hold it
|
||||
until the run completes.
|
||||
- For resumed runs, acquire the lock immediately on entry.
|
||||
|
||||
---
|
||||
|
||||
## 4. Event translation (Pi JSONL -> Takopi)
|
||||
|
||||
Pi emits `AgentSessionEvent` objects. Only a subset is required for Takopi.
|
||||
|
||||
### 4.1 `tool_execution_start`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"tool_execution_start","toolCallId":"tool_1","toolName":"bash","args":{"command":"ls"}}
|
||||
```
|
||||
|
||||
Mapping:
|
||||
- Emit `action` with `phase="started"`.
|
||||
- `action.id = toolCallId`.
|
||||
- `action.kind` from tool name (see section 5).
|
||||
- `action.title` derived from tool + args.
|
||||
|
||||
### 4.2 `tool_execution_end`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"tool_execution_end","toolCallId":"tool_1","toolName":"bash","result":{...},"isError":false}
|
||||
```
|
||||
|
||||
Mapping:
|
||||
- Emit `action` with `phase="completed"`.
|
||||
- `ok = !isError`.
|
||||
- Carry `result` and `isError` in `detail` for debugging.
|
||||
|
||||
### 4.3 `message_end` (assistant)
|
||||
|
||||
Pi emits message lifecycle events. For `message_end` where `message.role == "assistant"`:
|
||||
|
||||
- Store the latest assistant text as the **final answer fallback**.
|
||||
- If `stopReason` is `error` or `aborted`, store `errorMessage`.
|
||||
- Capture `usage` for `completed.usage`.
|
||||
|
||||
### 4.4 `agent_end`
|
||||
|
||||
Example:
|
||||
```json
|
||||
{"type":"agent_end","messages":[...]}
|
||||
```
|
||||
|
||||
Mapping:
|
||||
- Emit a single `completed` event:
|
||||
- `ok = true` unless the last assistant message has `stopReason` `error` or `aborted`.
|
||||
- `answer = last assistant text` (from `message_end` or `agent_end.messages`).
|
||||
- `error = errorMessage` if present.
|
||||
- `resume = ResumeToken(engine="pi", value=session_path)`.
|
||||
- `usage = last assistant usage`.
|
||||
|
||||
### 4.5 Other events
|
||||
|
||||
Ignore unknown events. If a JSONL line is malformed, emit a warning action and
|
||||
continue (default `JsonlSubprocessRunner` behavior).
|
||||
|
||||
---
|
||||
|
||||
## 5. Tool name -> ActionKind mapping heuristics
|
||||
|
||||
Pi tool names are lower-case by default. Suggested mapping:
|
||||
|
||||
| Tool name | ActionKind | Title logic |
|
||||
| --- | --- | --- |
|
||||
| `bash` | `command` | `args.command` |
|
||||
| `edit`, `write` | `file_change` | `args.path` |
|
||||
| `read` | `tool` | `read: <path>` |
|
||||
| `grep` | `tool` | `grep: <pattern>` |
|
||||
| `find` | `tool` | `find: <pattern>` |
|
||||
| `ls` | `tool` | `ls: <path>` |
|
||||
| (default) | `tool` | tool name |
|
||||
|
||||
For `file_change`, include `detail.changes = [{"path": <path>, "kind": "update"}]`.
|
||||
|
||||
---
|
||||
|
||||
## 6. Usage mapping
|
||||
|
||||
Takopi `completed.usage` should mirror Pi's assistant `usage` object without
|
||||
transformation.
|
||||
|
||||
---
|
||||
|
||||
## 7. Suggested Takopi config keys
|
||||
|
||||
A minimal TOML config for Pi:
|
||||
|
||||
```toml
|
||||
[pi]
|
||||
model = "..."
|
||||
provider = "..."
|
||||
extra_args = []
|
||||
```
|
||||
|
||||
Use `extra_args` for any Pi CLI flags not explicitly mapped.
|
||||
@@ -0,0 +1,518 @@
|
||||
# Takopi Specification v0.17.1 [2026-01-12]
|
||||
|
||||
This document is **normative**. The words **MUST**, **SHOULD**, and **MAY** express requirements.
|
||||
|
||||
## 1. Scope
|
||||
|
||||
Takopi v0.17.1 specifies:
|
||||
|
||||
- A **Telegram** bot bridge that runs an agent **Runner** and posts:
|
||||
- a throttled, edited **progress message**
|
||||
- a **final message** with the final answer and a resume line
|
||||
- **Thread continuation** via a **resume command** embedded in chat messages
|
||||
- **Parallel runs across different threads**
|
||||
- **Serialization within a thread** (no concurrent runs on the same thread)
|
||||
- **Automatic runner selection** among multiple engines based on ResumeLine (with a configurable default for new threads)
|
||||
- A Takopi-owned **normalized event model** produced by runners and consumed by renderers/bridge
|
||||
|
||||
Out of scope for v0.17.1:
|
||||
|
||||
- Non-Telegram clients (Slack/Discord/etc.)
|
||||
- Token-by-token streaming of the assistant’s final answer
|
||||
- Engines/runners that cannot provide **stable action IDs** within a run
|
||||
|
||||
## 2. Terminology
|
||||
|
||||
- **EngineId**: string identifier of an engine (e.g., `"codex"`, `"claude"`, `"pi"`).
|
||||
- **Runner**: Takopi adapter that executes an engine process and yields **Takopi events**.
|
||||
- **Thread**: a single engine-side conversation, identified in Takopi by a **ResumeToken**.
|
||||
- **ResumeToken**: Takopi-owned thread identifier `{ engine: EngineId, value: str }`.
|
||||
- **ResumeLine**: a runner-owned string embedded in chat that represents a ResumeToken.
|
||||
- **Run**: a single invocation of `Runner.run(prompt, resume)`.
|
||||
- **TakopiEvent**: a normalized event emitted by a runner and consumed by renderers/bridge.
|
||||
- **Progress message**: a Telegram message that is periodically edited during a run.
|
||||
- **Final message**: a Telegram message that includes run status, final answer, and resume line.
|
||||
|
||||
## 3. Resume tokens and resume lines
|
||||
|
||||
### 3.1 Decision: canonical resume line is the engine CLI resume command
|
||||
|
||||
The canonical ResumeLine embedded in chat MUST be the engine’s CLI resume command, e.g.:
|
||||
|
||||
- `codex resume <id>`
|
||||
- `claude --resume <id>`
|
||||
- `pi --session <token>`
|
||||
|
||||
ResumeLine MUST resume the interactive session when the engine offers both interactive and headless modes. It MUST NOT point to a headless/batch command that requires a new prompt (e.g., a `run` subcommand that errors without a message).
|
||||
|
||||
Takopi MUST treat the runner as authoritative for:
|
||||
|
||||
- formatting a ResumeToken into a ResumeLine
|
||||
- extracting a ResumeToken from message text
|
||||
|
||||
### 3.2 ResumeToken schema (Takopi-owned)
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class ResumeToken:
|
||||
engine: str # EngineId
|
||||
value: str
|
||||
```
|
||||
|
||||
### 3.3 Runner resume codec (MUST)
|
||||
|
||||
Each runner MUST implement:
|
||||
|
||||
* `format_resume(token: ResumeToken) -> str`
|
||||
* `extract_resume(text: str) -> ResumeToken | None`
|
||||
* `is_resume_line(line: str) -> bool`
|
||||
|
||||
Constraints:
|
||||
|
||||
* `format_resume()` MUST fail if `token.engine != runner.engine`.
|
||||
* `extract_resume()` MUST return `None` if it cannot **confidently** parse a resume line for its engine.
|
||||
|
||||
### 3.4 Bridge resume resolution (MUST)
|
||||
|
||||
Given `text` (user message), optional `reply_text` (the message being replied to), and an ordered list of available runners `runners`:
|
||||
|
||||
1. The bridge MUST attempt to extract a resume token by polling all runners in order:
|
||||
1. for each `r` in `runners`, attempt `r.extract_resume(text)`
|
||||
2. choose the **first** runner that returns a non-`None` token and stop
|
||||
2. If not found, it MUST repeat step (1) for `reply_text` if present.
|
||||
3. If still not found, the run MUST start with `resume=None` (new thread) on the default runner (per §8, including chat-level overrides).
|
||||
|
||||
## 4. Normalized event model
|
||||
|
||||
### 4.1 Decision: events are trusted after normalization
|
||||
|
||||
Runners are responsible for emitting well-formed Takopi events. Consumers (renderer/bridge) SHOULD assume validity and MAY fail fast on invariant violations.
|
||||
|
||||
### 4.2 Supported event types (minimum set)
|
||||
|
||||
Takopi MUST support:
|
||||
|
||||
* `started`
|
||||
* `action`
|
||||
* `completed`
|
||||
|
||||
Minimal runner mode is supported:
|
||||
|
||||
* A runner MAY emit only `started` and `completed`.
|
||||
* If `action` events are emitted, `phase="completed"` alone is valid (no requirement to emit `started`/`updated` phases).
|
||||
|
||||
### 4.3 Event schemas
|
||||
|
||||
All events MUST include `engine: EngineId` and `type`.
|
||||
|
||||
#### 4.3.1 `started`
|
||||
|
||||
Required:
|
||||
|
||||
* `type: "started"`
|
||||
* `engine: EngineId`
|
||||
* `resume: ResumeToken`
|
||||
|
||||
Optional:
|
||||
|
||||
* `title: str`
|
||||
* `meta: dict`
|
||||
|
||||
#### 4.3.2 `action`
|
||||
|
||||
Required:
|
||||
|
||||
* `type: "action"`
|
||||
* `engine: EngineId`
|
||||
* `action: Action`
|
||||
* `phase: "started" | "updated" | "completed"`
|
||||
|
||||
Optional:
|
||||
|
||||
* `ok: bool` (typically on `phase="completed"`)
|
||||
* `message: str`
|
||||
* `level: "debug" | "info" | "warning" | "error"`
|
||||
|
||||
Notes:
|
||||
|
||||
* `phase="completed"` alone is valid.
|
||||
|
||||
#### 4.3.3 `completed`
|
||||
|
||||
Required:
|
||||
|
||||
* `type: "completed"`
|
||||
* `engine: EngineId`
|
||||
* `ok: bool` (overall run success/failure)
|
||||
* `answer: str` (final assistant answer; MAY be empty)
|
||||
|
||||
Optional:
|
||||
|
||||
* `resume: ResumeToken` (final token; new or existing, if known)
|
||||
* `error: str | None` (fatal error message, if any)
|
||||
* `usage: dict` (telemetry/usage if available)
|
||||
|
||||
### 4.4 Action schema (MUST; stable IDs)
|
||||
|
||||
Actions MUST have stable IDs within a run:
|
||||
|
||||
```python
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Action:
|
||||
id: str
|
||||
kind: str
|
||||
title: str
|
||||
detail: dict[str, Any]
|
||||
```
|
||||
|
||||
Stability requirements:
|
||||
|
||||
* Within a single run, the same underlying action MUST keep the same `Action.id` across events.
|
||||
* `Action.id` values MUST be unique within a run.
|
||||
* IDs do **not** need to be stable across different runs/resumes.
|
||||
|
||||
Action kinds SHOULD come from an extensible stable set, e.g.:
|
||||
|
||||
* `command`, `tool`, `file_change`, `web_search`, `subagent`, `turn`, `warning`, `telemetry`, `note`
|
||||
|
||||
Unknown kinds MAY be rendered as `note`.
|
||||
|
||||
`detail` is freeform; no per-kind schema is required.
|
||||
|
||||
`ok` semantics are runner-defined.
|
||||
|
||||
User-visible warnings/errors SHOULD be surfaced as `action` events (typically `kind="warning"` or `kind="note"`, `phase="completed"`, `ok=False`) rather than introducing new event types.
|
||||
|
||||
## 5. Runner protocol and concurrency
|
||||
|
||||
### 5.1 Runner protocol (MUST)
|
||||
|
||||
```python
|
||||
class Runner(Protocol):
|
||||
engine: str # EngineId
|
||||
|
||||
def run(
|
||||
self,
|
||||
prompt: str,
|
||||
resume: ResumeToken | None,
|
||||
) -> AsyncIterator[TakopiEvent]: ...
|
||||
```
|
||||
|
||||
### 5.2 Per-thread serialization (MUST; core invariant)
|
||||
|
||||
Define:
|
||||
|
||||
* `ThreadKey(resume) := f"{resume.engine}:{resume.value}"`
|
||||
|
||||
Invariant:
|
||||
|
||||
* At most **one** active run may operate on the same `ThreadKey` at a time.
|
||||
|
||||
Rules:
|
||||
|
||||
* Runs for different ThreadKeys MAY run in parallel.
|
||||
* Runs for the same ThreadKey MUST be queued and executed sequentially.
|
||||
* This invariant MUST be enforced by the runner implementation even if used outside the Telegram bridge.
|
||||
|
||||
New thread rule (`resume is None`):
|
||||
|
||||
* When the runner learns the new thread’s ResumeToken, it MUST:
|
||||
|
||||
* acquire the per-thread lock for that token
|
||||
* do so **before emitting** `started(resume=token)`
|
||||
|
||||
### 5.3 `started` emission and ordering
|
||||
|
||||
* If the runner obtains a ResumeToken for the run, it MUST emit exactly one `started` event containing that token.
|
||||
* The runner MAY emit `action` events before `started` (e.g., pre-init warnings). Consumers MUST NOT assume `started` is the first event.
|
||||
|
||||
### 5.4 Completion
|
||||
|
||||
* If the run reaches `started`, and then terminates under the runner’s control (success or detected failure), the runner MUST emit exactly one `completed` event and it MUST be the last event.
|
||||
* If the runner never obtains a ResumeToken (e.g., fatal failure before session init), it MAY emit no `started` and no `completed`.
|
||||
|
||||
### 5.5 Event delivery semantics (MUST)
|
||||
|
||||
* Events MUST be yielded in the order produced by the runner.
|
||||
* The runner MUST NOT spawn unbounded background tasks per event.
|
||||
* If the consumer stops iterating early (cancel/break/exception), the runner MUST abort the run best-effort and release any held locks/resources.
|
||||
|
||||
## 6. Bridge (Telegram orchestration)
|
||||
|
||||
### 6.1 Responsibilities (MUST)
|
||||
|
||||
The bridge MUST:
|
||||
|
||||
* Receive Telegram updates
|
||||
* Resolve resume token (per §3.4)
|
||||
* Schedule runs per thread (per §6.2)
|
||||
* Start runner execution with cancellation support
|
||||
* Maintain a progress message while avoiding excessive edits
|
||||
* Publish a final message containing status, answer, and resume line (when known)
|
||||
* Support `/cancel` for in-flight runs
|
||||
|
||||
The bridge MUST NOT:
|
||||
|
||||
* parse engine-native streams/events
|
||||
* embed engine-specific rules beyond calling runner resume extraction/formatting
|
||||
|
||||
Queue depth:
|
||||
|
||||
* There is no queue depth limit; all prompts are accepted.
|
||||
|
||||
### 6.2 Scheduling (MUST)
|
||||
|
||||
Definitions:
|
||||
|
||||
* `Job := (chat_id, user_msg_id, text, resume: ResumeToken | None)`
|
||||
|
||||
Required behavior:
|
||||
|
||||
* For `resume != None`, the bridge MUST enqueue jobs into `pending_by_thread[ThreadKey(resume)]`.
|
||||
* For each ThreadKey, exactly one worker (or equivalent mechanism) MUST drain the queue sequentially.
|
||||
* A worker MUST exit when its queue is empty; the bridge SHOULD avoid retaining state for inactive threads.
|
||||
* The implementation MUST avoid spawning one long-lived task per queued job (bounded concurrency).
|
||||
|
||||
Runs that start as new threads:
|
||||
|
||||
* If a job starts with `resume=None` and later yields `started(resume=token)`, the bridge MUST treat that run as the in-flight job for `ThreadKey(token)` until it completes (for scheduling and cancellation routing).
|
||||
|
||||
### 6.3 Progress message behavior
|
||||
|
||||
* The bridge SHOULD send an initial progress message quickly (e.g., “Running…”).
|
||||
* The bridge SHOULD avoid excessive edits and respect transport constraints (implementation-defined).
|
||||
* The bridge SHOULD skip edits when rendered content is unchanged.
|
||||
* Once `started` is observed, the progress view SHOULD include the canonical ResumeLine.
|
||||
|
||||
### 6.4 Final message requirements (MUST)
|
||||
|
||||
The final output MUST include:
|
||||
|
||||
* a status line (`done` / `error` / `cancelled`)
|
||||
* the final `answer` (if any)
|
||||
* the ResumeLine if known (and MUST include it if `started` was received)
|
||||
|
||||
### 6.5 Cancellation `/cancel` (MUST)
|
||||
|
||||
* The bridge MUST allow users to cancel a run in progress by sending `/cancel` in reply to the progress message (or by an equivalent mapping defined by the bridge).
|
||||
* Cancellation MUST terminate the runner process via **SIGTERM**.
|
||||
* After cancellation, the bridge MUST stop further progress edits and publish a “cancelled” status message.
|
||||
* The bridge SHOULD include the ResumeLine if known.
|
||||
* Any additional text after `/cancel` is ignored.
|
||||
|
||||
### 6.6 Telegram markdown + truncation (MUST)
|
||||
|
||||
The bridge MUST:
|
||||
|
||||
* escape/prepare Telegram markdown correctly
|
||||
* enforce Telegram message length limits (including after escaping)
|
||||
* avoid truncating away the ResumeLine (using `runner.is_resume_line()`)
|
||||
|
||||
If truncation is required:
|
||||
|
||||
* the bridge MUST keep the ResumeLine intact
|
||||
* the bridge SHOULD preserve the beginning of the content and insert an ellipsis at the truncation point
|
||||
|
||||
### 6.7 Crash/error handling (MUST)
|
||||
|
||||
If the runner crashes or exits uncleanly:
|
||||
|
||||
* the bridge MUST publish an error status message
|
||||
* if `started` was received, the bridge MUST include the ResumeLine in that error message
|
||||
|
||||
## 7. Renderer
|
||||
|
||||
Renderers MUST:
|
||||
|
||||
* be deterministic functions/state machines over Takopi events + internal renderer state
|
||||
* produce Telegram-ready markdown (or markdown + entities)
|
||||
* tolerate `action` events that are “completed-only” (no prior `started`/`updated`)
|
||||
|
||||
Renderers MUST NOT:
|
||||
|
||||
* depend on engine-native event formats
|
||||
* call Telegram APIs
|
||||
* perform blocking I/O
|
||||
|
||||
Action update collapsing:
|
||||
|
||||
* If multiple `action` events share the same `Action.id`, renderers SHOULD treat later `started`/`updated` events as updates (replace the prior running line rather than appending).
|
||||
|
||||
## 8. Configuration and engine selection
|
||||
|
||||
Decision (v0.4.0):
|
||||
|
||||
* Takopi MUST support configuring a **default engine** used to start new threads (`resume=None`).
|
||||
* If not configured, the default engine is implementation-defined (non-normative: the reference implementation defaults to `codex`).
|
||||
* If no engine subcommand is provided, Takopi MUST run in **auto-router** mode:
|
||||
* new threads use the configured default engine
|
||||
* resumed threads are routed based on ResumeLine extraction (per §3.4)
|
||||
* If an engine subcommand is provided, Takopi MUST still use the auto-router, but it overrides the configured default engine for new threads.
|
||||
* Resume extraction MUST poll **all** available runners (per §3.4) and route to the first matching runner.
|
||||
* New thread engine override (chat-level):
|
||||
* Users MAY prefix the first non-empty line with `/{engine}` (e.g. `/claude`, `/codex`, or `/pi`) to select the engine for a **new** thread.
|
||||
* The bridge MUST strip that directive from the prompt before invoking the runner.
|
||||
* If a ResumeToken is resolved from the message or reply, it MUST take precedence and the `/{engine}` directive MUST be ignored.
|
||||
* Bridges MAY persist default engine overrides per Telegram scope:
|
||||
* **Topic default**: forum topic (`chat_id + thread_id`)
|
||||
* **Chat default**: chat (`chat_id`)
|
||||
* When no ResumeToken is resolved, engine selection MUST follow this precedence:
|
||||
1) explicit `/{engine}` directive
|
||||
2) topic default (if any)
|
||||
3) chat default (if any)
|
||||
4) project default engine (if configured for the resolved context)
|
||||
5) global default engine
|
||||
|
||||
### 8.1 Command menu (Telegram)
|
||||
|
||||
Takopi SHOULD keep the bot’s slash-command menu in sync at startup by calling
|
||||
`setMyCommands` with the canonical list of supported commands.
|
||||
|
||||
* The command list MUST include:
|
||||
* `cancel` — cancel the current run
|
||||
* one entry per configured engine
|
||||
* one entry per configured project alias that is a valid Telegram command
|
||||
* The command list MUST NOT include commands the bot does not support.
|
||||
* Command descriptions SHOULD be terse and lowercase.
|
||||
* The command list SHOULD be capped at 100 entries per Telegram's limit; if the
|
||||
config exceeds that limit, implementations SHOULD warn and truncate while
|
||||
still handling all commands at runtime.
|
||||
|
||||
## 9. Testing requirements (MUST)
|
||||
|
||||
Tests MUST cover:
|
||||
|
||||
1. **Runner contract**
|
||||
|
||||
* If a token is obtained: exactly one `started`
|
||||
* Action schema validity (required fields; stable unique IDs within run)
|
||||
* Event ordering preserved
|
||||
* `completed` emitted and last for controlled termination after `started`
|
||||
2. **Runner serialization**
|
||||
|
||||
* Concurrent runs for the same ResumeToken serialize
|
||||
* `resume=None` runs acquire the per-thread lock once token is known and before emitting `started`
|
||||
3. **Bridge per-thread scheduling**
|
||||
|
||||
* FIFO per ThreadKey
|
||||
* second job for same thread does not start until first completes
|
||||
4. **Progress throttling**
|
||||
|
||||
* edits not more frequent than configured interval
|
||||
* no edit when content unchanged
|
||||
* truncation preserves ResumeLine
|
||||
5. **Cancellation**
|
||||
|
||||
* `/cancel` terminates run and produces “cancelled”
|
||||
* ResumeLine included if known
|
||||
6. **Renderer formatting**
|
||||
|
||||
* completed-only actions render correctly
|
||||
* repeated events for same Action.id collapse as intended
|
||||
7. **Auto-router engine selection**
|
||||
|
||||
* resume lines for non-default engines are detected and routed correctly (poll all runners)
|
||||
* new threads use the configured default engine, with CLI subcommand overriding it
|
||||
|
||||
Test tooling SHOULD include event factories, deterministic/fake time, and a script/mock runner.
|
||||
|
||||
## 10. Lockfile (single-instance enforcement)
|
||||
|
||||
Takopi MUST prevent multiple instances from racing `getUpdates` offsets for the same bot token.
|
||||
|
||||
### 10.1 Lock file location
|
||||
|
||||
The lock file MUST be stored at `<config_path>.lock`. For the default config path, this resolves to `~/.takopi/takopi.lock`.
|
||||
|
||||
### 10.2 Lock file format
|
||||
|
||||
The lock file MUST contain JSON with:
|
||||
|
||||
* `pid: int` — the process ID holding the lock
|
||||
* `token_fingerprint: str` — SHA256 hash of the bot token, truncated to 10 characters
|
||||
|
||||
### 10.3 Lock acquisition rules
|
||||
|
||||
* If the lock file does not exist, acquire and write the lock.
|
||||
* If the lock file exists and the PID is dead (not running), replace the lock.
|
||||
* If the lock file exists and the token fingerprint differs (different bot), replace the lock.
|
||||
* If the lock file exists, the PID is alive, and the fingerprint matches, fail with an error instructing the user to stop the other instance.
|
||||
|
||||
### 10.4 Lock release
|
||||
|
||||
The lock file SHOULD be removed on clean shutdown. Stale locks from crashed processes are handled by the acquisition rules above.
|
||||
|
||||
## 11. Changelog
|
||||
|
||||
### v0.17.1 (2026-01-12)
|
||||
|
||||
- No normative changes; align spec version with the v0.17.1 release.
|
||||
|
||||
### v0.17.0 (2026-01-12)
|
||||
|
||||
- No normative changes; align spec version with the v0.17.0 release.
|
||||
|
||||
### v0.16.0 (2026-01-12)
|
||||
|
||||
- No normative changes; align spec version with the v0.16.0 release.
|
||||
|
||||
### v0.15.0 (2026-01-11)
|
||||
|
||||
- No normative changes; align spec version with the v0.15.0 release.
|
||||
|
||||
### v0.14.1 (2026-01-10)
|
||||
|
||||
- No normative changes; align spec version with the v0.14.1 release.
|
||||
|
||||
### v0.14.0 (2026-01-10)
|
||||
|
||||
- No normative changes; align spec version with the v0.14.0 release.
|
||||
|
||||
### v0.13.0 (2026-01-09)
|
||||
|
||||
- No normative changes; align spec version with the v0.13.0 release.
|
||||
|
||||
### v0.12.0 (2026-01-09)
|
||||
|
||||
- No normative changes; align spec version with the v0.12.0 release.
|
||||
|
||||
### v0.11.0 (2026-01-08)
|
||||
|
||||
- No normative changes; align spec version with the v0.11.0 release.
|
||||
|
||||
### v0.10.0 (2026-01-08)
|
||||
|
||||
- Require Telegram command menus to include valid project aliases and warn/truncate when exceeding 100 commands.
|
||||
|
||||
### v0.9.0 (2026-01-07)
|
||||
|
||||
- No normative changes; align spec version with the v0.9.0 release.
|
||||
|
||||
### v0.8.0 (2026-01-05)
|
||||
|
||||
- Add `subagent` action kind for agent/task delegation tools.
|
||||
- Add lockfile specification for single-instance enforcement (§10).
|
||||
|
||||
### v0.7.0 (2026-01-04)
|
||||
|
||||
- No normative changes; implementation migrated to structlog and msgspec schemas.
|
||||
|
||||
### v0.6.0 (2026-01-03)
|
||||
|
||||
- No normative changes; added interactive onboarding and lockfile implementation.
|
||||
|
||||
### v0.5.0 (2026-01-02)
|
||||
|
||||
- No normative changes; align spec version with the v0.5.0 release.
|
||||
|
||||
### v0.4.0 (2026-01-01)
|
||||
|
||||
- Add auto-router engine selection by polling all runners to decode resume lines; add configurable default engine for new threads (subcommand overrides default).
|
||||
|
||||
### v0.3.0 (2026-01-01)
|
||||
|
||||
- Require runners to implement explicit resume formatting/extraction/detection and treat runners as authoritative for resume tokens/lines.
|
||||
|
||||
### v0.2.0 (2025-12-31)
|
||||
|
||||
- Initial minimal Takopi specification (Telegram bridge + runner protocol + normalized events + resume support).
|
||||
@@ -0,0 +1,174 @@
|
||||
# Telegram Transport
|
||||
|
||||
## Overview
|
||||
|
||||
`TelegramClient` is the single transport for Telegram writes. It owns a
|
||||
`TelegramOutbox` that serializes send/edit/delete operations, applies
|
||||
coalescing, and enforces rate limits + retry-after backoff.
|
||||
|
||||
This document captures current behavior so transport changes stay intentional.
|
||||
|
||||
## Flow
|
||||
|
||||
1. Engine CLI emits JSONL events.
|
||||
2. We render progress on every step and diff against the last output.
|
||||
3. Only deltas enqueue a Telegram edit.
|
||||
4. High-value messages enqueue a send.
|
||||
5. All writes go through the outbox.
|
||||
|
||||
## Incoming messages
|
||||
|
||||
`parse_incoming_update` accepts text messages and voice notes.
|
||||
|
||||
If voice transcription is enabled, takopi downloads the voice payload from Telegram,
|
||||
transcribes it with OpenAI, and routes the transcript through the same command and
|
||||
directive pipeline as typed text.
|
||||
|
||||
Configuration (under `[transports.telegram]`):
|
||||
|
||||
```toml
|
||||
voice_transcription = true
|
||||
voice_transcription_model = "gpt-4o-mini-transcribe" # optional
|
||||
```
|
||||
|
||||
Set `OPENAI_API_KEY` in the environment. If transcription is enabled but the API key
|
||||
is missing or the audio download fails, takopi replies with a short error and skips
|
||||
the run.
|
||||
|
||||
To use a local OpenAI-compatible Whisper server, also set `OPENAI_BASE_URL` (for
|
||||
example, `http://localhost:8000/v1`) and a dummy `OPENAI_API_KEY` if your server
|
||||
ignores it. If your server requires a specific model name, set
|
||||
`voice_transcription_model` (for example, `whisper-1`).
|
||||
|
||||
## Chat sessions (optional)
|
||||
|
||||
Takopi is stateless by default unless you reply to a bot message containing a resume
|
||||
line. If you want auto-resume without replies, enable chat sessions.
|
||||
|
||||
Configuration (under `[transports.telegram]`):
|
||||
|
||||
```toml
|
||||
show_resume_line = true # set false to hide resume lines
|
||||
session_mode = "chat" # or "stateless"
|
||||
```
|
||||
|
||||
Behavior:
|
||||
|
||||
- Stores one resume token per chat (per sender in group chats).
|
||||
- Auto-resumes when no explicit resume token is present.
|
||||
- Reset with `/new`.
|
||||
|
||||
State is stored in `telegram_chat_sessions_state.json` alongside the config file.
|
||||
|
||||
Set `show_resume_line = false` to hide resume lines when takopi can auto-resume
|
||||
(topics or chat sessions) and a project context is resolved. Otherwise the resume
|
||||
line stays visible so reply-to-continue still works.
|
||||
|
||||
## Message overflow
|
||||
|
||||
By default, takopi trims long final responses to ~3500 characters to stay under
|
||||
Telegram's 4096 character limit after entity parsing. You can opt into splitting
|
||||
instead:
|
||||
|
||||
```toml
|
||||
[transports.telegram]
|
||||
message_overflow = "split" # trim | split
|
||||
```
|
||||
|
||||
Split mode sends multiple messages. Each chunk includes the footer; follow-up
|
||||
chunks add a "continued (N/M)" header.
|
||||
|
||||
## Forum topics (optional)
|
||||
|
||||
Takopi can bind Telegram forum topics to a project/branch and persist resume tokens
|
||||
per topic, so replies keep the right context even after restarts.
|
||||
|
||||
Configuration (under `[transports.telegram]`):
|
||||
|
||||
```toml
|
||||
[transports.telegram.topics]
|
||||
enabled = true
|
||||
scope = "auto" # auto | main | projects | all
|
||||
```
|
||||
|
||||
Requirements:
|
||||
|
||||
- `main`: `chat_id` must be a forum-enabled supergroup (topics enabled).
|
||||
- `projects`: each `projects.<alias>.chat_id` must point to a forum-enabled
|
||||
supergroup for that project.
|
||||
- `all`: both the main chat and each project chat must be forum-enabled.
|
||||
- `auto`: if any project chats are configured, uses `projects`; otherwise `main`.
|
||||
- The bot needs the **Manage Topics** permission in the relevant chat(s).
|
||||
|
||||
Commands:
|
||||
|
||||
- `main`: `/topic <project> @branch` creates a topic in the main chat and binds it.
|
||||
- `projects`: `/topic @branch` creates a topic in the project chat and binds it.
|
||||
- `all`: use `/topic <project> @branch` in the main chat, or `/topic @branch` in
|
||||
project chats.
|
||||
- `/ctx` inside a topic shows the bound context and stored session engines.
|
||||
`/ctx set ...` and `/ctx clear` update the binding.
|
||||
- `/new` inside a topic clears stored resume tokens for that topic.
|
||||
|
||||
State is stored in `telegram_topics_state.json` alongside the config file.
|
||||
Delete it to reset all topic bindings and stored sessions.
|
||||
|
||||
Note: main chat topics do not assume a default project; topics must be bound
|
||||
before running without directives.
|
||||
|
||||
## Outbox model
|
||||
|
||||
- Single worker processes one op at a time.
|
||||
- Each op is keyed; only one pending op per key.
|
||||
- New ops with the same key overwrite the payload but **do not** reset
|
||||
`queued_at` (fairness).
|
||||
|
||||
Keys (include `chat_id` to avoid cross-chat collisions):
|
||||
|
||||
- `("edit", chat_id, message_id)` for edits (coalesced).
|
||||
- `("delete", chat_id, message_id)` for deletes.
|
||||
- `("send", chat_id, replace_message_id)` when replacing a progress message.
|
||||
- Unique key for normal sends.
|
||||
|
||||
Scheduling:
|
||||
|
||||
- Ordered by `(priority, queued_at)`.
|
||||
- Priorities: send=0, delete=1, edit=2.
|
||||
- Within a priority tier, the oldest pending op runs first.
|
||||
|
||||
## Rate limiting + backoff
|
||||
|
||||
- Per-chat pacing is computed from `private_chat_rps` and `group_chat_rps`.
|
||||
Defaults: 1.0 msg/s for private, 20/60 msg/s for groups (≈1 message every 3s).
|
||||
- Pacing is currently enforced via a single global `next_at`; per-chat
|
||||
`next_at` is a future consideration if we ever run multiple chats in parallel.
|
||||
- The worker waits until `max(next_at, retry_at)` before executing the next op.
|
||||
- On 429, `RetryAfter` is raised using `parameters.retry_after` when present;
|
||||
if missing, we fall back to a 5s delay. The outbox sets `retry_at` and
|
||||
requeues the op if no newer op for the same key has arrived.
|
||||
|
||||
## Error handling
|
||||
|
||||
- Non-429 errors are logged and dropped (no retry).
|
||||
- On `RetryAfter`, the op is retried unless a newer op superseded the same key.
|
||||
|
||||
## Replace progress messages
|
||||
|
||||
`send_message(replace_message_id=...)`:
|
||||
|
||||
- Drops any pending edit for that progress message.
|
||||
- Enqueues the send at highest priority.
|
||||
- If the send succeeds, enqueues a delete for the old progress message.
|
||||
|
||||
This keeps the final message first and avoids deleting progress if the send
|
||||
fails.
|
||||
|
||||
## getUpdates
|
||||
|
||||
`get_updates` bypasses the outbox and retries on `RetryAfter` by sleeping
|
||||
for the provided delay.
|
||||
|
||||
## Close semantics
|
||||
|
||||
`TelegramClient.close()` shuts down the outbox and closes the HTTP client.
|
||||
Pending ops are failed with `None` (best-effort).
|
||||
Reference in New Issue
Block a user