Finalize Go-only migration, runtime hardening, and CI/container optimization
This commit consolidates the full repository transition to a Go-first codebase and captures the follow-up performance/reliability work completed in the same stream. Highlights: - Remove Python implementation and test suites (, , , ) and retire Python-specific docs/instructions. - Move and standardize static web assets under , updating Bun/TypeScript build paths and server static resolution logic. - Rewrite developer workflow to Makefile-first Go targets (vet/test/race/coverage/fuzz/build) and align repository guidance/docs accordingly. - Update Docker and CI/CD for leaner artifacts: - switch to Alpine-based multi-stage build with stripped Go binary - install only minimal runtime deps (, ) - tighten Docker build context via - ensure workflows build/publish the target. - Improve runtime correctness/latency and reduce duplication: - explicit WebSocket outbound frame typing (text vs binary) instead of payload-byte heuristics - SSE activity fan-out outside global lock and safer subscriber lifecycle - shared session output/snapshot helpers to reduce duplicated logic - restart-safe channel lifecycle for Docker watcher/stats start-stop-start flows - faster screenshot cold-start path (poll-until-ready within timeout vs fixed sleep). - Add/expand regression coverage for the above lifecycle and helper paths. Validation run: - bun run build [32mBundled 3 modules in 10ms[0m [34mterminal.js[33m 0.68 MB [2m(entry point)[0m (Bun typecheck + bundle) - cd go && go vet ./... cd go && go test ./... ok github.com/rcarmo/webterm-go-port/cmd/webterm (cached) ok github.com/rcarmo/webterm-go-port/internal/terminalstate (cached) ok github.com/rcarmo/webterm-go-port/webterm (cached) cd go && go test ./webterm -coverprofile=coverage.out && go tool cover -func=coverage.out ok github.com/rcarmo/webterm-go-port/webterm (cached) coverage: 81.0% of statements github.com/rcarmo/webterm-go-port/webterm/cli.go:14: RunCLI 51.6% github.com/rcarmo/webterm-go-port/webterm/config.go:25: DefaultConfig 100.0% github.com/rcarmo/webterm-go-port/webterm/config.go:29: LoadLandingYAML 82.6% github.com/rcarmo/webterm-go-port/webterm/config.go:70: LoadComposeManifest 76.9% github.com/rcarmo/webterm-go-port/webterm/config.go:114: extractLabel 92.3% github.com/rcarmo/webterm-go-port/webterm/config.go:138: asString 80.0% github.com/rcarmo/webterm-go-port/webterm/constants.go:27: EnvBool 50.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:30: Read 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:56: NewDockerExecSession 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:72: Open 90.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:99: Start 85.7% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:119: readLoop 83.3% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:143: handleOutput 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:153: createExec 75.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:183: startExecSocket 60.7% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:219: resizeExec 83.3% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:237: Close 90.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:250: Wait 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:257: SetTerminalSize 81.8% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:274: ForceRedraw 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:281: SendBytes 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:294: SendMeta 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:298: IsRunning 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:304: GetReplayBuffer 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:308: GetScreenSnapshot 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_exec_session.go:316: UpdateConnector 80.0% github.com/rcarmo/webterm-go-port/webterm/docker_http.go:23: DockerSocketPath 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_http.go:37: newUnixHTTPClient 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_http.go:47: sharedUnixClient 91.7% github.com/rcarmo/webterm-go-port/webterm/docker_http.go:64: unixJSONRequest 84.2% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:33: NewDockerStatsCollector 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:47: Available 72.7% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:64: Start 80.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:78: Stop 75.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:90: AddService 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:101: RemoveService 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:115: GetCPUHistory 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:124: pollLoop 95.2% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:160: discoverContainers 65.4% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:198: pollContainer 77.8% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:223: calculateCPUPercent 69.2% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:260: RenderSparklineSVG 89.3% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:299: max 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:306: toAnyMap 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:323: toStringMap 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:331: toAnySlice 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:340: toStringSlice 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:351: toUint 91.7% github.com/rcarmo/webterm-go-port/webterm/docker_stats.go:376: toInt 85.7% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:34: NewDockerWatcher 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:54: hasWebtermLabel 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:60: isAutoLabel 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:67: getContainerCommand 80.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:76: getContainerTheme 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:81: getContainerName 42.9% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:93: containerToSlug 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:98: addContainer 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:119: removeContainer 100.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:143: listLabeledContainers 94.1% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:168: handleEvent 80.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:202: watchEvents 85.7% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:239: ScanExisting 60.0% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:249: Start 83.3% github.com/rcarmo/webterm-go-port/webterm/docker_watcher.go:265: Stop 81.8% github.com/rcarmo/webterm-go-port/webterm/identity.go:12: GenerateID 94.1% github.com/rcarmo/webterm-go-port/webterm/normalize.go:13: FilterDASequences 83.3% github.com/rcarmo/webterm-go-port/webterm/replay.go:14: NewReplayBuffer 66.7% github.com/rcarmo/webterm-go-port/webterm/replay.go:21: Add 100.0% github.com/rcarmo/webterm-go-port/webterm/replay.go:43: Bytes 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:95: OnData 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go💯 OnBinary 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:105: OnMeta 0.0% github.com/rcarmo/webterm-go-port/webterm/server.go:107: OnClose 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:112: NewLocalServer 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:163: findStaticPath 62.5% github.com/rcarmo/webterm-go-port/webterm/server.go:184: markRouteActivity 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:207: enqueueWSFrame 77.8% github.com/rcarmo/webterm-go-port/webterm/server.go:233: stopWSClient 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:246: wsSender 80.0% github.com/rcarmo/webterm-go-port/webterm/server.go:256: createTerminalSession 66.7% github.com/rcarmo/webterm-go-port/webterm/server.go:278: clampInt 60.0% github.com/rcarmo/webterm-go-port/webterm/server.go:288: parseResizePayload 88.9% github.com/rcarmo/webterm-go-port/webterm/server.go:303: handleWebSocket 81.1% github.com/rcarmo/webterm-go-port/webterm/server.go:425: chooseRouteForScreenshot 50.0% github.com/rcarmo/webterm-go-port/webterm/server.go:440: screenshotTTL 66.7% github.com/rcarmo/webterm-go-port/webterm/server.go:457: handleScreenshot 55.7% github.com/rcarmo/webterm-go-port/webterm/server.go:541: handleCPUSparkline 94.4% github.com/rcarmo/webterm-go-port/webterm/server.go:566: handleEvents 76.0% github.com/rcarmo/webterm-go-port/webterm/server.go:602: toIntFromQuery 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:609: dashboardTiles 81.8% github.com/rcarmo/webterm-go-port/webterm/server.go:631: handleTiles 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:636: getWSURL 65.2% github.com/rcarmo/webterm-go-port/webterm/server.go:671: handleRoot 56.8% github.com/rcarmo/webterm-go-port/webterm/server.go:732: htmlEscape 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:736: htmlAttrEscape 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:740: handleHealth 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:744: setupDockerFeatures 40.0% github.com/rcarmo/webterm-go-port/webterm/server.go:791: shutdown 62.5% github.com/rcarmo/webterm-go-port/webterm/server.go:814: Handler 100.0% github.com/rcarmo/webterm-go-port/webterm/server.go:829: Run 77.8% github.com/rcarmo/webterm-go-port/webterm/session.go:31: OnData 0.0% github.com/rcarmo/webterm-go-port/webterm/session.go:32: OnBinary 0.0% github.com/rcarmo/webterm-go-port/webterm/session.go:33: OnMeta 0.0% github.com/rcarmo/webterm-go-port/webterm/session.go:34: OnClose 0.0% github.com/rcarmo/webterm-go-port/webterm/session.go:36: dispatchSessionOutput 100.0% github.com/rcarmo/webterm-go-port/webterm/session.go:47: snapshotFromTracker 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:20: NewSessionManager 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:34: SetSessionFactory 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:40: defaultSessionFactory 87.5% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:57: splitCommand 75.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:65: shlexSplit 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:69: AddApp 87.5% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:88: RemoveApp 87.5% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:101: Apps 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:107: AppBySlug 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:114: GetDefaultApp 80.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:123: NewSession 50.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:167: OnSessionEnd 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:176: CloseAll 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:189: CloseSession 87.5% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:201: GetSession 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:207: GetSessionByRouteKey 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:217: GetSessionIDByRouteKey 100.0% github.com/rcarmo/webterm-go-port/webterm/session_manager.go:223: GetFirstRunningSession 85.7% github.com/rcarmo/webterm-go-port/webterm/shellsplit.go:5: shlexSplitImpl 100.0% github.com/rcarmo/webterm-go-port/webterm/slugify.go:13: Slugify 100.0% github.com/rcarmo/webterm-go-port/webterm/svg_exporter.go:35: RenderTerminalSVG 92.6% github.com/rcarmo/webterm-go-port/webterm/svg_exporter.go:113: colorToHex 87.5% github.com/rcarmo/webterm-go-port/webterm/svg_exporter.go:139: isHex 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:37: NewTerminalSession 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:49: Open 86.7% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:90: Start 85.7% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:110: readLoop 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:132: handleOutput 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:142: Close 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:160: Wait 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:167: SetTerminalSize 80.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:190: ForceRedraw 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:198: SendBytes 88.9% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:211: SendMeta 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:215: IsRunning 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:221: GetReplayBuffer 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:225: GetScreenSnapshot 100.0% github.com/rcarmo/webterm-go-port/webterm/terminal_session.go:233: UpdateConnector 80.0% github.com/rcarmo/webterm-go-port/webterm/twoway.go:14: NewTwoWayMap 100.0% github.com/rcarmo/webterm-go-port/webterm/twoway.go:21: Set 88.9% github.com/rcarmo/webterm-go-port/webterm/twoway.go:36: DeleteKey 100.0% github.com/rcarmo/webterm-go-port/webterm/twoway.go:45: Get 100.0% github.com/rcarmo/webterm-go-port/webterm/twoway.go:52: GetKey 100.0% github.com/rcarmo/webterm-go-port/webterm/twoway.go:59: Keys 100.0% github.com/rcarmo/webterm-go-port/webterm/twoway.go:70: UnsafeForward 100.0% total: (statements) 81.0% - cd go && go test -race ./... ok github.com/rcarmo/webterm-go-port/cmd/webterm (cached) ok github.com/rcarmo/webterm-go-port/internal/terminalstate (cached) ok github.com/rcarmo/webterm-go-port/webterm (cached) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
GitHub Copilot
+42
-237
@@ -1,257 +1,62 @@
|
||||
# Architecture
|
||||
|
||||
This document describes the internal architecture of webterm.
|
||||
|
||||
## Overview
|
||||
|
||||
webterm is a web-based terminal server that exposes terminal sessions over HTTP and WebSocket. It's designed to run behind a reverse proxy with authentication.
|
||||
|
||||
> **Note:** As of v1.0.0, this project uses [ghostty-web](https://github.com/rcarmo/ghostty-web) (a patched fork with native theme support) instead of xterm.js.
|
||||
`webterm` is a Go HTTP/WebSocket server that hosts one or more terminal sessions and renders screenshot/telemetry surfaces for a dashboard UI.
|
||||
|
||||
```
|
||||
┌─────────────┐ ┌──────────────────────────────────────────────────┐
|
||||
│ Browser │─────▶│ local_server.py │
|
||||
│ (ghostty- │◀─────│ (aiohttp web server) │
|
||||
│ web) │ │ │
|
||||
└─────────────┘ │ ┌──────────────┐ ┌──────────────────────────┐ │
|
||||
│ │ │ session_ │ │ terminal_session.py │ │
|
||||
│ WebSocket │ │ manager.py │──│ (PTY + pyte emulator) │ │
|
||||
└────────────▶│ └──────────────┘ └──────────────────────────┘ │
|
||||
│ │
|
||||
│ ┌──────────────┐ ┌──────────────────────────┐ │
|
||||
│ │ poller.py │ │ docker_stats.py │ │
|
||||
│ │ (I/O thread) │ │ (CPU metrics via socket) │ │
|
||||
│ └──────────────┘ └──────────────────────────┘ │
|
||||
└──────────────────────────────────────────────────┘
|
||||
Browser (terminal.js + ghostty-vt.wasm)
|
||||
│
|
||||
│ WS / HTTP / SSE
|
||||
▼
|
||||
go/webterm/server.go (LocalServer)
|
||||
│
|
||||
├── session_manager.go (route/app/session registry)
|
||||
├── terminal_session.go (PTY-backed local sessions)
|
||||
├── docker_exec_session.go (Docker exec-backed sessions)
|
||||
├── docker_watcher.go (container add/remove discovery)
|
||||
├── docker_stats.go (CPU sampling + sparkline data)
|
||||
└── svg_exporter.go (terminal snapshot -> SVG)
|
||||
```
|
||||
|
||||
## Core Components
|
||||
## Packages
|
||||
|
||||
### local_server.py
|
||||
- `go/cmd/webterm`: CLI entrypoint
|
||||
- `go/webterm`: server/runtime/domain logic
|
||||
- `go/internal/terminalstate`: Go terminal emulator wrapper (`go-te`) used for screenshots
|
||||
|
||||
The main HTTP/WebSocket server built on aiohttp. Handles:
|
||||
## Runtime data flow
|
||||
|
||||
- **HTTP routes**: Dashboard, screenshots, sparklines, SSE events, health checks
|
||||
- **WebSocket connections**: Terminal I/O multiplexing with JSON protocol
|
||||
- **Screenshot caching**: Time-based and change-based cache invalidation
|
||||
- **SSE broadcasting**: Real-time activity notifications to dashboard
|
||||
1. Browser connects to `/ws/{route_key}`.
|
||||
2. `SessionManager` resolves or creates a session.
|
||||
3. Session reads PTY output and updates:
|
||||
- live WS stream (`stdout`)
|
||||
- replay buffer (reconnect support)
|
||||
- terminal-state tracker (`go-te`) for screenshots
|
||||
4. Dashboard pulls `/screenshot.svg` and listens on `/events` for activity.
|
||||
|
||||
Key classes:
|
||||
- `Server`: Main server class managing routes and session lifecycle
|
||||
## Static assets
|
||||
|
||||
### session_manager.py
|
||||
Assets live in `go/webterm/static`:
|
||||
|
||||
Manages the mapping between route keys and sessions:
|
||||
- `js/terminal.ts` source
|
||||
- `js/terminal.js` bundled client
|
||||
- `js/ghostty-vt.wasm`
|
||||
- `monospace.css`, icons, `manifest.json`
|
||||
|
||||
- **TwoWayDict**: Bidirectional mapping of RouteKey ↔ SessionID
|
||||
- **Session creation**: Creates TerminalSession on demand
|
||||
- **App registry**: Stores terminal configurations from manifest files
|
||||
The server resolves static files from:
|
||||
|
||||
### terminal_session.py
|
||||
1. `WEBTERM_STATIC_PATH` (if set)
|
||||
2. local repository-relative fallbacks rooted at `go/webterm/static`
|
||||
|
||||
Manages a single terminal session:
|
||||
## Docker integration
|
||||
|
||||
- **PTY management**: Fork/exec with pseudo-terminal
|
||||
- **pyte emulator**: Uses `AltScreen` (patched pyte) for ANSI interpretation
|
||||
- **Data pipeline**: C1 normalization → `expand_clear_sequences()` → `stream.feed()`
|
||||
- **Replay buffer**: 64KB ring buffer for reconnection support
|
||||
- **Resize handling**: Propagates window size changes to PTY
|
||||
- **Compose mode** loads services from a compose manifest and creates tiles for services carrying `webterm-command`.
|
||||
- **Watch mode** subscribes to Docker events and adds/removes tiles at runtime.
|
||||
- `webterm-theme` controls tile theme; default theme applies if unset.
|
||||
|
||||
The pyte screen buffer provides character-level access for screenshots.
|
||||
## Reliability notes
|
||||
|
||||
### poller.py
|
||||
|
||||
Background thread for non-blocking PTY I/O:
|
||||
|
||||
- **selector-based**: Uses `selectors.DefaultSelector` for efficient I/O
|
||||
- **Async queues**: Bridges sync I/O thread to async main loop
|
||||
- **Write queuing**: Handles backpressure for terminal input
|
||||
|
||||
### svg_exporter.py
|
||||
|
||||
Custom SVG renderer for terminal screenshots:
|
||||
|
||||
- **Per-character positioning**: Each character has explicit x coordinate
|
||||
- **Box-drawing scaling**: Vertical 1.2x scale for line-height alignment
|
||||
- **Color handling**: ANSI 16-color palette + 256-color + truecolor
|
||||
- **Wide character support**: Proper column tracking for CJK characters
|
||||
|
||||
### docker_stats.py
|
||||
|
||||
Collects CPU metrics from Docker containers:
|
||||
|
||||
- **Unix socket client**: Direct HTTP-over-Unix-socket to Docker API
|
||||
- **Compose awareness**: Filters containers by compose project label
|
||||
- **History buffer**: 180 samples (30 min at 10s intervals)
|
||||
- **Sparkline SVG**: Renders mini CPU graphs
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Terminal I/O
|
||||
|
||||
```
|
||||
Browser Server PTY
|
||||
│ │ │
|
||||
│──["stdin", "ls\n"]──────▶│ │
|
||||
│ │────write(b"ls\n")───────▶│
|
||||
│ │ │
|
||||
│ │◀───read(output)──────────│
|
||||
│◀─["stdout", "..."]───────│ │
|
||||
```
|
||||
|
||||
### Screenshot Generation
|
||||
|
||||
```
|
||||
Dashboard ──GET /screenshot.svg──▶ Server
|
||||
│
|
||||
▼
|
||||
TerminalSession
|
||||
│
|
||||
get_screen_state()
|
||||
│
|
||||
▼
|
||||
pyte.Screen
|
||||
.buffer
|
||||
│
|
||||
▼
|
||||
svg_exporter.py
|
||||
render_terminal_svg()
|
||||
│
|
||||
▼
|
||||
<svg>...</svg>
|
||||
```
|
||||
|
||||
### SSE Activity Updates
|
||||
|
||||
```
|
||||
Dashboard ──GET /events──▶ Server (SSE connection held open)
|
||||
│
|
||||
Terminal activity ───────────▶│
|
||||
│
|
||||
▼
|
||||
Broadcast to all SSE clients:
|
||||
data: {"route_key": "...", "type": "activity"}
|
||||
```
|
||||
|
||||
## Session Lifecycle
|
||||
|
||||
1. **Browser connects** to `/ws/{route_key}`
|
||||
2. **SessionManager** looks up or creates session for route_key
|
||||
3. **TerminalSession** forks PTY process, initializes pyte emulator
|
||||
4. **Poller** registers PTY fd for I/O events
|
||||
5. **WebSocket handler** bridges browser ↔ PTY via JSON messages
|
||||
6. **On disconnect**: Session stays alive; browser can reconnect
|
||||
7. **On reconnect**: Replay buffer restores recent output
|
||||
|
||||
## Configuration
|
||||
|
||||
### Manifest-based (--landing-manifest)
|
||||
|
||||
```yaml
|
||||
- name: Display Name
|
||||
slug: route-key
|
||||
command: /path/to/command
|
||||
```
|
||||
|
||||
### Compose-based (--compose-manifest)
|
||||
|
||||
Reads docker-compose.yaml, creates tiles for services with `webterm-command` label:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
myservice:
|
||||
labels:
|
||||
webterm-command: docker exec -it myservice bash
|
||||
```
|
||||
|
||||
## WebSocket Protocol
|
||||
|
||||
JSON-encoded messages over WebSocket:
|
||||
|
||||
| Direction | Message | Description |
|
||||
|-----------|---------|-------------|
|
||||
| Client→Server | `["stdin", "data"]` | Terminal input |
|
||||
| Client→Server | `["resize", {"width": N, "height": M}]` | Window resize |
|
||||
| Client→Server | `["ping", data]` | Keep-alive |
|
||||
| Server→Client | `["stdout", "data"]` | Terminal output |
|
||||
| Server→Client | `["pong", data]` | Keep-alive response |
|
||||
|
||||
## API Endpoints
|
||||
|
||||
| Endpoint | Method | Description |
|
||||
|----------|--------|-------------|
|
||||
| `/` | GET | Dashboard HTML or terminal redirect |
|
||||
| `/ws/{route_key}` | WS | WebSocket terminal connection |
|
||||
| `/screenshot.svg` | GET | SVG screenshot (query: `route_key`) |
|
||||
| `/cpu-sparkline.svg` | GET | CPU sparkline (query: `container`) |
|
||||
| `/events` | GET | SSE stream for activity updates |
|
||||
| `/health` | GET | Health check |
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
### Why bundle ghostty-web directly?
|
||||
|
||||
We bundle a [patched version of ghostty-web](https://github.com/rcarmo/ghostty-web) for:
|
||||
|
||||
- **Native theme support** - Theme colors are passed directly to WASM, no runtime remapping needed
|
||||
- **Production-tested VT100 parser** - Uses Ghostty's battle-tested parser via WebAssembly
|
||||
- **Full configuration control** - fontFamily, scrollback, theme are configurable via CLI
|
||||
- **IME support** - Proper input method support for CJK languages
|
||||
- **Mobile support** - Custom keyboard handling for iOS Safari and Android
|
||||
- **Smaller bundle** - ~0.67 MB (down from 1.16 MB with color patching)
|
||||
- **11 built-in themes** - xterm, monokai, ristretto, dark, light, dracula, catppuccin, nord, gruvbox, solarized, tokyo
|
||||
|
||||
The pre-built `terminal.js` bundle is committed to the repo so users can `pip install` without needing Node.js/Bun.
|
||||
|
||||
### Why custom SVG exporter?
|
||||
|
||||
Rich's `export_svg()` had alignment issues with box-drawing characters and varied font rendering across browsers. The custom exporter:
|
||||
|
||||
- Positions each character individually for pixel-perfect alignment
|
||||
- Scales box-drawing characters vertically to fill line height
|
||||
- Uses explicit x coordinates instead of relying on font metrics
|
||||
|
||||
### Why pyte?
|
||||
|
||||
pyte provides a pure-Python terminal emulator that tracks screen state character-by-character, enabling:
|
||||
|
||||
- Screenshot generation without screen scraping
|
||||
- Dirty tracking for efficient cache invalidation
|
||||
- Full ANSI/VT100 escape sequence support
|
||||
|
||||
pyte 0.8.2 has gaps (no alternate screen buffer, no CSI S/T scroll), so `AltScreen` in `alt_screen.py` monkeypatches pyte at import time to fill them — see [pyte-patches.md](pyte-patches.md) for details.
|
||||
|
||||
### Why session persistence?
|
||||
|
||||
Unlike traditional web terminals, sessions survive page refreshes:
|
||||
|
||||
- Replay buffer allows catching up on missed output
|
||||
- SessionManager keeps sessions alive until explicit close
|
||||
- Enables dashboard with multiple live terminal thumbnails
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
src/webterm/
|
||||
├── alt_screen.py # pyte Screen subclass with alt buffer + SU/SD patches
|
||||
├── cli.py # Click CLI entry point
|
||||
├── config.py # Configuration parsing (YAML manifests)
|
||||
├── local_server.py # Main HTTP/WebSocket server
|
||||
├── session_manager.py # Session registry and routing
|
||||
├── session.py # Abstract session interface
|
||||
├── terminal_session.py # PTY-based terminal session
|
||||
├── poller.py # Async I/O polling thread
|
||||
├── svg_exporter.py # Terminal→SVG renderer
|
||||
├── docker_stats.py # Docker CPU metrics collector
|
||||
├── exit_poller.py # Graceful shutdown handling
|
||||
├── identity.py # Session ID generation
|
||||
├── slugify.py # URL-safe slug generation
|
||||
├── types.py # Type aliases
|
||||
├── constants.py # Platform constants
|
||||
└── static/
|
||||
├── monospace.css # Font stack CSS variables
|
||||
└── js/
|
||||
├── terminal.ts # ghostty-web client source (TypeScript)
|
||||
├── terminal.js # Pre-built bundle (committed)
|
||||
└── ghostty-vt.wasm # Ghostty VT100 parser (WebAssembly)
|
||||
```
|
||||
- WebSocket writes are serialized through a sender queue.
|
||||
- Session-manager maps are lock-protected and race-tested.
|
||||
- Replay buffers are bounded to avoid unbounded memory growth.
|
||||
|
||||
-890
@@ -1,890 +0,0 @@
|
||||
# Roadmap: Migration to ghostty-web
|
||||
|
||||
This document outlines the completed migration from xterm.js/textual-serve to ghostty-web.
|
||||
|
||||
## Status: ✅ Complete (v1.0.0)
|
||||
|
||||
The migration has been completed and merged to `main`.
|
||||
|
||||
### What Was Done
|
||||
|
||||
- [x] **Phase 1: Tooling Setup** - Added `package.json`, `bunfig.toml`, Makefile targets
|
||||
- [x] **Phase 2: Terminal Client** - Created `terminal.ts` with full WebSocket protocol
|
||||
- [x] **Phase 3: Server Integration** - Updated HTML template, removed monkey-patch
|
||||
- [x] **Phase 4: Configuration** - Added CLI options for theme, font-family, font-size
|
||||
- [x] **Phase 5: Remove Dependency** - Dropped `textual-serve` from pyproject.toml
|
||||
- [x] **Phase 6: Documentation** - Updated README.md and ARCHITECTURE.md
|
||||
- [x] **Phase 7: Mobile Support** - Added hidden textarea for iOS Safari keyboard
|
||||
- [x] **Phase 8: Native Theme Support** - Upgraded to patched ghostty-web with WASM palette support
|
||||
|
||||
### Key Outcomes
|
||||
|
||||
| Metric | Before (xterm.js) | After (ghostty-web 0.4.0) |
|
||||
|--------|-------------------|---------------------------|
|
||||
| textual-serve dependency | Required | ❌ Removed |
|
||||
| Terminal engine | xterm.js 6.0 | ghostty-web (Ghostty VT parser via WASM) |
|
||||
| Theme handling | Runtime color remapping | Native WASM palette support |
|
||||
| Scrollback history | 0 (none) | 1000 (configurable) |
|
||||
| Theme configuration | None | 11 built-in themes via `--theme` |
|
||||
| Font configuration | Monkey-patch workaround | `--font-family` and `--font-size` CLI options |
|
||||
| Mobile Safari | No keyboard | ✅ Hidden textarea for keyboard input |
|
||||
| IME support | Limited | ✅ Full CJK input method support |
|
||||
| Bundle size | 1.16 MB | 0.67 MB |
|
||||
|
||||
### ghostty-web Fork
|
||||
|
||||
We use a [patched version of ghostty-web](https://github.com/rcarmo/ghostty-web) that adds:
|
||||
|
||||
- Native theme/palette support at the WASM level via `buildWasmConfig()`
|
||||
- Theme colors passed directly to `createTerminal()` config
|
||||
- No runtime color remapping needed
|
||||
- IME input fixes for CJK languages
|
||||
|
||||
### Files Changed
|
||||
|
||||
```
|
||||
Added:
|
||||
package.json # ghostty-web + TypeScript
|
||||
bunfig.toml # Bun configuration
|
||||
tsconfig.json # TypeScript configuration
|
||||
src/.../static/js/terminal.ts # TypeScript source
|
||||
src/.../static/js/terminal.js # Pre-built bundle (committed)
|
||||
src/.../static/js/ghostty-vt.wasm # Ghostty VT100 parser
|
||||
|
||||
Modified:
|
||||
pyproject.toml # Removed textual-serve dependency, version 1.0.0
|
||||
Makefile # Added bundle/bundle-watch/bump-patch targets
|
||||
.gitignore # Added node_modules/
|
||||
src/.../cli.py # Added --theme, --font-family, --font-size
|
||||
src/.../local_server.py # Pass theme/font config to HTML template
|
||||
docs/ARCHITECTURE.md # Updated for ghostty-web
|
||||
docs/ROADMAP.md # Migration complete
|
||||
README.md # Updated documentation
|
||||
```
|
||||
|
||||
### For Users
|
||||
|
||||
No action required. The pre-built `terminal.js` bundle is committed to the repo:
|
||||
|
||||
```bash
|
||||
pip install webterm
|
||||
# or
|
||||
pip install git+https://github.com/rcarmo/webterm.git
|
||||
```
|
||||
|
||||
Works without needing Node.js or Bun.
|
||||
|
||||
### For Developers
|
||||
|
||||
To modify the frontend:
|
||||
|
||||
```bash
|
||||
# Install Bun (https://bun.sh)
|
||||
curl -fsSL https://bun.sh/install | bash
|
||||
|
||||
# Install dependencies and build
|
||||
make bundle
|
||||
|
||||
# Or watch for changes during development
|
||||
make bundle-watch
|
||||
|
||||
# Bump patch version
|
||||
make bump-patch
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Background Analysis
|
||||
|
||||
The sections below document the original analysis that led to this migration.
|
||||
|
||||
### What textual-serve Provided
|
||||
|
||||
| Asset | Size | What We Used | Required? |
|
||||
|-------|------|--------------|-----------|
|
||||
| `static/js/textual.js` | 502 KB | xterm.js + WebSocket client | **Yes** |
|
||||
| `static/css/xterm.css` | 4.6 KB | Terminal styling | **Yes** |
|
||||
| `static/fonts/RobotoMono*.ttf` | 381 KB | Roboto Mono font | No (we override font) |
|
||||
| `static/images/background.png` | 58 KB | Background image | No |
|
||||
| **Total** | **948 KB** | | |
|
||||
|
||||
### Why We Switched to ghostty-web
|
||||
|
||||
| Benefit | Impact |
|
||||
|---------|--------|
|
||||
| **Production-tested VT100 parser** | Ghostty's parser handles edge cases correctly |
|
||||
| **xterm.js API compatibility** | Easy migration, familiar API |
|
||||
| **Full configuration control** | Theme, font, scrollback via CLI |
|
||||
| **Mobile Safari support** | Hidden textarea triggers keyboard |
|
||||
| **Modern features** | RGB colors, better cursor handling |
|
||||
|
||||
### WebSocket Protocol (Fully Compatible)
|
||||
|
||||
The protocol is simple JSON arrays. Our server already implements this:
|
||||
|
||||
| Direction | Message | Description |
|
||||
|-----------|---------|-------------|
|
||||
| Client → Server | `["stdin", "data"]` | Terminal input |
|
||||
| Client → Server | `["resize", {width: N, height: M}]` | Window resize |
|
||||
| Client → Server | `["ping", data]` | Keep-alive |
|
||||
| Server → Client | `["stdout", "data"]` | Terminal output (text) |
|
||||
| Server → Client | Binary frame | Terminal output (binary) |
|
||||
| Server → Client | `["pong", data]` | Keep-alive response |
|
||||
|
||||
### Current Workarounds
|
||||
|
||||
1. **Font override**: Canvas monkey-patch in HTML to replace hardcoded font family
|
||||
2. **No scrollback**: Users cannot scroll back through terminal history
|
||||
|
||||
---
|
||||
|
||||
## Tradeoffs Analysis
|
||||
|
||||
### Option A: Keep textual-serve Dependency
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Zero build tooling | Hardcoded font requires workaround |
|
||||
| Automatic updates via pip | No scrollback (scrollback: 0) |
|
||||
| Maintained by Textualize | No theme customization |
|
||||
| | Carries unused fonts/images (381 KB) |
|
||||
| | Tied to textual-serve release cycle |
|
||||
| | Unknown xterm.js version (likely 5.x) |
|
||||
|
||||
### Option B: Bundle xterm.js 6.0 Directly
|
||||
|
||||
| Pros | Cons |
|
||||
|------|------|
|
||||
| Full configuration control | Requires Bun toolchain |
|
||||
| Scrollback history support | ~150-200 KB bundle to maintain |
|
||||
| Custom themes/colors | Must track xterm.js updates |
|
||||
| Latest xterm.js 6.0 features | Initial setup effort (2-3 days) |
|
||||
| Smaller bundle (no unused fonts) | |
|
||||
| Can drop textual-serve dependency | |
|
||||
|
||||
### xterm.js 6.0 Features We'd Gain
|
||||
|
||||
| Feature | Benefit |
|
||||
|---------|---------|
|
||||
| Synchronized output (DEC 2026) | Smoother rapid output rendering |
|
||||
| Ligature support | Better programming font rendering |
|
||||
| Progress addon | Visual progress indicators |
|
||||
| Shadow DOM support | Better CSS encapsulation |
|
||||
| ESM support | Modern module loading |
|
||||
| Performance improvements | Faster search, less memory |
|
||||
| OSC 52 clipboard | Secure clipboard from terminal |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan (Completed)
|
||||
|
||||
### Phase 1: Tooling Setup ✅
|
||||
|
||||
**Goal**: Establish Bun-based build pipeline
|
||||
|
||||
```
|
||||
src/webterm/
|
||||
├── static/
|
||||
│ ├── js/
|
||||
│ │ └── terminal.ts # New: our xterm wrapper
|
||||
│ ├── css/
|
||||
│ │ └── xterm.css # Copied from xterm.js package
|
||||
│ └── monospace.css # Existing
|
||||
├── package.json # New: npm dependencies
|
||||
└── bunfig.toml # New: Bun configuration
|
||||
```
|
||||
|
||||
**Tasks**:
|
||||
- [x] Create `package.json` with xterm.js 6.0 dependencies
|
||||
- [x] Create `bunfig.toml` for build configuration
|
||||
- [x] Add `Makefile` targets: `make bundle`, `make bundle-watch`
|
||||
- [x] Add `.gitignore` entries for `node_modules/`
|
||||
- [x] Document Bun installation in README
|
||||
|
||||
**package.json** (final):
|
||||
```json
|
||||
{
|
||||
"name": "webterm-frontend",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"dependencies": {
|
||||
"@xterm/xterm": "^6.0.0",
|
||||
"@xterm/addon-fit": "^0.10.0",
|
||||
"@xterm/addon-webgl": "^0.18.0",
|
||||
"@xterm/addon-canvas": "^0.7.0",
|
||||
"@xterm/addon-unicode11": "^0.8.0",
|
||||
"@xterm/addon-web-links": "^0.11.0",
|
||||
"@xterm/addon-clipboard": "^0.2.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"typescript": "^5.7.0"
|
||||
},
|
||||
"scripts": {
|
||||
"build": "bun build src/webterm/static/js/terminal.ts --outfile=src/webterm/static/js/terminal.js --minify --target=browser",
|
||||
"watch": "bun build src/webterm/static/js/terminal.ts --outfile=src/webterm/static/js/terminal.js --watch --target=browser"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Phase 2: Terminal Client Implementation ✅
|
||||
|
||||
**Goal**: Create `terminal.ts` that replicates textual.js functionality
|
||||
|
||||
**Tasks**:
|
||||
- [x] Implement Terminal wrapper class
|
||||
- [x] WebSocket connection with reconnection logic
|
||||
- [x] Message protocol handling (stdin, resize, ping/pong)
|
||||
- [x] Addon initialization (fit, webgl, canvas, unicode11, web-links, clipboard)
|
||||
- [x] Configurable options via data attributes or window config
|
||||
|
||||
See `src/webterm/static/js/terminal.ts` for the full implementation (~230 lines).
|
||||
|
||||
### Phase 3: Server Integration ✅
|
||||
|
||||
**Goal**: Update local_server.py to use new bundle
|
||||
|
||||
**Tasks**:
|
||||
- [x] Update HTML template to load our bundle instead of textual.js
|
||||
- [x] Remove canvas monkey-patch workaround
|
||||
- [x] Add data attributes for scrollback, theme configuration
|
||||
- [x] Copy xterm.css to our static folder
|
||||
- [x] Update static file routes
|
||||
|
||||
### Phase 4: Configuration Support ✅
|
||||
|
||||
**Goal**: Make terminal appearance configurable
|
||||
|
||||
**Tasks**:
|
||||
- [x] Pass config to HTML template via data attributes (`data-scrollback`, `data-font-size`)
|
||||
- [ ] Add terminal config to CLI (--scrollback, --font-family) - *Future enhancement*
|
||||
- [ ] Add terminal config to TOML manifest files - *Future enhancement*
|
||||
|
||||
### Phase 5: Remove textual-serve Dependency ✅
|
||||
|
||||
**Goal**: Eliminate dependency once our bundle is stable
|
||||
|
||||
**Tasks**:
|
||||
- [x] Remove `textual-serve` from pyproject.toml dependencies
|
||||
- [x] Update ARCHITECTURE.md to document new frontend
|
||||
- [x] Update README.md with build instructions
|
||||
- [x] Commit pre-built bundle so users don't need Bun
|
||||
|
||||
### Phase 6: Testing & Polish
|
||||
|
||||
**Goal**: Ensure reliability across browsers
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Cross-browser testing (Chrome, Firefox, Safari, Edge)
|
||||
- [ ] Mobile browser testing (iOS Safari, Chrome Android)
|
||||
- [x] WebGL fallback to Canvas (implemented in terminal.ts)
|
||||
- [x] Reconnection logic (implemented with exponential backoff)
|
||||
- [ ] Performance comparison vs textual.js
|
||||
- [x] Bundle size: 560 KB (acceptable for full xterm.js + addons)
|
||||
|
||||
---
|
||||
|
||||
## Build Integration (Reference)
|
||||
|
||||
### Makefile Additions
|
||||
|
||||
```makefile
|
||||
# Frontend build
|
||||
.PHONY: bundle bundle-watch bundle-clean
|
||||
|
||||
bundle: node_modules
|
||||
bun run build
|
||||
|
||||
bundle-watch: node_modules
|
||||
bun run watch
|
||||
|
||||
bundle-clean:
|
||||
rm -rf node_modules src/webterm/static/js/terminal.js
|
||||
|
||||
node_modules: package.json
|
||||
bun install
|
||||
```
|
||||
|
||||
### Dockerfile Changes
|
||||
|
||||
```dockerfile
|
||||
# Add Bun for frontend build
|
||||
RUN curl -fsSL https://bun.sh/install | bash
|
||||
ENV PATH="/root/.bun/bin:${PATH}"
|
||||
|
||||
# Build frontend
|
||||
COPY package.json bunfig.toml ./
|
||||
RUN bun install
|
||||
COPY src/webterm/static/js/terminal.ts src/webterm/static/js/
|
||||
RUN bun run build
|
||||
```
|
||||
|
||||
### CI/CD Considerations
|
||||
|
||||
- Pre-commit hook to verify `terminal.js` matches `terminal.ts`
|
||||
- Or: commit built bundle to repo (simpler for users without Bun)
|
||||
- GitHub Actions step to build and verify bundle
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Mitigation |
|
||||
|------|------------|
|
||||
| xterm.js 6.0 breaking changes | Pin exact version, test thoroughly |
|
||||
| Bun compatibility issues | Fall back to esbuild if needed |
|
||||
| WebSocket protocol mismatch | Keep protocol identical to textual.js |
|
||||
| Performance regression | Benchmark before/after, keep WebGL |
|
||||
| Missing addon features | Test each addon explicitly |
|
||||
|
||||
---
|
||||
|
||||
## Timeline Estimate
|
||||
|
||||
| Phase | Effort | Dependencies |
|
||||
|-------|--------|--------------|
|
||||
| Phase 1: Tooling Setup | 0.5 days | None |
|
||||
| Phase 2: Terminal Client | 1-2 days | Phase 1 |
|
||||
| Phase 3: Server Integration | 0.5 days | Phase 2 |
|
||||
| Phase 4: Configuration | 0.5 days | Phase 3 |
|
||||
| Phase 5: Remove Dependency | 0.5 days | Phase 4 |
|
||||
| Phase 6: Testing | 1 day | Phase 5 |
|
||||
| **Total** | **4-5 days** | |
|
||||
|
||||
---
|
||||
|
||||
## Decision Checkpoints
|
||||
|
||||
1. **After Phase 2**: Verify terminal.ts works in isolation before integrating
|
||||
2. **After Phase 3**: Side-by-side comparison with textual.js
|
||||
3. **After Phase 5**: Confirm no regressions before removing dependency
|
||||
4. **After Phase 6**: Final sign-off for release
|
||||
|
||||
---
|
||||
|
||||
## Success Criteria
|
||||
|
||||
- [ ] Terminal renders correctly in Chrome, Firefox, Safari
|
||||
- [x] Scrollback history works (configurable limit)
|
||||
- [x] Custom fonts load without workarounds
|
||||
- [x] WebGL rendering enabled with Canvas fallback
|
||||
- [x] Bundle size: 560 KB (larger than target due to full addon suite, but acceptable)
|
||||
- [x] No textual-serve dependency in pyproject.toml
|
||||
- [x] All existing tests pass (302 tests)
|
||||
- [x] Documentation updated
|
||||
|
||||
---
|
||||
---
|
||||
|
||||
# Future: Go Reimplementation
|
||||
|
||||
This section analyzes what it would take to reimplement webterm in Go for lighter deployment.
|
||||
|
||||
## Status: 📋 Planning
|
||||
|
||||
Not yet started. This would be a separate project (`webterm-go`) providing a lightweight alternative.
|
||||
|
||||
## Executive Summary
|
||||
|
||||
**Most functionality can be reimplemented in Go** with mature libraries. The main challenge is the terminal emulator (pyte equivalent) - GoPyte exists but is less battle-tested than Python's pyte. Benefits would be a single static binary, lower memory footprint, and better concurrency.
|
||||
|
||||
---
|
||||
|
||||
## Component Mapping
|
||||
|
||||
| Python Component | Go Equivalent | Library | Maturity |
|
||||
|-----------------|---------------|---------|----------|
|
||||
| **aiohttp** (HTTP/WS server) | net/http + websocket | `gorilla/websocket` or `nhooyr.io/websocket` | ⭐⭐⭐⭐⭐ Excellent |
|
||||
| **pyte** (terminal emulator) | GoPyte | `github.com/scottpeterman/gopyte` | ⭐⭐⭐ Good |
|
||||
| **PTY handling** | go-pty | `github.com/aymanbagabas/go-pty` | ⭐⭐⭐⭐ Very Good |
|
||||
| **asyncio** (concurrency) | goroutines/channels | stdlib | ⭐⭐⭐⭐⭐ Native |
|
||||
| **SSE** | Custom handler | stdlib `net/http` | ⭐⭐⭐⭐ Simple |
|
||||
| **Docker stats** | Docker SDK | `github.com/docker/docker/client` | ⭐⭐⭐⭐⭐ Official |
|
||||
| **SVG generation** | SVGo | `github.com/ajstarks/svgo` | ⭐⭐⭐⭐⭐ Mature |
|
||||
| **YAML parsing** | yaml.v3 | `gopkg.in/yaml.v3` | ⭐⭐⭐⭐⭐ Standard |
|
||||
| **CLI** | cobra | `github.com/spf13/cobra` | ⭐⭐⭐⭐⭐ Standard |
|
||||
|
||||
---
|
||||
|
||||
## pyte vs GoPyte: Thorough Comparison
|
||||
|
||||
This section compares the Python **pyte** terminal emulator and the Go **GoPyte** emulator (per their upstream documentation/README). The focus is on capture accuracy, Unicode handling, performance expectations, and integration risk for webterm. Where GoPyte details are unclear, we call them out explicitly as validation items.
|
||||
|
||||
### Feature Matrix (Capture-Relevant)
|
||||
|
||||
| Capability | pyte (Python) | GoPyte (Go) | Capture Impact / Gaps |
|
||||
|-----------|---------------|-------------|------------------------|
|
||||
| **Terminal standards** | VTXXX/ANSI (VT100-style) | VT100/VT220/ANSI (claims) | Verify DEC private modes, OSC handling. |
|
||||
| **Screen buffer** | Screen + HistoryScreen (cells with attrs) | Screen buffer (claims) | Data model differences may affect attrs. |
|
||||
| **Alt screen** | Supported | Supported (claims) | Full-screen apps depend on correct switching. |
|
||||
| **Scrollback/history** | HistoryScreen (configurable) | Built-in scrollback (claims) | Semantics may differ; check memory cost. |
|
||||
| **Resize behavior** | Resizes + cursor + dirty state | Resizes + content (claims) | Must preserve content on resize. |
|
||||
| **SGR attributes** | Bold/underline/reverse/color | SGR attrs (claims) | Verify italics, faint, strikethrough, blink. |
|
||||
| **Underline styles** | Basic underline | Unknown | Double/curly underline may be missing. |
|
||||
| **Color depth** | ANSI + 256 + (truecolor in practice) | ANSI + 256? (verify) | Truecolor required for accurate screenshots. |
|
||||
| **Default colors** | SGR 39/49 respected | Unknown | Default fg/bg must be stable. |
|
||||
| **Reverse video** | Supported | Unknown | Affects screenshot accuracy. |
|
||||
| **DEC special graphics** | Supported (line drawing) | Unknown | Box-drawing is critical for TUIs. |
|
||||
| **Character sets (G0/G1)** | Supported | Unknown | Needed for legacy line-drawing. |
|
||||
| **Scroll regions** | Supported | Unknown | Needed for curses-style UI. |
|
||||
| **Insert/delete (IL/DL/ICH/DCH)** | Supported | Unknown | Affects screen fidelity during updates. |
|
||||
| **Tabs / tab stops** | Supported | Unknown | UI alignment depends on tabs. |
|
||||
| **Autowrap/origin modes** | Supported | Unknown | Impacts cursor positioning and layout. |
|
||||
| **Cursor save/restore** | Supported | Unknown | Common in TUIs and prompts. |
|
||||
| **Cursor visibility/style** | Basic visibility | Unknown | Useful for accurate snapshots. |
|
||||
| **Erase semantics (ED/EL/ECH)** | Supported | Unknown | Correct clearing is vital for screenshots. |
|
||||
| **Selective erase (DECSEL/DECSERA)** | Unknown | Unknown | Might matter for some TUIs. |
|
||||
| **Unicode width** | wcwidth-style width | go-runewidth | Width differences can misalign SVG. |
|
||||
| **Combining marks / ZWJ** | Unicode-aware | runewidth-based | Emoji sequences may differ in width. |
|
||||
| **Dirty tracking** | Exposes dirty set / DiffScreen | Unknown | Needed for efficient screenshot caching. |
|
||||
| **Images (sixel/kitty)** | Not supported | Not supported | Not required, but a capture limitation. |
|
||||
| **Performance** | Python, moderate | Go, claims high throughput | Benchmark under heavy output. |
|
||||
| **API stability** | Mature, widely used | Newer, smaller ecosystem | Risk of breaking changes. |
|
||||
| **Test maturity** | Established | Claims high coverage | Must run our own parity suite. |
|
||||
|
||||
### Unicode & Emoji Handling (Critical for SVG Capture)
|
||||
|
||||
**pyte**
|
||||
- Uses Python Unicode handling plus width calculation (wcwidth-style).
|
||||
- Generally robust for CJK and emoji, though edge cases exist (ZWJ, variation selectors).
|
||||
|
||||
**GoPyte**
|
||||
- Uses `go-runewidth` for width calculation.
|
||||
- Width differences vs wcwidth can shift glyph placement in SVG output.
|
||||
|
||||
**Capture Impact**
|
||||
- Width mismatches change x-coordinates -> visually incorrect screenshots.
|
||||
- Emoji sequences (ZWJ, VS16) may render as 1 cell in one emulator and 2 in another.
|
||||
- Must validate CJK + emoji + combining marks across sample workloads.
|
||||
|
||||
### Performance & Memory (Capture-Heavy Workloads)
|
||||
|
||||
**pyte**
|
||||
- Pure Python; adequate for typical workloads.
|
||||
- Predictable, but slower under heavy output.
|
||||
|
||||
**GoPyte**
|
||||
- Go implementation; upstream claims high throughput.
|
||||
- Performance depends on allocation strategy and screen model.
|
||||
|
||||
**Action**: Benchmark with fast-output scenarios, large scrollback, and frequent screenshots.
|
||||
|
||||
### Required Features for webterm Capture
|
||||
|
||||
We depend on the emulator for **accurate snapshot state**, not just live display:
|
||||
|
||||
- Stable **screen buffer** with per-cell fg/bg/bold/underline/reverse.
|
||||
- Correct **cursor position** after control sequences and resizes.
|
||||
- Accurate **alt screen** behavior for full-screen TUIs.
|
||||
- Consistent **Unicode width** for precise SVG positioning.
|
||||
- **Dirty tracking or diffability** to avoid re-rendering unchanged screens.
|
||||
- Reliable **color mapping** (ANSI 16 + 256 + truecolor).
|
||||
- Correct **DEC line drawing** (box-drawing characters).
|
||||
- Correct **erase semantics** (ED/EL/ECH) to avoid stale cells.
|
||||
|
||||
If any of these are missing, screenshots will be wrong or expensive to compute.
|
||||
|
||||
### Capture-Enhancing (Nice-to-Have) Features
|
||||
|
||||
These aren’t required for parity, but would improve capture quality or UX:
|
||||
|
||||
- **OSC 8 hyperlink metadata** to annotate clickable URLs in SVG output.
|
||||
- **OSC 52 clipboard** metadata for copy workflows.
|
||||
- **Underline styles** (double/curly) for richer text attributes.
|
||||
- **Cursor style/shape** metadata for accurate snapshots.
|
||||
- **Grapheme cluster awareness** (emoji sequences treated as single cells).
|
||||
- **Structured diffs** from emulator (explicit dirty regions instead of full screen).
|
||||
|
||||
### Known Gaps / Validation Checklist
|
||||
|
||||
Before relying on GoPyte for parity, verify:
|
||||
|
||||
- [ ] Full-screen app behavior (vim, htop, less) with alt screen.
|
||||
- [ ] DEC line drawing / special graphics set (box-drawing correctness).
|
||||
- [ ] Character set switching (G0/G1) for legacy graphics.
|
||||
- [ ] Scroll regions (DECSTBM) and origin mode behavior.
|
||||
- [ ] Insert/delete line/char (IL/DL/ICH/DCH) correctness.
|
||||
- [ ] Tab stops and tab clear (alignment in TUIs).
|
||||
- [ ] SGR coverage: bold/underline/italic/reverse + 256/truecolor.
|
||||
- [ ] Underline styles, faint, strikethrough, blink.
|
||||
- [ ] Default colors (SGR 39/49) and reverse video.
|
||||
- [ ] Unicode width parity with pyte (emoji + CJK samples).
|
||||
- [ ] Cursor save/restore (DECSC/DECRC) and visibility toggles.
|
||||
- [ ] Erase semantics (ED/EL/ECH) and autowrap correctness.
|
||||
- [ ] Resize correctness (content preservation, cursor placement).
|
||||
- [ ] Performance at high output rates (100k+ lines, low latency).
|
||||
|
||||
### Integration Implications for webterm
|
||||
|
||||
- **Screen buffer mapping**: GoPyte cells must map to our SVG exporter schema (fg/bg/bold/underline/reverse).
|
||||
- **Dirty tracking**: pyte exposes dirty state; GoPyte may need explicit diff tracking.
|
||||
- **Color translation**: Ensure SGR parsing aligns with our ANSI palette and truecolor handling.
|
||||
- **Replay buffer**: Replay must stay consistent with screen state for accurate screenshots.
|
||||
|
||||
### Alternatives (Brief)
|
||||
|
||||
If GoPyte does not meet parity, consider:
|
||||
|
||||
- **govte** (`github.com/cliofy/govte`): More comprehensive xterm-like parser/buffer, less documented.
|
||||
- **vito/vt100**: Lightweight VT100 emulator, limited scrollback/alt-screen.
|
||||
- **xyproto/vt100**: TUI canvas focus, not a drop-in emulator.
|
||||
|
||||
These alternatives still need capture parity validation.
|
||||
|
||||
|
||||
## What We'd Gain
|
||||
|
||||
| Benefit | Impact |
|
||||
|---------|--------|
|
||||
| **Single static binary** | No Python/pip dependency, simpler deployment |
|
||||
| **Lower memory** | ~10-20MB vs ~50-100MB for Python |
|
||||
| **Better concurrency** | Goroutines vs asyncio - more intuitive |
|
||||
| **Faster startup** | Instant vs Python interpreter load |
|
||||
| **Cross-compilation** | Easy builds for Linux/macOS/Windows/ARM |
|
||||
| **Smaller Docker image** | ~20MB vs ~200MB+ with Python |
|
||||
|
||||
## What We'd Lose
|
||||
|
||||
| Loss | Impact |
|
||||
|------|--------|
|
||||
| **Textual app support** | Removed in webterm |
|
||||
| **Rapid prototyping** | Go requires more boilerplate |
|
||||
| **pyte maturity** | GoPyte is less proven |
|
||||
|
||||
---
|
||||
|
||||
## Required Go Dependencies
|
||||
|
||||
```go
|
||||
// go.mod
|
||||
module github.com/rcarmo/webterm-go
|
||||
|
||||
go 1.22
|
||||
|
||||
require (
|
||||
// HTTP/WebSocket
|
||||
github.com/gorilla/websocket v1.5.1
|
||||
|
||||
// Terminal emulation
|
||||
github.com/scottpeterman/gopyte v0.1.0
|
||||
|
||||
// PTY handling
|
||||
github.com/aymanbagabas/go-pty v0.2.2
|
||||
|
||||
// Docker stats
|
||||
github.com/docker/docker v25.0.0
|
||||
|
||||
// SVG generation
|
||||
github.com/ajstarks/svgo v0.0.0-20211024235047-1546f124cd8b
|
||||
|
||||
// CLI
|
||||
github.com/spf13/cobra v1.8.0
|
||||
|
||||
// YAML parsing
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Implementation Plan
|
||||
|
||||
### Phase 1: Project Setup & Core Server (2 days)
|
||||
|
||||
**Goal**: Basic HTTP server with WebSocket support
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Initialize Go module with dependencies
|
||||
- [ ] Create basic HTTP server with routing
|
||||
- [ ] Implement WebSocket upgrade handler
|
||||
- [ ] Port JSON message protocol (stdin, resize, ping/pong)
|
||||
- [ ] Add graceful shutdown handling
|
||||
|
||||
**Files**:
|
||||
```
|
||||
cmd/
|
||||
webterm/
|
||||
main.go # Entry point
|
||||
internal/
|
||||
server/
|
||||
server.go # HTTP server
|
||||
websocket.go # WebSocket handler
|
||||
routes.go # Route definitions
|
||||
```
|
||||
|
||||
### Phase 2: PTY Session Management (2 days)
|
||||
|
||||
**Goal**: Spawn and manage terminal sessions
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Integrate go-pty for PTY creation
|
||||
- [ ] Implement session lifecycle (create, resize, close)
|
||||
- [ ] Build session manager with route mapping
|
||||
- [ ] Add replay buffer for reconnection
|
||||
- [ ] Handle concurrent session access
|
||||
|
||||
**Files**:
|
||||
```
|
||||
internal/
|
||||
session/
|
||||
manager.go # Session registry
|
||||
terminal.go # PTY session wrapper
|
||||
buffer.go # Replay ring buffer
|
||||
```
|
||||
|
||||
### Phase 3: Terminal Emulation (2 days)
|
||||
|
||||
**Goal**: Parse ANSI sequences for screen state
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Integrate GoPyte terminal emulator
|
||||
- [ ] Feed PTY output through emulator
|
||||
- [ ] Extract screen buffer for screenshots
|
||||
- [ ] Implement dirty tracking for cache invalidation
|
||||
- [ ] Handle resize events
|
||||
|
||||
**Files**:
|
||||
```
|
||||
internal/
|
||||
terminal/
|
||||
emulator.go # GoPyte wrapper
|
||||
screen.go # Screen buffer access
|
||||
```
|
||||
|
||||
### Phase 4: SVG Screenshot Generation (1.5 days)
|
||||
|
||||
**Goal**: Generate terminal screenshots as SVG
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Port character positioning logic from Python
|
||||
- [ ] Implement ANSI color palette (16 + 256 + truecolor)
|
||||
- [ ] Handle box-drawing character scaling
|
||||
- [ ] Add screenshot caching with ETag support
|
||||
- [ ] Implement cache TTL backoff
|
||||
|
||||
**Files**:
|
||||
```
|
||||
internal/
|
||||
screenshot/
|
||||
svg.go # SVG renderer
|
||||
colors.go # ANSI color handling
|
||||
cache.go # Screenshot cache
|
||||
```
|
||||
|
||||
### Phase 5: Dashboard & SSE (1.5 days)
|
||||
|
||||
**Goal**: Landing page with live updates
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Embed static assets (HTML, CSS, xterm.js bundle)
|
||||
- [ ] Implement SSE endpoint for activity notifications
|
||||
- [ ] Port dashboard HTML template
|
||||
- [ ] Add tile rendering with screenshots
|
||||
|
||||
**Files**:
|
||||
```
|
||||
internal/
|
||||
dashboard/
|
||||
handler.go # Dashboard HTTP handler
|
||||
sse.go # Server-Sent Events
|
||||
static/
|
||||
embed.go # Embedded assets
|
||||
```
|
||||
|
||||
### Phase 6: Docker Stats (1 day)
|
||||
|
||||
**Goal**: CPU sparklines for compose mode
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Integrate Docker SDK client
|
||||
- [ ] Implement container stats polling
|
||||
- [ ] Calculate CPU percentage from deltas
|
||||
- [ ] Generate sparkline SVGs
|
||||
- [ ] Filter by compose project label
|
||||
|
||||
**Files**:
|
||||
```
|
||||
internal/
|
||||
docker/
|
||||
stats.go # Stats collector
|
||||
sparkline.go # SVG sparkline
|
||||
```
|
||||
|
||||
### Phase 7: CLI & Configuration (1 day)
|
||||
|
||||
**Goal**: Feature-complete CLI
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Implement Cobra CLI with flags
|
||||
- [ ] Parse YAML landing manifests
|
||||
- [ ] Parse Docker Compose manifests
|
||||
- [ ] Add version command
|
||||
- [ ] Environment variable support
|
||||
|
||||
**Files**:
|
||||
```
|
||||
cmd/
|
||||
webterm/
|
||||
main.go # CLI entry point
|
||||
internal/
|
||||
config/
|
||||
config.go # Configuration types
|
||||
manifest.go # YAML parsing
|
||||
```
|
||||
|
||||
### Phase 8: Testing & Polish (2 days)
|
||||
|
||||
**Goal**: Production-ready release
|
||||
|
||||
**Tasks**:
|
||||
- [ ] Unit tests for core components
|
||||
- [ ] Integration tests for WebSocket protocol
|
||||
- [ ] Cross-browser testing
|
||||
- [ ] Build scripts for multiple platforms
|
||||
- [ ] Docker image (FROM scratch)
|
||||
- [ ] Documentation
|
||||
|
||||
**Files**:
|
||||
```
|
||||
Makefile # Build targets
|
||||
Dockerfile # Multi-stage build
|
||||
README.md # Usage docs
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Effort Summary
|
||||
|
||||
| Phase | Component | Days |
|
||||
|-------|-----------|------|
|
||||
| 1 | Project Setup & Core Server | 2 |
|
||||
| 2 | PTY Session Management | 2 |
|
||||
| 3 | Terminal Emulation | 2 |
|
||||
| 4 | SVG Screenshot Generation | 1.5 |
|
||||
| 5 | Dashboard & SSE | 1.5 |
|
||||
| 6 | Docker Stats | 1 |
|
||||
| 7 | CLI & Configuration | 1 |
|
||||
| 8 | Testing & Polish | 2 |
|
||||
| **Total** | | **13 days** |
|
||||
|
||||
---
|
||||
|
||||
## File Structure (Final)
|
||||
|
||||
```
|
||||
webterm-go/
|
||||
├── cmd/
|
||||
│ └── webterm/
|
||||
│ └── main.go
|
||||
├── internal/
|
||||
│ ├── config/
|
||||
│ │ ├── config.go
|
||||
│ │ └── manifest.go
|
||||
│ ├── dashboard/
|
||||
│ │ ├── handler.go
|
||||
│ │ └── sse.go
|
||||
│ ├── docker/
|
||||
│ │ ├── sparkline.go
|
||||
│ │ └── stats.go
|
||||
│ ├── screenshot/
|
||||
│ │ ├── cache.go
|
||||
│ │ ├── colors.go
|
||||
│ │ └── svg.go
|
||||
│ ├── server/
|
||||
│ │ ├── routes.go
|
||||
│ │ ├── server.go
|
||||
│ │ └── websocket.go
|
||||
│ ├── session/
|
||||
│ │ ├── buffer.go
|
||||
│ │ ├── manager.go
|
||||
│ │ └── terminal.go
|
||||
│ ├── static/
|
||||
│ │ └── embed.go
|
||||
│ └── terminal/
|
||||
│ ├── emulator.go
|
||||
│ └── screen.go
|
||||
├── static/
|
||||
│ ├── css/
|
||||
│ │ └── xterm.css
|
||||
│ └── js/
|
||||
│ └── terminal.js
|
||||
├── go.mod
|
||||
├── go.sum
|
||||
├── Makefile
|
||||
├── Dockerfile
|
||||
└── README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Build & Release
|
||||
|
||||
### Makefile Targets
|
||||
|
||||
```makefile
|
||||
.PHONY: build build-all test clean
|
||||
|
||||
BINARY := webterm
|
||||
VERSION := $(shell git describe --tags --always)
|
||||
LDFLAGS := -s -w -X main.version=$(VERSION)
|
||||
|
||||
build:
|
||||
go build -ldflags "$(LDFLAGS)" -o bin/$(BINARY) ./cmd/webterm
|
||||
|
||||
build-all:
|
||||
GOOS=linux GOARCH=amd64 go build -ldflags "$(LDFLAGS)" -o bin/$(BINARY)-linux-amd64 ./cmd/webterm
|
||||
GOOS=linux GOARCH=arm64 go build -ldflags "$(LDFLAGS)" -o bin/$(BINARY)-linux-arm64 ./cmd/webterm
|
||||
GOOS=darwin GOARCH=amd64 go build -ldflags "$(LDFLAGS)" -o bin/$(BINARY)-darwin-amd64 ./cmd/webterm
|
||||
GOOS=darwin GOARCH=arm64 go build -ldflags "$(LDFLAGS)" -o bin/$(BINARY)-darwin-arm64 ./cmd/webterm
|
||||
|
||||
test:
|
||||
go test -v ./...
|
||||
|
||||
clean:
|
||||
rm -rf bin/
|
||||
```
|
||||
|
||||
### Minimal Docker Image
|
||||
|
||||
```dockerfile
|
||||
FROM golang:1.22-alpine AS builder
|
||||
WORKDIR /app
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY . .
|
||||
RUN CGO_ENABLED=0 go build -ldflags "-s -w" -o webterm ./cmd/webterm
|
||||
|
||||
FROM scratch
|
||||
COPY --from=builder /app/webterm /webterm
|
||||
ENTRYPOINT ["/webterm"]
|
||||
```
|
||||
|
||||
**Result**: ~15-20MB Docker image vs ~200MB+ for Python version.
|
||||
|
||||
---
|
||||
|
||||
## Decision Criteria
|
||||
|
||||
Proceed with Go reimplementation if:
|
||||
|
||||
- [ ] Deployment size is critical (embedded, edge, IoT)
|
||||
- [x] No need for Textual app support
|
||||
- [ ] Want single-binary distribution
|
||||
- [ ] Memory constraints matter
|
||||
|
||||
Keep Python version if:
|
||||
|
||||
- [ ] Need Textual app support
|
||||
- [ ] Rapid iteration is priority
|
||||
- [ ] Team more familiar with Python
|
||||
- [ ] Current deployment size is acceptable
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- GoPyte: https://github.com/scottpeterman/gopyte
|
||||
- go-pty: https://github.com/aymanbagabas/go-pty
|
||||
- Gorilla WebSocket: https://github.com/gorilla/websocket
|
||||
- Docker Go SDK: https://pkg.go.dev/github.com/docker/docker/client
|
||||
- SVGo: https://github.com/ajstarks/svgo
|
||||
- Cobra CLI: https://github.com/spf13/cobra
|
||||
@@ -1,91 +0,0 @@
|
||||
# pyte Patches
|
||||
|
||||
webterm uses [pyte](https://github.com/selectel/pyte) (v0.8.2) as a headless terminal emulator to capture screen state for SVG screenshots. pyte doesn't implement several standard escape sequences, so `AltScreen` in `alt_screen.py` monkeypatches pyte at import time and provides additional methods.
|
||||
|
||||
## Patch 1: CSI S / CSI T — Scroll Up & Down (SU/SD)
|
||||
|
||||
**Status:** Primary fix for ghost content in screenshots.
|
||||
|
||||
### Problem
|
||||
|
||||
When `TERM=xterm-256color`, tmux uses `CSI n S` (Scroll Up) to shift content upward — for example, when Ink-based CLI apps (GitHub Copilot CLI) issue `/clear`. pyte does not implement CSI S or CSI T, silently ignoring the sequences. Old content remains in the screen buffer, appearing as ghost lines in SVG screenshots.
|
||||
|
||||
Real terminals (Ghostty, iTerm2, etc.) handle CSI S/T correctly, so the issue only manifests in pyte-rendered screenshots.
|
||||
|
||||
### Root cause
|
||||
|
||||
pyte's CSI dispatch table has no entry for `"S"` (SU — Scroll Up) or `"T"` (SD — Scroll Down). The `xterm-256color` terminfo entry includes `indn=\E[%p1%dS` and `rin=\E[%p1%dT`, so tmux sends these when the outer terminal advertises support. With simpler TERM values (e.g. `xterm-color`), tmux falls back to DECSTBM + index, which pyte handles correctly.
|
||||
|
||||
### Fix
|
||||
|
||||
At module load time, `alt_screen.py` patches pyte's dispatch tables and event set:
|
||||
|
||||
```python
|
||||
pyte.ByteStream.csi["S"] = "scroll_up"
|
||||
pyte.ByteStream.csi["T"] = "scroll_down"
|
||||
pyte.Stream.csi["S"] = "scroll_up"
|
||||
pyte.Stream.csi["T"] = "scroll_down"
|
||||
pyte.Stream.events = pyte.Stream.events | frozenset(["scroll_up", "scroll_down"])
|
||||
```
|
||||
|
||||
`AltScreen` implements `scroll_up(count)` and `scroll_down(count)`, which shift buffer lines within the current scroll region (respecting DECSTBM margins) and blank the vacated rows.
|
||||
|
||||
### Verification
|
||||
|
||||
```python
|
||||
from webterm.alt_screen import AltScreen
|
||||
import pyte
|
||||
|
||||
screen = AltScreen(80, 24)
|
||||
stream = pyte.ByteStream(screen)
|
||||
|
||||
# Fill screen with content
|
||||
for i in range(24):
|
||||
stream.feed(f"Line {i}\r\n".encode())
|
||||
|
||||
# CSI 6 S — scroll up 6 lines (top 6 lines lost, bottom 6 blanked)
|
||||
stream.feed(b"\x1b[6S")
|
||||
|
||||
# Lines 6-23 shifted to rows 0-17, rows 18-23 are blank
|
||||
assert screen.display[0].strip() == "Line 6"
|
||||
assert screen.display[17].strip() == "Line 23"
|
||||
assert screen.display[18].strip() == ""
|
||||
```
|
||||
|
||||
## Patch 2: Alternate Screen Buffer (DECSET/DECRST 1049)
|
||||
|
||||
**Status:** Core feature, implemented since v1.0.
|
||||
|
||||
### Problem
|
||||
|
||||
pyte doesn't implement private mode 1049 (or 47/1047/1048) for alternate screen buffers. Full-screen programs (tmux, vim, less) switch to an alternate buffer on entry and restore the main buffer on exit. Without this, exiting vim inside tmux would leave vim's screen content overlaid on the shell prompt in screenshots.
|
||||
|
||||
### Fix
|
||||
|
||||
`AltScreen` overrides `set_mode()` and `reset_mode()` to intercept private modes 47/1047/1048/1049. On entry, it deep-copies the current buffer and cursor; on exit, it restores them. The saved buffer is invalidated on resize.
|
||||
|
||||
## Patch 3: Ink Partial Clear Expansion (best-effort)
|
||||
|
||||
**Status:** Secondary defense; largely superseded by Patch 1.
|
||||
|
||||
### Problem
|
||||
|
||||
Ink-based CLI frameworks erase their previous output using repeated `EL2 + CUU1` (erase line + cursor up) pairs. When `/clear` resets Ink's internal line counter, the next frame erases fewer lines than needed. In a real terminal the old content has scrolled into the scrollback buffer, but pyte's fixed-size screen retains it.
|
||||
|
||||
### Fix
|
||||
|
||||
`AltScreen.expand_clear_sequences(data)` pre-processes incoming bytes before feeding to pyte. It detects runs of 3+ `EL2+CUU1` pairs that don't reach row 0 and extends them to cover all lines above the cursor.
|
||||
|
||||
Both `TerminalSession._update_screen()` and `DockerExecSession._update_screen()` call this method after C1 normalization and before `stream.feed()`.
|
||||
|
||||
### Why this is safe
|
||||
|
||||
- Only triggers on runs of **3 or more** `EL2+CUU1` pairs (normal editing uses 1–2).
|
||||
- Only adds erase operations for lines that would already be empty in a real terminal.
|
||||
- Short runs (< 3 pairs) and runs that already reach row 0 are left unchanged.
|
||||
|
||||
## Related
|
||||
|
||||
- `docs/tmux-da-response-filtering.md` — filtering Device Attributes responses from tmux.
|
||||
- `docs/ROADMAP.md` — future Go reimplementation would replace pyte entirely.
|
||||
- `WEBTERM_SCREENSHOT_FORCE_REDRAW` env var — sends SIGWINCH to force app redraw before screenshots.
|
||||
@@ -1,51 +0,0 @@
|
||||
# Fix for "1;10;0c" Display Issue in tmux
|
||||
|
||||
## Problem
|
||||
When entering or refreshing a webterm session running tmux, the text "1;10;0c"
|
||||
appears on the screen. This is a terminal Device Attributes (DA) response that
|
||||
should be filtered out.
|
||||
|
||||
## Root Cause
|
||||
1. **Tmux sends a DA2 query** (`ESC[>c`) when it initializes to detect terminal capabilities
|
||||
2. The terminal responds with `ESC[>1;10;0c` (DA2 response format)
|
||||
3. The filtering code only handled DA1 responses (`ESC[?...c`) but **not DA2** (`ESC[>...c`)
|
||||
4. The unfiltered DA2 response appeared as visible text "1;10;0c" in the replay buffer
|
||||
|
||||
## Solution
|
||||
Updated the DA response filtering patterns in three files to handle all DA variants:
|
||||
|
||||
- **DA1** (Primary): `ESC[?...c` - already handled ✓
|
||||
- **DA2** (Secondary): `ESC[>...c` - **NOW FIXED** ✓
|
||||
- **DA3** (Tertiary): `ESC[=...c` - added for completeness ✓
|
||||
|
||||
### Changed Files
|
||||
1. `src/webterm/terminal_session.py`
|
||||
2. `src/webterm/docker_exec_session.py`
|
||||
3. `src/webterm/local_server.py`
|
||||
|
||||
### Pattern Changes
|
||||
**Before:**
|
||||
```python
|
||||
DA_RESPONSE_PATTERN = re.compile(rb"\x1b\[\?[\d;]+c")
|
||||
DA_PARTIAL_PATTERN = re.compile(rb"\x1b(?:\[(?:\?[\d;]*)?)?$")
|
||||
```
|
||||
|
||||
**After:**
|
||||
```python
|
||||
DA_RESPONSE_PATTERN = re.compile(rb"\x1b\[[?>=][\d;]*c")
|
||||
DA_PARTIAL_PATTERN = re.compile(rb"\x1b(?:\[(?:[?>=][\d;]*)?)?$")
|
||||
```
|
||||
|
||||
## Testing
|
||||
- All 342 existing tests pass ✓
|
||||
- Verified the pattern matches DA1, DA2, and DA3 responses ✓
|
||||
- Confirmed partial buffering works for all variants ✓
|
||||
|
||||
## Technical Details
|
||||
Device Attributes queries/responses:
|
||||
- **DA1**: `ESC[c` → `ESC[?1;10;0c` (terminal capabilities)
|
||||
- **DA2**: `ESC[>c` → `ESC[>1;10;0c` (terminal version - **sent by tmux**)
|
||||
- **DA3**: `ESC[=c` → `ESC[=1;0c` (rarely used)
|
||||
|
||||
The fix ensures all three variants are filtered from both live output and
|
||||
replay buffers when clients reconnect.
|
||||
Reference in New Issue
Block a user