1. Overview
1.1 The vision
Rysh is an agentic execution environment: terminal panes rebuilt for the age of autonomous coding agents. In tmux, a pane is "a shell". In Rysh, a pane is a conversation — and a conversation can be backed by a shell, an LLM agent, a chat, or an external messaging channel. Because every pane type emits and consumes the same message protocol, the same primitives compose into far more than a pane layout:
- Run AI agents as first-class panes next to your shells, with a full filesystem/shell/git/web toolbox.
- Collaborate by sharing a pane — or a whole tab layout — to teammates over a cloud NATS workspace, in view or control mode.
- Reach the agent from anywhere: a "humanoid" agent can answer Slack threads or emails; a browser extension lets a cloud agent drive your Chrome; a website widget embeds the same engine as a customer chatbot.
- One agentic engine (
rysh-shared) powers all of it — local CLI agents, server-side browser panes, and chatbots — with streaming, context compaction, real sub-agents, and a permission policy. - Extend the toolbox two ways: point Rysh at an OpenAPI/GraphQL spec and Rysh Forge generates governed, context-budget-aware agent tools (plus SDKs, docs, and an MCP server); or
##mcp addan external MCP server and consume its tools directly.
1.2 The big picture
graph TB
subgraph Local["Local machine"]
TUI["rysh TUI
(Bubble Tea client)"]
Daemon["rysh daemon
(actor system + embedded NATS)"]
WebUI["Embedded Web UI
(Gin + WebSocket)"]
TUI <-->|NATS over TCP| Daemon
WebUI <-->|NATS| Daemon
end
subgraph Cloud["rysh-server (cloud backend)"]
API["Gin REST/WS API"]
NATSsrv["Embedded NATS
(+ JetStream)"]
SrvPanes["Server-side AI panes
(browser + chatbot)"]
DB[("PostgreSQL")]
API --- NATSsrv
SrvPanes --- NATSsrv
API --- DB
end
subgraph External["External surfaces"]
Chrome["Chrome extension
(browser control + chat)"]
Widget["Website chatbot widget"]
Slack["Slack / Email / WhatsApp"]
end
Daemon <-->|"upstream share
(NATS, ws.{id}.*)"| NATSsrv
Chrome <-->|"NATS over WS
(browser_pane_proxy)"| NATSsrv
Widget <-->|"closed {type,payload}
(chatbot_pane_proxy)"| NATSsrv
Daemon <-->|"channel creds (REST)
+ inbound msgs"| API
Slack <-->|"webhooks / Socket Mode"| External
Shared["rysh-shared
(agentic engine, msg, tools, provider)"]
Daemon -. imports .-> Shared
SrvPanes -. imports .-> Shared
LLM["LLM providers
Anthropic · OpenAI · Gemini · Ollama"]
Shared -->|CompleteWithTools| LLM
1.3 The five sub-projects in depth
rysh-cli — the execution environment + agent runtime (~124K LOC Go)
A client/daemon application. The daemon is a headless backend hosting a protoactor-go actor tree (workspaces → tabs → lanes → pane-groups → panes) glued by an embedded NATS server. The TUI is a Bubble Tea client that talks to the daemon only over NATS. Panes can run shells (real PTYs with a vendored vt10x terminal emulator) or AI agents. Ships the full agent toolbox (the registered list is in §6.4) (bash, file edit, git, web, email, durable project memory, pane control). Also hosts headless "agents" and "humanoids" (definable via declarative SKILL.md files), pane/tab sharing, voice input, an embedded browser web UI, Rysh Forge (API spec → governed agent tools / SDKs / docs / MCP server, internal/forge), and an MCP client (consume external MCP servers as tools, internal/mcp). The installed binary is named ry (with rysh as an alternate name).
rysh-server — the cloud backend (~31K LOC Go)
A single Go binary bundling: a Gin HTTP/WebSocket API, an embedded NATS server (the cross-machine messaging backbone), PostgreSQL via GORM, server-side agentic panes (for the Chrome extension and the chatbot), Stripe billing, Firebase social auth, and external channel integrations. Its defining responsibility is multi-tenant isolation: a streaming subject-ACL parser in the NATS WebSocket proxy confines each client to ws.{workspaceID}.>.
rysh-shared — the shared core (~15K LOC Go)
The reusable heart imported by both CLI and server:
msg— the unifiedConversationMessage, the JSONNATSEnvelopecodec, the subject/topic scheme, the agentic/memory/browser/chatbot message catalogs, and theNATSPublisher.agentic— theLLMPromptExecutionActor(per-pane manager) and theOrchestratorActor(the autonomous tool-use loop), plus the approval flow and memory injection.provider— theAgenticProvider/ChatProviderabstraction, the neutral conversation model, and the clients for every supported family: Anthropic Messages, the OpenAI Responses API, and OpenAI Chat Completions (Ollama, Gemini).tools— theToolExecutor/ToolRegistryframework and 5 host-agnostic tools (web_search,web_fetch,browser_action,page_context,list_tools).bridge— theNATSBridgethat delivers decoded NATS messages into actor mailboxes.
rysh-proto — reference schemas (~120 LOC Protobuf)
Four proto packages (conversationpb, panemsgpb, memorypb, envelopepb). These are documentation of message shapes only — the actual NATS wire format is JSON. The canonical serialization types are the Go structs in rysh-shared/msg.
rysh-chrome-plugin — browser control + chat (~5K LOC React/TS)
A Chrome MV3 side-panel app. Connects to rysh-server's NATS over a WebSocket. Two jobs: (1) chat with a server-side AI agent, and (2) let that agent control the user's browser — navigate, click, type, scroll, screenshot, extract content, manage tabs — via chrome.scripting injection driven by browser_action requests over NATS.
Beyond the five. Sample MCP servers ship separately: small demo services showing how to expose a REST API as a Model Context Protocol (MCP) server. They are sample code, not a product pillar.
rysh-shared,rysh-protoandrysh-chrome-pluginare each maintained as standalone repositories. A mobile app is not part of the current product; phones use the touch web UI.
1.4 Technology stack
| Concern | Technology |
|---|---|
| Actor runtime | asynkron/protoactor-go |
| Messaging | NATS (nats-server/v2 embedded, JetStream KV), JSON-in-envelope wire format |
| TUI | charmbracelet/bubbletea + lipgloss + bubbles, muesli/termenv |
| Terminal emulation | vendored fork of hinshun/vt10x (adds scrollback), creack/pty |
| LLM | Anthropic Messages API, OpenAI Responses API, OpenAI Chat Completions (Ollama, Gemini) — each streaming or non-streaming, with in-provider retry/backoff + model fallback; Anthropic adds prompt caching and extended thinking. Selectable per session, per scope or per pane |
| Web (server) | gin-gonic/gin, GORM + PostgreSQL, Gorilla WebSocket |
| Auth | Firebase Admin SDK, HS256 JWT, bcrypt, SHA-256 token hashing, AES-256-GCM secret encryption |
| Billing | Stripe (stripe-go v82) |
| Voice | sox/ffmpeg/arecord capture; Deepgram (nova-3) / OpenAI Whisper transcription |
| Extension | React + TypeScript + Zustand + Vite + Tailwind; Chrome MV3 |
| Channels | 13 built-in adapters (rysh-cli/internal/channels/factory.go:16–:64): slack-go (Socket Mode), hand-rolled IMAP/SMTP, Meta Graph API (WhatsApp), Twilio Programmable Messaging (phone/SMS), Telegram, Discord, Signal, iMessage, Teams, WeChat, WeCom, Google Chat, and the embeddable chatbot; other types via channel plugins |
1.5 Glossary (quick reference)
| Term | Meaning |
|---|---|
| Workspace | A configured set of tabs with its own upstream/API key. The cloud tenancy boundary (identified by UUID). |
| Tab | A screen of lanes (one visible at a time). Owns pipeline state. |
| Lane | A vertical column in a tab. Has a flex (horizontal width weight). |
| Pane group | A vertical slot within a lane holding a stack of panes. Has rowFlex (vertical height weight). |
| Pane | One conversation surface: a PTY shell and/or an LLM agent. Panes in a group form a rotating stack. |
| Grid | A tab seeded with R×C lanes/groups at once. |
| Agent | A headless autonomous LLM coding actor (no PTY) registered to a pane. |
| Humanoid | An agent plus external communication channels (Slack/email/etc.) and a memory of per-thread context. |
| Orchestrator | The actor that runs one prompt's tool-use loop to completion. |
| Share / Mirror | Publishing a pane/tab to an upstream NATS workspace (share) and reconstructing it on a subscriber (mirror). |
| Upstream | The remote NATS workspace a CLI connects to for sharing/collaboration (hosted by rysh-server). |
| Mode | A conversation type: shell, ai, rysh, chat, email, slack, chatbot. |
## command |
A Rysh system command typed in a pane (e.g. ##new grid 2x2, ##upstream subscribe <id>). |
| Envelope | The JSON wrapper {t: typeTag, r: replyTo, p: payload} for every NATS message. |
| Codec | The registry mapping type tags ↔ Go types for encode/decode. |
| Bridge | The component subscribing NATS subjects and delivering decoded messages into an actor mailbox. |
See 11. Glossary & Configuration for the full glossary and configuration reference.