Home › Docs › Architecture › 1. Overview

1. Overview

1.1 The vision

Rysh is an agentic execution environment: terminal panes rebuilt for the age of autonomous coding agents. In tmux, a pane is "a shell". In Rysh, a pane is a conversation — and a conversation can be backed by a shell, an LLM agent, a chat, or an external messaging channel. Because every pane type emits and consumes the same message protocol, the same primitives compose into far more than a pane layout:

  • Run AI agents as first-class panes next to your shells, with a full filesystem/shell/git/web toolbox.
  • Collaborate by sharing a pane — or a whole tab layout — to teammates over a cloud NATS workspace, in view or control mode.
  • Reach the agent from anywhere: a "humanoid" agent can answer Slack threads or emails; a browser extension lets a cloud agent drive your Chrome; a website widget embeds the same engine as a customer chatbot.
  • One agentic engine (rysh-shared) powers all of it — local CLI agents, server-side browser panes, and chatbots — with streaming, context compaction, real sub-agents, and a permission policy.
  • Extend the toolbox two ways: point Rysh at an OpenAPI/GraphQL spec and Rysh Forge generates governed, context-budget-aware agent tools (plus SDKs, docs, and an MCP server); or ##mcp add an external MCP server and consume its tools directly.

1.2 The big picture

graph TB
    subgraph Local["Local machine"]
        TUI["rysh TUI
(Bubble Tea client)"] Daemon["rysh daemon
(actor system + embedded NATS)"] WebUI["Embedded Web UI
(Gin + WebSocket)"] TUI <-->|NATS over TCP| Daemon WebUI <-->|NATS| Daemon end subgraph Cloud["rysh-server (cloud backend)"] API["Gin REST/WS API"] NATSsrv["Embedded NATS
(+ JetStream)"] SrvPanes["Server-side AI panes
(browser + chatbot)"] DB[("PostgreSQL")] API --- NATSsrv SrvPanes --- NATSsrv API --- DB end subgraph External["External surfaces"] Chrome["Chrome extension
(browser control + chat)"] Widget["Website chatbot widget"] Slack["Slack / Email / WhatsApp"] end Daemon <-->|"upstream share
(NATS, ws.{id}.*)"| NATSsrv Chrome <-->|"NATS over WS
(browser_pane_proxy)"| NATSsrv Widget <-->|"closed {type,payload}
(chatbot_pane_proxy)"| NATSsrv Daemon <-->|"channel creds (REST)
+ inbound msgs"| API Slack <-->|"webhooks / Socket Mode"| External Shared["rysh-shared
(agentic engine, msg, tools, provider)"] Daemon -. imports .-> Shared SrvPanes -. imports .-> Shared LLM["LLM providers
Anthropic · OpenAI · Gemini · Ollama"] Shared -->|CompleteWithTools| LLM

1.3 The five sub-projects in depth

rysh-cli — the execution environment + agent runtime (~124K LOC Go)

A client/daemon application. The daemon is a headless backend hosting a protoactor-go actor tree (workspaces → tabs → lanes → pane-groups → panes) glued by an embedded NATS server. The TUI is a Bubble Tea client that talks to the daemon only over NATS. Panes can run shells (real PTYs with a vendored vt10x terminal emulator) or AI agents. Ships the full agent toolbox (the registered list is in §6.4) (bash, file edit, git, web, email, durable project memory, pane control). Also hosts headless "agents" and "humanoids" (definable via declarative SKILL.md files), pane/tab sharing, voice input, an embedded browser web UI, Rysh Forge (API spec → governed agent tools / SDKs / docs / MCP server, internal/forge), and an MCP client (consume external MCP servers as tools, internal/mcp). The installed binary is named ry (with rysh as an alternate name).

rysh-server — the cloud backend (~31K LOC Go)

A single Go binary bundling: a Gin HTTP/WebSocket API, an embedded NATS server (the cross-machine messaging backbone), PostgreSQL via GORM, server-side agentic panes (for the Chrome extension and the chatbot), Stripe billing, Firebase social auth, and external channel integrations. Its defining responsibility is multi-tenant isolation: a streaming subject-ACL parser in the NATS WebSocket proxy confines each client to ws.{workspaceID}.>.

rysh-shared — the shared core (~15K LOC Go)

The reusable heart imported by both CLI and server:

  • msg — the unified ConversationMessage, the JSON NATSEnvelope codec, the subject/topic scheme, the agentic/memory/browser/chatbot message catalogs, and the NATSPublisher.
  • agentic — the LLMPromptExecutionActor (per-pane manager) and the OrchestratorActor (the autonomous tool-use loop), plus the approval flow and memory injection.
  • provider — the AgenticProvider/ChatProvider abstraction, the neutral conversation model, and the clients for every supported family: Anthropic Messages, the OpenAI Responses API, and OpenAI Chat Completions (Ollama, Gemini).
  • tools — the ToolExecutor/ToolRegistry framework and 5 host-agnostic tools (web_search, web_fetch, browser_action, page_context, list_tools).
  • bridge — the NATSBridge that delivers decoded NATS messages into actor mailboxes.

rysh-proto — reference schemas (~120 LOC Protobuf)

Four proto packages (conversationpb, panemsgpb, memorypb, envelopepb). These are documentation of message shapes only — the actual NATS wire format is JSON. The canonical serialization types are the Go structs in rysh-shared/msg.

rysh-chrome-plugin — browser control + chat (~5K LOC React/TS)

A Chrome MV3 side-panel app. Connects to rysh-server's NATS over a WebSocket. Two jobs: (1) chat with a server-side AI agent, and (2) let that agent control the user's browser — navigate, click, type, scroll, screenshot, extract content, manage tabs — via chrome.scripting injection driven by browser_action requests over NATS.

Beyond the five. Sample MCP servers ship separately: small demo services showing how to expose a REST API as a Model Context Protocol (MCP) server. They are sample code, not a product pillar. rysh-shared, rysh-proto and rysh-chrome-plugin are each maintained as standalone repositories. A mobile app is not part of the current product; phones use the touch web UI.

1.4 Technology stack

Concern Technology
Actor runtime asynkron/protoactor-go
Messaging NATS (nats-server/v2 embedded, JetStream KV), JSON-in-envelope wire format
TUI charmbracelet/bubbletea + lipgloss + bubbles, muesli/termenv
Terminal emulation vendored fork of hinshun/vt10x (adds scrollback), creack/pty
LLM Anthropic Messages API, OpenAI Responses API, OpenAI Chat Completions (Ollama, Gemini) — each streaming or non-streaming, with in-provider retry/backoff + model fallback; Anthropic adds prompt caching and extended thinking. Selectable per session, per scope or per pane
Web (server) gin-gonic/gin, GORM + PostgreSQL, Gorilla WebSocket
Auth Firebase Admin SDK, HS256 JWT, bcrypt, SHA-256 token hashing, AES-256-GCM secret encryption
Billing Stripe (stripe-go v82)
Voice sox/ffmpeg/arecord capture; Deepgram (nova-3) / OpenAI Whisper transcription
Extension React + TypeScript + Zustand + Vite + Tailwind; Chrome MV3
Channels 13 built-in adapters (rysh-cli/internal/channels/factory.go:16–:64): slack-go (Socket Mode), hand-rolled IMAP/SMTP, Meta Graph API (WhatsApp), Twilio Programmable Messaging (phone/SMS), Telegram, Discord, Signal, iMessage, Teams, WeChat, WeCom, Google Chat, and the embeddable chatbot; other types via channel plugins

1.5 Glossary (quick reference)

Term Meaning
Workspace A configured set of tabs with its own upstream/API key. The cloud tenancy boundary (identified by UUID).
Tab A screen of lanes (one visible at a time). Owns pipeline state.
Lane A vertical column in a tab. Has a flex (horizontal width weight).
Pane group A vertical slot within a lane holding a stack of panes. Has rowFlex (vertical height weight).
Pane One conversation surface: a PTY shell and/or an LLM agent. Panes in a group form a rotating stack.
Grid A tab seeded with R×C lanes/groups at once.
Agent A headless autonomous LLM coding actor (no PTY) registered to a pane.
Humanoid An agent plus external communication channels (Slack/email/etc.) and a memory of per-thread context.
Orchestrator The actor that runs one prompt's tool-use loop to completion.
Share / Mirror Publishing a pane/tab to an upstream NATS workspace (share) and reconstructing it on a subscriber (mirror).
Upstream The remote NATS workspace a CLI connects to for sharing/collaboration (hosted by rysh-server).
Mode A conversation type: shell, ai, rysh, chat, email, slack, chatbot.
## command A Rysh system command typed in a pane (e.g. ##new grid 2x2, ##upstream subscribe <id>).
Envelope The JSON wrapper {t: typeTag, r: replyTo, p: payload} for every NATS message.
Codec The registry mapping type tags ↔ Go types for encode/decode.
Bridge The component subscribing NATS subjects and delivering decoded messages into an actor mailbox.

See 11. Glossary & Configuration for the full glossary and configuration reference.