InfiniteEcho
Project

AgentSSH

A local-first SSH terminal and AI operator workspace for inspecting Linux servers with command approvals, live audit history, cancellable agent runs, and session-only credentials.

AgentSSH puts the terminal and the assistant in the same browser workspace. Ask what you want to know, review the proposed action, watch the live terminal output, and keep a clear timeline of what happened. The agent can help, but the operator stays in charge.

MVP built Local-first Human-approved commands React + Node + WebSockets + ssh2
View technical walkthrough See safety model
APPROVE Terminal Workspace user@host:~$ ssh remote-srv Establishing connection... OK remote-srv:~$ policy_load Loaded 20+ regex checks GATED: [df -h] needs OK remote-srv:~$ _ [10:04:12] prompt: inspect disk [10:04:13] policy check: GATED [10:04:15] proposal: df -h [10:04:18] command approved [10:04:19] ws stream: stdout bytes [10:04:22] loop complete (idle) [10:04:24] waiting for operator... OPERATOR CONSOLE AI CO-PILOT CORE ACTIVE AGENT PLAN REMOTE TARGET HOST
Why this exists

Remote work spreads across too many surfaces

Remote server work often spreads across too many surfaces: a terminal, notes, browser searches, documentation, and a separate AI chat. That split makes it easy to lose context, repeat commands, or accept advice without seeing exactly what will happen on the machine.

AgentSSH pulls those pieces into one local workspace. The terminal remains the source of truth, while the assistant becomes a guided layer beside it: proposing one step at a time, explaining why, waiting for approval when needed, and leaving behind a timeline of requests, decisions, output, errors, and cancellations.

The important idea is not "AI runs shell commands." The important idea is AI can help operate a machine only when the product gives the human clear brakes, context, visibility, and a reliable approval path.

What you can do

Features at a glance

Open a real SSH terminal in the browserConnect to a Linux host through a local Node backend and interact with a full xterm.js terminal session that resizes with the workspace.
Ask the agent for operational helpUse natural language for common inspection tasks like checking disk usage, memory pressure, running services, listening ports, logs, system identity, or package state.
Review proposed actions before they runAgent actions become approval cards with a command, risk label, and reason. The operator can approve, deny, or cancel the whole task.
Auto-run known read-only checksFor low-risk inspection commands, the app can skip repetitive approvals when the local policy allows it. Elevated or unknown command shapes stay gated.
Stop work in progressThe UI includes task cancellation and command interruption so a long-running or unwanted agent action can be stopped from the interface.
Search the web with controlsOptional DuckDuckGo search and public page fetching let the agent look up external documentation or version-specific context, with configurable limits and approvals.
Watch the audit trailA live timeline records user requests, proposals, approvals, denials, cancellations, command output events, connection changes, search/fetch events, and agent messages.
Use local or OpenAI-compatible modelsThe backend speaks to any OpenAI-compatible chat endpoint when configured, and falls back to keyword-matched common admin tasks when no model is set up.
Architecture

How it works

AgentSSH is a two-process local web app: a Vite/React client for the operator interface and a Node/Express backend that owns SSH, agent orchestration, policy decisions, and WebSocket messaging.

flowchart TD operator["Operator
Browser UI"] --> ui["React Terminal + Chat
Approvals • Thinking Panel
Audit Timeline • Context Meter"] ui -->|"SSH credentials
(session setup only)"| ws["WebSocket Channel"] ui -->|"terminal input
chat requests
approval decisions
settings"| ws ws --> handler["Node WebSocket Handler"] handler --> ssh["SSH Service
ssh2 shell session"] ssh -->|"terminal bytes"| remote["Remote Linux Host"] remote -->|"stdout/stderr
shell prompts"| ssh ssh -->|"live terminal output
command results"| handler handler --> ai["AI Service
OpenAI-compatible agent loop"] ai -->|"proposed command/search/fetch"| handler handler --> policy["Command Policy Classifier
20+ regex patterns"] policy -->|"auto_run"| handler policy -->|"approval_required"| handler policy -->|"blocked"| handler handler --> approvals["Approval Queue"] approvals -->|"approve / deny / cancel"| handler handler --> search["Search Service
DuckDuckGo HTML + page fetch"] search -->|"public results/excerpts"| handler handler --> audit["Audit Timeline
Immutable event log"] handler -->|"status, thoughts, proposals,
terminal output, audit events"| ws ws --> ui ai -.->|no credentials| ssh ai -.->|summarized command history
and approved tool results| handler classDef browser fill:#2f8cff14,stroke:#2f8cff40,color:#f2f7ff,stroke-width:1px classDef transport fill:#67d9ff14,stroke:#67d9ff40,color:#f2f7ff,stroke-width:1px classDef backend fill:#2dd4bf14,stroke:#2dd4bf40,color:#f2f7ff,stroke-width:1px classDef external fill:#9fb2c814,stroke:#9fb2c840,color:#9fb2c8,stroke-width:1px classDef policy fill:#f59e0b14,stroke:#f59e0b40,color:#f2f7ff,stroke-width:1px class operator,ui browser class ws transport class handler,ssh,ai,policy,approvals,search,audit backend class remote external

User input enters through the browser, but execution decisions happen in the local backend. SSH credentials are used only by the SSH service, while the agent receives task context, approved tool results, and bounded command history.

The browser never connects directly to SSH. The local backend owns the SSH session, agent loop, command policy, and audit stream, then mirrors terminal and agent state to the UI over WebSockets.

Safety model

Built around human approval

AgentSSH treats command execution as a product safety problem. The assistant does not receive SSH credentials, does not directly own the terminal, and does not silently run state-changing work. It proposes an action, labels the risk, gives a reason, and waits for the operator or the configured local policy.

Layer What it protects How it shows up in the UI
Session-only credentials Prevents secrets from becoming project data or model context Password/key fields are used for connection setup and not displayed in logs
Risk classification Separates inspection from state-changing or dangerous operations Approval cards show a risk label and reason
Approval gates Keeps the human in control of meaningful changes Approve, deny, and cancel actions are visible on each proposal
Task cancellation Gives the operator an escape hatch Stop Agent and Stop Command controls appear during active work
Audit timeline Makes the workflow inspectable after the fact Requests, proposals, decisions, output events, errors, and cancellations are timestamped
Web search controls Prevents surprise external lookups or excessive page reads Search can be disabled, capped, and gated before page fetches
Output limits Prevents runaway output from overwhelming the interface Large output is throttled, suppressed, or truncated with visible status
Engineering highlights

Under the hood

WebSocket-first interaction

Terminal bytes, agent status, command proposals, approvals, audit events, and context estimates move through a single live channel.

Session-scoped SSH handling

Credentials are accepted for connection setup but are not persisted, logged, or passed into the AI prompt.

Command policy layer

Proposed shell actions are classified into read-only, write, or destructive categories using 20+ regex patterns, with policy outcomes such as auto-run, approval-required, or blocked.

One-action-at-a-time agent loop

The model is asked to choose a single next action, then output is fed back into the next step. This keeps the workflow inspectable and bounded (max 5 iterations).

Cancellable long runs

Agent tasks have an action cap and must ask before continuing; users can cancel the whole task or interrupt a running command.

Output safety limits

Command output is buffered, throttled, and truncated at 500 characters in the context block so runaway commands do not overwhelm the UI or prompt context.

Sensitive input handling

While an agent command is active, terminal input is locked unless the shell is waiting for a password or passphrase-style prompt.

Context awareness

The UI shows estimated context usage so long troubleshooting runs do not silently drift toward model limits.

Fallback mode

Without an AI endpoint, 25+ keyword-mapped commands cover common server-inspection requests with safe, predictable suggestions.

Web search boundaries

Public web search is opt-in, capped at 5 results and 2 page fetches, cache-aware, and avoids fetching local or private network URLs.

Walkthrough

A typical session

1
Connect

Open AgentSSH locally, enter a host, username, port, and either password or private-key credentials. The connection status turns green when the SSH shell is ready.

2
Ask

Ask a plain-language question: "check disk space", "summarize web server status", or "what changed in the logs."

3
Review

The agent proposes one action at a time. Each proposal includes the command or tool action, a short reason, and a risk label.

4
Run

Approved actions run through the SSH service. Output streams into the terminal and is summarized back into the agent loop.

5
Decide

Approve the next step, deny it, cancel the task, or use the terminal directly. The audit timeline keeps a record of the entire session.

Screenshots

Reserved for future capture

All screenshots require a redaction pass before public display. Captions describe what each frame will contain once cleared. Replace hostnames, usernames, IP addresses, and paths with generic demo values.

Screenshot slot 01 Connection modal — Session-only SSH setup with password or private-key authentication. Use redacted placeholder values.
Screenshot slot 02 Main workspace — A live browser terminal next to the agent chat, with status and context usage in the top bar.
Screenshot slot 03 Command approval card — The agent proposes a single action with a risk label, reason, and approve/deny/cancel controls.
Screenshot slot 04 Agent thinking panel — Reasoning, planning, and analysis events stream into a collapsible panel while the task runs.
Screenshot slot 05 Audit timeline — Requests, proposals, approvals, denials, cancellations, output events, and errors are recorded chronologically.
Screenshot slot 06 Settings modal — Local policy controls for AI endpoint, context window, read-only auto-run, approval memory, and web search limits.
Lessons

What stuck

The hardest part was not wiring a model to a terminal. The harder and more interesting part was designing the control surface around it: when to ask, when to block, how to make progress visible, and how to stop cleanly when the user changes their mind.

Agentic tools need observable state. A spinner is not enough when a tool can touch a real machine.

Approval flows work best when they are part of the main workspace, not a last-second warning.

The terminal should remain the source of truth; the assistant should explain and propose around it.

Safety policy needs layers: credential boundaries, command classification, output limits, cancellation, and auditability all cover different failure modes.

Fallback behavior matters because a useful local tool should degrade gracefully when an AI endpoint is not configured.

Current status

What's built and what's next

AgentSSH is an MVP with a working React/Vite frontend, Node/Express backend, browser terminal, SSH session service, OpenAI-compatible agent loop, command approval cards, cancellable tasks, local policy settings, optional web search, context usage display, and a live audit timeline.

Built

  • React + Vite frontend
  • xterm.js terminal with fit addon and responsive resizing
  • Node.js + Express backend
  • WebSocket message routing for realtime terminal, chat, status, approvals, and audit events
  • SSH connections through the ssh2 library
  • Password and private-key authentication support
  • OpenAI-compatible AI service with streaming responses
  • Fallback keyword mapper for 25+ common server inspection tasks
  • Command policy classifier with read-only, approval-required, and blocked outcomes
  • Approval, denial, cancellation, and remembered session approval paths
  • Optional auto-run for known read-only command shapes
  • Optional elevated read-only auto-run setting (sudo inspection commands)
  • Agent task action cap with approval before continuation
  • Stop Agent and Stop Command controls
  • Web search and page fetch service with limits
  • Context usage estimator
  • Markdown rendering for assistant responses
  • Resizable agent thinking panel
  • Audit timeline for operator-visible history