Codebase Lowdown

Hyperbolic Gateway — architecture, structure and findings Version 1.4.5 · 22 October 2026

This document explains what the code does, how the parts connect, and what to fix next. It uses short sentences and simple terms. It is a map, not a tutorial. For install and build steps, see BUILDING.md. For the tool list, see the README.

1. What this program does

Hyperbolic Gateway is a desktop application. It runs a real Chromium browser on the local machine. An AI agent controls that browser through the Model Context Protocol (MCP).

The agent does not use a remote browser. It does not use a headless browser. The browser is visible, and it keeps cookies, tabs, downloads and memory between sessions.

The program has two processes:

1. The Electron app. It owns the window, the browser tabs, the HUD, the database and the local files. It listens on http://127.0.0.1:4111.
2. The MCP server. Each agent client starts its own copy (bin/mcp-server.cjs). It speaks JSON-RPC over stdin/stdout and forwards each tool call to the Electron app over HTTP.

The program also has one optional decision path. If a decision model is configured, the agent can call decide or drive. These tools ask a model for a typed choice. The model never writes text for the page. The agent supplies text.

2. System map

flowchart LR
  Agent["Agent / MCP client<br/>(OpenCode, Claude, Cursor, ...)"]
  MCP["MCP server<br/>bin/mcp-server.cjs<br/>stdio JSON-RPC"]
  Bridge["HTTP bridge<br/>127.0.0.1:4111"]
  Chassis["BrowserChassis<br/>orchestrator"]
  Tabs["WebContentsView tabs"]
  Prim["Primitives<br/>observe · act · wait_for · extract"]
  Marks["Set-of-Marks overlay"]
  Input["CDP input + Bézier telemetry"]
  Cortex[("LanceDB Cortex")]
  Vaults[("Session / script / download vaults")]
  HUD["HUD renderer<br/>settings · recorder · decisions"]
  Web[("Live web")]

  Agent -->|stdio| MCP
  MCP -->|HTTP| Bridge
  Bridge --> Chassis
  Chassis --> Tabs
  Chassis --> Prim
  Chassis --> Marks
  Chassis --> Input
  Chassis --> Cortex
  Chassis --> Vaults
  Tabs --> Web
  Input --> Tabs
  HUD -. IPC .-> Chassis

The bridge is the only control surface. The MCP server is stateless. All state lives in the Electron app.

3. Process and window model

The HUD and the web pages are different renderers. The HUD is not a web page. A tool call cannot reach window.electronAPI. This separation is deliberate: page JavaScript (evaluate_javascript) must not be able to change settings or read local files.

4. Call path

sequenceDiagram
  participant A as Agent
  participant M as MCP server
  participant B as HTTP bridge
  participant C as BrowserChassis
  participant P as Page (WebContentsView)

  A->>M: tools/call { name: "act", arguments }
  M->>B: POST /act (JSON)
  B->>C: chassis.act(...)
  C->>P: CDP Input.dispatchMouseEvent
  P-->>C: DOM changes / events
  C-->>B: result object
  B-->>M: JSON response
  M-->>A: text content (+ image block)

Rules that the code depends on:

• stdout carries JSON-RPC only. Logs go to data/logs/mcp-server.log.
• stderr is a short boot banner by default. A client that does not read stderr must not be able to block the server. See HANDOFF_ORCHA_MCP_STDERR.md.
• The bridge binds to 127.0.0.1 only.
• KUANG_BRIDGE_PORT changes the port. KUANG_BRIDGE_URL points the MCP server at it.
• KUANG_BRIDGE_TIMEOUT_MS caps one bridge round-trip (default 300 000 ms).

5. Directory map

src/
  main/            Electron main process
    index.ts       entry point, window, IPC, service wiring
    browser-chassis.ts   central orchestrator (1 865 lines)
    http-bridge.ts       HTTP routes (1 162 lines)
    primitives/          observe, act, wait_for, extract, ...
    marks/               Set-of-Marks overlay and targeting
    telemetry/           Bézier pointer, typing, fingerprints
    tabs/                multi-tab WebContentsView manager
    cortex/              LanceDB memory and embedder
    harvester/           Readability + Turndown markdown
    shield/              consent/GDPR dismissal
    hil/                 human-in-the-loop requests
    decide/              optional typed decision client + guard
    drive/               observe → decide → act loop
    recording/           timelapse capture service
    downloads/           download routing and parsing
    session/             cookie/storage vaults
    scripts/             script vault
    network/             CDP network + stream capture
    search/ crawl/ research/ prerender/ snapshots/ llm/ updates/ tray/ menu/ utils/
  mcp/             MCP server and tool catalog
    tools-catalog.ts     tool schemas, endpoints, formatters
    bridge-server.ts     stdio server, resources, dispatch
  preload/         contextBridge API
  renderer/        HUD (main.ts) and timelapse encoder (recording.ts)
  shared/          shared types and version

public/            welcome manual, assets, icons
scripts/           build, release, tests, one-off tools
docs/              architecture notes, field notes, this file
bin/               bundled MCP server (unpacked in installers)

All 46 TypeScript files are reachable from four entry points: src/main/index.ts, src/preload/index.ts, src/renderer/main.ts and src/mcp/bridge-server.ts. There are no unreachable TypeScript modules.

6. Main subsystems

6.1 BrowserChassis

src/main/browser-chassis.ts is the coordinator. It owns the active tab, the viewport bounds, the primitives object, the network capture, the harvester, the download manager and the memory writes. It is the largest file in the project. New code should not make it larger. See finding F4.

6.2 Orthogonal primitives

src/main/primitives/primitives.ts exposes the preferred tool surface:

observe · act · wait_for · expect · find · extract · collect_all · page_state · diff · export_har.

The rules are:

• Read with observe, not screenshots, unless layout matters.
• Act with act(mark_id), a selector, or coordinates. Do not inject JavaScript to click.
• Wait with wait_for, not a fixed sleep.
• Assert with expect; compare with page_state / diff.

6.3 Set-of-Marks

src/main/marks/marks-overlay.ts adds numbered badges to interactive elements. It traverses open shadow roots and computes a safe click point. The mark table is cheap text. The image is optional.

6.4 Input and telemetry

src/main/telemetry/input-simulator.ts sends input over CDP. It uses Bézier pointer paths and Gaussian typing delays. CDP input is focus-independent: the app does not need the OS foreground. The fallback is webContents.sendInputEvent.

6.5 Memory

src/main/cortex/cortex-db.ts uses LanceDB with a vector column and a BM25 full-text index. A deterministic 128-dimension embedder keeps the install small. Visited pages, downloads, bookmarks and research results are indexed.

6.6 Decision layer (optional)

src/main/decide/decision-client.ts calls a hosted TypeSafe model or a local ONNX model. The result is a typed choice: a component and an operation. decision-guard.ts rejects unknown targets, low confidence and impossible actions. src/main/drive/driver.ts repeats observe → decide → act until it finishes, needs a human, or runs out of steps. Age gates and CAPTCHAs are human hand-offs. They are detected before any model call.

6.7 Recording

The main process owns the file and the config. The HUD renderer owns capture and encoding. Frames are captured at intervalMs and stamped 1000/fps apart. This is true timelapse. MediaRecorder cannot retime frames, so the code uses WebCodecs and webm-muxer.

6.8 Bridge and catalog

tools-catalog.ts maps each tool name to a Zod schema, an HTTP route and a formatter. http-bridge.ts serves those routes. npm run test:mcp:contract parses both files and fails if a tool points at a route that does not exist. Keep this test green.

7. Findings

Orphans and dead weight

Irregularities and risks

Optimisations, in order of value

1. Split the monoliths (F4). This reduces review risk more than any micro-optimisation.
2. Move state to userData (R1). This fixes a class of macOS and packaged-app failures.
3. Prune scripts and duplicate bundles (F1, F2). Less code to scan, smaller repository.
4. Add the contract test to CI (R3). One minute of CI prevents a class of broken tools.
5. Trim assets (R7). About 5 MB saved in the repository, and a faster README.
6. Enable noUnusedLocals / noUnusedParameters. The current tsconfig disables both. Turn them on file by file, or fix the existing warnings first.
7. Lazy-load heavy dependencies. @huggingface/transformers and the ONNX runtime dominate install size. Load them only when the local decision model or embedder is used.

8. Walkthrough: search a flight on an airline site

This example shows the normal tool order. The site is generic. Selectors are illustrative. Always prefer a mark ID from observe or find over a guessed CSS selector.

Goal: find one-way economy flights from London to New York on 14 November 2026, and return the cheapest three.

// 1. Confirm the bridge and the active tab.
{ "tool": "bridge_health" }

// 2. Open the airline. Consent banners are dismissed automatically by default.
{ "tool": "navigate_to", "arguments": { "url": "https://www.example-airline.com" } }

// 3. Read the page as compact markdown. Look for the "Book a flight" entry point.
{ "tool": "observe", "arguments": { "as": "markdown" } }

// 4. Find the link by role and name. This returns a mark ID.
{ "tool": "find", "arguments": { "role": "link", "name": "Book a flight" } }

// 5. Click the mark. `act` reports what changed: URL, revision, marks.
{ "tool": "act", "arguments": { "mark_id": 4, "action": "click" } }

// 6. Wait for the booking form, not for a fixed number of seconds.
{ "tool": "wait_for", "arguments": { "conditions": [
  { "selector": "form[data-testid=flight-search]" },
  { "network_idle": true }
] } }

// 7. Choose one-way, then type the origin. Pick the suggestion; do not just press Enter.
{ "tool": "act", "arguments": { "selector": "label[for=trip-oneway]", "action": "click" } }
{ "tool": "act", "arguments": { "selector": "#origin", "action": "type", "text": "London" } }
{ "tool": "wait_for", "arguments": { "conditions": [ { "selector": "[role=option]", "text": "London (LHR)" } ] } }
{ "tool": "act", "arguments": { "selector": "[role=option]", "action": "click" } }

// 8. Type the destination and the date, then submit.
{ "tool": "act", "arguments": { "selector": "#destination", "action": "type", "text": "New York" } }
{ "tool": "act", "arguments": { "selector": "#depart-date", "action": "type", "text": "2026-11-14" } }
{ "tool": "act", "arguments": { "selector": "button[type=submit]", "action": "click" } }

// 9. Wait for results. If the page paginates, `collect_all` harvests the list.
{ "tool": "wait_for", "arguments": { "conditions": [
  { "selector": "[data-testid=flight-card]" },
  { "count": { "selector": "[data-testid=flight-card]", "min": 1 } }
] } }

// 10. Extract structured rows, then sort by price in the agent.
{ "tool": "extract", "arguments": {
  "kind": "list",
  "selector": "[data-testid=flight-card]"
} }

Rules for this workflow:

• navigate_to returns a result, but the page may still load. Use wait_for after a navigation when the next action depends on new content.
• act returns evidence. Read ok, delta and target before you continue.
• Use find or a mark ID when the DOM is complex. Do not guess generated class names.
• If the site shows a CAPTCHA or an age gate, stop. Call request_human_intervention, or let Hyperbolic Gateway raise the hand-off. Do not retry the form.
• Use evaluate_javascript only when the primitives cannot express the read or the action.

9. Commands

Release flow: push a v* tag. CI builds Windows, macOS and Linux. The publish job uploads the assets. A patch tag publishes itself; a minor or major tag stays a draft for a human. See BUILDING.md §4.

10. How to change the code safely

Add a tool

1. Add the entry to src/mcp/tools-catalog.ts. Give it an endpoint, a Zod schema and a formatter.
2. Add the route to src/main/http-bridge.ts.
3. Run npm run test:mcp:contract.
4. Run npm run test:mcp:surface with the app open.
5. Add a row to the README tool table. Run npm run manual:tools.

Change the browser behaviour

1. Prefer the primitives layer. Add a primitive before you add a one-off tool.
2. Keep act honest: return evidence, and a typed error when the action did nothing.
3. Add a live test in scripts/test-*.mjs.

Change settings

Settings are IPC-only. The HUD calls window.electronAPI. Do not add an HTTP route that writes settings. The recording tool gate depends on this rule.

Release

1. Update package.json, src/shared/version.ts and CHANGELOG.md.
2. Run npx tsc --noEmit, npm run build, npm run test:mcp:contract.
3. Commit, tag vX.Y.Z, push the tag.
4. Watch the CI run. For a minor or major tag, publish the draft after a check.

11. Glossary