⚡ Hypagon Gateway

The local browser for AI agents.

"I'll hack the Gibson"
— Dade Murphy (AKA Crash Override), Circa 1995

🧭 Index

• About — What it is · A punk-ethos alternative to browser clouds
• The deck — Theme & Cyberdeck UI · Ghost cursor telemetry · Focus-independent input · Multi-tab browsing
• Reading the web — Orthogonal primitive layer · Construct harvester · JEV page compaction · Set-of-Marks
• Driving the web — Shield-breaker · Form filler · Human-in-the-loop
• Memory & data — Cortex · Downloads · Media download integration · Session vault · Script vault · Timelapse recorder
• Inspection — Network & API streams · WebSocket & SSE
• Operating it — System tray · MCP tools (16 advertised · +2 decision · +3 JEV) · Agent skills · REST wrappers
• Building on it — Architecture lowdown · Codebase & docs · Quickstart · Testing · OKF knowledge
• Optional decision layer — decide / drive design notes · Control layer: targeting, consent, human takeover · Shadow DOM & iframes: what Hypagon Gateway can see and click · Shadow Walk: sealed-content plan · Field notes from dogfooding · Changelog
• Release notes — What shipped · All releases
• Lore — Neuromancer inspiration

🕶️ What it is

Hypagon Gateway is a visible, stateful web browser that any MCP agent can drive.

Most agent browsers are a library you integrate, or a cloud you rent. Hypagon Gateway is a desktop application you own: a real Chromium window on your machine, controlled over the Model Context Protocol. Your agent runs where you already run it — local or remote — and Hypagon Gateway supplies the browser, the input, and the memory.

• A real browser, not a simulation. Visible Chromium with hardware acceleration, persistent cookies, real tabs, downloads, and a human-sized viewport. No headless fingerprint.
• Structured first, pixels when needed. observe returns a compact, addressable view of the page (markdown, text, accessibility tree, or Set-of-Marks). take_screenshot is there when layout matters.
• Input that looks human. Clicks and keys travel over the Chrome DevTools Protocol with Bézier paths and Gaussian typing cadence — and they land even when another application owns the foreground.
• State that survives. LanceDB memory, bookmark and session vaults, download harvesting and a script vault mean the next session starts where the last one stopped.
• Self-hosted by construction. stdio MCP plus a localhost bridge. No account, no dashboard, and no telemetry leaving the machine.

_{Recorded from the running app: Google → “hypagon definition” → AI Mode. Open the MP4 if your browser does not animate WebP.}

[!WARNING] EXPERIMENTAL PROTOTYPE: Hypagon Gateway is currently under active, experimental development. Tool contracts, telemetry tuning, and database schemas may evolve rapidly. Built for developers experimenting with multimodal local AI agents.

[!IMPORTANT] RESEARCH PURPOSES ONLY: This software is provided strictly for educational, testing, and local AI agent research purposes. It is not intended for malicious use, automated credential stuffing, unauthorized exploitation, or violating third-party websites' Terms of Service. Users are responsible for complying with all applicable laws and terms.

Installers: grab them from Releases — Windows Setup/Portable/zip, macOS dmg/zip (x64 + arm64), Linux AppImage/deb/tar.gz. macOS is unsigned for now: right-click → Open (or xattr -dr com.apple.quarantine "/Applications/Hypagon Gateway.app"). Built by GitHub Actions; see docs/BUILDING.md.

🤘 A Punk-Ethos, DIY Alternative to browser-use, Playwright Agents & Hosted Browser Clouds

Most "AI browser" tooling today is a library you integrate, or a cloud you rent. browser-use is a popular MIT-licensed Python framework that drives Playwright with its own agent loop (plus an optional hosted cloud); Playwright / Puppeteer / Selenium are general-purpose drivers where you write the loop; and managed browser clouds run a remote Chromium and bill per session. They're good tools — this isn't a rivalry, it's a different shape.

Hypagon Gateway is the other road: a single, self-hosted desktop app you own (Electron, no account, no dashboard). It runs a real, visible Chromium on your own machine and speaks MCP over stdio (+ a localhost bridge), so it drops into whatever agent runner and whichever model you already use — local or remote, with no cloud account, no API key, no seat pricing, and no telemetry leaving your box.

That's the punk ethos: one operator, one machine, a tool you can read, fork and keep. Built by one person, not a company. No VC, no roadmap-by-committee, no dashboard.

How it differs, honestly

What that buys you in practice

• Self-hosted and local-first. The browser, the memory, the logs and the model all stay on the box.
• Built for a machine you're also using. Input goes over CDP and is focus-independent — clicks and typing land while your editor owns the foreground — and occluded windows keep hydrating and scrolling, so long harvests keep running behind your other apps.
• Tier-graded tool surface, not a token bonfire. Reading is observe(as:"markdown") with inline [#id] marks (no screenshot tax); interaction is one act; [TIER 1/2/3] prefixes plus a tools/recommended resource tell the agent what to prefer.
• Honest about its limits. Detached reads are gated, evaluate_javascript is "use deliberately" rather than banned, and failures are machine-coded (STALE_REF, TIMEOUT, NO_TARGET, BAD_REQUEST, NOT_SUPPORTED) instead of vibes.
• No framework lock-in. Because the interface is MCP, your runner stays yours: swap the agent, swap the model, keep the tool.

New here? Plain-English FAQ

What is this, in one sentence? A web browser that an AI can drive for you: it opens real pages, reads them, clicks, types, fills in forms, and remembers what it found.

Is it a browser I use myself? It's a real browser window. You can watch it, and you can take over at any time — most people just let their AI do the driving.

Do I need to be a programmer? To use it with an AI assistant you already have: no — you paste one line of configuration into that assistant. To change how it behaves internally: yes, it's a code project.

What's an "MCP server", in plain terms? MCP is a standard plug shape for AI tools. If your AI app accepts MCP plugins, it can use this one. Claude Desktop, Cursor and several other apps speak it.

Can I install an agent skill too? Yes. The hb-browser and hb-wrap skills are at skills/.

hermes skills install https://raw.githubusercontent.com/hyperbolictonic/hypagon/main/skills/hb-browser/SKILL.md
hermes skills install https://raw.githubusercontent.com/hyperbolictonic/hypagon/main/skills/hb-wrap/SKILL.md

npx skills add hyperbolictonic/hypagon --skill hb-browser
npx skills add hyperbolictonic/hypagon --skill hb-wrap

Add -g -y to the npx commands for a non-interactive global install.

Which AI can drive it? Any assistant that supports MCP — a cloud service, or a model running on your own computer. You choose, and you aren't locked to one provider.

Do I need to pay for anything? No subscription and no account. The software is free and open source. The only cost is whatever you already pay for your AI, if anything.

Where does my data go? Nowhere. The pages it reads, the notes it keeps and the logs all stay on your computer. There's no company server in the middle and no usage tracking.

Could a website tell it's a bot? It's built to be hard to spot: a normal visible browser, mouse movement and typing that look human, and it clears cookie banners politely. Nothing is unbeatable, but it isn't the obvious "automated script" that gets blocked.

What happens at a CAPTCHA? It doesn't solve them. It asks you to step in (it can send a notification and bring the window forward), then continues once you're done.

Does it run headless on a server? Not really, and that's on purpose. It runs a visible browser on a normal desktop, which is exactly why it looks human. For large-scale server-side scraping, a different tool fits better.

Is using it allowed? It's for personal research and testing. You still have to follow each website's rules and the law — running it yourself doesn't change that. Don't use it to break into accounts or ignore terms of service.

What do I need to run it? A normal Windows desktop (that's the tested target). If your computer can run a browser and an AI assistant, it can run this.

Is it finished? No. It's an experimental project by one developer — expect rough edges and things that change.

FAQ for developers — browsers, memory, and the tricky bits

Is there a self-hosted, open-source alternative to browser-use? Yes — MIT-licensed and entirely local. But note the shape: this is a tool surface, not an agent framework. If you want a Python framework that owns the loop, browser-use is the right tool; if you want a self-hosted browser that your own agent drives, this is it.

Can I use it with Claude Desktop, Cursor, or a local LLM? Yes. It's an MCP server, so any MCP client works - point your client at bin/mcp-server.cjs and the hb-browser tool namespace appears (legacy kuang alias still works).

Does it need a cloud account, API key, or per-session billing? No. There is no cloud tier. The only network calls are the ones your agent makes to the pages it visits (plus, optionally, a search engine via search_web).

Is it headless? Deliberately the opposite. It runs a visible Chromium so a human can watch, take over at a CAPTCHA, and so the session stays an honest, real browser. Headless-at-scale scraping is a different job; this is a single operator's console.

Will Cloudflare block it like Puppeteer / Playwright? It's built for exactly that fight: real CDP input with isTrusted: true, cubic Bézier pointer kinematics, Gaussian typing cadence, a genuine cookie jar, and an autonomous consent-modal breaker. No promises — anti-bot is an arms race — but it is not naive headless emulation.

Can I point it at my existing Playwright test suite? No. This isn't a test driver; Playwright remains excellent for deterministic suites. Grade Eleven is an agent-facing browser an LLM operates.

Where does memory live? Locally, in LanceDB ("Cortex"): harvested pages and vaulted scripts are embedded and semantically recallable, so your agent keeps research context across sessions without a hosted vector store.

Is it production-ready? No — it's an experimental prototype for local AI research. That's rather the point.

Punk ethos, not carte blanche. Owning your tooling doesn't change the law: the RESEARCH PURPOSES ONLY terms at the top of this README still apply. The point is ownership and readability, not immunity.

🎯 Core Architecture Principles

1. 100% Real Browser Chassis: No headless emulation, synthetic user-agent spoofing, or sandboxed scraping. Hypagon Browser runs a full-fledged, visible Chromium instance (headless: false) with hardware acceleration, audio/video capabilities, and authentic cookie/session persistence.
2. Multimodal Vision Integration: Specifically tailored for multimodal models (e.g. Bonsai 2 27B). The agent takes high-res viewport screenshots (take_screenshot) to visually understand layouts, buttons, and dynamic canvas states, and can act directly by coordinate, selector, or mark badge (act).
3. Full Developer Console & Script Execution: Gives the agent full console access (get_console_logs) to inspect runtime errors and API output, plus arbitrary JavaScript evaluation (evaluate_javascript) to interact with complex SPAs.
4. Behavioral Bot Detection Bypass: Bypasses Cloudflare, Akamai, Datadome, and behavioral bot sensors by avoiding synthetic element.click() events (isTrusted: false). Instead, it dispatches real input events over the Chrome DevTools Protocol (Input.dispatchMouseEvent / Input.dispatchKeyEvent / mouseWheel) accompanied by organic cubic Bézier pointer movement and human Gaussian typing cadence. CDP delivery is focus-independent — it lands even when another application owns the OS foreground — with webContents.sendInputEvent kept only as a fallback.

🎨 Theme & Cyberdeck Interface

Built with Tailwind CSS v4 and DaisyUI 5:

• Custom Theme: mark-eleven
• Color Inspiration: Antigravity electric cyan (oklch(75% 0.18 215)), VS Code workspace slate (oklch(21% 0.012 250)), and Photoshop deep charcoal neutrals.
• DaisyUI Components:
  • navbar & mockup-browser address bar with real-time navigation and SSL/protocol status.
  • status and badge indicators reflecting live agent lifecycle (IDLE, TRAVERSING, SIMULATING_INPUT, HARVESTING).
  • tabs (tabs-border) for switching between Telemetry Kinematics, Construct Terminal, and MCP Matrix Logs.
  • stats / stat blocks tracking coordinates, Bézier velocity, tokens saved, and interaction counters.
  • mockup-code for viewing synthesized Markdown memory constructs with token downsampling ratios.

🕹️ Ghost Cursor Telemetry Engine

• Organic Bézier Trajectories: Natural cubic Bézier curve calculation with quintic ease-in/ease-out acceleration, micro-jitter, and target deceleration.
• Focus-Independent CDP Delivery: Input events are dispatched over the tab's already-attached Chrome DevTools Protocol debugger (Input.dispatchMouseEvent / Input.dispatchKeyEvent / Input.mouseWheel), with Emulation.setFocusEmulationEnabled so focus-gated page JS still responds. Clicks, keystrokes, and momentum wheel scrolling land regardless of which window owns the OS foreground. webContents.sendInputEvent remains only as a fallback (and only that path needs the window focused). The active transport is reported by bridge_health → input.transport (cdp | native).
• Anti-Bot Defense Bypass: Avoids heuristic bot triggers by behaving like an authentic human operator; every interaction is isTrusted: true because it is genuine browser input, not a synthetic DOM .click().

🎛️ Focus-Independent Input & Background Rendering

Two real-desktop failure modes are handled explicitly so long harvests and HIL flows stay reliable:

• Input is not tied to OS focus. Synthetic input is delivered over CDP and therefore reaches the renderer even while you work in another application. If the debugger is unavailable the simulator falls back to webContents.sendInputEvent and brings the window forward first (that path is the only one that steals the foreground). bridge_health exposes input.transport so an agent can tell which path is live.
• Occluded windows keep rendering. Browsing tabs run with backgroundThrottling: false, and the app boots with --disable-backgrounding-occluded-windows, --disable-renderer-backgrounding, and --disable-background-timer-throttling. Without these, Chromium reports the page hidden, throttles timers to ≥1s, and stops producing frames — so scroll events never fire and IntersectionObserver/rAF hydration stalls (the classic "lazy list stops growing" bug).
• Viewport gating is explicit. Real input needs a non-zero viewport; a fresh or hidden tab can briefly report 0x0. Call set_window_visibility({ visible: true }) and wait for a non-zero viewport before acting. Showing the window is otherwise only needed to make it visible to a human (HIL).

🧩 Orthogonal Primitive Layer

The older tool list is being consolidated onto a few orthogonal verbs - one observation model, one action model, one wait model - with the flat tools becoming projections. All primitives run on the TIER-1 humanlike path (real input events; no page DOM mutation beyond Hypagon Browser's own data-mark-id stamp).

Refs are marks. A ref is a Set-of-Marks badge id — find returns rows carrying the mark, and act(mark_id=…) consumes it (re-resolving the live centre each time). When there is no badge, the same act also takes selector="…" or x=/y= — one action tool, three address modes.

Wait vocabulary: selector · gone · text/regex · url · role_name · count(min/equals/max) · network_idle · stable(marks|height|text|nodes) · revision.

Structured errors: new endpoints return a machine code (HTTP 400) — STALE_REF (re-scan and re-issue), TIMEOUT, NO_TARGET, BAD_REQUEST, NOT_SUPPORTED.

Full recipes (R1–R5), the wait table, and the error catalogue live in the OKF partition hypagon_docs/partitions/primitives_and_workflows.md, also readable as the MCP resource mark-eleven://okf/partitions/primitives-and-workflows.

📄 Token-Efficient Construct Harvester

• Pure Node.js: Powered by Mozilla Readability and Turndown. No external Python or platform-specific binaries required.
• Composed-DOM capture: Serialisation walks through open shadow roots and bakes live form state (value, checked, selected) into attributes, so web-component forms — signup and login pages especially — are actual content instead of empty custom-element tags. Password values are recorded as a length, never a value.
• Semantic Flattening: Strips ads, navigation sidebars, and script clutter. Links and images resolve to absolute URLs.
• Honest, stable envelope. Front-matter carries only what changes the reading of the body — no timestamp (so two harvests of an unchanged page are byte-identical and can be diffed or paged), no byte-ratio vanity metric:

  ---
  url: "https://example.com"
  title: "Example Domain"
  reader: "readability"
  extracted_chars: 2140
  page_chars: 5410
  shadow_roots: 2
  ---
  

  reader names the extraction path; extracted_chars and page_chars are the same unit (characters of page text), so coverage is comparable. A partly-visible page says so, and names the alternates, instead of looking short.
• Pageable by default: observe(as:"markdown") returns a 25,000-character view; budget + next_cursor page through the rest, and budget: 0 returns the whole construct. On very large pages, JEV adds a section index on top.

🗜️ JEV Page Compaction

Just-Enough-View (JEV) keeps a large page out of the agent's context window. Instead of a full Markdown dump, the agent gets a compact section index — headings, token weights and one-line digests — and fetches only the sections it needs from an in-memory depth store.

• Where: Settings → Decision Model → JEV page compaction. Optional, not experimental; a master switch plus a read mode. Both are disabled until a decision model is configured. Persisted to jev.json over IPC.
• prefer (default): jev_index, read_section and search_page lead the tool surface; full-page reads still work.
• enforce: unscoped observe(as:"markdown"|"text") and GET /markdown return JEV_REQUIRED (HTTP 409). Region-scoped reads, budgeted reads (budget > 0 or cursor), full: true and pages under 1,500 tokens still work. Every other projection — som, components, ax, list, table, app_data, find/extract/act, screenshots — is untouched.
• A funnel, not a trap: a cheap precheck refuses an obviously oversized read before the harvest (~15 ms). After two refusals for the same URL in 60 seconds, the next read is served with a note rather than blocking the turn. Harvest errors and unhelpful indexes fail open.
• Measured: a 14,823-token Wikipedia article becomes a ~396-token index; index plus one section is ~2,045 tokens (86–97% saved).
• Free when off: the three tools are not advertised at all, so their schemas (~1k tokens per request) are not billed.

Tools: jev_index, read_section, search_page. Full contract and limits: docs/JEV.md.

🧠 LanceDB Persistent Cortex & Semantic Recall

Hypagon Gateway features an embedded, serverless vector database (LanceDB) combined with Tantivy-powered Full-Text Search (BM25) stored locally under ./data/cortex/.

• Auto-Harvest on Navigation: Every web page visited is automatically chunked, embedded, and indexed into the page_history table in the background.
• Hybrid Semantic Recall: Models can invoke recall_memory to execute hybrid vector + BM25 search across past research sessions, retrieving verbatim snippets, page titles, and relevance match scores.
• Persistent Bookmarks: Save critical research nodes, documentation hubs, or matrix targets with semantic tags and research notes.
• DaisyUI 5 Cortex HUD: Open the Cortex modal (🧠 CORTEX) from the navbar to browse recent history, execute live semantic queries, or manage bookmarked cards.

📑 Hardware-Accelerated Multi-Tab Browsing

• Electron WebContentsView Isolation: Independent navigation state, separate history stacks, and zero-flicker view swapping.
• Automatic Popup Trapping: Links configured with target="_blank" or window.open are cleanly trapped into managed child tabs rather than escaping as detached OS windows.
• Dynamic DaisyUI 5 Tab Bar Strip: Real-time loading spinners, active indicators, title truncations, and memory reclamation upon tab closure.

🛡️ Autonomous Shield-Breaker (Cookie & GDPR Destroyer)

• Zero-Token Modal Clearance: Saves 1–2 LLM tool calls on almost every new domain.
• Preload Framework Detection: Detects OneTrust, Cookiebot, Google GDPR (#L2AGLb), TrustArc, and generic consent frameworks.
• Bézier Dismissal: Dispatches authentic Bézier cursor movement and hardware clicks to "Accept All" or "Agree" before notifying the agent that the page is ready.
• Manual Directive: Agents can also manually call dismiss_consent whenever unexpected overlays block the viewport.

🎯 Set-of-Marks (SoM) Visual Grounding

• Zero Coordinate Hallucination: Completely eliminates hallucinated pixel coordinates or fragile CSS selectors.

• Shadow-DOM aware: Badges are collected through open shadow roots, so controls inside web components (login/signup forms, design-system dialogs) get marks like any light-DOM element — occlusion is tested with the deepest hit and composed containment, not elementFromPoint alone. Only mode:"closed" roots stay opaque.

• High-Contrast Neon Badges: Ephemerally injects high-contrast amber neon numbered badges ([1], [2], [3], etc.) over every visible, interactable DOM element (<a>, <button>, <input>, select, [role="button"]).

• Precision Multimodal Actuation:

  1. take_screenshot({ marks: true }) captures the badge overlay and returns an element lookup table ({ id: 1, tag: "BUTTON", text: "Submit", bbox: [...] }).
  2. act(1) moves the Bézier ghost cursor to the element's centroid with sub-pixel accuracy and clicks.
  3. act(2, action: "type", text: "cyberdeck") focuses the input and types text with human cadence.

• One tool, three cost tiers via take_screenshot({ marks: true, detail: … }) (pick the cheapest that fits your model):

  detail	Payload	Use when
  "full" (default)	Marks table + full-resolution PNG	Vision model, fine visual detail matters
  "economy"	Marks table + 640px-wide JPEG (q45), roughly 10-20x smaller	Vision model, tokens/bandwidth matter
  "markdown"	Marks table only, no image	Text-only models (e.g. local GGUFs) or the cheapest page scan

  All three return the same mark IDs, so act(mark_id, …) works after any of them. Text-only models must not be sent images: a base64 PNG inlined into the prompt can be 100k+ tokens and will stall local inference servers.

📝 Single-Turn Batch Form Filler

• Instantaneous Multi-Input Population: Fills complex checkout forms, flight booking screens, or multi-field search bars in a single turn without round-trip LLM latency.
• Human Typing Cadence: Simulates variable human inter-key delays (60–240ms) with Gaussian jitter between fields.
• Flexible Field Types: Supports text inputs, password fields, dropdown <select> menus, checkboxes, and radio buttons with optional automatic form submission (submit_selector).

📥 Download Manager & Cortex Harvester

• Workspace-Local Routing: Intercepts Chromium's will-download session events, routing all files into ./data/downloads/ rather than the OS user folder.
• Automated Text Extraction: Automatically extracts clean Markdown text from downloaded PDFs (pdf-parse), CSVs, and JSON files.
• LanceDB Vector Ingestion: Automatically ingests downloaded document text into the LanceDB Cortex vector store for instant conversational recall.
• MCP Inspection: list_downloads and read_download(filename) allow agents to inspect and read downloaded files on demand.

📥 Media download integration

• Preview before committing: media preview returns the title, duration, a transcript excerpt and the poster image without downloading the media. media transcript returns the full captions as text, paged with max_chars/cursor, and indexes them into Cortex so memory recall can answer from the video later.
• Queue, don't block: media download starts a job and returns an id; media jobs reports percent, bytes, speed, ETA and output files; media cancel kills the process tree and removes the partial output.
• Managed binary: the downloader is pinned in the build, verified with SHA-256, and updates itself on launch and on an interval (default 60 minutes, configurable). It runs on Electron's own Node runtime, so no extra runtime is installed.
• Signed-in downloads are opt-in: off by default. A site that needs a session returns LOGIN_REQUIRED and raises a desktop notification; a human enables Settings → Media → "Use browser session" and retries. Cookies are scoped to the requested host and deleted after the run.
• ffmpeg when you want it: auto uses a configured binary, a managed copy or PATH; path and managed are explicit. Managed installs are checksum-verified and not shipped in the installer.
• Save folder: a picker in Settings → Media; empty keeps the app's media folder.

Read docs/MEDIA.md for the design notes and threat model.

🎥 Timelapse Recorder

• Records the deck itself: the REC button (next to SETTINGS) captures the Cyberdeck window as a timelapse — frames every intervalMs, played back at a chosen fps. One minute of work becomes two seconds of video at the 1 fps / 30 fps defaults.
• True re-timing, not a sped-up screen recording: the renderer uses WebCodecs VideoEncoder with explicit frame timestamps and muxes VP9 into WebM with webm-muxer (MediaRecorder cannot retime frames).
• Settings → Recording: enable/disable, output directory with BROWSE…, capture interval, playback fps, max width and bitrate; finished files are listed with COPY PATH.
• Agent access is a separate, human-only switch: "Let agents use the recording tools" removes record_start, record_stop and list_recordings from tools/list and makes the bridge refuse them with RECORDING_TOOLS_DISABLED. Only the Settings panel can flip it — no tool or HTTP route writes that flag.
• Programmatic hand-off: record_start → record_stop returns the absolute .webm path (plus size, frames and speed-up), and list_recordings lists recent files newest-first. That path is a plain string you can type into a file-upload field; if the field only opens a native picker, hand it to a human.
• Why a path matters: upload forms usually cannot be driven by setting a value - the browser requires a real file selection. Hypagon Gateway gives the agent (or the human) the exact file to choose instead of guessing.

🔐 Session & Cookie Jar Vault

• Stateful Auth Persistence: Captures and restores active session cookies, localStorage, and sessionStorage into encrypted/structured snapshots under ./data/sessions/<name>.json.
• Bypass 2FA & Logins: Resume authenticated states on GitHub, airline portals, news sites, or internal dashboards across agent sessions without triggering repetitive logins or CAPTCHAs.
• MCP Tools: context_save(name), context_open(name), context_list(), and context_delete(name).
• Legacy aliases: save_session(name) / restore_session(name) / list_sessions() still work (same vault) but are hidden from tools/list.

📡 Live Network & API Stream Interceptor

• Direct Chrome DevTools Protocol Wire: Attaches directly via CDP (Network.enable) to monitor all incoming and outgoing network traffic.
• Direct JSON Extraction: Bypasses DOM scraping entirely for modern Single-Page Applications (SPAs like Twitter, Reddit, Skyscanner). Captures completed XHR/Fetch response bodies (Network.getResponseBody).
• MCP Inspection: get_network_traffic filters requests by URL pattern or method, and query_json_data({ source: "network_payload", source_id }) returns parsed JSON (sliced by path/view) directly into the LLM context.
• Action-Correlated Capture: get_network_traffic({ since: 'last_action' }) returns only the requests fired by the most recent mutating action (also accepts an epoch-ms timestamp or a page revision). export_har writes a HAR 1.2 log of captured traffic for offline replay.

⚡ Real-Time WebSocket & SSE Interceptor

• Duplex Stream Surveillance: Captures duplex frames across persistent wss:// WebSockets and Server-Sent Events (text/event-stream).
• Auto-Parsed Payloads: Automatically detects and parses JSON streaming frames from live crypto tickers, chat platforms, financial feeds, and streaming LLM APIs.
• In-Memory Ring Buffer: Rolling buffer of 500 frames with filtering by stream ID, direction (sent / received), and message limit.
• MCP Tools: list_realtime_streams and clear_capture({ scope: "streams" }); inspect frames with query_json_data({ source: "stream_messages", source_id }).

💾 Script Vault & LanceDB Semantic Macro Engine

• Reusable Automation Macros: Store, recall, and execute custom JavaScript DOM extraction and automation scripts under ./data/scripts/<name>.json.
• Vector-Indexed Semantic Recall: Every vaulted script is automatically embedded into the LanceDB scripts table. Models can recall scripts using conversational natural language queries (list_scripts({ query: "extract table rows as json" })).
• Parameter Injection: run_script(name, args) executes vaulted scripts inside the active tab with dynamic runtime arguments and closure isolation.

🗔 System Tray & Background Minimization ("Out of the Way")

• Unobtrusive Background Operation: Keeps the Cyberdeck running completely out of the way in the Windows notification tray during long-running automated tasks, batch scrapes, and research pipelines.
• Single-Click Summon: Left-clicking or double-clicking the system tray icon toggles window visibility instantly.
• Close-to-Tray Protection: Closing the window (X) minimizes to the system tray rather than killing active background tasks and MCP sessions.
• MCP Tools: set_window_visibility({ visible?, focus? }) and configure_system_tray({ close_to_tray }).

🚨 Human-In-The-Loop (HIL) Intervention Protocol

• Graceful Human Collaboration: When the agent hits Cloudflare Turnstile, CAPTCHAs, 2FA/SMS OTPs, or payment confirmations, it invokes request_human_intervention.
• Multi-Channel Alerting:
  1. OS Native Desktop Notification: Fires an urgent desktop notification banner with sound.
  2. Window Summon & Taskbar Flash: Automatically restores the window from the tray, brings it to the front, and flashes the Windows taskbar amber (flashFrame(true)).
  3. Interactive In-Page Cyberdeck Banner: Injects an amber neon cyberdeck banner (#__kuang_hil_banner) directly into the active webpage with a "RESUME AGENT (DONE)" button.
  4. HUD Action Indicator: Changes the status badge to AWAITING_INPUT with live prompt text.
• Instant Unpause: When the user completes the action and clicks "DONE", the agent immediately resumes its autonomous turn.
• MCP Tools: request_human_intervention, resolve_human_intervention, get_human_intervention_status, and non-blocking send_notification.

🔌 Consolidated Semantic MCP Tools (16 advertised · +2 decision · +3 JEV)

Advertised surface: 17 grouped tools, plus drive and decide when a decision model is configured, and jev_index, read_section and search_page when JEV page compaction is on. The old flat names remain callable by name but are hidden from tools/list; they are listed below as the underlying catalog.

Tool manager: Settings → Tools controls enablement, order, per-tool instructions, per-action instructions for grouped tools, and Core/Research/Automation/Full presets. The default order starts with navigate → observe → act → wait_for → find → extract.

Tier map: TIER 1 · prefer: navigate, observe, act, wait_for, find, extract; TIER 2 · fallback: capture; TIER 3 · deliberate: evaluate_javascript. drive and decide are Tier 1 when a decision model is configured.

Underlying catalog (hidden aliases)

The table below documents the flat catalog entries. Most are hidden from tools/list behind the groups above; they remain callable by name for compatibility and are useful when editing tool instructions.

🔌 REST wrappers (hb-rest)

A second, separate MCP server for turning a REST endpoint into a saved, typed tool. It is deliberately not mixed into the browser server: hb-rest ships its own binary, its own stdio process, its own config entry and its own bridge token.

What it is for: agent access to JSON APIs (dashboards, internal tools, SaaS). The wrapper author — human or agent — defines the request once: method, URL, typed arguments, a response recipe, and a credential name. The agent then calls flat arguments; it never assembles a URL, a header set, or a body envelope, and it never sees a credential value.

Credentials are human-only and host-bound: a static value, or an OAuth2 client-credentials client that mints and caches a token. Secrets are encrypted with the OS keychain (safeStorage), injected only on an approved host, and never enter the MCP process.

// Settings → MCP Clients → REST wrappers generates an entry like this
{
  "mcpServers": {
    "hb-rest": {
      "command": "node",
      "args": ["<path-to-repo>/bin/wrap-server.cjs"],
      "env": { "HB_WRAP_TOKEN_FILE": "<userData>/wrap/token" },
    },
  },
}

Meta-tools: wrap_list, wrap_get, wrap_secret_list, wrap_derive, wrap_create, wrap_update, wrap_delete, wrap_run, wrap_test, wrap_reload.

[!WARNING] Read the limits before relying on it. No human-approval gate for mutating calls · OAuth2 is client_credentials only · the wrap server does not raise HIL itself (a rejected credential returns AUTH_REQUIRED) · exposeAsTool has no Settings control · 40-tool cap · DNS-rebinding residual risk · the token URL is outside allowedHosts. The full, blunt breakdown — what works, what doesn't, what is untested, what is risky — is docs/HYPAGON_REST.md.

Tests: npm run test:wrap (50 offline) · npm run test:wrap:mcp (11 lifecycle) · npm run test:wrap:live (46 against DummyJSON).

🗺️ Codebase Directory Structure & Documentation

Each folder contains a dedicated README.md detailing its responsibilities, dependencies, and internal workflows:

🚀 Quickstart

Development Mode (with Live Reloading)

pnpm dev

One bridge per port, one profile per process. A dev run and an installed build derive the same userData directory, so they share Electron's single-instance lock and the default bridge port 4111. To run both at once, give the dev run its own profile and port:

HB_USER_DATA_DIR="$APPDATA/hypagon-gateway-dev" HB_BRIDGE_PORT=4113 pnpm dev

Then point a second MCP client config at HB_BRIDGE_URL=http://127.0.0.1:4113 and HB_WRAP_TOKEN_FILE=<that profile>/wrap/token. The dev window's Settings → MCP Clients tab generates the config for you, with the right port and profile paths.

pnpm dev:isolated does all of that in one step: it starts the dev build on its own profile and port 4113, and prints a ready-to-paste MCP config.

Production Build & Launch

pnpm run build
pnpm start

MCP Client Configuration (OpenCode, Claude, Cursor, Antigravity, Atomic Chat)

The HUD's Settings → MCP Clients tab generates a correct, copy-ready config for this installation — OpenCode, Claude Desktop, Claude Code, Cursor, VS Code, Windsurf, Google Antigravity, Gemini CLI, Atomic Chat, Zed, and the generic mcpServers shape. It fills in the absolute server path and bridge port, and offers two runtimes:

• Bundled (default) - runs the server on Hypagon Gateway's own Electron runtime with ELECTRON_RUN_AS_NODE=1, so Node does not need to be installed. Installed builds unpack bin/ out of the asar precisely so this works.
• System Node — node bin/mcp-server.cjs, requiring Node 20+ on PATH.

A minimal Claude-style entry looks like this (the tab generates the real paths):

{
  "mcpServers": {
    "hb-browser": {
      "command": "<path-to-app>/Hypagon Gateway.exe",
      "args": ["<userData>\\bin\\mcp-server.cjs"],
      "env": { "ELECTRON_RUN_AS_NODE": "1" }
    }
  }
}

Note: the Cyberdeck must be running (pnpm run dev or pnpm start) so the stdio server can reach the browser instance on http://127.0.0.1:4111. If you run on another port with HB_BRIDGE_PORT, set HB_BRIDGE_URL for the MCP server — the generated snippet includes it automatically. Installed builds copy the server into userData/bin/ (refreshed on update), because resources/ is temporary for portable and AppImage builds.

Agent Skills

The MCP servers provide the tools; the skills teach the agent which grouped tool to use and give exact recipes. Install either skill directly from GitHub:

hermes skills install https://raw.githubusercontent.com/hyperbolictonic/hypagon/main/skills/hb-browser/SKILL.md
hermes skills install https://raw.githubusercontent.com/hyperbolictonic/hypagon/main/skills/hb-wrap/SKILL.md

Or with the Skills CLI:

npx skills add hyperbolictonic/hypagon --skill hb-browser
npx skills add hyperbolictonic/hypagon --skill hb-wrap

For a non-interactive global install, add -g -y. The source files are skills/hb-browser/SKILL.md and skills/hb-wrap/SKILL.md.

hb-browser covers the grouped tools plus the JEV read path (jev_index → read_section, including the enforce refusal); hb-wrap covers wrapper discovery, wrap_test dry-runs, recipes and credential hand-off.

🧪 Testing

All suites run against the live Cyberdeck on http://127.0.0.1:4111 (start the app first):

Type-check and bundles: npx tsc --noEmit, npm run build, npm run build:mcp.

📚 Documentation & Knowledge (OKF)

• developer/ — the standard OKF v0.2 developer bundle: MCP setup, browser and REST APIs, wrapper/recipe formats, and examples.
• CHANGELOG.md — release history and migration-relevant changes.
• docs/ — the documentation hub: concept lore, subsystem specs, field notes, and the abstraction roadmap.
• docs/PROPOSED_ABSTRACTIONS.md — the primitive-layer design and its status (A1–A6, B, C, D).
• docs/FIELD_NOTES_2026-10-18.md - root-caused findings from hardening the TIER-1 path on real SPAs.
• docs/FIELD_NOTES_2026-10-20_SIGNUP.md - the signup dogfood run: three silent no-ops, and the plan they produced.
• docs/DECISION_MODELS.md - the optional decision layer: decide / drive, guard + elimination, gates, screenshots, and who decides what.
• docs/JEV.md - Just-Enough-View: the surface index, prefer / enforce modes, refusals and the preferred reading path.
• docs/BUILDING.md - installers for Windows/macOS/Linux, signing (macOS unsigned first), GitHub Releases, and auto-update.
• hypagon_docs/ — the OKF bundle (hypagon_browser_docs): playbook/reference partitions, also served to agents as standard MCP resources:
  • mark-eleven://skills/guide — the full operational manual.
  • mark-eleven://okf/manifest — the bundle manifest.
  • mark-eleven://okf/partitions/selector-and-som · captcha-and-anti-bot · browser-navigation · primitives-and-workflows.

📦 Release notes

[!NOTE] What's new in 2.1.1 — Cortex recall for video, and a VS Code-style Settings. media transcript now indexes the full transcript into Cortex under the video URL, so memory recall can answer from a video later (pass index:false to skip). Settings navigation moved from a wrapping top tab strip to a scrollable left rail: the nav scrolls on its own, the active panel scrolls independently, and narrow windows collapse the rail into a horizontal strip. Earlier releases are in CHANGELOG.md.

📄 License

This project is licensed under the MIT License © 2026 hyperbolictonic.

🌌 Neuromancer Inspiration & Cyberpunk Canon

"A Kuang Grade Mark Eleven penetration program. Icebreaker. The military stuff, the heavy artillery."
— William Gibson, Neuromancer (1984)

In William Gibson's genre-defining cyberpunk masterpiece Neuromancer, console cowboys Case and the ROM personality construct of the Dixie Flatline jack into the Matrix using an Ono-Sendai Cyberspace 7 deck. Their ultimate weapon: the Kuang Grade Mark Eleven—a mythical military-grade icebreaker designed to slice through the impenetrable corporate ICE surrounding the Tessier-Ashpool mainframe.

Hypagon Gateway began as a project inspired by that fictional lineage and has been renamed since; the current name is its own. The quote and the original Kuang Grade Mark Eleven reference are preserved here as inspiration; Hypagon Gateway is an independent project and is not affiliated with William Gibson, Neuromancer, or the fictional device.

It carries that inspiration into the modern web:

• Piercing the Modern Web's "ICE": Headless automation (Puppeteer, Playwright) is routinely flagged and blocked by Cloudflare, Datadome, and behavioral bot sensors. Hypagon Browser provides a tactile, visible Chromium cyberdeck that operates with authentic human telemetry.
• The Ghost Cursor as Neural Trodes: Dispatches organic cubic Bézier pointer movement and Gaussian typing cadences, behaving like an authentic human operator seated at the console.
• The LanceDB Cortex as ROM Personality Construct: Provides persistent semantic recall and memory traces so your local AI agent retains research context and bookmarks across browsing sessions.