th3ist/architecture.md

Architecture & system design of th3ist

1. Overview

th3ist is a high-performance, zero-dependency Go gateway proxy that provides a standard OpenAI-compatible API (/v1/chat/completions and /v1/models) backed by the t3.chat web backend.

The service is designed to solve two core security constraints enforced by t3.chat and its hosting infrastructure (Vercel Edge):

1. Vercel security checkpoint (TLS fingerprinting): Vercel Edge validates incoming TLS Client Hello handshakes (JA3/JA4). Direct HTTP requests originating from non-browser runtimes (Go net/http, curl, Python requests) trigger an HTML security checkpoint (HTTP 429 / HTTP 403).
2. hCaptcha Enterprise protection: The t3.chat backend requires a valid, unconsumed single-use hCaptcha token on every guest request.

th3ist solves both challenges using an isolated, in-process Browser Bridge Engine that coordinates with a private Chromium instance via Chrome DevTools Protocol (CDP).

---

2. High-level architecture diagram

+-----------------------------------------------------------------------------------+
|                                OpenAI Client                                      |
|            (curl / Python openai / LangChain / Open-WebUI / LibreChat)            |
+-----------------------------------------------------------------------------------+
                                         |
                                         | HTTP POST /v1/chat/completions
                                         v
+-----------------------------------------------------------------------------------+
|                                th3ist Gateway                                     |
|                                                                                   |
|  +---------------------------+  +-----------------------------------------------+ |
|  | Request & Tool Converter  |  | Dynamic Token Ingestion & Auth (/v1/token)    | |
|  | - Tool prompt injection   |  | - Live token push                             | |
|  | - Message schema mapping  |  | - Per-request header overrides                | |
|  +---------------------------+  +-----------------------------------------------+ |
|                                         |                                         |
|                                         v                                         |
|  +------------------------------------------------------------------------------+ |
|  |                          BrowserBridge Engine                                | |
|  |                                                                              | |
|  |  1. Mint fresh token via in-browser hCaptcha (c3102294-b06e-444a-a0c8-...)    | |
|  |  2. Execute fetch("https://t3.chat/api/chat", payload) inside page context   | |
|  |  3. Stream raw SSE / Vercel AI SDK lines back to Go via CDP frame reader    | |
|  +------------------------------------------------------------------------------+ |
+-----------------------------------------------------------------------------------+
             |                                              |
             | WebSocket CDP (127.0.0.1:<dynamic_port>)     |
             v                                              |
+------------------------------------------+                |
|  Isolated Chromium Process               |                |
|  - Ephemeral Profile: /tmp/th3ist_bridge_*|                |
|  - Offscreen: -3000, -3000               |                |
|  - Real Chrome 133 TLS Handshake         |                |
+------------------------------------------+                |
             |                                              |
             | HTTPS fetch() (Native Chrome JA3 Fingerprint)|
             v                                              v
+-----------------------------------------------------------------------------------+
|                          Vercel Edge & t3.chat Backend                            |
|             (Bypasses TLS Checkpoint & Verifies Fresh hCaptcha Token)             |
+-----------------------------------------------------------------------------------+

---

3. Core components

3.1. Browser bridge engine (BrowserBridge)

The BrowserBridge manages an active, isolated Chromium instance over standard Chrome DevTools Protocol (CDP) WebSocket framing implemented directly in pure Go standard library.

Lifecycle & startup

• Dynamic port allocation: Uses getFreePort() to bind to an available local TCP port on 127.0.0.1, preventing port conflicts with other tools or existing Chrome debug sessions.
• Isolated user data directory: Generates a dedicated temporary directory (/tmp/th3ist_bridge_*) on startup via os.MkdirTemp.
• Automatic cleanup: On shutdown or bridge termination, bb.cleanup() kills the browser process and deletes the temporary directory via os.RemoveAll.

Launch arguments & offscreen rendering

args := []string{
  "--window-position=-3000,-3000",
  "--window-size=1280,800",
  "--remote-debugging-port=" + port,
  "--user-data-dir=" + tmpDir,
  "--disable-gpu",
  "--no-sandbox",
  "--no-first-run",
  "--no-default-browser-check",
  "--disable-extensions",
  "--disable-default-apps",
  "https://t3.chat",
}

• Offscreen positioning (-3000, -3000): Keeps the browser window completely off the visible desktop area without taking focus, stealing cursor input, or disrupting user workflow.
• Tiled window manager isolation (-xvfb / -display):
  • Tiling window managers (i3, bspwm, sway, dwm) can capture offscreen windows on the main display. th3ist provides -xvfb to automatically launch and attach to a virtual X server (Xvfb :<free_display> -screen 0 1280x800x24), ensuring 100% isolation from the desktop environment.
  • Users can also supply a custom -display :99 (for existing Xvfb, Xephyr, or Xnest sessions).
  • Sets window class --class=th3ist_hidden and --app=https://t3.chat for easy floating/scratchpad filtering.
• Headless detection evasion: Running in an active display context (real or virtual Xvfb) prevents hCaptcha Enterprise's client heuristics from detecting headless browser environments, ensuring automated invisible token generation succeeds.
• Strict headless fallback (-headless): When the -headless flag is passed or when DISPLAY is unset (e.g. Docker or server environments), it automatically uses --headless=new.

Why Chromium stays active for upstream requests (TLS fingerprinting vs. captcha)

An important architectural nuance is that Chromium is required for both token generation and request dispatch:

1. Single-use token lifecycle: t3.chat backend invalidates the hCaptcha token upon every single completion. Each new chat prompt or message requires a freshly generated token.
2. Vercel Edge JA3/JA4 TLS checkpoint: Even when provided with a 100% valid token and session cookies, any direct HTTP request initiated outside of Chromium (e.g., via Go net/http, Node.js fetch(), curl) is intercepted by Vercel Edge with HTTP 429: Vercel Security Checkpoint because non-browser TLS handshakes do not match Chromium's TLS Client Hello signature.
3. Zero-overhead persistence: Rather than launching and killing a browser process on every single prompt (which adds 3-5s cold-start latency), th3ist maintains a single, lightweight offscreen Chromium instance. Incoming completion requests execute fetch() and mint tokens concurrently inside the existing page context with sub-300ms latency.

---

3.2. Automated hCaptcha token generation pipeline

t3.chat uses hCaptcha Enterprise with the active sitekey:

c3102294-b06e-444a-a0c8-79d0f3b04b5a

Token minting procedure (GetFreshHcaptchaToken)

1. CDP interaction injection: Dispatches simulated mouse movement (Input.dispatchMouseEvent) into the page to prime interaction context.
2. Container injection: Creates a hidden DOM element offscreen (left: -9999px).
3. Widget execution:
   • Waits for window.hcaptcha.render and window.hcaptcha.execute to become ready.
   • Invokes window.hcaptcha.render(container, { sitekey, size: "invisible", callback: ... }).
   • Executes window.hcaptcha.execute(widgetId).
4. Promise await & token extraction:
   • Uses CDP Runtime.evaluate with awaitPromise: true and returnByValue: true.
   • Extracts the generated single-use JWT token (P1_...) and attaches it directly to the outgoing T3ChatPayload.

---

3.3. Request translation & tool calling engine

th3ist translates standard OpenAI completion requests into the schema expected by t3.chat:

OpenAI Request (messages, tools, stream, model)
  │
  ├─► TransformMessages()
  │     ├─► Injects tool definitions into system instructions
  │     ├─► Maps tool calls to <tool_call> XML syntax
  │     └─► Maps tool results to <tool_response> XML syntax
  │
  ├─► Generates UUIDs & Session IDs (convex-session-id, threadId, responseMessageId)
  │
  └─► Constructs T3ChatPayload

Function calling parsing & interception

When the upstream model produces tool calls:

• In non-streaming mode: <tool_call>...</tool_call> XML and raw JSON blocks are extracted via DetectToolCalls(), populating choice.Message.ToolCalls with ID: "call_...", Type: "function", Function: { Name: "...", Arguments: "..." }, while cleaning tool call tags out of Content.
• In streaming mode: StreamToolCallFilter intercepts <tool_call> blocks in real-time, preventing them from leaking into delta.content, and emits standard OpenAI delta.tool_calls chunks followed by finish_reason: "tool_calls".

---

3.4. Stream processing, reasoning, & tool call filters

t3.chat streams responses using the Vercel AI SDK Data Stream protocol and standard SSE:

• 0:"<text_chunk>": Assistant text tokens.
• b:"<reasoning_chunk>" / g:"...": Deep thinking and reasoning content.
• 8:[{...}]: Native tool call invocations.
• data: {"type":"text-delta","delta":"..."}: Standard SSE deltas.

Streaming pipeline architecture

Stream Lines (0: "...", data: {...}, b: "...")
  │
  ├─► StreamThinkingFilter (Stateful sliding window)
  │     ├─► Emits reasoning content to delta.reasoning_content
  │     └─► Passes non-thinking text forward
  │
  └─► StreamToolCallFilter (Stateful tool tag parser)
        ├─► Buffers & parses <tool_call>...</tool_call> blocks
        ├─► Emits structured OpenAI delta.tool_calls
        └─► Emits clean assistant text to delta.content

1. StreamThinkingFilter: Stateful filter intercepting <think>...</think> tags and routing to reasoning_content.
2. StreamToolCallFilter: Stateful filter intercepting <tool_call>...</tool_call> tags, preventing raw XML leaks in content and emitting OpenAI delta.tool_calls.
3. Flushes data: [DONE] on stream termination.

---

3.5. Token ingestion API

To allow external token management or custom API keys:

• Inspect active credentials:

  GET /v1/token
  

  Returns masked session cookies, active deployment ID, and token status.

• Push fresh token / credentials:

  POST /v1/token
  Content-Type: application/json
  
  {
    "hcaptchaToken": "<new-token>",
    "cookie": "<optional-new-cookie>",
    "deploymentId": "dpl_...",
    "clientContext": "..."
  }
  

• Per-request header overrides:

  • X-Hcaptcha-Token: Override token for a single completion request.
  • X-Cookie / Authorization: Bearer <cookie>: Custom session cookies.
  • X-Deployment-Id: Upstream deployment ID.
  • X-Client-Context: Upstream client context.

---

3.6. Error handling & fast-fail semantics

• Client errors (400, 401, 403): Non-transient errors (such as captcha_failed or invalid_params) bypass retry loops and return immediately in <300ms with original error bodies and HTTP status codes.
• Server errors (5xx): Direct HTTP fallback requests utilize DoWithFibonacciRetry with bounded Fibonacci delays (1s, 1s, 2s, 3s, 5s) before failing.

---

3.7. Rate limiting mechanism & automated hardware fingerprint rotation

Empirical testing confirmed that rate limiting on t3.chat is tracked strictly per FingerprintJS visitor ID (t3-anon-visitor), rather than by IP address:

1. How visitor identities are minted (/api/identity)

• t3.chat loads assets/fp.esm-*.js to collect 42 hardware entropy components: canvas 2D render geometry, AudioContext base latency, WebGL parameters, CPU hardware concurrency, and screen dimensions.
• The browser submits { fingerprint: { visitorId, confidence, components, version } } to /api/identity.
• The server evaluates the component entropy and signs an HMAC cookie:

  t3-anon-visitor = v1.visitor_<hash>.<timestamp>.<hmacSignature>
  

• Free tier quota is tracked in Convex against this signed visitor_<hash>. When exhausted, Convex returns HTTP 429 ratelimit_hit.

2. Automated hardware fingerprint mutation engine (RotateIdentity)

Because th3ist controls Chromium via CDP, BrowserBridge.RotateIdentity() mutates client hardware entropy on demand:

• Canvas entropy: Injects pseudo-random variations into 2D canvas geometry hashing.
• Audio entropy: Adds subtle jitter to AudioContext frequency responses.
• Hardware concurrency & resolution: Mutates CPU core counts (4, 6, 8, 12, 16) and screen resolutions.
• Submits the modified components to /api/identity, which returns:

  {
    "visitorId": "visitor_<fresh_hash>",
    "requiresSignIn": false,
    "confidence": 1
  }
  

• Zero-downtime auto-recovery: If ExecuteFetch() receives an HTTP 429 ratelimit_hit, it automatically invokes RotateIdentity(), mints a fresh hCaptcha token, and retries the completion request transparently.

3. Bypassing guest limits via BYOK (bring your own key)

th3ist also supports passing custom provider API keys directly in standard OpenAI format:

curl http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer <your-key>" \
  -d '{"model":"gemini-3.5-flash-lite","messages":[{"role":"user","content":"Hi"}]}'

th3ist automatically detects the provider (sk-ant- -> Anthropic, sk-or- -> OpenRouter, AIza -> Google, sk- -> OpenAI) and attaches the key to the upstream T3ChatPayload, completely bypassing anonymous guest quotas and captcha requirements.

---

4. Source file index

File	Purpose
th3ist.go	Main server, BrowserBridge implementation, CDP protocol handler, stream parsers, tool calling converter, and HTTP handlers.
th3ist_test.go	Unit test suite covering message transforms, thinking tag filters, tool calling detection, mock upstream completions, and live bridge tests.
Makefile	Build, test, run, and cleanup targets.
README.md	User documentation, quickstart guide, CLI flags, and API examples.
architecture.md	In-depth architectural design, reverse engineering findings, and security mechanisms.