271 lines
16 KiB
Markdown
271 lines
16 KiB
Markdown
# Architecture & system design of th3ist
|
|
|
|
## 1. Overview
|
|
|
|
`th3ist` is a high-performance, zero-dependency Go gateway proxy that provides a standard OpenAI-compatible API (`/v1/chat/completions` and `/v1/models`) backed by the `t3.chat` web backend.
|
|
|
|
The service is designed to solve two core security constraints enforced by `t3.chat` and its hosting infrastructure (Vercel Edge):
|
|
1. **Vercel security checkpoint (TLS fingerprinting)**: Vercel Edge validates incoming TLS Client Hello handshakes (JA3/JA4). Direct HTTP requests originating from non-browser runtimes (Go `net/http`, curl, Python `requests`) trigger an HTML security checkpoint (`HTTP 429` / `HTTP 403`).
|
|
2. **hCaptcha Enterprise protection**: The `t3.chat` backend requires a valid, unconsumed single-use hCaptcha token on every guest request.
|
|
|
|
`th3ist` solves both challenges using an isolated, in-process **Browser Bridge Engine** that coordinates with a private Chromium instance via Chrome DevTools Protocol (CDP).
|
|
|
|
---
|
|
|
|
## 2. High-level architecture diagram
|
|
|
|
```
|
|
+-----------------------------------------------------------------------------------+
|
|
| OpenAI Client |
|
|
| (curl / Python openai / LangChain / Open-WebUI / LibreChat) |
|
|
+-----------------------------------------------------------------------------------+
|
|
|
|
|
| HTTP POST /v1/chat/completions
|
|
v
|
|
+-----------------------------------------------------------------------------------+
|
|
| th3ist Gateway |
|
|
| |
|
|
| +---------------------------+ +-----------------------------------------------+ |
|
|
| | Request & Tool Converter | | Dynamic Token Ingestion & Auth (/v1/token) | |
|
|
| | - Tool prompt injection | | - Live token push | |
|
|
| | - Message schema mapping | | - Per-request header overrides | |
|
|
| +---------------------------+ +-----------------------------------------------+ |
|
|
| | |
|
|
| v |
|
|
| +------------------------------------------------------------------------------+ |
|
|
| | BrowserBridge Engine | |
|
|
| | | |
|
|
| | 1. Mint fresh token via in-browser hCaptcha (c3102294-b06e-444a-a0c8-...) | |
|
|
| | 2. Execute fetch("https://t3.chat/api/chat", payload) inside page context | |
|
|
| | 3. Stream raw SSE / Vercel AI SDK lines back to Go via CDP frame reader | |
|
|
| +------------------------------------------------------------------------------+ |
|
|
+-----------------------------------------------------------------------------------+
|
|
| |
|
|
| WebSocket CDP (127.0.0.1:<dynamic_port>) |
|
|
v |
|
|
+------------------------------------------+ |
|
|
| Isolated Chromium Process | |
|
|
| - Ephemeral Profile: /tmp/th3ist_bridge_*| |
|
|
| - Offscreen: -3000, -3000 | |
|
|
| - Real Chrome 133 TLS Handshake | |
|
|
+------------------------------------------+ |
|
|
| |
|
|
| HTTPS fetch() (Native Chrome JA3 Fingerprint)|
|
|
v v
|
|
+-----------------------------------------------------------------------------------+
|
|
| Vercel Edge & t3.chat Backend |
|
|
| (Bypasses TLS Checkpoint & Verifies Fresh hCaptcha Token) |
|
|
+-----------------------------------------------------------------------------------+
|
|
```
|
|
|
|
---
|
|
|
|
## 3. Core components
|
|
|
|
### 3.1. Browser bridge engine (`BrowserBridge`)
|
|
|
|
The `BrowserBridge` manages an active, isolated Chromium instance over standard Chrome DevTools Protocol (CDP) WebSocket framing implemented directly in pure Go standard library.
|
|
|
|
#### Lifecycle & startup
|
|
- **Dynamic port allocation**: Uses `getFreePort()` to bind to an available local TCP port on `127.0.0.1`, preventing port conflicts with other tools or existing Chrome debug sessions.
|
|
- **Isolated user data directory**: Generates a dedicated temporary directory (`/tmp/th3ist_bridge_*`) on startup via `os.MkdirTemp`.
|
|
- **Automatic cleanup**: On shutdown or bridge termination, `bb.cleanup()` kills the browser process and deletes the temporary directory via `os.RemoveAll`.
|
|
|
|
#### Launch arguments & offscreen rendering
|
|
```go
|
|
args := []string{
|
|
"--window-position=-3000,-3000",
|
|
"--window-size=1280,800",
|
|
"--remote-debugging-port=" + port,
|
|
"--user-data-dir=" + tmpDir,
|
|
"--disable-gpu",
|
|
"--no-sandbox",
|
|
"--no-first-run",
|
|
"--no-default-browser-check",
|
|
"--disable-extensions",
|
|
"--disable-default-apps",
|
|
"https://t3.chat",
|
|
}
|
|
```
|
|
|
|
- **Offscreen positioning (`-3000, -3000`)**: Keeps the browser window completely off the visible desktop area without taking focus, stealing cursor input, or disrupting user workflow.
|
|
- **Tiled window manager isolation (`-xvfb` / `-display`)**:
|
|
- Tiling window managers (i3, bspwm, sway, dwm, awesome, hyprland, xmonad) can capture offscreen windows on the main display. `th3ist` defaults to managing an isolated virtual X server (`-xvfb`, default: true) via `Xvfb :<free_display> -screen 0 1280x800x24`, ensuring 100% isolation from the desktop environment out of the box.
|
|
- Users can also supply a custom `-display :99` (for existing `Xvfb`, `Xephyr`, or `Xnest` sessions) or pass `-no-xvfb` to attach to `$DISPLAY`.
|
|
- Sets window class `--class=th3ist_hidden` and `--app=https://t3.chat` for easy floating/scratchpad filtering.
|
|
- **Headless detection evasion**: Running in an active display context (real or virtual Xvfb) prevents hCaptcha Enterprise's client heuristics from detecting headless browser environments, ensuring automated invisible token generation succeeds.
|
|
- **Strict headless fallback (`-headless`)**: When the `-headless` flag is passed or when `DISPLAY` is unset (e.g. Docker or server environments), it automatically uses `--headless=new`.
|
|
|
|
#### Why Chromium stays active for upstream requests (TLS fingerprinting vs. captcha)
|
|
An important architectural nuance is that Chromium is required for **both** token generation and request dispatch:
|
|
1. **Single-use token lifecycle**: `t3.chat` backend invalidates the hCaptcha token upon every single completion. Each new chat prompt or message requires a freshly generated token.
|
|
2. **Vercel Edge JA3/JA4 TLS checkpoint**: Even when provided with a 100% valid token and session cookies, any direct HTTP request initiated outside of Chromium (e.g., via Go `net/http`, Node.js `fetch()`, curl) is intercepted by Vercel Edge with `HTTP 429: Vercel Security Checkpoint` because non-browser TLS handshakes do not match Chromium's TLS Client Hello signature.
|
|
3. **Zero-overhead persistence**: Rather than launching and killing a browser process on every single prompt (which adds 3-5s cold-start latency), `th3ist` maintains a single, lightweight offscreen Chromium instance. Incoming completion requests execute `fetch()` and mint tokens concurrently inside the existing page context with sub-300ms latency.
|
|
|
|
---
|
|
|
|
### 3.2. Automated hCaptcha token generation pipeline
|
|
|
|
`t3.chat` uses hCaptcha Enterprise with the active sitekey:
|
|
```
|
|
c3102294-b06e-444a-a0c8-79d0f3b04b5a
|
|
```
|
|
|
|
#### Token minting procedure (`GetFreshHcaptchaToken`)
|
|
1. **CDP interaction injection**: Dispatches simulated mouse movement (`Input.dispatchMouseEvent`) into the page to prime interaction context.
|
|
2. **Container injection**: Creates a hidden DOM element offscreen (`left: -9999px`).
|
|
3. **Widget execution**:
|
|
- Waits for `window.hcaptcha.render` and `window.hcaptcha.execute` to become ready.
|
|
- Invokes `window.hcaptcha.render(container, { sitekey, size: "invisible", callback: ... })`.
|
|
- Executes `window.hcaptcha.execute(widgetId)`.
|
|
4. **Promise await & token extraction**:
|
|
- Uses CDP `Runtime.evaluate` with `awaitPromise: true` and `returnByValue: true`.
|
|
- Extracts the generated single-use JWT token (`P1_...`) and attaches it directly to the outgoing `T3ChatPayload`.
|
|
|
|
---
|
|
|
|
### 3.3. Request translation & tool calling engine
|
|
|
|
`th3ist` translates standard OpenAI completion requests into the schema expected by `t3.chat`:
|
|
|
|
```
|
|
OpenAI Request (messages, tools, stream, model)
|
|
│
|
|
├─► TransformMessages()
|
|
│ ├─► Injects tool definitions into system instructions
|
|
│ ├─► Maps tool calls to <tool_call> XML syntax
|
|
│ └─► Maps tool results to <tool_response> XML syntax
|
|
│
|
|
├─► Generates UUIDs & Session IDs (convex-session-id, threadId, responseMessageId)
|
|
│
|
|
└─► Constructs T3ChatPayload
|
|
```
|
|
|
|
#### Function calling parsing & interception
|
|
When the upstream model produces tool calls:
|
|
- In non-streaming mode: `<tool_call>...</tool_call>` XML and raw JSON blocks are extracted via `DetectToolCalls()`, populating `choice.Message.ToolCalls` with `ID: "call_..."`, `Type: "function"`, `Function: { Name: "...", Arguments: "..." }`, while cleaning tool call tags out of `Content`.
|
|
- In streaming mode: `StreamToolCallFilter` intercepts `<tool_call>` blocks in real-time, preventing them from leaking into `delta.content`, and emits standard OpenAI `delta.tool_calls` chunks followed by `finish_reason: "tool_calls"`.
|
|
|
|
---
|
|
|
|
### 3.4. Stream processing, reasoning, & tool call filters
|
|
|
|
`t3.chat` streams responses using the Vercel AI SDK Data Stream protocol and standard SSE:
|
|
- `0:"<text_chunk>"`: Assistant text tokens.
|
|
- `b:"<reasoning_chunk>"` / `g:"..."`: Deep thinking and reasoning content.
|
|
- `8:[{...}]`: Native tool call invocations.
|
|
- `data: {"type":"text-delta","delta":"..."}`: Standard SSE deltas.
|
|
|
|
#### Streaming pipeline architecture
|
|
```
|
|
Chromium ReadableStream (window.th3istStreamChunk)
|
|
│
|
|
├─► CDP Runtime.bindingCalled Event
|
|
│
|
|
├─► StreamLineBuffer (Reassembles complete stream lines across TCP chunks)
|
|
│
|
|
├─► StreamThinkingFilter (Stateful sliding window)
|
|
│ ├─► Emits reasoning content to delta.reasoning_content
|
|
│ └─► Passes non-thinking text forward
|
|
│
|
|
└─► StreamToolCallFilter (Stateful tool tag parser)
|
|
├─► Buffers & parses <tool_call>...</tool_call> blocks
|
|
├─► Emits structured OpenAI delta.tool_calls
|
|
└─► Emits clean assistant text to delta.content in real time
|
|
```
|
|
1. `Runtime.addBinding`: Exposes `window.th3istStreamChunk` to the browser context upon bridge startup, relaying in-page `ReadableStream` chunks over CDP in real time without buffering.
|
|
2. `StreamLineBuffer`: Reassembles partial TCP packets into complete protocol lines across chunk boundaries.
|
|
3. `StreamThinkingFilter`: Stateful filter intercepting `<think>...</think>` tags and routing to `reasoning_content`.
|
|
4. `StreamToolCallFilter`: Stateful filter intercepting `<tool_call>...</tool_call>` tags, preventing raw XML leaks in `content` and emitting OpenAI `delta.tool_calls`.
|
|
5. Flushes `data: [DONE]` on stream termination.
|
|
|
|
---
|
|
|
|
### 3.5. Token ingestion API
|
|
|
|
To allow external token management or custom API keys:
|
|
|
|
- **Inspect active credentials**:
|
|
```http
|
|
GET /v1/token
|
|
```
|
|
Returns masked session cookies, active deployment ID, and token status.
|
|
|
|
- **Push fresh token / credentials**:
|
|
```http
|
|
POST /v1/token
|
|
Content-Type: application/json
|
|
|
|
{
|
|
"hcaptchaToken": "<new-token>",
|
|
"cookie": "<optional-new-cookie>",
|
|
"deploymentId": "dpl_...",
|
|
"clientContext": "..."
|
|
}
|
|
```
|
|
|
|
- **Per-request header overrides**:
|
|
- `X-Hcaptcha-Token`: Override token for a single completion request.
|
|
- `X-Cookie` / `Authorization: Bearer <cookie>`: Custom session cookies.
|
|
- `X-Deployment-Id`: Upstream deployment ID.
|
|
- `X-Client-Context`: Upstream client context.
|
|
|
|
---
|
|
|
|
### 3.6. Error handling & fast-fail semantics
|
|
|
|
- **Client errors (`400`, `401`, `403`)**: Non-transient errors (such as `invalid_params`) bypass retry loops and return immediately in `<300ms` with original error bodies and HTTP status codes.
|
|
- **Server errors (`5xx`)**: Direct HTTP fallback requests utilize `DoWithFibonacciRetry` with bounded Fibonacci delays (1s, 1s, 2s, 3s, 5s) before failing.
|
|
|
|
---
|
|
|
|
### 3.7. Rate limiting mechanism & automated hardware fingerprint rotation
|
|
|
|
Empirical testing confirmed that rate limiting on `t3.chat` is **tracked strictly per FingerprintJS visitor ID (`t3-anon-visitor`)**, rather than by IP address:
|
|
|
|
#### 1. How visitor identities are minted (`/api/identity`)
|
|
- `t3.chat` loads `assets/fp.esm-*.js` to collect 42 hardware entropy components: canvas 2D render geometry, AudioContext base latency, WebGL parameters, CPU hardware concurrency, and screen dimensions.
|
|
- The browser submits `{ fingerprint: { visitorId, confidence, components, version } }` to `/api/identity`.
|
|
- The server evaluates the component entropy and signs an HMAC cookie:
|
|
```
|
|
t3-anon-visitor = v1.visitor_<hash>.<timestamp>.<hmacSignature>
|
|
```
|
|
- Free tier quota is tracked in Convex against this signed `visitor_<hash>`. When exhausted, Convex returns `HTTP 429 ratelimit_hit`.
|
|
|
|
#### 2. Automated hardware fingerprint mutation engine (`RotateIdentity`)
|
|
Because `th3ist` controls Chromium via CDP, `BrowserBridge.RotateIdentity()` mutates client hardware entropy on demand:
|
|
- **Dynamic module discovery & synthetic fallback**: Searches `document.scripts` and `performance.getEntriesByType('resource')` for active FingerprintJS modules before falling back to asset paths. If unavailable, synthesizes a complete set of high-entropy components.
|
|
- **Canvas entropy**: Injects pseudo-random variations into 2D canvas geometry & text hashing.
|
|
- **Audio entropy**: Adds subtle jitter to `AudioContext` frequency responses.
|
|
- **Hardware concurrency, resolution & memory**: Mutates CPU core counts (4, 6, 8, 12, 16, 24, 32), screen resolutions, and device memory values (4, 8, 16, 32 GB).
|
|
- **Storage reset**: Clears cached visitor markers from `localStorage` and `sessionStorage`.
|
|
- Submits the modified components to `/api/identity`, which returns a fresh signed `visitor_<hash>` and updates the browser cookie jar.
|
|
- **Zero-downtime multi-attempt auto-recovery**: When `ExecuteFetch()` or `ExecuteStreamFetch()` receives an `HTTP 429` or rate limit response, it immediately invokes `RotateIdentity()`, mints a fresh hCaptcha token, resets session IDs, and retries the request transparently (up to 4 attempts) without returning 429 errors to the client right away.
|
|
|
|
#### 3. Bypassing guest limits via BYOK (bring your own key)
|
|
`th3ist` also supports passing custom provider API keys directly in standard OpenAI format:
|
|
```bash
|
|
curl http://localhost:8080/v1/chat/completions \
|
|
-H "Authorization: Bearer <your-key>" \
|
|
-d '{"model":"gemini-3.5-flash-lite","messages":[{"role":"user","content":"Hi"}]}'
|
|
```
|
|
`th3ist` automatically detects the provider (`sk-ant-` -> Anthropic, `sk-or-` -> OpenRouter, `AIza` -> Google, `sk-` -> OpenAI) and attaches the key to the upstream `T3ChatPayload`, completely bypassing anonymous guest quotas and captcha requirements.
|
|
|
|
---
|
|
|
|
## 4. Source file index
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| [`th3ist.go`](file:///home/lux/proj/th3ist/th3ist.go) | Main server, BrowserBridge implementation, CDP protocol handler, stream parsers, tool calling converter, and HTTP handlers. |
|
|
| [`th3ist_test.go`](file:///home/lux/proj/th3ist/th3ist_test.go) | Unit test suite covering message transforms, thinking tag filters, tool calling detection, mock upstream completions, and live bridge tests. |
|
|
| [`Makefile`](file:///home/lux/proj/th3ist/Makefile) | Build, test, run, and cleanup targets. |
|
|
| [`README.md`](file:///home/lux/proj/th3ist/README.md) | User documentation, quickstart guide, CLI flags, and API examples. |
|
|
| [`architecture.md`](file:///home/lux/proj/th3ist/architecture.md) | In-depth architectural design, reverse engineering findings, and security mechanisms. |
|
|
|
|
---
|
|
|
|
## 5. Credits
|
|
|
|
Created by Luxferre in 2026, released into the public domain with no warranties.
|