121 lines
5.2 KiB
Markdown
121 lines
5.2 KiB
Markdown
# hygate-c
|
||
|
||
A small, dependency-light C port of the Hygate gateway that exposes a
|
||
**Gradio-hosted Hunyuan model (`hy3`)** behind an **OpenAI-compatible** REST API.
|
||
|
||
It translates OpenAI `chat/completions` requests into Gradio's queue API
|
||
(`POST /gradio_api/call/chat` → `GET /gradio_api/call/chat/{event_id}` SSE
|
||
stream) and translates the Gradio stream back into OpenAI streaming and
|
||
non-streaming responses, including tool calls and reasoning content.
|
||
|
||
## Build
|
||
|
||
Requires `libcurl` and `json-c` (auto-detected via `pkg-config`); falls back
|
||
to `-lcurl -ljson-c` if `pkg-config` is unavailable.
|
||
|
||
```sh
|
||
make # dynamic binary -> bin/hygate
|
||
make static # static binary -> bin/hygate.static
|
||
make clean
|
||
```
|
||
|
||
Build is POSIX-portable (Linux, FreeBSD, macOS) and compiles clean with
|
||
`-Wall -Wextra`.
|
||
|
||
## Run
|
||
|
||
```sh
|
||
./bin/hygate [options]
|
||
```
|
||
|
||
### Options
|
||
|
||
| Flag | Default | Description |
|
||
|------|---------|-------------|
|
||
| `-port <n>` | `8080` | Listen port (binds `0.0.0.0`). |
|
||
| `-endpoint <url>` | `https://tencent-hy3.hf.space` | Upstream Gradio Space base URL (trailing slash trimmed). |
|
||
| `-model <name>` | `hy3` | Model name reported in `/v1/models` and responses. |
|
||
| `-user-agent <ua>` / `-ua <ua>` | built-in Firefox UA | Outgoing `User-Agent`. Empty string => default. |
|
||
| `-max-body <bytes>` | `5242880` (5 MiB) | Maximum request body size. Oversized requests are rejected with `413`. |
|
||
| `-max-conn <n>` | `16` | Maximum concurrent connections (worker threads). Excess connections queue in the kernel listen backlog. |
|
||
|
||
Flag forms: `-key value` or `-key=value`.
|
||
|
||
Example — listen on `9090`, allow 8 MiB bodies, cap at 32 concurrent connections:
|
||
|
||
```sh
|
||
./bin/hygate -port 9090 -max-body 8388608 -max-conn 32
|
||
```
|
||
|
||
## Endpoints
|
||
|
||
| Method | Path | Notes |
|
||
|--------|------|-------|
|
||
| `GET` | `/v1/models`, `/models` | Returns the configured model (served locally, no upstream call). |
|
||
| `POST` | `/v1/chat/completions`, `/chat/completions` | OpenAI-style chat completion. Supports `stream`, `tools`/`tool_calls`, and `reasoning_content`. |
|
||
| `OPTIONS` | any of the above | CORS preflight. |
|
||
|
||
CORS is enabled for all origins (`Access-Control-Allow-Origin: *`).
|
||
|
||
## Request / response notes
|
||
|
||
- OpenAI `messages` are mapped into Gradio's `[message, system, history,
|
||
"high", temperature, max_tokens, top_p, null, functions_json]` payload.
|
||
- `max_tokens` / `max_completion_tokens` are honored; defaults to `131072`
|
||
when omitted.
|
||
- Upstream calls use Fibonacci-backoff retries (up to `MAX_RETRIES = 5`,
|
||
300 s timeout).
|
||
|
||
## Token usage counting
|
||
|
||
Responses include an OpenAI-style `usage` object with `prompt_tokens`,
|
||
`completion_tokens`, `total_tokens`, and `completion_tokens_details`
|
||
(`reasoning_tokens`). These are computed locally by an **embedded Tencent Hy3
|
||
ByteLevel BPE tokenizer** (`tencent/Hy3`, 120,000 BPE tokens + 818 added
|
||
special/control tokens; vocabulary data in `tokenizer_data.c`, 4-stage
|
||
pre-tokenizer and BPE logic in `tokenizer.c`) — no upstream token counts are
|
||
available from the Gradio backend.
|
||
|
||
- `prompt_tokens` sums the active `system` prompt, each message's `content`
|
||
(or individual multi-part content parts), `reasoning_content` when
|
||
`preserved_thinking` is active (enabled by default when `tools` are
|
||
present, matching `tencent/Hy3`'s `chat_template.jinja`), `tool_calls` /
|
||
`function_call` (`name` and unpacked `arguments` key/value pairs), the
|
||
final `tool` result wrapper, and any request `tools` function definitions.
|
||
- `completion_tokens` sums the assistant `content`, `reasoning_content`, and
|
||
any emitted `tool_calls` (`name` and unpacked `arguments` key/value pairs),
|
||
with `reasoning_tokens` broken out under `completion_tokens_details`.
|
||
- `total_tokens = prompt_tokens + completion_tokens`.
|
||
|
||
The embedded tokenizer implements `tencent/Hy3`'s exact 4-stage `Sequence`
|
||
pre-tokenizer (`\p{N}{1,3}` digit splitting, CJK/Hiragana/Katakana isolation,
|
||
word/punctuation/whitespace regex splitting, and `ByteLevel` byte mapping),
|
||
added special-token matching (`<｜hy_...｜>`, `<think:opensource>`, tool tags,
|
||
etc.), and rank-ordered BPE merges. No per-message chat-template wrapper
|
||
overhead is added.
|
||
|
||
## Security considerations
|
||
|
||
|
||
- The server binds all interfaces (`0.0.0.0`) and currently enforces **no
|
||
authentication** — it is an open relay to the configured upstream. Do not
|
||
expose it to untrusted networks without a fronting proxy or auth layer.
|
||
- `-max-body` and `-max-conn` exist to bound resource usage (memory and
|
||
thread count) against abusive clients.
|
||
|
||
## Source layout
|
||
|
||
| File | Role |
|
||
|------|------|
|
||
| `main.c` | Entry point, flag parsing, TCP listen, thread-per-connection with concurrency cap. |
|
||
| `server.c` | HTTP parsing, routing, SSE emitters, libcurl upstream calls, streaming state machine. |
|
||
| `jsonapi.c` | OpenAI <-> Gradio payload translation, stream parsing, and message token counting. |
|
||
| `tokenizer.c` | Embedded Tencent Hy3 4-stage pre-tokenizer, added-token matcher, and BPE engine. |
|
||
| `tokenizer_data.c` | Generated Tencent Hy3 120k BPE vocabulary tables. |
|
||
| `util.c` | Growable string buffer (`strbuf`), UUID, backoff, helpers. |
|
||
| `hygate.h` | Shared types and declarations. |
|
||
|
||
## Credits
|
||
|
||
Created by Luxferre in 2026, released into the public domain with no warranties.
|