luxferre/dynagate-cpp
1
0
Fork 0
Code Actions Activity

init commit

Browse Source
This commit is contained in:
Luxferre
2026-10-14 09:18:19 +03:00
commit b8b03f3b41
10 changed files with 6159 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+210
View File
@@ -0,0 +1,210 @@
# Dynagate LLM gateway (C++ port)
Dynagate is a lightweight, high-performance LLM gateway that acts as a proxy for OpenAI-compatible APIs. This directory is a modern **C++20** port of the Go reference implementation (`dynagate`), using a native RFC 8259 JSON engine (`json.cpp`) and **libcurl** for HTTP networking.
It provides seamless model routing, automatic key and model failover, live configuration hot-reloading without restarts, token authentication, and custom header injection.
## Features
1. **Multi-API Protocol Endpoints (Stateless)**:
- `/v1/models` and `/models` (GET): Dynamically lists unique active model IDs.
- `/v1/chat/completions` and `/chat/completions` (POST): Proxies non-streaming and streaming (`text/event-stream`) completions.
- `/v1/responses` and `/responses` (POST): Stateless-style OpenAI Responses API endpoint (automatically routed to upstream chat completions endpoints).
- `/v1/messages` and `/messages` (POST): Anthropic-compatible Messages API endpoint (automatically routed to upstream chat completions endpoints).
- `/v1/images/generations` and `/images/generations` (POST): Proxies image generation requests.
> **Note** Dynagate operates in a completely stateless manner across all API formats (`/v1/chat/completions`, `/v1/responses`, `/v1/messages`). Dynagate does not store or persist server-side conversation state. Clients must include the full conversation history in each request for multi-turn dialogues.
2. **Zero-downtime live-reloading**:
- Watches `models.csv` (overridable via command-line flags) continuously using a background thread and automatically reloads configuration updates without dropping active connections.
3. **Fallback and retry orchestration**:
- Attempts keys configured for the requested model sequentially using Fibonacci backoff delays (`-retry-delay`).
- Cascades automatically to the next model in the priority chain (wrapping around to cover all entries) if all keys for the requested model fail.
- Updates the `"model"` field dynamically in the outgoing request body before proxying, ensuring upstream compatibility.
4. **Flexible authorization**:
- Optional gateway token authentication via command-line key file.
- Key options (`""`, `"-"`) to strip authorization headers when proxying to local or open upstreams.
- Special `"-blank-"` key config to pass an empty `Authorization: Bearer` header.
5. **External config updater**:
- Executes a periodic CLI command (`-csv-updater`) at a custom interval (`-csv-update-interval`) to fetch, generate, or download fresh model listings.
6. **Custom header injection (`extra` field)**:
- Sets or overrides custom headers dynamically per model deployment using a JSON configuration string.
## Tech stack
- **Language**: C++20 (`-std=c++20`)
- **Dependencies**:
- `libcurl` (HTTP/HTTPS upstream requests and SSE streaming)
- Native RFC 8259 JSON parser and serializer in `json.cpp` (no extra or vendored third-party JSON dependency)
- POSIX sockets and standard C++20 threads/atomics/filesystem
- **Target Platforms**: Linux, FreeBSD, macOS, Haiku, and any POSIX-compliant OS (64-bit and 32-bit hosts)
## Repository layout
```text
dynagate.hpp Shared declarations, data structures, and interfaces
json.cpp Native RFC 8259 JSON parser and serializer
config.cpp RFC 4180 CSV parser, ConfigManager, hot-reloader, CSV updater
handler.cpp Auth, normalization, protocol conversions, libcurl proxying
server.cpp Multi-threaded POSIX HTTP/1.1 server and streaming writer
main.cpp CLI flag parsing, signal handling, server lifecycle
gateway_test.cpp Comprehensive unit and end-to-end test suite
Makefile POSIX-compliant Makefile (bmake, gmake, smake compatible)
models.csv Example model configuration file
free-model-collector.sh Free model discovery and verification script
freekeys.sh Helper script for collecting free keys
```
## Prerequisites
- A C++20 compiler (`c++`, `g++`, or `clang++`)
- Any POSIX `make` (`bmake`, `gmake`, `make`)
- `libcurl` development headers and library:
```bash
# Debian / Ubuntu
apt-get install build-essential libcurl4-openssl-dev
# Fedora / RHEL
dnf install gcc-c++ libcurl-devel
# FreeBSD
pkg install curl
# macOS (Xcode Command Line Tools include libcurl, or via Homebrew)
xcode-select --install
```
## Getting started
### 1. Building from source
```bash
make # builds ./dynagate
make 32bit # builds a 32-bit ./dynagate binary with large file support
make debug # builds ./dynagate with AddressSanitizer + UBSan
make test # builds and runs the test suite
make clean # removes build artifacts
make install # installs to $(DESTDIR)/usr/local/bin/dynagate
```
If `libcurl` or custom headers live in a non-standard prefix, override flags on the `make` command line:
```bash
make CXX=clang++ CPPFLAGS="-I/opt/homebrew/include -I." LDLIBS="-L/opt/homebrew/lib -lcurl -pthread"
```
### 2. Configure models
Create a `models.csv` file in the same directory. The file supports both header and header-less formats.
**Example `models.csv` with Header**:
```csv
model,key,endpoint,extra
gpt-4o,sk-proj-key1,https://api.openai.com/v1,"{""Openai-Organization"":""org-123""}"
gpt-4o,sk-proj-key2,https://api.openai.com/v1,
gpt-3.5-turbo,-blank-,https://api.openai.com/v1,
deepseek-chat,sk-ds-key,https://api.deepseek.com/v1,"{""X-Custom-Env"":""production""}"
ollama-llama3,-,http://localhost:11434/v1,
```
### 3. Setup gateway key (optional)
To restrict access to your gateway, write one or more plaintext keys (one per line) to a file (e.g. `gatekey.txt`):
```text
my-secure-gateway-token-1
my-secure-gateway-token-2
```
### 4. Run the server
```bash
./dynagate -port 8080 -csv models.csv -key gatekey.txt
```
## Configuration reference
### Command line flags
| Flag | Type | Default | Description |
| ---- | ---- | ------- | ----------- |
| `-port` | int | `8080` | Port the gateway listens on. |
| `-csv` | string | `models.csv` | Path to the model configuration CSV. |
| `-key` | string | `""` | Path to a file containing the gateway's access tokens (one per line). If blank, authentication is disabled. |
| `-csv-updater` | string | `""` | Command/script to run periodically to update `models.csv`. |
| `-csv-update-interval`| int | `10` | Frequency in minutes to invoke `-csv-updater`. |
| `-retry-delay` | int | `100` | Base delay in milliseconds for Fibonacci failover retries (0 to disable). |
### Model CSV format rules
The gateway maps columns dynamically by looking at the header row. If no header is present, it defaults to:
- Column 0: `model` (Model ID)
- Column 1: `key` (API Key / Authorization Token)
- Column 2: `endpoint` (Upstream base URL endpoint)
- Column 3: `extra` (Optional JSON string object containing header mappings)
#### Auth override key values:
- **`""` or `"-"`**: The gateway will strip the `Authorization` header completely (useful for local LLM endpoints or unauthenticated proxies).
- **`"-blank-"`**: The gateway will attach `Authorization: Bearer` without any key appended.
- **Any other string**: The gateway will attach `Authorization: Bearer <key>`.
### Stateless Operation & Multi-Turn Conversations
Dynagate is designed as a lightweight, zero-state proxy gateway. All supported request formats (`/v1/chat/completions`, `/v1/responses`, `/v1/messages`) are processed statelessly without storing session context or conversation state on disk or in memory.
To maintain context in multi-turn conversations, clients **must include the entire conversation history** in every request payload:
- **OpenAI Chat Completions (`/v1/chat/completions`)**: Pass the full array of system, user, and assistant messages in `messages`.
- **OpenAI Responses API (`/v1/responses`)**: Pass previous conversation turns in `input` (or `messages`) along with `instructions`.
- **Anthropic Messages API (`/v1/messages`)**: Pass the full history of user and assistant messages in `messages` along with `system`.
### Manual testing with curl
#### 1. Model listing (authenticated)
```bash
curl -i -H "Authorization: Bearer my-secure-gateway-token" http://localhost:8080/v1/models
```
#### 2. Chat completions proxying
```bash
curl -i -X POST http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer my-secure-gateway-token" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Explain gravity in one sentence"}]
}'
```
#### 3. Chat completions streaming
```bash
curl -i -X POST http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer my-secure-gateway-token" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"stream": true,
"messages": [{"role": "user", "content": "Count from 1 to 5"}]
}'
```
#### 4. OpenAI-compatible Responses API (`/v1/responses`)
```bash
curl -i -X POST http://localhost:8080/v1/responses \
-H "Authorization: Bearer my-secure-gateway-token" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o",
"instructions": "Be concise",
"input": "Explain relativity in one sentence"
}'
```
#### 5. Anthropic-compatible Messages API (`/v1/messages`)
```bash
curl -i -X POST http://localhost:8080/v1/messages \
-H "Authorization: Bearer my-secure-gateway-token" \
-H "Content-Type: application/json" \
-d '{
"model": "claude-3-5-sonnet",
"system": "You are a helpful assistant",
"messages": [{"role": "user", "content": "Hello world"}]
}'
```
## Credits
Created by Luxferre in 2026, released into the public domain with no warranties.