dynagate-cpp/README.md

Dynagate LLM gateway (C++ port)

Dynagate is a lightweight, high-performance LLM gateway that acts as a proxy for OpenAI-compatible APIs. This directory is a modern C++20 port of the Go reference implementation (dynagate), using a native RFC 8259 JSON engine (json.cpp) and libcurl for HTTP networking.

It provides seamless model routing, automatic key and model failover, live configuration hot-reloading without restarts, token authentication, and custom header injection.

Features

1. Multi-API Protocol Endpoints (Stateless):

   • /v1/models and /models (GET): Dynamically lists unique active model IDs.
   • /v1/chat/completions and /chat/completions (POST): Proxies non-streaming and streaming (text/event-stream) completions.
   • /v1/responses and /responses (POST): Stateless-style OpenAI Responses API endpoint (automatically routed to upstream chat completions endpoints).
   • /v1/messages and /messages (POST): Anthropic-compatible Messages API endpoint (automatically routed to upstream chat completions endpoints).
   • /v1/images/generations and /images/generations (POST): Proxies image generation requests.

   Note

   Dynagate operates in a completely stateless manner across all API formats (/v1/chat/completions, /v1/responses, /v1/messages). Dynagate does not store or persist server-side conversation state. Clients must include the full conversation history in each request for multi-turn dialogues.

2. Zero-downtime live-reloading:

   • Watches models.csv (overridable via command-line flags) continuously using a background thread and automatically reloads configuration updates without dropping active connections.

3. Fallback and retry orchestration:

   • Attempts keys configured for the requested model sequentially using Fibonacci backoff delays (-retry-delay).
   • Cascades automatically to the next model in the priority chain (wrapping around to cover all entries) if all keys for the requested model fail.
   • Updates the "model" field dynamically in the outgoing request body before proxying, ensuring upstream compatibility.

4. Flexible authorization:

   • Optional gateway token authentication via command-line key file.
   • Key options ("", "-") to strip authorization headers when proxying to local or open upstreams.
   • Special "-blank-" key config to pass an empty Authorization: Bearer header.

5. External config updater:

   • Executes a periodic CLI command (-csv-updater) at a custom interval (-csv-update-interval) to fetch, generate, or download fresh model listings.

6. Custom header injection (extra field):

   • Sets or overrides custom headers dynamically per model deployment using a JSON configuration string.

Tech stack

• Language: C++20 (-std=c++20)
• Dependencies:
  • libcurl (HTTP/HTTPS upstream requests and SSE streaming)
  • Native RFC 8259 JSON parser and serializer in json.cpp (no extra or vendored third-party JSON dependency)
  • POSIX sockets and standard C++20 threads/atomics/filesystem
• Target Platforms: Linux, FreeBSD, macOS, Haiku, and any POSIX-compliant OS (64-bit and 32-bit hosts)

Repository layout

dynagate.hpp             Shared declarations, data structures, and interfaces
json.cpp                 Native RFC 8259 JSON parser and serializer
config.cpp               RFC 4180 CSV parser, ConfigManager, hot-reloader, CSV updater
handler.cpp              Auth, normalization, protocol conversions, libcurl proxying
server.cpp               Multi-threaded POSIX HTTP/1.1 server and streaming writer
main.cpp                 CLI flag parsing, signal handling, server lifecycle
gateway_test.cpp         Comprehensive unit and end-to-end test suite
Makefile                 POSIX-compliant Makefile (bmake, gmake, smake compatible)
models.csv               Example model configuration file
free-model-collector.sh  Free model discovery and verification script
freekeys.sh              Helper script for collecting free keys

Prerequisites

• A C++20 compiler (c++, g++, or clang++)
• Any POSIX make (bmake, gmake, make)
• libcurl development headers and library:

  # Debian / Ubuntu
  apt-get install build-essential libcurl4-openssl-dev
  # Fedora / RHEL
  dnf install gcc-c++ libcurl-devel
  # FreeBSD
  pkg install curl
  # macOS (Xcode Command Line Tools include libcurl, or via Homebrew)
  xcode-select --install
  

Getting started

1. Building from source

make            # builds ./dynagate
make 32bit      # builds a 32-bit ./dynagate binary with large file support
make debug      # builds ./dynagate with AddressSanitizer + UBSan
make test       # builds and runs the test suite
make clean      # removes build artifacts
make install    # installs to $(DESTDIR)/usr/local/bin/dynagate

If libcurl or custom headers live in a non-standard prefix, override flags on the make command line:

make CXX=clang++ CPPFLAGS="-I/opt/homebrew/include -I." LDLIBS="-L/opt/homebrew/lib -lcurl -pthread"

2. Configure models

Create a models.csv file in the same directory. The file supports both header and header-less formats.

Example models.csv with Header:

model,key,endpoint,extra
gpt-4o,sk-proj-key1,https://api.openai.com/v1,"{""Openai-Organization"":""org-123""}"
gpt-4o,sk-proj-key2,https://api.openai.com/v1,
gpt-3.5-turbo,-blank-,https://api.openai.com/v1,
deepseek-chat,sk-ds-key,https://api.deepseek.com/v1,"{""X-Custom-Env"":""production""}"
ollama-llama3,-,http://localhost:11434/v1,

3. Setup gateway key (optional)

To restrict access to your gateway, write one or more plaintext keys (one per line) to a file (e.g. gatekey.txt):

my-secure-gateway-token-1
my-secure-gateway-token-2

4. Run the server

./dynagate -port 8080 -csv models.csv -key gatekey.txt

Configuration reference

Command line flags

Flag	Type	Default	Description
-port	int	8080	Port the gateway listens on.
-csv	string	models.csv	Path to the model configuration CSV.
-key	string	""	Path to a file containing the gateway's access tokens (one per line). If blank, authentication is disabled.
-csv-updater	string	""	Command/script to run periodically to update models.csv.
-csv-update-interval	int	10	Frequency in minutes to invoke -csv-updater.
-retry-delay	int	100	Base delay in milliseconds for Fibonacci failover retries (0 to disable).

Model CSV format rules

The gateway maps columns dynamically by looking at the header row. If no header is present, it defaults to:

• Column 0: model (Model ID)
• Column 1: key (API Key / Authorization Token)
• Column 2: endpoint (Upstream base URL endpoint)
• Column 3: extra (Optional JSON string object containing header mappings)

Auth override key values:

• "" or "-": The gateway will strip the Authorization header completely (useful for local LLM endpoints or unauthenticated proxies).
• "-blank-": The gateway will attach Authorization: Bearer without any key appended.
• Any other string: The gateway will attach Authorization: Bearer <key>.

Stateless Operation & Multi-Turn Conversations

Dynagate is designed as a lightweight, zero-state proxy gateway. All supported request formats (/v1/chat/completions, /v1/responses, /v1/messages) are processed statelessly without storing session context or conversation state on disk or in memory.

To maintain context in multi-turn conversations, clients must include the entire conversation history in every request payload:

• OpenAI Chat Completions (/v1/chat/completions): Pass the full array of system, user, and assistant messages in messages.
• OpenAI Responses API (/v1/responses): Pass previous conversation turns in input (or messages) along with instructions.
• Anthropic Messages API (/v1/messages): Pass the full history of user and assistant messages in messages along with system.

Manual testing with curl

1. Model listing (authenticated)

curl -i -H "Authorization: Bearer my-secure-gateway-token" http://localhost:8080/v1/models

2. Chat completions proxying

curl -i -X POST http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer my-secure-gateway-token" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Explain gravity in one sentence"}]
  }'

3. Chat completions streaming

curl -i -X POST http://localhost:8080/v1/chat/completions \
  -H "Authorization: Bearer my-secure-gateway-token" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "stream": true,
    "messages": [{"role": "user", "content": "Count from 1 to 5"}]
  }'

4. OpenAI-compatible Responses API (/v1/responses)

curl -i -X POST http://localhost:8080/v1/responses \
  -H "Authorization: Bearer my-secure-gateway-token" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "instructions": "Be concise",
    "input": "Explain relativity in one sentence"
  }'

5. Anthropic-compatible Messages API (/v1/messages)

curl -i -X POST http://localhost:8080/v1/messages \
  -H "Authorization: Bearer my-secure-gateway-token" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "claude-3-5-sonnet",
    "system": "You are a helpful assistant",
    "messages": [{"role": "user", "content": "Hello world"}]
  }'

Credits

Created by Luxferre in 2026, released into the public domain with no warranties.