# groqqer `groqqer` is a lightweight, zero-dependency Go OpenAI-compatible API gateway and proxy for the Groq Streamlit chatbot space (`https://dromerosm-groq-chatbot.hf.space`). It converts standard OpenAI chat completions and model requests into automated headless browser interactions, providing fast inference with Groq models without requiring an API key. ## Features - **OpenAI Compatible API**: Exposes standard `/v1/models` and `/v1/chat/completions` endpoints. Drop-in replacement for OpenAI SDKs, Open-WebUI, LiteLLM, LibreChat, and LangChain. - **Zero Third-Party Dependencies**: Written entirely in pure Go using only the standard library (`net/http`, `encoding/json`, `os/exec`, `crypto/rand`, etc.). Includes a built-in RFC 6455 WebSocket CDP client and RFC 1928 SOCKS5 proxy client. - **System Prompt Support**: Translates developer/system messages into natural instruction framing. - **Streaming & Non-Streaming**: Supports Server-Sent Events (`stream: true`) with real-time token delivery, as well as synchronous JSON responses. - **Tool / Function Calling**: Fully supports OpenAI `tools`, `tool_choice`, and multi-turn execution (`role: "tool"`). Automatically injects schemas and parses `` responses into OpenAI `tool_calls` payloads with `finish_reason: "tool_calls"`. - **Reasoning Content**: Automatically parses `` and `` tags from reasoning models (such as Qwen 3.6/3.8) and streams or populates `reasoning_content` in accordance with OpenAI O-series conventions. - **Dynamic Model Discovery**: Scrapes active models directly from the Streamlit UI on startup and allows seamless model switching between requests (`qwen/qwen3.6-27b`, `openai/gpt-oss-120b`, `openai/gpt-oss-20b`, `groq/compound`, etc.). - **Display Isolation via Xvfb**: Automatically discovers and spawns a virtual X display (`:100` - `:199`) to keep browser automation completely isolated from host desktop displays. Gracefully falls back to offscreen coordinates if Xvfb is not present. - **SOCKS5 Proxy Tunneling**: Tunnel browser traffic through SOCKS5 proxies using the `-socks` flag or `ALL_PROXY` / `SOCKS5_PROXY` environment variables. - **Clean Session Isolation**: Automatically clears Streamlit conversation state between requests to prevent cumulative token window exhaustion or stale error states. ## Architecture ``` +---------------------------+ | OpenAI Client / SDK / App | +---------------------------+ | HTTP POST /v1/chat/completions v +---------------------------+ | groqqer | | (Pure Go Standard Lib) | +---------------------------+ | RFC 6455 WebSocket CDP v +---------------------------+ | Chromium / Chrome | | (Display :100+ via Xvfb) | +---------------------------+ | HTTPS v +---------------------------+ | Groq Streamlit Space | | (dromerosm-groq-chatbot) | +---------------------------+ ``` ## Requirements - **Linux** (x86_64 or aarch64) - **Go 1.20+** - **Chromium** or **Google Chrome** installed (`/usr/bin/chromium`, `/usr/bin/google-chrome`, or in `$PATH`) - *(Optional)* **Xvfb** (`xorg-server-xvfb`) for display isolation ## Installation & Build Clone or navigate to the repository, then build the binary: ```bash make ``` The optimized binary will be placed at `bin/groqqer`. ## Running the Server Start the gateway with default settings: ```bash ./bin/groqqer -port 8080 ``` ### CLI Flags | Flag | Default | Description | |------|---------|-------------| | `-port` | `8080` | HTTP server listening port | | `-target` | `https://dromerosm-groq-chatbot.hf.space` | Target Groq Streamlit space URL | | `-default-model` | `llama-3.3-70b-versatile` | Fallback model if request does not specify one | | `-browser` | `""` | Custom path to Chromium/Chrome binary | | `-xvfb` | `true` | Enable virtual X server display isolation | | `-no-xvfb` | `false` | Disable virtual X server (run offscreen) | | `-socks` | `""` | SOCKS5 proxy URL (e.g. `socks5://127.0.0.1:1080`) | | `-user-agent` | *(Chrome 133 UA)* | Custom browser User-Agent string | ## API Usage Examples ### 1. List Available Models ```bash curl -s http://127.0.0.1:8080/v1/models ``` **Response:** ```json { "object": "list", "data": [ {"id": "qwen/qwen3.6-27b", "object": "model", "created": 1788869470, "owned_by": "groq"}, {"id": "openai/gpt-oss-120b", "object": "model", "created": 1788869470, "owned_by": "groq"}, {"id": "openai/gpt-oss-20b", "object": "model", "created": 1788869470, "owned_by": "groq"}, {"id": "groq/compound", "object": "model", "created": 1788869470, "owned_by": "groq"} ] } ``` --- ### 2. Chat Completion (Non-Streaming) ```bash curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "qwen/qwen3.6-27b", "messages": [ {"role": "system", "content": "You are a concise assistant. Reply in one sentence."}, {"role": "user", "content": "What is the capital of France?"} ] }' ``` **Response:** ```json { "id": "chatcmpl-9691bbf8-8623-4155-a40f-a00e6b6903ac", "object": "chat.completion", "created": 1788869478, "model": "qwen/qwen3.6-27b", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "The capital of France is Paris.", "reasoning_content": "Analyze User Input: Question: 'What is the capital of France?'..." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 0, "completion_tokens": 0, "total_tokens": 0 } } ``` --- ### 3. Streaming Chat Completion (SSE) ```bash curl -N -s -X POST http://127.0.0.1:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "qwen/qwen3.6-27b", "messages": [ {"role": "user", "content": "Count from 1 to 4 with commas."} ], "stream": true }' ``` **Output:** ``` data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"role":"assistant"}}]} data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"reasoning_content":"Thinking process..."}}]} data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":"1, 2, 3,"}}]} data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":" 4"}}]} data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]} data: [DONE] ``` --- ### 4. Tool / Function Calling ```bash curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "qwen/qwen3.6-27b", "messages": [ {"role": "user", "content": "What is the current weather in Tokyo?"} ], "tools": [ { "type": "function", "function": { "name": "get_current_weather", "description": "Get current weather for a city", "parameters": { "type": "object", "properties": { "location": {"type": "string", "description": "The city and country, e.g. Tokyo, Japan"} }, "required": ["location"] } } } ], "tool_choice": "auto" }' ``` **Response:** ```json { "id": "chatcmpl-cb050ba0-2573-403f-a1db-e76cdfb8f99d", "object": "chat.completion", "created": 1788869493, "model": "qwen/qwen3.6-27b", "choices": [ { "index": 0, "message": { "role": "assistant", "content": null, "tool_calls": [ { "id": "call_ff437271", "type": "function", "function": { "name": "get_current_weather", "arguments": "{\"location\":\"Tokyo, Japan\"}" } } ] }, "finish_reason": "tool_calls" } ] } ``` --- ### 5. Multi-Turn Tool Response Execution Once you run your tool locally, send the tool result back using `role: "tool"`: ```bash curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \ -H "Content-Type: application/json" \ -d '{ "model": "qwen/qwen3.6-27b", "messages": [ {"role": "user", "content": "What is the current weather in Tokyo?"}, { "role": "assistant", "tool_calls": [ { "id": "call_ff437271", "type": "function", "function": { "name": "get_current_weather", "arguments": "{\"location\":\"Tokyo, Japan\"}" } } ] }, { "role": "tool", "tool_call_id": "call_ff437271", "name": "get_current_weather", "content": "{\"temperature\": \"18°C\", \"condition\": \"Sunny\", \"humidity\": \"45%\"}" } ] }' ``` **Response:** ```json { "id": "chatcmpl-aad3ed3e-b401-41b8-9b3b-a9455070def5", "object": "chat.completion", "created": 1788869501, "model": "qwen/qwen3.6-27b", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "It is currently sunny in Tokyo with a temperature of 18°C and 45% humidity." }, "finish_reason": "stop" } ] } ``` ## Python OpenAI SDK Example ```python from openai import OpenAI client = OpenAI( base_url="http://127.0.0.1:8080/v1", api_key="not-needed" ) response = client.chat.completions.create( model="qwen/qwen3.6-27b", messages=[ {"role": "system", "content": "You are a helpful coding assistant."}, {"role": "user", "content": "Write a Python function to reverse a string."} ] ) print(response.choices[0].message.content) ``` ## License MIT License