luxferre/groqqer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: enforce tool whitelisting and isolate prompt tool call markers

Browse Source
This commit is contained in:
Luxferre
2026-09-08 16:59:17 +03:00
parent 354db0afcb
commit c99ffcac23
4 changed files with 2210 additions and 1092 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+65 -63
View File
@@ -1,21 +1,21 @@
# groqqer
`groqqer` is a lightweight, zero-dependency Go OpenAI-compatible API gateway and proxy for the Groq Streamlit chatbot space (`https://dromerosm-groq-chatbot.hf.space`).
`groqqer` is a lightweight, zero-dependency Go OpenAI-compatible API gateway and proxy for the Groq Streamlit space (`https://dromerosm-groq-chatbot.hf.space`).
It converts standard OpenAI chat completions and model requests into automated headless browser interactions, providing fast inference with Groq models without requiring an API key.
It connects directly to Streamlit's binary WebSocket engine (`/_stcore/stream`) using a pure Go Protobuf wire-format encoder/decoder. It requires **no browser**, **no Chromium**, and **no Xvfb**, allowing it to run smoothly on minimal headless servers and low-resource containers.
## Features
## Key Features
- **OpenAI Compatible API**: Exposes standard `/v1/models` and `/v1/chat/completions` endpoints. Drop-in replacement for OpenAI SDKs, Open-WebUI, LiteLLM, LibreChat, and LangChain.
- **Zero Third-Party Dependencies**: Written entirely in pure Go using only the standard library (`net/http`, `encoding/json`, `os/exec`, `crypto/rand`, etc.). Includes a built-in RFC 6455 WebSocket CDP client and RFC 1928 SOCKS5 proxy client.
- **System Prompt Support**: Translates developer/system messages into natural instruction framing.
- **Streaming & Non-Streaming**: Supports Server-Sent Events (`stream: true`) with real-time token delivery, as well as synchronous JSON responses.
- **Tool / Function Calling**: Fully supports OpenAI `tools`, `tool_choice`, and multi-turn execution (`role: "tool"`). Automatically injects schemas and parses `<tool_call>` responses into OpenAI `tool_calls` payloads with `finish_reason: "tool_calls"`.
- **Reasoning Content**: Automatically parses `<think>` and `</think>` tags from reasoning models (such as Qwen 3.6/3.8) and streams or populates `reasoning_content` in accordance with OpenAI O-series conventions.
- **Dynamic Model Discovery**: Scrapes active models directly from the Streamlit UI on startup and allows seamless model switching between requests (`qwen/qwen3.6-27b`, `openai/gpt-oss-120b`, `openai/gpt-oss-20b`, `groq/compound`, etc.).
- **Display Isolation via Xvfb**: Automatically discovers and spawns a virtual X display (`:100` - `:199`) to keep browser automation completely isolated from host desktop displays. Gracefully falls back to offscreen coordinates if Xvfb is not present.
- **SOCKS5 Proxy Tunneling**: Tunnel browser traffic through SOCKS5 proxies using the `-socks` flag or `ALL_PROXY` / `SOCKS5_PROXY` environment variables.
- **Clean Session Isolation**: Automatically clears Streamlit conversation state between requests to prevent cumulative token window exhaustion or stale error states.
- **Direct WebSocket Protobuf Protocol**: Communicates directly with Streamlit's internal engine over RFC 6455 WebSockets and Protocol Buffers wire format.
- **Completely Headless & Zero Dependencies**: Written entirely in pure Go using only the standard library (`net/http`, `crypto/tls`, `encoding/binary`, `encoding/json`, etc.). No Chromium, Chrome, Xvfb, Puppeteer, or external Go modules required.
- **Sub-Second Latency**: Bypasses browser rendering and DOM parsing entirely, delivering responses with minimal overhead.
- **OpenAI Compatible API**: Exposes standard `/v1/models` and `/v1/chat/completions` endpoints. Drop-in replacement for OpenAI SDKs, LiteLLM, Open-WebUI, LibreChat, and LangChain.
- **System Prompt Support**: Formats developer/system instructions cleanly for the target model.
- **Streaming & Non-Streaming**: Supports Server-Sent Events (`stream: true`) with real-time token streaming and synchronous JSON responses.
- **Tool / Function Calling**: Fully supports OpenAI `tools`, `tool_choice`, and multi-turn execution (`role: "tool"`). Automatically injects schemas and parses `<tool_call>` outputs into standard OpenAI `tool_calls` payloads with `finish_reason: "tool_calls"`.
- **Reasoning Content**: Extracts `<think>` and `</think>` tags from reasoning models (e.g., Qwen 3.6/3.8) and streams or populates `reasoning_content` following OpenAI O-series conventions.
- **Dynamic Model Discovery & Switching**: Discovers active models directly from the Streamlit space on startup and allows seamless model switching between requests (`qwen/qwen3.6-27b`, `openai/gpt-oss-120b`, `openai/gpt-oss-20b`, `groq/compound`, etc.).
- **SOCKS5 Proxy Support**: Native pure Go SOCKS5 proxy client supporting authentication (`-socks` flag or `ALL_PROXY` / `SOCKS5_PROXY` environment variables).
## Architecture
@@ -28,14 +28,9 @@ It converts standard OpenAI chat completions and model requests into automated h
+---------------------------+
| groqqer |
| (Pure Go Standard Lib) |
| RFC 6455 WS + Protobuf |
+---------------------------+
| RFC 6455 WebSocket CDP
v
+---------------------------+
| Chromium / Chrome |
| (Display :100+ via Xvfb) |
+---------------------------+
| HTTPS
| WSS (TLS WebSocket)
v
+---------------------------+
| Groq Streamlit Space |
@@ -45,10 +40,9 @@ It converts standard OpenAI chat completions and model requests into automated h
## Requirements
- **Linux** (x86_64 or aarch64)
- **Go 1.20+**
- **Chromium** or **Google Chrome** installed (`/usr/bin/chromium`, `/usr/bin/google-chrome`, or in `$PATH`)
- *(Optional)* **Xvfb** (`xorg-server-xvfb`) for display isolation
- **Linux / macOS / Windows**
- **Go 1.20+** (for building from source)
- No browser or graphics packages needed.
## Installation & Build
@@ -58,7 +52,7 @@ Clone or navigate to the repository, then build the binary:
make
```
The optimized binary will be placed at `bin/groqqer`.
The optimized binary will be created at `bin/groqqer`.
## Running the Server
@@ -74,12 +68,11 @@ Start the gateway with default settings:
|------|---------|-------------|
| `-port` | `8080` | HTTP server listening port |
| `-target` | `https://dromerosm-groq-chatbot.hf.space` | Target Groq Streamlit space URL |
| `-default-model` | `llama-3.3-70b-versatile` | Fallback model if request does not specify one |
| `-browser` | `""` | Custom path to Chromium/Chrome binary |
| `-xvfb` | `true` | Enable virtual X server display isolation |
| `-no-xvfb` | `false` | Disable virtual X server (run offscreen) |
| `-default-model` | `qwen/qwen3.6-27b` | Fallback model if request does not specify one |
| `-socks` | `""` | SOCKS5 proxy URL (e.g. `socks5://127.0.0.1:1080`) |
| `-user-agent` | *(Chrome 133 UA)* | Custom browser User-Agent string |
| `-user-agent` | *(Chrome UA)* | Custom User-Agent string |
*(Note: legacy flags `-browser`, `-xvfb`, and `-headless` are retained for backward compatibility but are ignored, as groqqer operates completely headless via direct WebSocket).*
## API Usage Examples
@@ -94,10 +87,13 @@ curl -s http://127.0.0.1:8080/v1/models
{
"object": "list",
"data": [
{"id": "qwen/qwen3.6-27b", "object": "model", "created": 1788869470, "owned_by": "groq"},
{"id": "openai/gpt-oss-120b", "object": "model", "created": 1788869470, "owned_by": "groq"},
{"id": "openai/gpt-oss-20b", "object": "model", "created": 1788869470, "owned_by": "groq"},
{"id": "groq/compound", "object": "model", "created": 1788869470, "owned_by": "groq"}
{"id": "qwen/qwen3.6-27b", "object": "model", "created": 1788870683, "owned_by": "groq"},
{"id": "qwen/qwen3.8-27b", "object": "model", "created": 1788870683, "owned_by": "groq"},
{"id": "openai/gpt-oss-120b", "object": "model", "created": 1788870683, "owned_by": "groq"},
{"id": "openai/gpt-oss-20b", "object": "model", "created": 1788870683, "owned_by": "groq"},
{"id": "groq/compound", "object": "model", "created": 1788870683, "owned_by": "groq"},
{"id": "groq/compound-mini", "object": "model", "created": 1788870683, "owned_by": "groq"},
{"id": "allam-2-7b", "object": "model", "created": 1788870683, "owned_by": "groq"}
]
}
```
@@ -110,7 +106,7 @@ curl -s http://127.0.0.1:8080/v1/models
curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen/qwen3.6-27b",
"model": "openai/gpt-oss-20b",
"messages": [
{"role": "system", "content": "You are a concise assistant. Reply in one sentence."},
{"role": "user", "content": "What is the capital of France?"}
@@ -121,17 +117,16 @@ curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
**Response:**
```json
{
"id": "chatcmpl-9691bbf8-8623-4155-a40f-a00e6b6903ac",
"id": "chatcmpl-893498c9-2808-4bec-8c6a-2a0352ed69e9",
"object": "chat.completion",
"created": 1788869478,
"model": "qwen/qwen3.6-27b",
"created": 1788870700,
"model": "openai/gpt-oss-20b",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "The capital of France is Paris.",
"reasoning_content": "Analyze User Input: Question: 'What is the capital of France?'..."
"content": "The capital of France is Paris."
},
"finish_reason": "stop"
}
@@ -154,7 +149,7 @@ curl -N -s -X POST http://127.0.0.1:8080/v1/chat/completions \
-d '{
"model": "qwen/qwen3.6-27b",
"messages": [
{"role": "user", "content": "Count from 1 to 4 with commas."}
{"role": "user", "content": "Count from 1 to 4 separated by commas."}
],
"stream": true
}'
@@ -162,15 +157,19 @@ curl -N -s -X POST http://127.0.0.1:8080/v1/chat/completions \
**Output:**
```
data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"role":"assistant"}}]}
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"role":"assistant"}}]}
data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"reasoning_content":"Thinking process..."}}]}
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"reasoning_content":"Thinking process..."}}]}
data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":"1, 2, 3,"}}]}
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":"1,"}}]}
data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":" 4"}}]}
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":" 2,"}}]}
data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":" 3,"}}]}
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":" 4"}}]}
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: [DONE]
```
@@ -185,7 +184,7 @@ curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
-d '{
"model": "qwen/qwen3.6-27b",
"messages": [
{"role": "user", "content": "What is the current weather in Tokyo?"}
{"role": "user", "content": "What is the weather in Tokyo right now?"}
],
"tools": [
{
@@ -196,23 +195,23 @@ curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string", "description": "The city and country, e.g. Tokyo, Japan"}
"location": {"type": "string", "description": "City and country"}
},
"required": ["location"]
}
}
}
],
"tool_choice": "auto"
"tool_choice": "required"
}'
```
**Response:**
```json
{
"id": "chatcmpl-cb050ba0-2573-403f-a1db-e76cdfb8f99d",
"id": "chatcmpl-29f63b3b-7ab6-444c-8579-49f906007d43",
"object": "chat.completion",
"created": 1788869493,
"created": 1788870711,
"model": "qwen/qwen3.6-27b",
"choices": [
{
@@ -220,13 +219,14 @@ curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
"message": {
"role": "assistant",
"content": null,
"reasoning_content": "The user is asking for the current weather in Tokyo...",
"tool_calls": [
{
"id": "call_ff437271",
"id": "call_163e8f97",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"location\":\"Tokyo, Japan\"}"
"arguments": "{\"location\":\"Tokyo\"}"
}
}
]
@@ -241,7 +241,7 @@ curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
### 5. Multi-Turn Tool Response Execution
Once you run your tool locally, send the tool result back using `role: "tool"`:
Send back the tool execution results using `role: "tool"`:
```bash
curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
@@ -249,25 +249,25 @@ curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
-d '{
"model": "qwen/qwen3.6-27b",
"messages": [
{"role": "user", "content": "What is the current weather in Tokyo?"},
{"role": "user", "content": "What is the weather in Tokyo right now?"},
{
"role": "assistant",
"tool_calls": [
{
"id": "call_ff437271",
"id": "call_163e8f97",
"type": "function",
"function": {
"name": "get_current_weather",
"arguments": "{\"location\":\"Tokyo, Japan\"}"
"arguments": "{\"location\":\"Tokyo\"}"
}
}
]
},
{
"role": "tool",
"tool_call_id": "call_ff437271",
"tool_call_id": "call_163e8f97",
"name": "get_current_weather",
"content": "{\"temperature\": \"18°C\", \"condition\": \"Sunny\", \"humidity\": \"45%\"}"
"content": "{\"temperature\": \"19°C\", \"weather\": \"Sunny with clear skies\"}"
}
]
}'
@@ -276,16 +276,16 @@ curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
**Response:**
```json
{
"id": "chatcmpl-aad3ed3e-b401-41b8-9b3b-a9455070def5",
"id": "chatcmpl-23354bba-33c7-45be-8807-9c84243037b8",
"object": "chat.completion",
"created": 1788869501,
"created": 1788870718,
"model": "qwen/qwen3.6-27b",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "It is currently sunny in Tokyo with a temperature of 18°C and 45% humidity."
"content": "It's currently 19°C and sunny with clear skies in Tokyo"
},
"finish_reason": "stop"
}
@@ -293,7 +293,9 @@ curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
}
```
## Python OpenAI SDK Example
---
### 6. Python OpenAI SDK Example
```python
from openai import OpenAI