338 lines
10 KiB
Markdown
338 lines
10 KiB
Markdown
# groqqer
|
|
|
|
`groqqer` is a lightweight, zero-dependency Go OpenAI-compatible API gateway and proxy for the Groq Streamlit space (`https://dromerosm-groq-chatbot.hf.space`).
|
|
|
|
It connects directly to Streamlit's binary WebSocket engine (`/_stcore/stream`) using a pure Go Protobuf wire-format encoder/decoder. It requires **no browser**, **no Chromium**, and **no Xvfb**, allowing it to run smoothly on minimal headless servers and low-resource containers.
|
|
|
|
## Key features
|
|
|
|
- **Direct WebSocket Protobuf protocol**: communicates directly with Streamlit's internal engine over RFC 6455 WebSockets and Protocol Buffers wire format.
|
|
- **Completely headless and zero dependencies**: written entirely in pure Go using only the standard library (`net/http`, `crypto/tls`, `encoding/binary`, `encoding/json`, etc.). No Chromium, Chrome, Xvfb, Puppeteer, or external Go modules required.
|
|
- **Sub-second latency**: bypasses browser rendering and DOM parsing entirely, delivering responses with minimal overhead.
|
|
- **OpenAI-compatible API**: exposes standard `/v1/models` and `/v1/chat/completions` endpoints. Drop-in replacement for OpenAI SDKs, LiteLLM, Open-WebUI, LibreChat, and LangChain.
|
|
- **System prompt support**: formats developer/system instructions cleanly for the target model.
|
|
- **Streaming and non-streaming**: supports Server-Sent Events (`stream: true`) with real-time token streaming and synchronous JSON responses.
|
|
- **Tool and function calling**: fully supports OpenAI `tools`, `tool_choice`, and multi-turn execution (`role: "tool"`). Automatically injects schemas and parses `<tool_call>` outputs into standard OpenAI `tool_calls` payloads with `finish_reason: "tool_calls"`.
|
|
- **Reasoning content**: extracts `<think>` and `</think>` tags from reasoning models (e.g., Qwen 3.6/3.8) and streams or populates `reasoning_content` following OpenAI O-series conventions.
|
|
- **Dynamic model discovery and switching**: discovers active models directly from the Streamlit space on startup and allows seamless model switching between requests (`qwen/qwen3.6-27b`, `openai/gpt-oss-120b`, `openai/gpt-oss-20b`, `groq/compound`, etc.).
|
|
- **SOCKS5 proxy support**: native pure Go SOCKS5 proxy client supporting authentication (`-socks` flag or `ALL_PROXY` / `SOCKS5_PROXY` environment variables).
|
|
|
|
## Architecture
|
|
|
|
```
|
|
+---------------------------+
|
|
| OpenAI Client / SDK / App |
|
|
+---------------------------+
|
|
| HTTP POST /v1/chat/completions
|
|
v
|
|
+---------------------------+
|
|
| groqqer |
|
|
| (Pure Go Standard Lib) |
|
|
| RFC 6455 WS + Protobuf |
|
|
+---------------------------+
|
|
| WSS (TLS WebSocket)
|
|
v
|
|
+---------------------------+
|
|
| Groq Streamlit Space |
|
|
| (dromerosm-groq-chatbot) |
|
|
+---------------------------+
|
|
```
|
|
|
|
## Requirements
|
|
|
|
- Linux, macOS, or Windows
|
|
- Go 1.20+ (for building from source)
|
|
- No browser or graphics packages needed
|
|
|
|
## Installation and build
|
|
|
|
### Using `go install`
|
|
|
|
Install the binary directly to `$GOPATH/bin` (or `~/go/bin`):
|
|
|
|
```bash
|
|
go install code.luxferre.top/luxferre/groqqer@latest
|
|
```
|
|
|
|
Make sure `$GOPATH/bin` is in your `PATH`.
|
|
|
|
### Building from source
|
|
|
|
Clone or navigate to the repository, then compile the binary:
|
|
|
|
```bash
|
|
make
|
|
```
|
|
|
|
The optimized binary will be created at `bin/groqqer`. Alternatively, install into `$GOPATH/bin` from the local directory:
|
|
|
|
```bash
|
|
go install .
|
|
```
|
|
|
|
## Running the server
|
|
|
|
Start the gateway with default settings:
|
|
|
|
```bash
|
|
./bin/groqqer -port 8080
|
|
```
|
|
|
|
### CLI flags
|
|
|
|
| Flag | Default | Description |
|
|
|------|---------|-------------|
|
|
| `-port` | `8080` | HTTP server listening port |
|
|
| `-target` | `https://dromerosm-groq-chatbot.hf.space` | Target Groq Streamlit space URL |
|
|
| `-default-model` | `qwen/qwen3.6-27b` | Fallback model if request does not specify one |
|
|
| `-socks` | `""` | SOCKS5 proxy URL (e.g. `socks5://127.0.0.1:1080`) |
|
|
| `-user-agent` | *(Chrome UA)* | Custom User-Agent string |
|
|
|
|
*(Note: legacy flags `-browser`, `-xvfb`, and `-headless` are retained for backward compatibility but are ignored, as groqqer operates completely headless via direct WebSocket).*
|
|
|
|
## API usage examples
|
|
|
|
### 1. List available models
|
|
|
|
```bash
|
|
curl -s http://127.0.0.1:8080/v1/models
|
|
```
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"object": "list",
|
|
"data": [
|
|
{"id": "qwen/qwen3.6-27b", "object": "model", "created": 1788870683, "owned_by": "groq"},
|
|
{"id": "qwen/qwen3.8-27b", "object": "model", "created": 1788870683, "owned_by": "groq"},
|
|
{"id": "openai/gpt-oss-120b", "object": "model", "created": 1788870683, "owned_by": "groq"},
|
|
{"id": "openai/gpt-oss-20b", "object": "model", "created": 1788870683, "owned_by": "groq"},
|
|
{"id": "groq/compound", "object": "model", "created": 1788870683, "owned_by": "groq"},
|
|
{"id": "groq/compound-mini", "object": "model", "created": 1788870683, "owned_by": "groq"},
|
|
{"id": "allam-2-7b", "object": "model", "created": 1788870683, "owned_by": "groq"}
|
|
]
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 2. Chat completion (non-streaming)
|
|
|
|
```bash
|
|
curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"model": "openai/gpt-oss-20b",
|
|
"messages": [
|
|
{"role": "system", "content": "You are a concise assistant. Reply in one sentence."},
|
|
{"role": "user", "content": "What is the capital of France?"}
|
|
]
|
|
}'
|
|
```
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"id": "chatcmpl-893498c9-2808-4bec-8c6a-2a0352ed69e9",
|
|
"object": "chat.completion",
|
|
"created": 1788870700,
|
|
"model": "openai/gpt-oss-20b",
|
|
"choices": [
|
|
{
|
|
"index": 0,
|
|
"message": {
|
|
"role": "assistant",
|
|
"content": "The capital of France is Paris."
|
|
},
|
|
"finish_reason": "stop"
|
|
}
|
|
],
|
|
"usage": {
|
|
"prompt_tokens": 0,
|
|
"completion_tokens": 0,
|
|
"total_tokens": 0
|
|
}
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 3. Streaming chat completion (SSE)
|
|
|
|
```bash
|
|
curl -N -s -X POST http://127.0.0.1:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"model": "qwen/qwen3.6-27b",
|
|
"messages": [
|
|
{"role": "user", "content": "Count from 1 to 4 separated by commas."}
|
|
],
|
|
"stream": true
|
|
}'
|
|
```
|
|
|
|
**Output:**
|
|
```
|
|
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"role":"assistant"}}]}
|
|
|
|
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"reasoning_content":"Thinking process..."}}]}
|
|
|
|
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":"1,"}}]}
|
|
|
|
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":" 2,"}}]}
|
|
|
|
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":" 3,"}}]}
|
|
|
|
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":" 4"}}]}
|
|
|
|
data: {"id":"chatcmpl-7a06d0fb","object":"chat.completion.chunk","created":1788870705,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
|
|
|
|
data: [DONE]
|
|
```
|
|
|
|
---
|
|
|
|
### 4. Tool and function calling
|
|
|
|
```bash
|
|
curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"model": "qwen/qwen3.6-27b",
|
|
"messages": [
|
|
{"role": "user", "content": "What is the weather in Tokyo right now?"}
|
|
],
|
|
"tools": [
|
|
{
|
|
"type": "function",
|
|
"function": {
|
|
"name": "get_current_weather",
|
|
"description": "Get current weather for a city",
|
|
"parameters": {
|
|
"type": "object",
|
|
"properties": {
|
|
"location": {"type": "string", "description": "City and country"}
|
|
},
|
|
"required": ["location"]
|
|
}
|
|
}
|
|
}
|
|
],
|
|
"tool_choice": "required"
|
|
}'
|
|
```
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"id": "chatcmpl-29f63b3b-7ab6-444c-8579-49f906007d43",
|
|
"object": "chat.completion",
|
|
"created": 1788870711,
|
|
"model": "qwen/qwen3.6-27b",
|
|
"choices": [
|
|
{
|
|
"index": 0,
|
|
"message": {
|
|
"role": "assistant",
|
|
"content": null,
|
|
"reasoning_content": "The user is asking for the current weather in Tokyo...",
|
|
"tool_calls": [
|
|
{
|
|
"id": "call_163e8f97",
|
|
"type": "function",
|
|
"function": {
|
|
"name": "get_current_weather",
|
|
"arguments": "{\"location\":\"Tokyo\"}"
|
|
}
|
|
}
|
|
]
|
|
},
|
|
"finish_reason": "tool_calls"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 5. Multi-turn tool response execution
|
|
|
|
Send back the tool execution results using `role: "tool"`:
|
|
|
|
```bash
|
|
curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{
|
|
"model": "qwen/qwen3.6-27b",
|
|
"messages": [
|
|
{"role": "user", "content": "What is the weather in Tokyo right now?"},
|
|
{
|
|
"role": "assistant",
|
|
"tool_calls": [
|
|
{
|
|
"id": "call_163e8f97",
|
|
"type": "function",
|
|
"function": {
|
|
"name": "get_current_weather",
|
|
"arguments": "{\"location\":\"Tokyo\"}"
|
|
}
|
|
}
|
|
]
|
|
},
|
|
{
|
|
"role": "tool",
|
|
"tool_call_id": "call_163e8f97",
|
|
"name": "get_current_weather",
|
|
"content": "{\"temperature\": \"19°C\", \"weather\": \"Sunny with clear skies\"}"
|
|
}
|
|
]
|
|
}'
|
|
```
|
|
|
|
**Response:**
|
|
```json
|
|
{
|
|
"id": "chatcmpl-23354bba-33c7-45be-8807-9c84243037b8",
|
|
"object": "chat.completion",
|
|
"created": 1788870718,
|
|
"model": "qwen/qwen3.6-27b",
|
|
"choices": [
|
|
{
|
|
"index": 0,
|
|
"message": {
|
|
"role": "assistant",
|
|
"content": "It's currently 19°C and sunny with clear skies in Tokyo"
|
|
},
|
|
"finish_reason": "stop"
|
|
}
|
|
]
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### 6. Python OpenAI SDK example
|
|
|
|
```python
|
|
from openai import OpenAI
|
|
|
|
client = OpenAI(
|
|
base_url="http://127.0.0.1:8080/v1",
|
|
api_key="not-needed"
|
|
)
|
|
|
|
response = client.chat.completions.create(
|
|
model="qwen/qwen3.6-27b",
|
|
messages=[
|
|
{"role": "system", "content": "You are a helpful coding assistant."},
|
|
{"role": "user", "content": "Write a Python function to reverse a string."}
|
|
]
|
|
)
|
|
|
|
print(response.choices[0].message.content)
|
|
```
|
|
|
|
## License
|
|
|
|
Created by Luxferre in 2026, released into the public domain with no warranties.
|