groqqer/README.md

# groqqer

`groqqer` is a lightweight, zero-dependency Go OpenAI-compatible API gateway and proxy for the Groq Streamlit chatbot space (`https://dromerosm-groq-chatbot.hf.space`).

It converts standard OpenAI chat completions and model requests into automated headless browser interactions, providing fast inference with Groq models without requiring an API key.

## Features

- **OpenAI Compatible API**: Exposes standard `/v1/models` and `/v1/chat/completions` endpoints. Drop-in replacement for OpenAI SDKs, Open-WebUI, LiteLLM, LibreChat, and LangChain.
- **Zero Third-Party Dependencies**: Written entirely in pure Go using only the standard library (`net/http`, `encoding/json`, `os/exec`, `crypto/rand`, etc.). Includes a built-in RFC 6455 WebSocket CDP client and RFC 1928 SOCKS5 proxy client.
- **System Prompt Support**: Translates developer/system messages into natural instruction framing.
- **Streaming & Non-Streaming**: Supports Server-Sent Events (`stream: true`) with real-time token delivery, as well as synchronous JSON responses.
- **Tool / Function Calling**: Fully supports OpenAI `tools`, `tool_choice`, and multi-turn execution (`role: "tool"`). Automatically injects schemas and parses `<tool_call>` responses into OpenAI `tool_calls` payloads with `finish_reason: "tool_calls"`.
- **Reasoning Content**: Automatically parses `<think>` and `</think>` tags from reasoning models (such as Qwen 3.6/3.8) and streams or populates `reasoning_content` in accordance with OpenAI O-series conventions.
- **Dynamic Model Discovery**: Scrapes active models directly from the Streamlit UI on startup and allows seamless model switching between requests (`qwen/qwen3.6-27b`, `openai/gpt-oss-120b`, `openai/gpt-oss-20b`, `groq/compound`, etc.).
- **Display Isolation via Xvfb**: Automatically discovers and spawns a virtual X display (`:100` - `:199`) to keep browser automation completely isolated from host desktop displays. Gracefully falls back to offscreen coordinates if Xvfb is not present.
- **SOCKS5 Proxy Tunneling**: Tunnel browser traffic through SOCKS5 proxies using the `-socks` flag or `ALL_PROXY` / `SOCKS5_PROXY` environment variables.
- **Clean Session Isolation**: Automatically clears Streamlit conversation state between requests to prevent cumulative token window exhaustion or stale error states.

## Architecture

```
+---------------------------+
| OpenAI Client / SDK / App |
+---------------------------+
              |  HTTP POST /v1/chat/completions
              v
+---------------------------+
|          groqqer          |
|  (Pure Go Standard Lib)   |
+---------------------------+
              |  RFC 6455 WebSocket CDP
              v
+---------------------------+
|    Chromium / Chrome      |
| (Display :100+ via Xvfb)  |
+---------------------------+
              |  HTTPS
              v
+---------------------------+
|   Groq Streamlit Space    |
| (dromerosm-groq-chatbot)  |
+---------------------------+
```

## Requirements

- **Linux** (x86_64 or aarch64)
- **Go 1.20+**
- **Chromium** or **Google Chrome** installed (`/usr/bin/chromium`, `/usr/bin/google-chrome`, or in `$PATH`)
- *(Optional)* **Xvfb** (`xorg-server-xvfb`) for display isolation

## Installation & Build

Clone or navigate to the repository, then build the binary:

```bash
make
```

The optimized binary will be placed at `bin/groqqer`.

## Running the Server

Start the gateway with default settings:

```bash
./bin/groqqer -port 8080
```

### CLI Flags

| Flag | Default | Description |
|------|---------|-------------|
| `-port` | `8080` | HTTP server listening port |
| `-target` | `https://dromerosm-groq-chatbot.hf.space` | Target Groq Streamlit space URL |
| `-default-model` | `llama-3.3-70b-versatile` | Fallback model if request does not specify one |
| `-browser` | `""` | Custom path to Chromium/Chrome binary |
| `-xvfb` | `true` | Enable virtual X server display isolation |
| `-no-xvfb` | `false` | Disable virtual X server (run offscreen) |
| `-socks` | `""` | SOCKS5 proxy URL (e.g. `socks5://127.0.0.1:1080`) |
| `-user-agent` | *(Chrome 133 UA)* | Custom browser User-Agent string |

## API Usage Examples

### 1. List Available Models

```bash
curl -s http://127.0.0.1:8080/v1/models
```

**Response:**
```json
{
  "object": "list",
  "data": [
    {"id": "qwen/qwen3.6-27b", "object": "model", "created": 1788869470, "owned_by": "groq"},
    {"id": "openai/gpt-oss-120b", "object": "model", "created": 1788869470, "owned_by": "groq"},
    {"id": "openai/gpt-oss-20b", "object": "model", "created": 1788869470, "owned_by": "groq"},
    {"id": "groq/compound", "object": "model", "created": 1788869470, "owned_by": "groq"}
  ]
}
```

---

### 2. Chat Completion (Non-Streaming)

```bash
curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3.6-27b",
    "messages": [
      {"role": "system", "content": "You are a concise assistant. Reply in one sentence."},
      {"role": "user", "content": "What is the capital of France?"}
    ]
  }'
```

**Response:**
```json
{
  "id": "chatcmpl-9691bbf8-8623-4155-a40f-a00e6b6903ac",
  "object": "chat.completion",
  "created": 1788869478,
  "model": "qwen/qwen3.6-27b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "The capital of France is Paris.",
        "reasoning_content": "Analyze User Input: Question: 'What is the capital of France?'..."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 0,
    "completion_tokens": 0,
    "total_tokens": 0
  }
}
```

---

### 3. Streaming Chat Completion (SSE)

```bash
curl -N -s -X POST http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3.6-27b",
    "messages": [
      {"role": "user", "content": "Count from 1 to 4 with commas."}
    ],
    "stream": true
  }'
```

**Output:**
```
data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"role":"assistant"}}]}

data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"reasoning_content":"Thinking process..."}}]}

data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":"1, 2, 3,"}}]}

data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{"content":" 4"}}]}

data: {"id":"chatcmpl-341d6d05","object":"chat.completion.chunk","created":1788869487,"model":"qwen/qwen3.6-27b","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: [DONE]
```

---

### 4. Tool / Function Calling

```bash
curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3.6-27b",
    "messages": [
      {"role": "user", "content": "What is the current weather in Tokyo?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_current_weather",
          "description": "Get current weather for a city",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {"type": "string", "description": "The city and country, e.g. Tokyo, Japan"}
            },
            "required": ["location"]
          }
        }
      }
    ],
    "tool_choice": "auto"
  }'
```

**Response:**
```json
{
  "id": "chatcmpl-cb050ba0-2573-403f-a1db-e76cdfb8f99d",
  "object": "chat.completion",
  "created": 1788869493,
  "model": "qwen/qwen3.6-27b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_ff437271",
            "type": "function",
            "function": {
              "name": "get_current_weather",
              "arguments": "{\"location\":\"Tokyo, Japan\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}
```

---

### 5. Multi-Turn Tool Response Execution

Once you run your tool locally, send the tool result back using `role: "tool"`:

```bash
curl -s -X POST http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen3.6-27b",
    "messages": [
      {"role": "user", "content": "What is the current weather in Tokyo?"},
      {
        "role": "assistant",
        "tool_calls": [
          {
            "id": "call_ff437271",
            "type": "function",
            "function": {
              "name": "get_current_weather",
              "arguments": "{\"location\":\"Tokyo, Japan\"}"
            }
          }
        ]
      },
      {
        "role": "tool",
        "tool_call_id": "call_ff437271",
        "name": "get_current_weather",
        "content": "{\"temperature\": \"18°C\", \"condition\": \"Sunny\", \"humidity\": \"45%\"}"
      }
    ]
  }'
```

**Response:**
```json
{
  "id": "chatcmpl-aad3ed3e-b401-41b8-9b3b-a9455070def5",
  "object": "chat.completion",
  "created": 1788869501,
  "model": "qwen/qwen3.6-27b",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "It is currently sunny in Tokyo with a temperature of 18°C and 45% humidity."
      },
      "finish_reason": "stop"
    }
  ]
}
```

## Python OpenAI SDK Example

```python
from openai import OpenAI

client = OpenAI(
  base_url="http://127.0.0.1:8080/v1",
  api_key="not-needed"
)

response = client.chat.completions.create(
  model="qwen/qwen3.6-27b",
  messages=[
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": "Write a Python function to reverse a string."}
  ]
)

print(response.choices[0].message.content)
```

## License

MIT License