qorona/README.md

# qorona

Standalone, zero-dependency OpenAI-compatible proxy gateway in Go for the **Qwen 3.8 Max** model (`Qwen/Qwen3.8-Max`) hosted on Hugging Face Spaces (`harpreetsahota-qwen38-max-openlogo-demo.hf.space`).

## Overview

`qorona` reverse-engineers the FiftyOne plugin backend operator interface of the Hugging Face space and transforms it into a standard, production-ready OpenAI API endpoint (`/v1/chat/completions` and `/v1/models`).

### Features

- **OpenAI Standard Compatibility**: Full drop-in replacement for OpenAI API clients (Curl, Python `openai`, LangChain, LiteLLM, Open-WebUI).
- **Zero External Dependencies**: Pure standard library Go implementation (`net/http`, `encoding/json`, `crypto/rand`, `time`).
- **Fully Headless & Browserless**: No Chromium, Playwright, or X11 required. Runs directly on bare servers, containers, or embedded systems.
- **Live Streaming SSE & Reasoning**: Streams real-time tokens with separation of reasoning content (`delta.reasoning_content`) and message content (`delta.content`).
- **Function / Tool Calling Interception**: Supports OpenAI `tools` specification, system prompt tool schema injection, and stateful streaming interception of tool calls (`delta.tool_calls` and `finish_reason: "tool_calls"`).
- **Session Lifecycle Management**: Thread-safe automatic session creation (`/__session/start`), periodic background heartbeats (`/__session/heartbeat`), and auto-reconnect recovery.
- **Fibonacci Backoff Retry**: Resilient against network hiccups and transient timeouts.

---

## Architecture & Upstream Protocol

```
+---------------------------+       OpenAI HTTP / SSE       +------------------------+
| Client (Python / Curl /   |  ===========================> |     qorona Gateway     |
| OpenAI SDK / Open-WebUI)  |                               |    (localhost:8080)    |
+---------------------------+                               +------------------------+
                                                                        |
                                                                        | FiftyOne Session &
                                                                        | Operator API
                                                                        v
                                                            +------------------------+
                                                            |  HuggingFace Space     |
                                                            |  FiftyOne Backend      |
                                                            |  (Qwen 3.8 Max Model)  |
                                                            +------------------------+
```

### Upstream Flow:
1. `POST /__session/start` -> Allocates an ephemeral session token `X-FiftyOne-Session` and dataset clone.
2. `POST /operators/execute` -> Dispatches the `@harpreetsahota/qwen38-max/qwen38_chat` operator with method `"ask"`.
3. Polling Loops:
   - `get_thinking_chunk`: Extracts newly generated reasoning tokens in real-time.
   - `get_stream_chunk`: Extracts newly generated message content tokens in real-time.

---

## Build & Run

### Build
```bash
make qorona
```

Binary is output to `bin/qorona`.

### Run
```bash
./bin/qorona -port 8080
```

### CLI Flags

| Flag | Default | Description |
|------|---------|-------------|
| `-port` | `8080` | Port to listen on |
| `-space-url` | `https://harpreetsahota-qwen38-max-openlogo-demo.hf.space` | Upstream Hugging Face Space URL |
| `-sample-path` | `/home/user/datasets/openlogo/data/data_0/logos32plus_002359.jpg` | Container image sample path |
| `-model` | `qwen-3.8-max` | Default model identifier |
| `-timeout` | `300` | Upstream timeout in seconds |
| `-user-agent` / `-ua` | `""` | Custom User-Agent header |
| `-hf-token` | `""` | Optional Hugging Face token |

---

## API Usage Examples

### 1. List Models
```bash
curl http://localhost:8080/v1/models
```

### 2. Non-Streaming Chat Completion
```bash
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qorona",
    "messages": [
      {"role": "user", "content": "What is the capital of Germany? Answer in 1 word."}
    ],
    "reasoning_effort": "none",
    "max_tokens": 50
  }'
```

### 3. Streaming Chat Completion with Reasoning
```bash
curl -N -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qorona",
    "messages": [
      {"role": "user", "content": "Calculate 25 * 25 and explain in one sentence."}
    ],
    "stream": true,
    "reasoning_effort": "medium",
    "max_tokens": 150
  }'
```

### 4. Function / Tool Calling
```bash
curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qorona",
    "messages": [
      {"role": "user", "content": "What is the weather in Berlin?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a city",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {"type": "string"}
            },
            "required": ["location"]
          }
        }
      }
    ],
    "reasoning_effort": "none"
  }'
```

### 5. Python OpenAI Client Example
```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="none"
)

response = client.chat.completions.create(
    model="qorona",
    messages=[
        {"role": "user", "content": "Write a short haiku about computers."}
    ],
    stream=True
)

for chunk in response:
    delta = chunk.choices[0].delta
    if hasattr(delta, "reasoning_content") and delta.reasoning_content:
        print(f"[Thinking] {delta.reasoning_content}", end="", flush=True)
    if delta.content:
        print(delta.content, end="", flush=True)
print()
```

---

## Testing

Run unit tests:
```bash
make test
```

Run end-to-end integration tests:
```bash
./xtest.sh 8080
```

---

## License

Public Domain / Unlicense