qorona/README.md

q38max

Standalone, zero-dependency OpenAI-compatible proxy gateway in Go for the Qwen 3.8 Max model (Qwen/Qwen3.8-Max) hosted on Hugging Face Spaces (harpreetsahota-qwen38-max-openlogo-demo.hf.space).

Overview

q38max reverse-engineers the FiftyOne plugin backend operator interface of the Hugging Face space and transforms it into a standard, production-ready OpenAI API endpoint (/v1/chat/completions and /v1/models).

Features

• OpenAI Standard Compatibility: Full drop-in replacement for OpenAI API clients (Curl, Python openai, LangChain, LiteLLM, Open-WebUI).
• Zero External Dependencies: Pure standard library Go implementation (net/http, encoding/json, crypto/rand, time).
• Live Streaming SSE & Reasoning: Streams real-time tokens with separation of reasoning content (delta.reasoning_content) and message content (delta.content).
• Function / Tool Calling Interception: Supports OpenAI tools specification, system prompt tool schema injection, and stateful streaming interception of tool calls (delta.tool_calls and finish_reason: "tool_calls").
• Session Lifecycle Management: Thread-safe automatic session creation (/__session/start), periodic background heartbeats (/__session/heartbeat), and auto-reconnect recovery.
• Fibonacci Backoff Retry: Resilient against network hiccups and transient timeouts.

---

Architecture & Upstream Protocol

+---------------------------+       OpenAI HTTP / SSE       +------------------------+
| Client (Python / Curl /   |  ===========================> |     q38max Gateway     |
| OpenAI SDK / Open-WebUI)  |                               |    (localhost:8080)    |
+---------------------------+                               +------------------------+
                                                                        |
                                                                        | FiftyOne Session &
                                                                        | Operator API
                                                                        v
                                                            +------------------------+
                                                            |  HuggingFace Space     |
                                                            |  FiftyOne Backend      |
                                                            |  (Qwen 3.8 Max Model)  |
                                                            +------------------------+

Upstream Flow:

1. POST /__session/start -> Allocates an ephemeral session token X-FiftyOne-Session and dataset clone.
2. POST /operators/execute -> Dispatches the @harpreetsahota/qwen38-max/qwen38_chat operator with method "ask".
3. Polling Loops:
   • get_thinking_chunk: Extracts newly generated reasoning tokens in real-time.
   • get_stream_chunk: Extracts newly generated message content tokens in real-time.

---

Build & Run

Build

make q38max

Binary is output to bin/q38max.

Run

./bin/q38max -port 8080

CLI Flags

Flag	Default	Description
-port	8080	Port to listen on
-space-url	https://harpreetsahota-qwen38-max-openlogo-demo.hf.space	Upstream Hugging Face Space URL
-sample-path	/home/user/datasets/openlogo/data/data_0/logos32plus_002359.jpg	Container image sample path
-model	qwen-3.8-max	Default model identifier
-timeout	300	Upstream timeout in seconds
-user-agent / -ua	""	Custom User-Agent header
-hf-token	""	Optional Hugging Face token

---

API Usage Examples

1. List Models

curl http://localhost:8080/v1/models

2. Non-Streaming Chat Completion

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-3.8-max",
    "messages": [
      {"role": "user", "content": "What is the capital of Germany? Answer in 1 word."}
    ],
    "reasoning_effort": "none",
    "max_tokens": 50
  }'

3. Streaming Chat Completion with Reasoning

curl -N -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-3.8-max",
    "messages": [
      {"role": "user", "content": "Calculate 25 * 25 and explain in one sentence."}
    ],
    "stream": true,
    "reasoning_effort": "medium",
    "max_tokens": 150
  }'

4. Function / Tool Calling

curl -X POST http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen-3.8-max",
    "messages": [
      {"role": "user", "content": "What is the weather in Berlin?"}
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a city",
          "parameters": {
            "type": "object",
            "properties": {
              "location": {"type": "string"}
            },
            "required": ["location"]
          }
        }
      }
    ],
    "reasoning_effort": "none"
  }'

5. Python OpenAI Client Example

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="none"
)

response = client.chat.completions.create(
    model="qwen-3.8-max",
    messages=[
        {"role": "user", "content": "Write a short haiku about computers."}
    ],
    stream=True
)

for chunk in response:
    delta = chunk.choices[0].delta
    if hasattr(delta, "reasoning_content") and delta.reasoning_content:
        print(f"[Thinking] {delta.reasoning_content}", end="", flush=True)
    if delta.content:
        print(delta.content, end="", flush=True)
print()

---

Testing

Run unit tests:

make test

Run end-to-end integration tests:

./xtest.sh 8080

---

License

Public Domain / Unlicense