luxferre/qorona
1
0
Fork 0
Code Actions Activity

feat: initial release of q38max gateway

Browse Source
This commit is contained in:
Luxferre
2026-08-27 13:54:10 +03:00
commit d68efdbf59
7 changed files with 2028 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.gitignore
+2
View File
@@ -0,0 +1,2 @@
bin/
*.log
Makefile
+13
View File
@@ -0,0 +1,13 @@
all: q38max
q38max:
mkdir -p bin
go build -trimpath -ldflags="-s -w" -o bin/q38max main.go
test:
go test -v ./...
clean:
rm -rf bin
.PHONY: all q38max test clean
README.md
+184
View File
@@ -0,0 +1,184 @@
# q38max
Standalone, zero-dependency OpenAI-compatible proxy gateway in Go for the **Qwen 3.8 Max** model (`Qwen/Qwen3.8-Max`) hosted on Hugging Face Spaces (`harpreetsahota-qwen38-max-openlogo-demo.hf.space`).
## Overview
`q38max` reverse-engineers the FiftyOne plugin backend operator interface of the Hugging Face space and transforms it into a standard, production-ready OpenAI API endpoint (`/v1/chat/completions` and `/v1/models`).
### Features
- **OpenAI Standard Compatibility**: Full drop-in replacement for OpenAI API clients (Curl, Python `openai`, LangChain, LiteLLM, Open-WebUI).
- **Zero External Dependencies**: Pure standard library Go implementation (`net/http`, `encoding/json`, `crypto/rand`, `time`).
- **Live Streaming SSE & Reasoning**: Streams real-time tokens with separation of reasoning content (`delta.reasoning_content`) and message content (`delta.content`).
- **Function / Tool Calling Interception**: Supports OpenAI `tools` specification, system prompt tool schema injection, and stateful streaming interception of tool calls (`delta.tool_calls` and `finish_reason: "tool_calls"`).
- **Session Lifecycle Management**: Thread-safe automatic session creation (`/__session/start`), periodic background heartbeats (`/__session/heartbeat`), and auto-reconnect recovery.
- **Fibonacci Backoff Retry**: Resilient against network hiccups and transient timeouts.
---
## Architecture & Upstream Protocol
```
+---------------------------+ OpenAI HTTP / SSE +------------------------+
| Client (Python / Curl / | ===========================> | q38max Gateway |
| OpenAI SDK / Open-WebUI) | | (localhost:8080) |
+---------------------------+ +------------------------+
|
| FiftyOne Session &
| Operator API
v
+------------------------+
| HuggingFace Space |
| FiftyOne Backend |
| (Qwen 3.8 Max Model) |
+------------------------+
```
### Upstream Flow:
1. `POST /__session/start` -> Allocates an ephemeral session token `X-FiftyOne-Session` and dataset clone.
2. `POST /operators/execute` -> Dispatches the `@harpreetsahota/qwen38-max/qwen38_chat` operator with method `"ask"`.
3. Polling Loops:
- `get_thinking_chunk`: Extracts newly generated reasoning tokens in real-time.
- `get_stream_chunk`: Extracts newly generated message content tokens in real-time.
---
## Build & Run
### Build
```bash
make q38max
```
Binary is output to `bin/q38max`.
### Run
```bash
./bin/q38max -port 8080
```
### CLI Flags
| Flag | Default | Description |
|------|---------|-------------|
| `-port` | `8080` | Port to listen on |
| `-space-url` | `https://harpreetsahota-qwen38-max-openlogo-demo.hf.space` | Upstream Hugging Face Space URL |
| `-sample-path` | `/home/user/datasets/openlogo/data/data_0/logos32plus_002359.jpg` | Container image sample path |
| `-model` | `qwen-3.8-max` | Default model identifier |
| `-timeout` | `300` | Upstream timeout in seconds |
| `-user-agent` / `-ua` | `""` | Custom User-Agent header |
| `-hf-token` | `""` | Optional Hugging Face token |
---
## API Usage Examples
### 1. List Models
```bash
curl http://localhost:8080/v1/models
```
### 2. Non-Streaming Chat Completion
```bash
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-max",
"messages": [
{"role": "user", "content": "What is the capital of Germany? Answer in 1 word."}
],
"reasoning_effort": "none",
"max_tokens": 50
}'
```
### 3. Streaming Chat Completion with Reasoning
```bash
curl -N -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-max",
"messages": [
{"role": "user", "content": "Calculate 25 * 25 and explain in one sentence."}
],
"stream": true,
"reasoning_effort": "medium",
"max_tokens": 150
}'
```
### 4. Function / Tool Calling
```bash
curl -X POST http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-max",
"messages": [
{"role": "user", "content": "What is the weather in Berlin?"}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a city",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
}
}
}
],
"reasoning_effort": "none"
}'
```
### 5. Python OpenAI Client Example
```python
from openai import OpenAI
client = OpenAI(
base_url="http://localhost:8080/v1",
api_key="none"
)
response = client.chat.completions.create(
model="qwen-3.8-max",
messages=[
{"role": "user", "content": "Write a short haiku about computers."}
],
stream=True
)
for chunk in response:
delta = chunk.choices[0].delta
if hasattr(delta, "reasoning_content") and delta.reasoning_content:
print(f"[Thinking] {delta.reasoning_content}", end="", flush=True)
if delta.content:
print(delta.content, end="", flush=True)
print()
```
---
## Testing
Run unit tests:
```bash
make test
```
Run end-to-end integration tests:
```bash
./xtest.sh 8080
```
---
## License
Public Domain / Unlicense
go.mod
+3
View File
@@ -0,0 +1,3 @@
module q38max
go 1.22
main.go
+1640
View File
File diff suppressed because it is too large Load Diff
q38max_test.go
+119
View File
@@ -0,0 +1,119 @@
// Unit and integration tests for q38max
// Created by Luxferre in 2026, released into the public domain
package main
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
"time"
)
func TestChatMessageGetContentString(t *testing.T) {
msg1 := ChatMessage{Role: "user", Content: "Hello world"}
if msg1.GetContentString() != "Hello world" {
t.Fatalf("expected 'Hello world', got %q", msg1.GetContentString())
}
msg2 := ChatMessage{
Role: "user",
Content: []interface{}{
map[string]interface{}{"type": "text", "text": "Part 1 "},
map[string]interface{}{"type": "text", "text": "Part 2"},
},
}
if msg2.GetContentString() != "Part 1 Part 2" {
t.Fatalf("expected 'Part 1 Part 2', got %q", msg2.GetContentString())
}
}
func TestExtractThinkingContent(t *testing.T) {
raw := "<think>\nAnalyzing the user's request...\n</think>\nHere is the answer."
thinking, clean := ExtractThinkingContent(raw)
if thinking != "Analyzing the user's request..." {
t.Fatalf("unexpected thinking extraction: %q", thinking)
}
if clean != "Here is the answer." {
t.Fatalf("unexpected clean text: %q", clean)
}
}
func TestDetectToolCalls(t *testing.T) {
xmlInput := "Let me check the weather.\n<tool_call>{\"name\": \"get_weather\", \"arguments\": {\"location\": \"Tokyo\"}}</tool_call>"
calls, rem := DetectToolCalls(xmlInput)
if len(calls) != 1 {
t.Fatalf("expected 1 tool call, got %d", len(calls))
}
if calls[0].Function.Name != "get_weather" {
t.Fatalf("expected function name 'get_weather', got %q", calls[0].Function.Name)
}
if !strings.Contains(calls[0].Function.Arguments, "Tokyo") {
t.Fatalf("expected argument with Tokyo, got %q", calls[0].Function.Arguments)
}
if strings.TrimSpace(rem) != "Let me check the weather." {
t.Fatalf("unexpected remaining text: %q", rem)
}
}
func TestPrepareConversation(t *testing.T) {
req := ChatCompletionRequest{
Model: "qwen-3.8-max",
Messages: []ChatMessage{
{Role: "system", Content: "You are a helpful assistant."},
{Role: "user", Content: "Tell me a joke."},
{Role: "assistant", Content: "Why did the chicken cross the road?"},
{Role: "user", Content: "Why?"},
},
ReasoningEffort: "medium",
}
history, question, thinkingMode := PrepareConversation(req)
if thinkingMode != "true" {
t.Fatalf("expected thinkingMode 'true', got %q", thinkingMode)
}
if len(history) != 2 {
t.Fatalf("expected 2 history items, got %d", len(history))
}
if question != "Why?" {
t.Fatalf("expected question 'Why?', got %q", question)
}
if !strings.Contains(history[0]["content"].(string), "You are a helpful assistant.") {
t.Fatalf("expected system prompt inside first turn, got %v", history[0]["content"])
}
}
func TestModelsHandler(t *testing.T) {
gw := NewQ38Gateway("https://mock.hf.space", "/dummy/path.jpg", "qwen-3.8-max", 5*time.Second)
req := httptest.NewRequest("GET", "/v1/models", nil)
w := httptest.NewRecorder()
gw.HandleModels(w, req)
if w.Code != http.StatusOK {
t.Fatalf("expected 200 OK, got %d", w.Code)
}
var res ModelsResponse
if err := json.NewDecoder(w.Body).Decode(&res); err != nil {
t.Fatalf("failed to decode response: %v", err)
}
if len(res.Data) == 0 {
t.Fatalf("expected at least 1 model in response")
}
found := false
for _, m := range res.Data {
if m.ID == "qwen-3.8-max" {
found = true
break
}
}
if !found {
t.Fatalf("qwen-3.8-max not found in models list")
}
}
xtest.sh
Executable
+67
View File
@@ -0,0 +1,67 @@
#!/usr/bin/env bash
set -e
PORT=${1:-18080}
BASE_URL="http://localhost:${PORT}"
echo "=== 1. Testing Models Endpoint ==="
curl -s "${BASE_URL}/v1/models" | jq .
echo ""
echo "=== 2. Testing Non-Streaming Chat Completion ==="
curl -s -X POST "${BASE_URL}/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-max",
"messages": [
{"role": "user", "content": "What is the capital of Italy? Answer in 1 word."}
],
"reasoning_effort": "none",
"max_tokens": 50
}' | jq .
echo ""
echo "=== 3. Testing Streaming SSE Completion (with reasoning) ==="
curl -N -s -X POST "${BASE_URL}/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-max",
"messages": [
{"role": "user", "content": "Calculate 25 * 25 and explain briefly in one sentence."}
],
"stream": true,
"reasoning_effort": "medium",
"max_tokens": 150
}'
echo ""
echo "=== 4. Testing Function/Tool Calling ==="
curl -s -X POST "${BASE_URL}/v1/chat/completions" \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-3.8-max",
"messages": [
{"role": "user", "content": "What is the weather in Berlin?"}
],
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"}
},
"required": ["location"]
}
}
}
],
"reasoning_effort": "none",
"max_tokens": 200
}' | jq .
echo ""
echo "=== All integration tests finished successfully! ==="