qflash/README.md

Qflash: OpenAI-compatible gateway for Qwen3.8 models

About

Qflash is a standalone, single-binary gateway that exposes Qwen3.8 model spaces on Hugging Face through a standard OpenAI-compatible API. It automatically detects and supports multiple upstream protocols:

• gradio /chat_response endpoints (such as https://halvo78-qwen3-8-flash-next-playground.hf.space)
• direct OpenAI /v1/chat/completions endpoints (such as https://wanyamaelis-qwen3-8-27b.hf.space and https://apathy-exe-qwen3-8-flash-next.hf.space, running persistent llama-server instances on CPU with zero quota limits)
• gradio /respond endpoints (supported for custom spaces such as https://microhero-qwen3-8-27b-uncensored-chat.hf.space)

It translates standard /v1/chat/completions and /v1/models requests and Server-Sent Events (SSE) stream protocols into upstream formats, allowing any standard OpenAI-compatible client, agent, or IDE to interface with Qwen3.8 models without modification.

Features

• openai-compatible chat completions (streaming and non-streaming)
• automatic upstream failover from primary playground to non-ZeroGPU endpoints
• smart endpoint cooldown (5 minutes on quota exhaustion, 30 seconds on network errors)
• automatic upstream endpoint detection (respond, openai, chat_response)
• deep reasoning extraction with thinking trace passthrough (<think> tags and blockquotes mapped to reasoning_content)
• stateful streaming tool call interception (StreamToolCallFilter) with zero XML or JSON leakage into delta.content
• real-time token streaming with incremental SSE delivery
• support for instruct mode via standard reasoning_effort: "none"
• zero-dependency SOCKS5 proxy client (RFC 1928 / RFC 1929) with domain resolution (socks5h://), IPv4, IPv6, and auth
• bring your own key (BYOK) pass-through support via Authorization: Bearer or CLI flags
• zero-gpu quota authentication via free Hugging Face personal tokens (HF_TOKEN)
• fibonacci backoff retry on transient upstream errors
• zero external dependencies (Go standard library only)

Supported upstream spaces

Space	Model	Hardware	Protocol	Notes
Halvo78/qwen3-8-flash-next-playground (primary)	Qwen3.8-Flash-Next	CPU basic	Gradio /chat_response	Default endpoint; auto-fails over to live endpoints if unavailable
Wanyamaelis/Qwen3.8-27B (fallback)	Qwen3.8-27B (MTP)	CPU basic (8 vCPU)	Native OpenAI /v1	Non-ZeroGPU, no quota limits, fast speculative decoding (~5s)
apathy-exe/Qwen3.8-27B (fallback)	Qwen3.8-27B (MTP)	CPU (OpenMP/AVX512)	Native OpenAI /v1	Non-ZeroGPU, no quota limits, speculative decoding
apathy-exe/Qwen3.8-Flash-Next (fallback)	Qwen3.8-Flash-Next (~177B)	CPU (OpenMP/AVX512)	Native OpenAI /v1	Non-ZeroGPU, no quota limits, full 177B Flash-Next model
MicroHERO/qwen3.8-27b-uncensored-chat	Qwen3.8-27B Uncensored	ZeroGPU (A10G)	Gradio /respond	Fast live GPU inference, can be configured via custom -endpoints

Auto-failover and quota handling

By default, Qflash attempts the primary Halvo78 playground endpoint. If the primary space is unavailable, offline, or lacks live inference credentials, it automatically fails over to the persistent CPU endpoints (wanyamaelis and apathy-exe), ensuring uninterrupted service:

• with auto-failover enabled (default), when an endpoint encounters an error, timeout, or unavailable backend, it is placed on cooldown and the gateway automatically fails over to the next configured endpoint
• transient network errors trigger a 30-second cooldown, while quota limits or unavailable sandbox spaces trigger a 5-minute cooldown
• optional ZeroGPU or private spaces can be authenticated using a Hugging Face personal access token (https://huggingface.co/settings/tokens) via the HF_TOKEN environment variable, the -hf-token CLI flag, or the Authorization: Bearer hf_... header

Installation

You need Go 1.22 or newer (tested on Go 1.26). Install the latest release straight from the Git repository with go install:

go install code.luxferre.top/luxferre/qflash@latest

This fetches the module from https://code.luxferre.top/luxferre/qflash.git and places the qflash binary in $(go env GOPATH)/bin. Make sure that directory is on your PATH.

(Note: If installing right after a new commit has been pushed, bypass any proxy cache with GOPROXY=direct go install code.luxferre.top/luxferre/qflash@latest).

If you prefer to build from a local checkout instead:

git clone https://code.luxferre.top/luxferre/qflash.git
cd qflash
go install .

Alternatively, build directly from source using make:

make qflash

This produces the bin/qflash binary for your platform.

Models served

The gateway advertises the following models under /v1/models:

Model ID	Target model	Description
Qwen/Qwen3.8-Flash-Next	Qwen/Qwen3.8-Flash-Next	Default primary model identifier
Qwen/Qwen3.8-27B	Qwen/Qwen3.8-27B	27B model identifier
Qwen/Qwen3.8-27B-Uncensored	Qwen/Qwen3.8-27B-Uncensored	Uncensored model identifier
qwen3.8-flash-next	Qwen/Qwen3.8-Flash-Next	Flash-Next lowercase alias
qwen-flash-next	Qwen/Qwen3.8-Flash-Next	Shorthand alias
qwen-flash	Qwen/Qwen3.8-Flash-Next	Quick convenience alias
qwen3.8-27b	Qwen/Qwen3.8-27B	Standard 27B lowercase alias
qwen3.8-27b-uncensored	Qwen/Qwen3.8-27B-Uncensored	Uncensored lowercase alias
qwen	Default model	Generic shorthand alias

Any unlisted custom model name requested by the client is passed through directly.

Usage

Run the gateway with default auto-failover endpoints:

qflash

By default, this listens on http://127.0.0.1:8080 targeting Halvo78 by default with automatic failover to wanyamaelis and apathy-exe persistent endpoints.

Available flags:

• -port — tcp port to listen on (default 8080)
• -endpoints / -endpoint — comma-separated upstream space or OpenAI URLs (default list of 4 endpoints, QFLASH_ENDPOINTS / QFLASH_ENDPOINT env)
• -failover / -auto-failover — enable automatic failover across endpoints on quota exhaustion or error (default true, QFLASH_FAILOVER env)
• -mode — upstream protocol mode: auto, respond, chat_response, openai (default auto, QFLASH_MODE env)
• -model — exposed model name (default Qwen/Qwen3.8-Flash-Next, QFLASH_MODEL env)
• -thinking / -enable-thinking — enable chain-of-thought reasoning by default (default true)
• -hf-token / -token — Hugging Face API token for ZeroGPU quota or private spaces (HF_TOKEN env)
• -api-key — upstream inference engine API key for BYOK mode (OPENAI_API_KEY / QWEN_API_KEY env)
• -base-url — upstream inference engine base URL for BYOK mode (OPENAI_BASE_URL / QWEN_BASE_URL env)
• -socks / -proxy / -socks5 — SOCKS5 proxy URL, e.g. socks5://127.0.0.1:1080 (ALL_PROXY env)
• -user-agent / -ua — custom User-Agent sent to upstream

Endpoints served:

• GET /models and GET /v1/models
• POST /chat/completions and POST /v1/chat/completions

Example request with curl:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen","messages":[{"role":"user","content":"Explain quantum superposition in one sentence."}],"stream":false}'

Streaming example:

curl -N http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen","messages":[{"role":"user","content":"Count from 1 to 5."}],"stream":true}'

Instruct mode example (disables thinking):

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen","messages":[{"role":"user","content":"Hello!"}],"reasoning_effort":"none"}'

Using custom upstream spaces or a single endpoint:

# Point to a single CPU llama-server without failover
qflash -endpoint https://wanyamaelis-qwen3-8-27b.hf.space -failover=false

# Point to Halvo78 playground with your own API credentials
qflash -endpoint https://halvo78-qwen3-8-flash-next-playground.hf.space -api-key "sk-..." -base-url "https://api.openai.com/v1"

Python OpenAI SDK integration:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="sk-dummy"
)

stream = client.chat.completions.create(
    model="qwen",
    messages=[
        {"role": "user", "content": "Prove that the sum of the first n odd numbers is n^2."}
    ],
    stream=True
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if hasattr(delta, "reasoning_content") and delta.reasoning_content:
        print(f"[THINK] {delta.reasoning_content}", end="", flush=True)
    if delta.content:
        print(delta.content, end="", flush=True)
print()

Credits

Created by Luxferre in 2026, released into the public domain with no warranties.