qflash/README.md

Qflash: OpenAI-compatible gateway for Qwen3.8 models

About

Qflash is a standalone, single-binary gateway that exposes Qwen3.8 model spaces on Hugging Face through a standard OpenAI-compatible API. It automatically detects and supports multiple upstream protocols:

• gradio /chat_response endpoints (such as https://halvo78-qwen3-8-flash-next-playground.hf.space)
• direct OpenAI /v1/chat/completions endpoints (such as https://wanyamaelis-qwen3-8-27b.hf.space, https://apathy-exe-qwen3-8-27b.hf.space, and https://mikelalda-qwen3-8-euskera-gguf-q4.hf.space, running persistent llama-server instances on CPU with zero quota limits)
• gradio /respond endpoints (supported for spaces such as https://jickman125-qwen3-8-27b-uncensored-demo.hf.space and https://microhero-qwen3-8-27b-uncensored-chat.hf.space)

It translates standard /v1/chat/completions and /v1/models requests and Server-Sent Events (SSE) stream protocols into upstream formats, allowing any standard OpenAI-compatible client, agent, or IDE to interface with Qwen3.8 models without modification.

Features

• openai-compatible chat completions (streaming and non-streaming)
• automatic upstream failover across persistent non-ZeroGPU endpoints
• smart endpoint cooldown (5 minutes on quota exhaustion, 30 seconds on network errors)
• automatic upstream endpoint detection (respond, openai, chat_response)
• deep reasoning extraction with thinking trace passthrough (<think> tags and blockquotes mapped to reasoning_content)
• stateful streaming tool call interception (StreamToolCallFilter) with zero XML or JSON leakage into delta.content
• real-time token streaming with incremental SSE delivery
• support for instruct mode via standard reasoning_effort: "none"
• zero-dependency SOCKS5 proxy client (RFC 1928 / RFC 1929) with domain resolution (socks5h://), IPv4, IPv6, and auth
• bring your own key (BYOK) pass-through support via Authorization: Bearer or CLI flags
• zero-gpu quota authentication via free Hugging Face personal tokens (HF_TOKEN)
• fibonacci backoff retry on transient upstream errors
• zero external dependencies (Go standard library only)

Supported upstream spaces

Space	Model	Hardware	Protocol	Notes
Wanyamaelis/Qwen3.8-27B (primary)	Qwen3.8-27B (MTP)	CPU basic (8 vCPU)	Native OpenAI /v1	Default endpoint; non-ZeroGPU, no quota limits, fast speculative decoding
apathy-exe/Qwen3.8-27B (fallback)	Qwen3.8-27B	CPU (AVX-512)	Native OpenAI /v1	Non-ZeroGPU, no quota limits, persistent llama-server
apathy-exe/Qwen3.8-Flash-Next (fallback)	Qwen3.8-Flash-Next (~177B)	CPU (AVX-512)	Native OpenAI /v1	Non-ZeroGPU, no quota limits, full 177B Flash-Next model
mikelalda/Qwen3.8-euskera-GGUF-Q4 (fallback)	Qwen3.8-27B	CPU basic	Native OpenAI /v1	Non-ZeroGPU, no quota limits, persistent llama-server (Q4_K_M)
jickman125/Qwen3.8-27B-Uncensored-Demo	Qwen3.8-27B Uncensored	ZeroGPU (A10G)	Gradio /respond	Fast live GPU inference, can be configured via custom -endpoints
MicroHERO/qwen3.8-27b-uncensored-chat	Qwen3.8-27B Uncensored	ZeroGPU (A10G)	Gradio /respond	Fast live GPU inference, can be configured via custom -endpoints
Halvo78/qwen3-8-flash-next-playground	Qwen3.8-Flash-Next	CPU basic	Gradio /chat_response	Sandbox playground, supports BYOK upstream inference credentials

Auto-failover and quota handling

By default, Qflash attempts the primary wanyamaelis endpoint with CPU MTP speculative decoding. If any endpoint is unavailable, offline, or exhausts resources, it automatically fails over across the persistent CPU endpoints (apathy-exe and mikelalda), ensuring uninterrupted service:

• with auto-failover enabled (default), when an endpoint encounters an error, timeout, or unavailable backend, it is placed on cooldown and the gateway automatically fails over to the next configured endpoint
• transient network errors trigger a 30-second cooldown, while quota limits or unavailable sandbox spaces trigger a 5-minute cooldown
• optional ZeroGPU or private spaces can be authenticated using a Hugging Face personal access token (https://huggingface.co/settings/tokens) via the HF_TOKEN environment variable, the -hf-token CLI flag, or the Authorization: Bearer hf_... header

Installation

You need Go 1.22 or newer (tested on Go 1.26). Install the latest release straight from the Git repository with go install:

go install code.luxferre.top/luxferre/qflash@latest

This fetches the module from https://code.luxferre.top/luxferre/qflash.git and places the qflash binary in $(go env GOPATH)/bin. Make sure that directory is on your PATH.

(Note: If installing right after a new commit has been pushed, bypass any proxy cache with GOPROXY=direct go install code.luxferre.top/luxferre/qflash@latest).

If you prefer to build from a local checkout instead:

git clone https://code.luxferre.top/luxferre/qflash.git
cd qflash
go install .

Alternatively, build directly from source using make:

make qflash

This produces the bin/qflash binary for your platform.

Models served

The gateway advertises the following models under /v1/models:

Model ID	Target model	Description
Qwen/Qwen3.8-27B	Qwen/Qwen3.8-27B	Default primary model identifier
Qwen/Qwen3.8-Flash-Next	Qwen/Qwen3.8-Flash-Next	Flash-Next model identifier
Qwen/Qwen3.8-27B-Uncensored	Qwen/Qwen3.8-27B-Uncensored	Uncensored model identifier
qwen3.8-27b	Qwen/Qwen3.8-27B	Standard 27B lowercase alias
qwen3.8-flash-next	Qwen/Qwen3.8-Flash-Next	Flash-Next lowercase alias
qwen-flash-next	Qwen/Qwen3.8-Flash-Next	Shorthand alias
qwen-flash	Qwen/Qwen3.8-Flash-Next	Quick convenience alias
qwen3.8-27b-uncensored	Qwen/Qwen3.8-27B-Uncensored	Uncensored lowercase alias
qwen	Default model	Generic shorthand alias

Any unlisted custom model name requested by the client is passed through directly.

Usage

Run the gateway with default auto-failover endpoints:

qflash

By default, this listens on http://127.0.0.1:8080 targeting wanyamaelis by default with automatic failover to apathy-exe and mikelalda persistent endpoints.

Available flags:

• -port — tcp port to listen on (default 8080)
• -endpoints / -endpoint — comma-separated upstream space or OpenAI URLs (default list of 4 persistent endpoints, QFLASH_ENDPOINTS / QFLASH_ENDPOINT env)
• -failover / -auto-failover — enable automatic failover across endpoints on quota exhaustion or error (default true, QFLASH_FAILOVER env)
• -mode — upstream protocol mode: auto, respond, chat_response, openai (default auto, QFLASH_MODE env)
• -model — exposed model name (default Qwen/Qwen3.8-27B, QFLASH_MODEL env)
• -thinking / -enable-thinking — enable chain-of-thought reasoning by default (default true)
• -hf-token / -token — Hugging Face API token for ZeroGPU quota or private spaces (HF_TOKEN env)
• -api-key — upstream inference engine API key for BYOK mode (OPENAI_API_KEY / QWEN_API_KEY env)
• -base-url — upstream inference engine base URL for BYOK mode (OPENAI_BASE_URL / QWEN_BASE_URL env)
• -socks / -proxy / -socks5 — SOCKS5 proxy URL, e.g. socks5://127.0.0.1:1080 (ALL_PROXY env)
• -user-agent / -ua — custom User-Agent sent to upstream

Endpoints served:

• GET /models and GET /v1/models
• POST /chat/completions and POST /v1/chat/completions

Example request with curl:

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen","messages":[{"role":"user","content":"Explain quantum superposition in one sentence."}],"stream":false}'

Streaming example:

curl -N http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen","messages":[{"role":"user","content":"Count from 1 to 5."}],"stream":true}'

Instruct mode example (disables thinking):

curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"qwen","messages":[{"role":"user","content":"Hello!"}],"reasoning_effort":"none"}'

Using custom upstream spaces or a single endpoint:

# Point to a single CPU llama-server without failover
qflash -endpoint https://wanyamaelis-qwen3-8-27b.hf.space -failover=false

# Point to Halvo78 playground with your own API credentials
qflash -endpoint https://halvo78-qwen3-8-flash-next-playground.hf.space -api-key "sk-..." -base-url "https://api.openai.com/v1"

Python OpenAI SDK integration:

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8080/v1",
    api_key="sk-dummy"
)

stream = client.chat.completions.create(
    model="qwen",
    messages=[
        {"role": "user", "content": "Prove that the sum of the first n odd numbers is n^2."}
    ],
    stream=True
)

for chunk in stream:
    delta = chunk.choices[0].delta
    if hasattr(delta, "reasoning_content") and delta.reasoning_content:
        print(f"[THINK] {delta.reasoning_content}", end="", flush=True)
    if delta.content:
        print(delta.content, end="", flush=True)
print()

Credits

Created by Luxferre in 2026, released into the public domain with no warranties.