139 lines
5.1 KiB
Markdown
139 lines
5.1 KiB
Markdown
# Qflash
|
|
|
|
## About
|
|
|
|
Qflash is a standalone, single-binary gateway that exposes the Qwen3.8-Flash-Next Gradio space (`https://halvo78-qwen3-8-flash-next-playground.hf.space`) through an OpenAI-compatible API. It translates the standard `/v1/chat/completions` and `/v1/models` endpoints into Gradio's `/gradio_api/call/chat_response` request and Server-Sent Events (SSE) stream protocol, allowing any standard OpenAI-compatible client, agent, or IDE to interface with Qwen3.8-Flash-Next without modification.
|
|
|
|
## Features
|
|
|
|
- OpenAI-compatible chat completions (streaming and non-streaming)
|
|
- Deep reasoning extraction with thinking trace passthrough (`<think>` tags and blockquotes mapped to `reasoning_content`)
|
|
- Stateful streaming tool call interception (`StreamToolCallFilter`) with zero XML/JSON leakage into `delta.content`
|
|
- Real-time token streaming with incremental SSE delivery
|
|
- Support for instruct mode via standard `reasoning_effort: "none"`
|
|
- Zero-dependency SOCKS5 proxy client (RFC 1928 / RFC 1929) with domain resolution (`socks5h://`), IPv4, IPv6, and auth
|
|
- Bring Your Own Key (BYOK) pass-through support via `Authorization: Bearer` or CLI flags
|
|
- Fibonacci backoff retry on transient upstream errors
|
|
- Zero external dependencies (Go standard library only)
|
|
|
|
## Installation
|
|
|
|
You need Go 1.22 or newer (tested on Go 1.26). Install the latest release straight from the Git repository with `go install`:
|
|
|
|
```bash
|
|
go install code.luxferre.top/luxferre/qflash@latest
|
|
```
|
|
|
|
This fetches the module from `https://code.luxferre.top/luxferre/qflash.git` and places the `qflash` binary in `$(go env GOPATH)/bin`. Make sure that directory is on your `PATH`.
|
|
|
|
*(Note: If installing right after a new commit has been pushed, bypass any proxy cache with `GOPROXY=direct go install code.luxferre.top/luxferre/qflash@latest`).*
|
|
|
|
If you prefer to build from a local checkout instead:
|
|
|
|
```bash
|
|
git clone https://code.luxferre.top/luxferre/qflash.git
|
|
cd qflash
|
|
go install .
|
|
```
|
|
|
|
Alternatively, build directly from source using `make`:
|
|
|
|
```bash
|
|
make qflash
|
|
```
|
|
|
|
This produces the `bin/qflash` binary for your platform.
|
|
|
|
## Models served
|
|
|
|
The gateway advertises the following models under `/v1/models`:
|
|
|
|
| Model ID | Target model | Description |
|
|
|---|---|---|
|
|
| `Qwen/Qwen3.8-Flash-Next` | `Qwen/Qwen3.8-Flash-Next` | Primary playground model (125B MoE, 6B activated) |
|
|
| `qwen3.8-flash-next` | `Qwen/Qwen3.8-Flash-Next` | Standard lowercase alias |
|
|
| `qwen-flash-next` | `Qwen/Qwen3.8-Flash-Next` | Shorthand alias |
|
|
| `qwen-flash` | `Qwen/Qwen3.8-Flash-Next` | Quick convenience alias |
|
|
|
|
Any unlisted custom model name requested by the client is passed through directly.
|
|
|
|
## Usage
|
|
|
|
Run the gateway:
|
|
|
|
```bash
|
|
qflash [-port 8080] [-endpoint https://halvo78-qwen3-8-flash-next-playground.hf.space] [-model Qwen/Qwen3.8-Flash-Next]
|
|
```
|
|
|
|
Available flags:
|
|
|
|
- `-port` — TCP port to listen on (default `8080`)
|
|
- `-endpoint` — root URL of the Gradio space (default `https://halvo78-qwen3-8-flash-next-playground.hf.space`)
|
|
- `-model` — exposed model name (default `Qwen/Qwen3.8-Flash-Next`)
|
|
- `-thinking` / `-enable-thinking` — enable chain-of-thought reasoning by default (default `true`)
|
|
- `-hf-token` — optional Hugging Face API token for authenticated spaces (`HF_TOKEN` env)
|
|
- `-api-key` — upstream inference engine API key for BYOK mode (`OPENAI_API_KEY` / `QWEN_API_KEY` env)
|
|
- `-base-url` — upstream inference engine base URL for BYOK mode (`OPENAI_BASE_URL` / `QWEN_BASE_URL` env)
|
|
- `-socks` / `-proxy` / `-socks5` — SOCKS5 proxy URL, e.g. `socks5://127.0.0.1:1080` (`ALL_PROXY` env)
|
|
- `-user-agent` / `-ua` — custom User-Agent sent to upstream
|
|
|
|
Endpoints served:
|
|
|
|
- `GET /models` and `GET /v1/models`
|
|
- `POST /chat/completions` and `POST /v1/chat/completions`
|
|
|
|
Example request with curl:
|
|
|
|
```bash
|
|
curl http://localhost:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"qwen3.8-flash-next","messages":[{"role":"user","content":"Explain QSA micro-blocks in one sentence."}],"stream":false}'
|
|
```
|
|
|
|
Streaming example:
|
|
|
|
```bash
|
|
curl -N http://localhost:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"qwen-flash","messages":[{"role":"user","content":"Count from 1 to 5."}],"stream":true}'
|
|
```
|
|
|
|
Instruct mode example (disables thinking):
|
|
|
|
```bash
|
|
curl http://localhost:8080/v1/chat/completions \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"model":"qwen-flash","messages":[{"role":"user","content":"Hello!"}],"reasoning_effort":"none"}'
|
|
```
|
|
|
|
Python OpenAI SDK integration:
|
|
|
|
```python
|
|
from openai import OpenAI
|
|
|
|
client = OpenAI(
|
|
base_url="http://localhost:8080/v1",
|
|
api_key="sk-dummy"
|
|
)
|
|
|
|
stream = client.chat.completions.create(
|
|
model="qwen3.8-flash-next",
|
|
messages=[
|
|
{"role": "user", "content": "Prove that the sum of the first n odd numbers is n^2."}
|
|
],
|
|
stream=True
|
|
)
|
|
|
|
for chunk in stream:
|
|
delta = chunk.choices[0].delta
|
|
if hasattr(delta, "reasoning_content") and delta.reasoning_content:
|
|
print(f"[THINK] {delta.reasoning_content}", end="", flush=True)
|
|
if delta.content:
|
|
print(delta.content, end="", flush=True)
|
|
print()
|
|
```
|
|
|
|
## Credits
|
|
|
|
Created by Luxferre in 2026, released into the public domain with no warranties.
|