luxferre/qflash
1
0
Fork 0
Code Actions Activity

docs: use sentence case for subheadings and list items

Browse Source
This commit is contained in:
Luxferre
2026-09-05 15:47:25 +03:00
parent 58eb5fd463
commit 28af9d49a3
1 changed files with 61 additions and 94 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+61 -94
View File
@@ -1,66 +1,54 @@
# qflash: OpenAI Proxy Gateway for Qwen3.8-Flash-Next
# Qflash
Standalone, performant, zero-dependency Go OpenAI proxy gateway for the **Qwen3.8-Flash-Next** Hugging Face Gradio space (`https://halvo78-qwen3-8-flash-next-playground.hf.space`).
## About
Created by Luxferre in 2026, released into the public domain.
---
Qflash is a standalone, single-binary gateway that exposes the Qwen3.8-Flash-Next Gradio space (`https://halvo78-qwen3-8-flash-next-playground.hf.space`) through an OpenAI-compatible API. It translates the standard `/v1/chat/completions` and `/v1/models` endpoints into Gradio's `/gradio_api/call/chat_response` request and Server-Sent Events (SSE) stream protocol, allowing any standard OpenAI-compatible client, agent, or IDE to interface with Qwen3.8-Flash-Next without modification.
## Features
- **Zero External Dependencies**: Built entirely with Go standard library packages (`net/http`, `encoding/json`, `bufio`, etc.).
- **OpenAI-Compatible API**: Implements standard `/v1/chat/completions` (streaming & non-streaming) and `/v1/models`.
- **Real-Time Token Streaming**: Streams SSE chunks with incremental token delivery directly to clients.
- **Deep Reasoning Separation**:
- Automatically isolates thinking traces from both standard `<think>...</think>` tags and the playground's blockquote thinking blocks (`> 💭 **Thinking Process...**`).
- Emits pure thought traces to `delta.reasoning_content` (streaming) and `message.reasoning_content` (non-streaming).
- Keeps `delta.content` and `message.content` clean.
- **Stateful Streaming Tool Call Interception**:
- Injects tool schemas into system instructions.
- Intercepts `<tool_call>` blocks in real time via `StreamToolCallFilter` without leaking raw XML or JSON into `delta.content`.
- Emits structured `delta.tool_calls` chunks and sets `finish_reason: "tool_calls"`.
- **Reasoning Effort Control**: Respects standard `reasoning_effort: "none"` to switch dynamically into high-speed Instruct Mode.
- **Zero-Dependency SOCKS5 Proxy Client**:
- RFC 1928 and RFC 1929 compliant client with domain resolution (`socks5h://`), IPv4, IPv6, and authentication.
- Wireable via `-socks` CLI flag or `ALL_PROXY` / `SOCKS5_PROXY` environment variables.
- **Bring Your Own Key (BYOK) Pass-through**:
- Passes client API keys or custom base URLs directly to upstream inference engines when provided.
---
- OpenAI-compatible chat completions (streaming and non-streaming)
- Deep reasoning extraction with thinking trace passthrough (`<think>` tags and blockquotes mapped to `reasoning_content`)
- Stateful streaming tool call interception (`StreamToolCallFilter`) with zero XML/JSON leakage into `delta.content`
- Real-time token streaming with incremental SSE delivery
- Support for instruct mode via standard `reasoning_effort: "none"`
- Zero-dependency SOCKS5 proxy client (RFC 1928 / RFC 1929) with domain resolution (`socks5h://`), IPv4, IPv6, and auth
- Bring Your Own Key (BYOK) pass-through support via `Authorization: Bearer` or CLI flags
- Fibonacci backoff retry on transient upstream errors
- Zero external dependencies (Go standard library only)
## Installation
### Via `go install`
Install directly to `$(go env GOPATH)/bin`:
You need Go 1.22 or newer (tested on Go 1.26). Install the latest release straight from the Git repository with `go install`:
```bash
go install code.luxferre.top/luxferre/qflash@latest
```
*Note: If installing right after pushing to Gitea, bypass any public module proxy cache by using `GOPROXY=direct`:*
This fetches the module from `https://code.luxferre.top/luxferre/qflash.git` and places the `qflash` binary in `$(go env GOPATH)/bin`. Make sure that directory is on your `PATH`.
```bash
GOPROXY=direct go install code.luxferre.top/luxferre/qflash@latest
```
*(Note: If installing right after a new commit has been pushed, bypass any proxy cache with `GOPROXY=direct go install code.luxferre.top/luxferre/qflash@latest`).*
### From Source
If you prefer to build from a local checkout instead:
```bash
git clone https://code.luxferre.top/luxferre/qflash.git
cd qflash
go install .
```
Alternatively, build directly from source using `make`:
```bash
make qflash
```
The resulting binary will be placed at `bin/qflash`.
This produces the `bin/qflash` binary for your platform.
---
## Models served
## Architecture & Model Aliases
The gateway advertises the following models under `/v1/models`:
The gateway serves the following models under `/v1/models`:
| Model ID | Target Model | Description |
| Model ID | Target model | Description |
|---|---|---|
| `Qwen/Qwen3.8-Flash-Next` | `Qwen/Qwen3.8-Flash-Next` | Primary playground model (125B MoE, 6B activated) |
| `qwen3.8-flash-next` | `Qwen/Qwen3.8-Flash-Next` | Standard lowercase alias |
@@ -69,87 +57,62 @@ The gateway serves the following models under `/v1/models`:
Any unlisted custom model name requested by the client is passed through directly.
---
## Usage
## Configuration Flags & Environment Variables
| Flag | Shorthand | Environment Variable | Default | Description |
|---|---|---|---|---|
| `-port` | | `PORT` | `8080` | Port to bind the HTTP server |
| `-endpoint` | | | `https://halvo78-qwen3-8-flash-next-playground.hf.space` | Upstream Gradio space base URL |
| `-model` | | | `Qwen/Qwen3.8-Flash-Next` | Default model ID |
| `-thinking` | `-enable-thinking` | | `true` | Enable chain-of-thought reasoning by default |
| `-hf-token` | | `HF_TOKEN` | `""` | Hugging Face user access token |
| `-api-key` | | `OPENAI_API_KEY` / `QWEN_API_KEY` | `""` | Upstream inference engine API key |
| `-base-url` | | `OPENAI_BASE_URL` / `QWEN_BASE_URL` | `""` | Upstream inference engine base URL |
| `-user-agent` | `-ua` | `USER_AGENT` | Firefox 153 on Linux | Custom User-Agent header |
| `-socks` | `-proxy`, `-socks5` | `ALL_PROXY`, `SOCKS5_PROXY` | `""` | SOCKS5 proxy URL (`socks5://127.0.0.1:1080`) |
---
## Usage Examples
### 1. Launch Gateway
Run the gateway:
```bash
qflash -port 8080
qflash [-port 8080] [-endpoint https://halvo78-qwen3-8-flash-next-playground.hf.space] [-model Qwen/Qwen3.8-Flash-Next]
```
### 2. List Models
Available flags:
- `-port` — TCP port to listen on (default `8080`)
- `-endpoint` — root URL of the Gradio space (default `https://halvo78-qwen3-8-flash-next-playground.hf.space`)
- `-model` — exposed model name (default `Qwen/Qwen3.8-Flash-Next`)
- `-thinking` / `-enable-thinking` — enable chain-of-thought reasoning by default (default `true`)
- `-hf-token` — optional Hugging Face API token for authenticated spaces (`HF_TOKEN` env)
- `-api-key` — upstream inference engine API key for BYOK mode (`OPENAI_API_KEY` / `QWEN_API_KEY` env)
- `-base-url` — upstream inference engine base URL for BYOK mode (`OPENAI_BASE_URL` / `QWEN_BASE_URL` env)
- `-socks` / `-proxy` / `-socks5` — SOCKS5 proxy URL, e.g. `socks5://127.0.0.1:1080` (`ALL_PROXY` env)
- `-user-agent` / `-ua` — custom User-Agent sent to upstream
Endpoints served:
- `GET /models` and `GET /v1/models`
- `POST /chat/completions` and `POST /v1/chat/completions`
Example request with curl:
```bash
curl http://127.0.0.1:8080/v1/models
```
### 3. Non-Streaming Chat Completion
```bash
curl -X POST http://127.0.0.1:8080/v1/chat/completions \
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen3.8-flash-next",
"messages": [
{"role": "user", "content": "Explain QSA micro-blocks in one sentence."}
],
"stream": false
}'
-d '{"model":"qwen3.8-flash-next","messages":[{"role":"user","content":"Explain QSA micro-blocks in one sentence."}],"stream":false}'
```
### 4. Streaming Chat Completion (Real-Time SSE)
Streaming example:
```bash
curl -N -X POST http://127.0.0.1:8080/v1/chat/completions \
curl -N http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-flash",
"messages": [
{"role": "user", "content": "Write a quick Python countdown loop."}
],
"stream": true
}'
-d '{"model":"qwen-flash","messages":[{"role":"user","content":"Count from 1 to 5."}],"stream":true}'
```
### 5. Instruct Mode (Disable Thinking)
Instruct mode example (disables thinking):
```bash
curl -X POST http://127.0.0.1:8080/v1/chat/completions \
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen-flash",
"messages": [
{"role": "user", "content": "Hello!"}
],
"reasoning_effort": "none"
}'
-d '{"model":"qwen-flash","messages":[{"role":"user","content":"Hello!"}],"reasoning_effort":"none"}'
```
### 6. Python OpenAI SDK Integration
Python OpenAI SDK integration:
```python
from openai import OpenAI
client = OpenAI(
base_url="http://127.0.0.1:8080/v1",
base_url="http://localhost:8080/v1",
api_key="sk-dummy"
)
@@ -169,3 +132,7 @@ for chunk in stream:
print(delta.content, end="", flush=True)
print()
```
## Credits
Created by Luxferre in 2026, released into the public domain with no warranties.