feat: use Halvo78 playground as primary default endpoint with automatic failover

README.md

@@ -4,16 +4,16 @@
 
 Qflash is a standalone, single-binary gateway that exposes Qwen3.8 model spaces on Hugging Face through a standard OpenAI-compatible API. It automatically detects and supports multiple upstream protocols:
 
+- gradio `/chat_response` endpoints (such as `https://halvo78-qwen3-8-flash-next-playground.hf.space`)
 - direct OpenAI `/v1/chat/completions` endpoints (such as `https://wanyamaelis-qwen3-8-27b.hf.space` and `https://apathy-exe-qwen3-8-flash-next.hf.space`, running persistent `llama-server` instances on CPU with zero quota limits)
 - gradio `/respond` endpoints (supported for custom spaces such as `https://microhero-qwen3-8-27b-uncensored-chat.hf.space`)
-- gradio `/chat_response` endpoints (such as `https://halvo78-qwen3-8-flash-next-playground.hf.space`)
 
 It translates standard `/v1/chat/completions` and `/v1/models` requests and Server-Sent Events (SSE) stream protocols into upstream formats, allowing any standard OpenAI-compatible client, agent, or IDE to interface with Qwen3.8 models without modification.
 
 ## Features
 
 - openai-compatible chat completions (streaming and non-streaming)
-- automatic upstream failover across configured non-ZeroGPU endpoints
+- automatic upstream failover from primary playground to non-ZeroGPU endpoints
 - smart endpoint cooldown (5 minutes on quota exhaustion, 30 seconds on network errors)
 - automatic upstream endpoint detection (`respond`, `openai`, `chat_response`)
 - deep reasoning extraction with thinking trace passthrough (`<think>` tags and blockquotes mapped to `reasoning_content`)
@@ -30,20 +30,18 @@ It translates standard `/v1/chat/completions` and `/v1/models` requests and Serv
 
 | Space | Model | Hardware | Protocol | Notes |
 |---|---|---|---|---|
-| `Wanyamaelis/Qwen3.8-27B` *(primary)* | Qwen3.8-27B (MTP) | CPU basic (8 vCPU) | Native OpenAI `/v1` | Default endpoint, non-ZeroGPU, no quota limits, fast speculative decoding (~5s) |
+| `Halvo78/qwen3-8-flash-next-playground` *(primary)* | Qwen3.8-Flash-Next | CPU basic | Gradio `/chat_response` | Default endpoint; auto-fails over to live endpoints if unavailable |
+| `Wanyamaelis/Qwen3.8-27B` *(fallback)* | Qwen3.8-27B (MTP) | CPU basic (8 vCPU) | Native OpenAI `/v1` | Non-ZeroGPU, no quota limits, fast speculative decoding (~5s) |
 | `apathy-exe/Qwen3.8-27B` *(fallback)* | Qwen3.8-27B (MTP) | CPU (OpenMP/AVX512) | Native OpenAI `/v1` | Non-ZeroGPU, no quota limits, speculative decoding |
 | `apathy-exe/Qwen3.8-Flash-Next` *(fallback)* | Qwen3.8-Flash-Next (~177B) | CPU (OpenMP/AVX512) | Native OpenAI `/v1` | Non-ZeroGPU, no quota limits, full 177B Flash-Next model |
 | `MicroHERO/qwen3.8-27b-uncensored-chat` | Qwen3.8-27B Uncensored | ZeroGPU (A10G) | Gradio `/respond` | Fast live GPU inference, can be configured via custom `-endpoints` |
-| `Halvo78/qwen3-8-flash-next-playground` | Qwen3.8-Flash-Next | CPU basic | Gradio `/chat_response` | Sandbox client; requires BYOK API key/base URL |
 
 ### Auto-failover and quota handling
 
-Default endpoints run on persistent non-ZeroGPU hardware with no daily runs limit. When custom ZeroGPU spaces are configured, anonymous requests share a small pool per IP address. When runs limit is reached, upstream returns a quota error (`429` or `ZeroGPU runs limit`).
+By default, Qflash attempts the primary `Halvo78` playground endpoint. If the primary space is unavailable, offline, or lacks live inference credentials, it automatically fails over to the persistent CPU endpoints (`wanyamaelis` and `apathy-exe`), ensuring uninterrupted service:
 
-Qflash handles upstream reliability seamlessly:
+- with auto-failover enabled (default), when an endpoint encounters an error, timeout, or unavailable backend, it is placed on cooldown and the gateway automatically fails over to the next configured endpoint
-
+- transient network errors trigger a 30-second cooldown, while quota limits or unavailable sandbox spaces trigger a 5-minute cooldown
-- with auto-failover enabled (default), when an endpoint encounters an error or timeout, it is placed on cooldown and the gateway automatically fails over to the next configured endpoint
-- transient network errors trigger a 30-second cooldown, while quota limits trigger a 5-minute cooldown
 - optional ZeroGPU or private spaces can be authenticated using a Hugging Face personal access token (`https://huggingface.co/settings/tokens`) via the `HF_TOKEN` environment variable, the `-hf-token` CLI flag, or the `Authorization: Bearer hf_...` header
 
 ## Installation
@@ -80,14 +78,14 @@ The gateway advertises the following models under `/v1/models`:
 
 | Model ID | Target model | Description |
 |---|---|---|
-| `Qwen/Qwen3.8-27B` | `Qwen/Qwen3.8-27B` | Default primary live model |
+| `Qwen/Qwen3.8-Flash-Next` | `Qwen/Qwen3.8-Flash-Next` | Default primary model identifier |
+| `Qwen/Qwen3.8-27B` | `Qwen/Qwen3.8-27B` | 27B model identifier |
 | `Qwen/Qwen3.8-27B-Uncensored` | `Qwen/Qwen3.8-27B-Uncensored` | Uncensored model identifier |
-| `Qwen/Qwen3.8-Flash-Next` | `Qwen/Qwen3.8-Flash-Next` | Flash-Next model identifier |
-| `qwen3.8-27b` | `Qwen/Qwen3.8-27B` | Standard lowercase alias |
-| `qwen3.8-27b-uncensored` | `Qwen/Qwen3.8-27B-Uncensored` | Uncensored lowercase alias |
 | `qwen3.8-flash-next` | `Qwen/Qwen3.8-Flash-Next` | Flash-Next lowercase alias |
 | `qwen-flash-next` | `Qwen/Qwen3.8-Flash-Next` | Shorthand alias |
 | `qwen-flash` | `Qwen/Qwen3.8-Flash-Next` | Quick convenience alias |
+| `qwen3.8-27b` | `Qwen/Qwen3.8-27B` | Standard 27B lowercase alias |
+| `qwen3.8-27b-uncensored` | `Qwen/Qwen3.8-27B-Uncensored` | Uncensored lowercase alias |
 | `qwen` | Default model | Generic shorthand alias |
 
 Any unlisted custom model name requested by the client is passed through directly.
@@ -100,15 +98,15 @@ Run the gateway with default auto-failover endpoints:
 qflash
 ```
 
-By default, this listens on `http://127.0.0.1:8080` with failover configured across persistent non-ZeroGPU endpoints (`wanyamaelis` and `apathy-exe`).
+By default, this listens on `http://127.0.0.1:8080` targeting `Halvo78` by default with automatic failover to `wanyamaelis` and `apathy-exe` persistent endpoints.
 
 Available flags:
 
 - `-port` — tcp port to listen on (default `8080`)
-- `-endpoints` / `-endpoint` — comma-separated upstream space or OpenAI URLs (default list of 3 non-ZeroGPU endpoints, `QFLASH_ENDPOINTS` / `QFLASH_ENDPOINT` env)
+- `-endpoints` / `-endpoint` — comma-separated upstream space or OpenAI URLs (default list of 4 endpoints, `QFLASH_ENDPOINTS` / `QFLASH_ENDPOINT` env)
 - `-failover` / `-auto-failover` — enable automatic failover across endpoints on quota exhaustion or error (default `true`, `QFLASH_FAILOVER` env)
 - `-mode` — upstream protocol mode: `auto`, `respond`, `chat_response`, `openai` (default `auto`, `QFLASH_MODE` env)
-- `-model` — exposed model name (default `Qwen/Qwen3.8-27B`, `QFLASH_MODEL` env)
+- `-model` — exposed model name (default `Qwen/Qwen3.8-Flash-Next`, `QFLASH_MODEL` env)
 - `-thinking` / `-enable-thinking` — enable chain-of-thought reasoning by default (default `true`)
 - `-hf-token` / `-token` — Hugging Face API token for ZeroGPU quota or private spaces (`HF_TOKEN` env)
 - `-api-key` — upstream inference engine API key for BYOK mode (`OPENAI_API_KEY` / `QWEN_API_KEY` env)

main.go

@@ -32,6 +32,7 @@ var (
   ConfiguredModel       string
   EnableThinkingDefault = true
   DefaultEndpoints      = []string{
+    "https://halvo78-qwen3-8-flash-next-playground.hf.space",
     "https://wanyamaelis-qwen3-8-27b.hf.space",
     "https://apathy-exe-qwen3-8-27b.hf.space",
     "https://apathy-exe-qwen3-8-flash-next.hf.space",
@@ -516,7 +517,7 @@ func EffectiveModelID(reqModel string, defaultModel string) string {
     if defaultModel != "" {
       return defaultModel
     }
-    return "Qwen/Qwen3.8-27B"
+    return "Qwen/Qwen3.8-Flash-Next"
   default:
     return clean
   }
@@ -1251,7 +1252,7 @@ func parseEndpointList(rawList []string) []*EndpointNode {
 
 func NewQwenService(endpoints []string, modelName, mode, token, apiKey, baseURL, socksProxy string, enableThinking, autoFailover bool) *QwenService {
   if modelName == "" {
-    modelName = "Qwen/Qwen3.8-27B"
+    modelName = "Qwen/Qwen3.8-Flash-Next"
   }
   if mode == "" {
     mode = "auto"
@@ -1288,19 +1289,19 @@ func (s *QwenService) ListModels() []ModelItem {
   now := time.Now().Unix()
   primaryID := s.modelName
   if primaryID == "" {
-    primaryID = "Qwen/Qwen3.8-27B"
+    primaryID = "Qwen/Qwen3.8-Flash-Next"
   }
 
   candidates := []string{
     primaryID,
+    "Qwen/Qwen3.8-Flash-Next",
     "Qwen/Qwen3.8-27B",
     "Qwen/Qwen3.8-27B-Uncensored",
-    "Qwen/Qwen3.8-Flash-Next",
-    "qwen3.8-27b",
-    "qwen3.8-27b-uncensored",
     "qwen3.8-flash-next",
     "qwen-flash-next",
     "qwen-flash",
+    "qwen3.8-27b",
+    "qwen3.8-27b-uncensored",
     "qwen",
   }
 
@@ -1400,6 +1401,9 @@ func (s *QwenService) markEndpointFailure(node *EndpointNode, err error) {
   if strings.Contains(errStr, "zerogpu") || strings.Contains(errStr, "quota") || strings.Contains(errStr, "429") {
     node.CooldownUntil = time.Now().Add(5 * time.Minute)
     log.Printf("Endpoint %s hit quota/rate limit, cooling down until %s", node.URL, node.CooldownUntil.Format("15:04:05"))
+  } else if strings.Contains(errStr, "sandbox") {
+    node.CooldownUntil = time.Now().Add(5 * time.Minute)
+    log.Printf("Endpoint %s is in sandbox mode (live inference unavailable), cooling down until %s", node.URL, node.CooldownUntil.Format("15:04:05"))
   } else {
     // For other transient errors, cool down for 30 seconds
     node.CooldownUntil = time.Now().Add(30 * time.Second)
@@ -2089,6 +2093,10 @@ func (s *QwenService) chatGradioChatResponse(endpointURL string, w http.Response
       }
     }
 
+    if s.autoFailover && (strings.Contains(finalRawText, "QSA Micro-block Reasoning") || strings.Contains(finalRawText, "Sandbox Response") || strings.Contains(finalRawText, "To connect to a live inference engine")) {
+      return fmt.Errorf("upstream space %s is in sandbox simulation mode (live inference not configured)", endpointURL)
+    }
+
     cleanedReasoning, cleanedContent := SeparateReasoningAndContent(finalRawText)
     toolCalls, remContent, hasToolCalls := DetectToolCalls(cleanedContent)
 
@@ -2114,26 +2122,26 @@ func (s *QwenService) chatGradioChatResponse(endpointURL string, w http.Response
   }
 
   // Streaming completion
-  flusher, _ := w.(http.Flusher)
+  var streamer *Streamer
-  streamer := NewStreamer(w, flusher, completionID, createdTime, resolvedModel)
-  streamer.Role()
-
-  reader := bufio.NewReader(streamResp.Body)
-  var currentEvent string
   var emittedReasoning string
   var emittedContent string
 
   toolFilter := &StreamToolCallFilter{}
 
   onContentChunk := func(text string) {
-    if text != "" {
+    if text != "" && streamer != nil {
       streamer.Content(text)
     }
   }
 
   onToolCallChunk := func(tc ToolCall) {
+    if streamer != nil {
       streamer.ToolCallDelta(tc)
     }
+  }
+
+  reader := bufio.NewReader(streamResp.Body)
+  var currentEvent string
 
   for {
     line, err := reader.ReadString('\n')
@@ -2153,10 +2161,23 @@ func (s *QwenService) chatGradioChatResponse(endpointURL string, w http.Response
     if strings.HasPrefix(line, "data: ") {
       dataJSON := strings.TrimPrefix(line, "data: ")
       if currentEvent == "error" {
+        if streamer != nil {
           break
         }
+        return fmt.Errorf("gradio upstream error: %s", dataJSON)
+      }
 
       if fullAssistantText, ok := parseAssistantText(dataJSON); ok {
+        if s.autoFailover && (strings.Contains(fullAssistantText, "QSA Micro-block Reasoning") || strings.Contains(fullAssistantText, "Sandbox Response") || strings.Contains(fullAssistantText, "To connect to a live inference engine")) {
+          return fmt.Errorf("upstream space %s is in sandbox simulation mode (live inference not configured)", endpointURL)
+        }
+
+        if streamer == nil {
+          flusher, _ := w.(http.Flusher)
+          streamer = NewStreamer(w, flusher, completionID, createdTime, resolvedModel)
+          streamer.Role()
+        }
+
         currentReasoning, currentContent := SeparateReasoningAndContent(fullAssistantText)
 
         // Stream reasoning tokens incrementally
@@ -2176,6 +2197,10 @@ func (s *QwenService) chatGradioChatResponse(endpointURL string, w http.Response
     }
   }
 
+  if streamer == nil {
+    return fmt.Errorf("upstream space %s produced no valid output", endpointURL)
+  }
+
   // Flush remaining buffer in tool filter
   toolFilter.Flush(onContentChunk, onToolCallChunk)
 
@@ -2207,7 +2232,7 @@ func main() {
   autoFailover := flag.Bool("failover", true, "Enable automatic failover across endpoints on error or quota limit")
   flag.BoolVar(autoFailover, "auto-failover", true, "Alias for -failover")
 
-  defaultModelVal := "Qwen/Qwen3.8-27B"
+  defaultModelVal := "Qwen/Qwen3.8-Flash-Next"
   if envModel := os.Getenv("QFLASH_MODEL"); envModel != "" {
     defaultModelVal = envModel
   }

qflash_test.go

@@ -10,6 +10,7 @@ import (
   "net/http/httptest"
   "strings"
   "testing"
+  "time"
 )
 
 func TestChatMessageGetContentString(t *testing.T) {
@@ -652,5 +653,95 @@ func TestResolveMaxTokens(t *testing.T) {
   }
 }
 
+func TestQwenServiceSandboxFailover(t *testing.T) {
+  server1Hits := 0
+  server2Hits := 0
 
+  // Server 1: Returns Gradio chat_response sandbox simulation
+  server1 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+    server1Hits++
+    if r.URL.Path == "/gradio_api/call/chat_response" {
+      w.Header().Set("Content-Type", "application/json")
+      w.Write([]byte(`{"event_id": "sandbox_event"}`))
+      return
+    }
+    if r.URL.Path == "/gradio_api/call/chat_response/sandbox_event" {
+      w.Header().Set("Content-Type", "text/event-stream")
+      flusher, _ := w.(http.Flusher)
+      chunk := "event: complete\n" +
+        "data: [[{\"role\": \"user\", \"content\": [{\"text\": \"hello\", \"type\": \"text\"}]}, {\"role\": \"assistant\", \"content\": [{\"text\": \"> 💭 **Thinking Process (QSA Micro-block Reasoning):**\\n\\n### Qwen3.8-Flash-Next Sandbox Response\\n\\nTo connect to a live inference engine, enter your endpoint credentials.\", \"type\": \"text\"}]}]]\n\n"
+      w.Write([]byte(chunk))
+      flusher.Flush()
+      return
+    }
+    http.NotFound(w, r)
+  }))
+  defer server1.Close()
+
+  // Server 2: Standard OpenAI server returning live completion
+  server2 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+    server2Hits++
+    w.Header().Set("Content-Type", "application/json")
+    w.Write([]byte(`{"choices":[{"message":{"role":"assistant","content":"Live completion from fallback server"},"finish_reason":"stop"}]}`))
+  }))
+  defer server2.Close()
+
+  svc := NewQwenService([]string{server1.URL, server2.URL}, "Qwen/Qwen3.8-Flash-Next", "auto", "", "", "", "", true, true)
+  // Explicitly assign modes for mock URLs
+  svc.endpoints[0].Mode = "chat_response"
+  svc.endpoints[1].Mode = "openai"
+
+  // 1. Non-streaming failover test
+  rec := httptest.NewRecorder()
+  req := ChatCompletionRequest{
+    Model: "qwen",
+    Messages: []ChatMessage{{Role: "user", Content: "hello"}},
+    Stream: false,
+  }
+
+  err := svc.Chat(rec, nil, req)
+  if err != nil {
+    t.Fatalf("expected failover to succeed, got error: %v", err)
+  }
+
+  var res ChatCompletionResponse
+  if err := json.Unmarshal(rec.Body.Bytes(), &res); err != nil {
+    t.Fatalf("failed to decode response: %v", err)
+  }
+  if res.Choices[0].Message.Content != "Live completion from fallback server" {
+    t.Fatalf("unexpected content from failover: %v", res.Choices[0].Message.Content)
+  }
+  if server1Hits != 2 { // 1 for /gradio_api/call, 1 for /gradio_api/call/sandbox_event
+    t.Fatalf("expected server1 to be attempted, got %d hits", server1Hits)
+  }
+  if server2Hits != 1 {
+    t.Fatalf("expected server2 to be reached on failover, got %d hits", server2Hits)
+  }
+
+  // 2. Verify server1 is in cooldown and next request goes directly to server2
+  rec2 := httptest.NewRecorder()
+  err2 := svc.Chat(rec2, nil, req)
+  if err2 != nil {
+    t.Fatalf("expected request during cooldown to succeed on server2: %v", err2)
+  }
+  if server2Hits != 2 {
+    t.Fatalf("expected server2 to receive the second request directly, got %d hits", server2Hits)
+  }
+
+  // 3. Test streaming failover when server1 is out of cooldown
+  svc.endpoints[0].CooldownUntil = time.Time{}
+  streamReq := ChatCompletionRequest{
+    Model: "qwen",
+    Messages: []ChatMessage{{Role: "user", Content: "hello"}},
+    Stream: true,
+  }
+  recStream := httptest.NewRecorder()
+  errStream := svc.Chat(recStream, nil, streamReq)
+  if errStream != nil {
+    t.Fatalf("expected streaming failover to succeed, got error: %v", errStream)
+  }
+  if server2Hits != 3 {
+    t.Fatalf("expected server2 to receive the streaming failover, got %d hits", server2Hits)
+  }
+}