omniforge support added

README.md

@@ -1,13 +1,26 @@
-# gr2gw: Universal Gradio to OpenAI LLM gateway
+# gr2gw: universal Gradio to OpenAI LLM gateway
 
 A zero-dependency, high-performance Go proxy server that introspects any Gradio chat space (such as Hugging Face Spaces or custom deployments) and exposes a standards-compliant OpenAI `/v1/chat/completions` and `/v1/models` HTTP API.
 
-Default demo space: `https://ghost2513-openai-gpt-oss-120b.hf.space`
+Default demo space: `https://tencent-hy3.hf.space`
 
 ## Features
 
-- **Zero external dependencies**: Pure Go standard library (`net/http`, `encoding/json`, `bufio`, etc.).
-- **Automatic space introspection**: Dynamically queries `/gradio_api/info`, `/config`, and Hugging Face space metadata to discover models, endpoints, and input parameter mappings.
+- **Zero external dependencies**: pure Go standard library (`net/http`, `encoding/json`, `bufio`, etc.).
+- **Universal heuristic discovery engine**:
+  - Automatically interrogates `/gradio_api/info`, `/config`, and Hugging Face metadata without endpoint-specific hardcoding.
+  - Detects Gradio runtime versions across v3, v4, v5, and v6.
+  - Classifies application architecture into `ChatInterface`, `Blocks (Chat)`, `Blocks (Multimodal Chat)`, `Interface`, and `Generic`.
+  - Disambiguation scoring engine evaluates candidate endpoints, filtering out UI resets, clears, retries, likes, and utility triggers to pinpoint primary conversational completion functions.
+  - Correlates semantic parameter names from `/gradio_api/info` with component IDs from `/config` to reconstruct parameter mappings even when component labels are obfuscated.
+- **Dual protocol support with auto-fallback**:
+  - Supports modern Gradio 4/5/6 `/call` SSE protocol with persistent session hashes.
+  - Supports legacy Gradio 3 `/run/predict` and `/api/predict` direct execution protocol.
+  - Instant zero-latency fallback from `/call` to `/run/predict` upon HTTP 404 or 405 status codes.
+- **Tool calling support classification**:
+  - Automatically identifies tool calling mechanisms: `native_slot`, `prompt_augmented_system`, `prompt_augmented_first_turn`, or `prompt_augmented_single_prompt`.
+  - Upstream tool call recovery intercepts `tool_use_failed` errors and extracts function names and JSON arguments.
+  - Real-time sliding-window `<tool_call>` tag interceptor emits structured OpenAI tool call chunks.
 - **Native Tencent Hunyuan 3 (`tencent-hy3`) support**:
   - Full native zero-degradation handling for official spaces like `https://tencent-hy3.hf.space`.
   - Maps `functions_json_str` natively without polluting the system prompt.
@@ -17,6 +30,7 @@ Default demo space: `https://ghost2513-openai-gpt-oss-120b.hf.space`
   - Automatically formats conversation history into structured inputs when the space supports them.
   - Transparently composes multi-turn dialogue (`System`, `User`, `Assistant`) into single prompt inputs when the space only accepts a single message textbox.
   - Automatically pads hidden/State inputs (e.g. Gradio State components) to prevent backend argument count mismatches.
+  - Automatically handles multimodal textbox inputs (`MultimodalData` with `{text, files}`) and space component defaults (radios, sliders, checkboxes).
 - **Real-time streaming & accumulation filter**:
   - Automatically computes token deltas from cumulative or incremental Gradio SSE output streams (including 2D Hy3 frames `[[content, reasoning, tool_calls, history]]`).
   - Emits standards-compliant `chat.completion.chunk` SSE events in real time.
@@ -25,16 +39,111 @@ Default demo space: `https://ghost2513-openai-gpt-oss-120b.hf.space`
   - Detects `<think>...</think>` tags in real time as fallback for standard spaces.
   - Separates reasoning into `delta.reasoning_content` (streaming) and `message.reasoning_content` (non-streaming).
   - Keeps `content` clean without tag leakage.
-- **Full tool calling & function interception**:
-  - Formats schemas into native `functions_json_str` (Hy3) or system prompts (standard spaces).
-  - **`StreamToolCallFilter`**: Stateful sliding-window filter that prevents `<tool_call>` tags from leaking into `delta.content`. Emits structured OpenAI `delta.tool_calls` chunks and sets `finish_reason: "tool_calls"`.
-  - Seamlessly maintains multi-turn context when tool results are submitted back via `role: "tool"`.
 - **Built-in SOCKS5 proxy client**:
   - Full RFC 1928 / RFC 1929 implementation with domain resolution (`socks5h://`), IPv4, IPv6, and username/password auth.
 - **Dynamic space override**:
   - Switch the target Gradio space on-the-fly per request using the `X-Gradio-Space` or `X-Space-URL` HTTP headers.
 - **Fibonacci retry engine**:
-  - Resilient backoff retry mechanism (1s, 1s, 2s, 3s, 5s) for transient network hiccups.
+  - Resilient backoff retry mechanism (1s, 1s, 2s, 3s, 5s) for transient network hiccups with immediate break on 404/405 errors.
+
+## Heuristic discovery engine
+
+The gateway implements an autonomous heuristic engine that discovers and configures the optimal completion path for any target Gradio space at startup.
+
+### Space introspection and version detection
+
+Upon initialization, `gr2gw` inspects the space metadata:
+1. Queries `/gradio_api/info` and `/config` endpoints.
+2. Extracts the Gradio runtime version (v3, v4, v5, or v6).
+3. Detects API routing prefixes (e.g. `/gradio_api` on modern versions, or root on Gradio 3).
+
+### UI flavor classification
+
+The engine classifies the application structure into architectural flavors:
+- **`ChatInterface`**: Standard Gradio chat interfaces equipped with chatbot, textbox, and optional additional inputs.
+- **`Blocks (Chat)`**: Custom `gr.Blocks` layouts containing conversational components.
+- **`Blocks (Multimodal Chat)`**: Blocks architectures featuring `MultimodalTextbox` components that accept `{text, files}` JSON payloads.
+- **`Interface`**: Classic input-output `gr.Interface` instances.
+- **`Generic`**: Spaces with custom or unclassified component topologies.
+
+### Candidate endpoint scoring
+
+Gradio spaces frequently expose dozens of internal endpoints for UI actions (e.g. clearing text, retrying responses, voting/liking, adjusting sliders). The scoring algorithm identifies the true conversational endpoint by:
+- Penalizing non-conversational triggers (e.g. `-600` for clear/reset/undo/retry/like endpoints).
+- Rewarding chat semantics (`+150` for `/chat`, `/predict`, `/generate`, `/respond`).
+- Rewarding message inputs (`+120` for `Textbox` or `MultimodalTextbox`).
+- Rewarding chat history slots (`+80` for `Chatbot` or `State` components).
+- Rewarding generator and streaming dependencies (`+50`).
+
+### Dual protocol execution and auto-fallback
+
+- **`call` protocol**: Modern Gradio 4/5/6 execution via `POST /call/{endpoint}` returning an event ID, followed by `GET /call/{endpoint}/{event_id}` SSE streaming.
+- **`predict` protocol**: Gradio 3 and legacy execution via direct `POST /run/predict` or `POST /api/predict`.
+- **Runtime failover**: If a space returns HTTP 404 or 405 when calling the modern protocol, the gateway breaks immediately from the retry loop and falls back to `/run/predict`.
+
+### Tool calling support modes
+
+The gateway evaluates available input components to determine how tool schemas and function calls should be delivered:
+- **`native_slot`**: The space provides a dedicated parameter slot for tool definitions (e.g. `functions_json_str` on Hunyuan 3). Function definitions are passed cleanly without prompt alteration.
+- **`prompt_augmented_system`**: The space provides a separate `system_prompt` input slot. Tool definitions and invocation schemas are injected directly into the system prompt.
+- **`prompt_augmented_first_turn`**: The space accepts chat history pairs but lacks a dedicated system prompt slot. Tool definitions are prepended to the user prompt on the first dialogue turn.
+- **`prompt_augmented_single_prompt`**: The space accepts only a single textbox input. Full multi-turn dialogue, tool definitions, and system guidance are synthesized into a single cohesive prompt.
+
+### Startup resolution diagnostics
+
+Whenever the gateway starts or inspects a new space, it prints the complete resolution picture:
+
+```text
+================================================================================
+Gradio Space Resolution Picture
+--------------------------------------------------------------------------------
+Space URL:         https://tencent-hy3.hf.space
+Title:             Hunyuan 3 Chat
+Gradio Version:    5.29.0
+UI Flavor:         Blocks (Chat)
+Protocol:          call
+API Prefix:        /gradio_api
+Resolved Endpoint: /chat_fn
+Function Index:    1
+Primary Model:     hy3
+Exposed Models:    hy3, hunyuan3, tencent/Hy3
+History Format:    tuples
+Tool Call Support: native_slot
+Total Input Slots: 9
+Input Slot Mappings:
+  [0] Component ID 1    textbox (label="message")      -> message
+  [1] Component ID 2    textbox (label="system")       -> system_prompt
+  [2] Component ID 3    chatbot (label="chatbot")      -> history
+  [3] Component ID 4    radio (label="think_level")    -> think_level
+  [4] Component ID 5    slider (label="temperature")   -> temperature
+  [5] Component ID 6    slider (label="max_tokens")    -> max_tokens
+  [6] Component ID 7    slider (label="top_p")         -> top_p
+  [7] Component ID 8    state (label="preserved")      -> preserved_thinking
+  [8] Component ID 9    textbox (label="functions")    -> functions_json_str
+================================================================================
+```
+
+### Gateway status endpoint
+
+Making a `GET` request to `/` returns a JSON summary of the running gateway and the discovered space profile:
+
+```bash
+curl http://localhost:8080/
+```
+
+Response:
+```json
+{
+  "name": "gr2gw",
+  "status": "ready",
+  "space_url": "https://tencent-hy3.hf.space",
+  "gradio_version": "5.29.0",
+  "flavor": "Blocks (Chat)",
+  "protocol": "call",
+  "tool_call_mode": "native_slot",
+  "models": ["hy3", "hunyuan3", "tencent/Hy3"]
+}
+```
 
 ## Installation
 
@@ -59,26 +168,27 @@ make test
 
 ### Quick start
 
-Run with the default space (`https://ghost2513-openai-gpt-oss-120b.hf.space`):
+Run with the default space (`https://tencent-hy3.hf.space`):
 ```bash
 ./bin/gr2gw -port 8080
 ```
 
 Target any other Gradio space:
 ```bash
-./bin/gr2gw -space https://ericsqin-hy3.hf.space -port 8080
+./bin/gr2gw -space https://lucasmarchettidelima-digital-twin.hf.space -port 8080
 ```
 
 With SOCKS5 proxy:
 ```bash
-./bin/gr2gw -space https://ghost2513-openai-gpt-oss-120b.hf.space -socks socks5://127.0.0.1:1080
+./bin/gr2gw -space https://tencent-hy3.hf.space -socks socks5://127.0.0.1:1080
 ```
 
 ### CLI flags
 
 | Flag | Default | Description |
 |------|---------|-------------|
-| `-space`, `-url` | `https://ghost2513-openai-gpt-oss-120b.hf.space` | Target Gradio space URL |
+| `-space`, `-url`, `-endpoint` | `https://tencent-hy3.hf.space` | Target Gradio space URL |
+| `-model` | `""` | Exposed model name override (default: auto-detected) |
 | `-port` | `8080` | Port to listen on |
 | `-host` | `0.0.0.0` | Host interface to bind to |
 | `-socks`, `-proxy`, `-socks5` | `""` | SOCKS5 proxy URL (`socks5://user:pass@host:port`) |
@@ -104,13 +214,19 @@ Response:
   "object": "list",
   "data": [
     {
-      "id": "openai/gpt-oss-120b",
+      "id": "hy3",
       "object": "model",
       "created": 1788756307,
       "owned_by": "gradio"
     },
     {
-      "id": "gpt-oss-120b",
+      "id": "hunyuan3",
+      "object": "model",
+      "created": 1788756307,
+      "owned_by": "gradio"
+    },
+    {
+      "id": "tencent/Hy3",
       "object": "model",
       "created": 1788756307,
       "owned_by": "gradio"
@@ -125,7 +241,7 @@ Response:
 curl http://localhost:8080/v1/chat/completions \
   -H "Content-Type: application/json" \
   -d '{
-    "model": "openai/gpt-oss-120b",
+    "model": "hy3",
     "messages": [
       {"role": "user", "content": "What is the capital of France?"}
     ]
@@ -138,13 +254,13 @@ Response:
   "id": "chatcmpl-16425f9d-c350-47a1-9a6d-e9ce10871545",
   "object": "chat.completion",
   "created": 1788756310,
-  "model": "openai/gpt-oss-120b",
+  "model": "hy3",
   "choices": [
     {
       "index": 0,
       "message": {
         "role": "assistant",
-        "content": "Paris is the capital of France."
+        "content": "The capital of France is Paris."
       },
       "finish_reason": "stop"
     }
@@ -163,7 +279,7 @@ Response:
 curl -N http://localhost:8080/v1/chat/completions \
   -H "Content-Type: application/json" \
   -d '{
-    "model": "openai/gpt-oss-120b",
+    "model": "hy3",
     "messages": [
       {"role": "user", "content": "Count from 1 to 5."}
     ],
@@ -177,7 +293,7 @@ curl -N http://localhost:8080/v1/chat/completions \
 curl http://localhost:8080/v1/chat/completions \
   -H "Content-Type: application/json" \
   -d '{
-    "model": "openai/gpt-oss-120b",
+    "model": "hy3",
     "messages": [
       {"role": "user", "content": "What is the weather in Tokyo?"}
     ],
@@ -204,7 +320,7 @@ Response:
   "id": "chatcmpl-32727c62-ef2a-4866-855b-f1c7ec2b8023",
   "object": "chat.completion",
   "created": 1788756322,
-  "model": "openai/gpt-oss-120b",
+  "model": "hy3",
   "choices": [
     {
       "index": 0,
@@ -239,7 +355,7 @@ Response:
 curl http://localhost:8080/v1/chat/completions \
   -H "Content-Type: application/json" \
   -d '{
-    "model": "openai/gpt-oss-120b",
+    "model": "hy3",
     "messages": [
       {"role": "user", "content": "What is the weather in Tokyo?"},
       {
@@ -284,7 +400,7 @@ Response:
   "id": "chatcmpl-4903ba12-f12b-4cd3-a801-7290bc91a421",
   "object": "chat.completion",
   "created": 1788756335,
-  "model": "openai/gpt-oss-120b",
+  "model": "hy3",
   "choices": [
     {
       "index": 0,