luxferre/gr2gw
1
0
Fork 0
Code Actions Projects Activity

feat: universal heuristic engine for gradio version, flavor, protocol and tool calling resolution

Browse Source
This commit is contained in:
Luxferre
2026-09-07 11:53:44 +03:00
parent 491114b2bd
commit 751fe3c54f
3 changed files with 1027 additions and 211 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+114 -7
View File
@@ -7,7 +7,20 @@ Default demo space: `https://tencent-hy3.hf.space`
## Features
- **Zero external dependencies**: pure Go standard library (`net/http`, `encoding/json`, `bufio`, etc.).
- **Automatic space introspection**: dynamically queries `/gradio_api/info`, `/config`, and Hugging Face space metadata to discover models, endpoints, and input parameter mappings.
- **Universal heuristic discovery engine**:
- Automatically interrogates `/gradio_api/info`, `/config`, and Hugging Face metadata without endpoint-specific hardcoding.
- Detects Gradio runtime versions across v3, v4, v5, and v6.
- Classifies application architecture into `ChatInterface`, `Blocks (Chat)`, `Blocks (Multimodal Chat)`, `Interface`, and `Generic`.
- Disambiguation scoring engine evaluates candidate endpoints, filtering out UI resets, clears, retries, likes, and utility triggers to pinpoint primary conversational completion functions.
- Correlates semantic parameter names from `/gradio_api/info` with component IDs from `/config` to reconstruct parameter mappings even when component labels are obfuscated.
- **Dual protocol support with auto-fallback**:
- Supports modern Gradio 4/5/6 `/call` SSE protocol with persistent session hashes.
- Supports legacy Gradio 3 `/run/predict` and `/api/predict` direct execution protocol.
- Instant zero-latency fallback from `/call` to `/run/predict` upon HTTP 404 or 405 status codes.
- **Tool calling support classification**:
- Automatically identifies tool calling mechanisms: `native_slot`, `prompt_augmented_system`, `prompt_augmented_first_turn`, or `prompt_augmented_single_prompt`.
- Upstream tool call recovery intercepts `tool_use_failed` errors and extracts function names and JSON arguments.
- Real-time sliding-window `<tool_call>` tag interceptor emits structured OpenAI tool call chunks.
- **Native Tencent Hunyuan 3 (`tencent-hy3`) support**:
- Full native zero-degradation handling for official spaces like `https://tencent-hy3.hf.space`.
- Maps `functions_json_str` natively without polluting the system prompt.
@@ -26,17 +39,111 @@ Default demo space: `https://tencent-hy3.hf.space`
- Detects `<think>...</think>` tags in real time as fallback for standard spaces.
- Separates reasoning into `delta.reasoning_content` (streaming) and `message.reasoning_content` (non-streaming).
- Keeps `content` clean without tag leakage.
- **Full tool calling & function interception**:
- Formats schemas into native `functions_json_str` (Hy3) or system prompts (standard spaces).
- Intercepts and recovers tool calls from upstream `tool_use_failed` errors containing `failed_generation`.
- **`StreamToolCallFilter`**: stateful sliding-window filter that prevents `<tool_call>` tags from leaking into `delta.content`. Emits structured OpenAI `delta.tool_calls` chunks and sets `finish_reason: "tool_calls"`.
- Seamlessly maintains multi-turn context when tool results are submitted back via `role: "tool"`.
- **Built-in SOCKS5 proxy client**:
- Full RFC 1928 / RFC 1929 implementation with domain resolution (`socks5h://`), IPv4, IPv6, and username/password auth.
- **Dynamic space override**:
- Switch the target Gradio space on-the-fly per request using the `X-Gradio-Space` or `X-Space-URL` HTTP headers.
- **Fibonacci retry engine**:
- Resilient backoff retry mechanism (1s, 1s, 2s, 3s, 5s) for transient network hiccups.
- Resilient backoff retry mechanism (1s, 1s, 2s, 3s, 5s) for transient network hiccups with immediate break on 404/405 errors.
## Heuristic discovery engine
The gateway implements an autonomous heuristic engine that discovers and configures the optimal completion path for any target Gradio space at startup.
### Space introspection and version detection
Upon initialization, `gr2gw` inspects the space metadata:
1. Queries `/gradio_api/info` and `/config` endpoints.
2. Extracts the Gradio runtime version (v3, v4, v5, or v6).
3. Detects API routing prefixes (e.g. `/gradio_api` on modern versions, or root on Gradio 3).
### UI flavor classification
The engine classifies the application structure into architectural flavors:
- **`ChatInterface`**: Standard Gradio chat interfaces equipped with chatbot, textbox, and optional additional inputs.
- **`Blocks (Chat)`**: Custom `gr.Blocks` layouts containing conversational components.
- **`Blocks (Multimodal Chat)`**: Blocks architectures featuring `MultimodalTextbox` components that accept `{text, files}` JSON payloads.
- **`Interface`**: Classic input-output `gr.Interface` instances.
- **`Generic`**: Spaces with custom or unclassified component topologies.
### Candidate endpoint scoring
Gradio spaces frequently expose dozens of internal endpoints for UI actions (e.g. clearing text, retrying responses, voting/liking, adjusting sliders). The scoring algorithm identifies the true conversational endpoint by:
- Penalizing non-conversational triggers (e.g. `-600` for clear/reset/undo/retry/like endpoints).
- Rewarding chat semantics (`+150` for `/chat`, `/predict`, `/generate`, `/respond`).
- Rewarding message inputs (`+120` for `Textbox` or `MultimodalTextbox`).
- Rewarding chat history slots (`+80` for `Chatbot` or `State` components).
- Rewarding generator and streaming dependencies (`+50`).
### Dual protocol execution and auto-fallback
- **`call` protocol**: Modern Gradio 4/5/6 execution via `POST /call/{endpoint}` returning an event ID, followed by `GET /call/{endpoint}/{event_id}` SSE streaming.
- **`predict` protocol**: Gradio 3 and legacy execution via direct `POST /run/predict` or `POST /api/predict`.
- **Runtime failover**: If a space returns HTTP 404 or 405 when calling the modern protocol, the gateway breaks immediately from the retry loop and falls back to `/run/predict`.
### Tool calling support modes
The gateway evaluates available input components to determine how tool schemas and function calls should be delivered:
- **`native_slot`**: The space provides a dedicated parameter slot for tool definitions (e.g. `functions_json_str` on Hunyuan 3). Function definitions are passed cleanly without prompt alteration.
- **`prompt_augmented_system`**: The space provides a separate `system_prompt` input slot. Tool definitions and invocation schemas are injected directly into the system prompt.
- **`prompt_augmented_first_turn`**: The space accepts chat history pairs but lacks a dedicated system prompt slot. Tool definitions are prepended to the user prompt on the first dialogue turn.
- **`prompt_augmented_single_prompt`**: The space accepts only a single textbox input. Full multi-turn dialogue, tool definitions, and system guidance are synthesized into a single cohesive prompt.
### Startup resolution diagnostics
Whenever the gateway starts or inspects a new space, it prints the complete resolution picture:
```text
================================================================================
Gradio Space Resolution Picture
--------------------------------------------------------------------------------
Space URL: https://tencent-hy3.hf.space
Title: Hunyuan 3 Chat
Gradio Version: 5.29.0
UI Flavor: Blocks (Chat)
Protocol: call
API Prefix: /gradio_api
Resolved Endpoint: /chat_fn
Function Index: 1
Primary Model: hy3
Exposed Models: hy3, hunyuan3, tencent/Hy3
History Format: tuples
Tool Call Support: native_slot
Total Input Slots: 9
Input Slot Mappings:
[0] Component ID 1 textbox (label="message") -> message
[1] Component ID 2 textbox (label="system") -> system_prompt
[2] Component ID 3 chatbot (label="chatbot") -> history
[3] Component ID 4 radio (label="think_level") -> think_level
[4] Component ID 5 slider (label="temperature") -> temperature
[5] Component ID 6 slider (label="max_tokens") -> max_tokens
[6] Component ID 7 slider (label="top_p") -> top_p
[7] Component ID 8 state (label="preserved") -> preserved_thinking
[8] Component ID 9 textbox (label="functions") -> functions_json_str
================================================================================
```
### Gateway status endpoint
Making a `GET` request to `/` returns a JSON summary of the running gateway and the discovered space profile:
```bash
curl http://localhost:8080/
```
Response:
```json
{
"name": "gr2gw",
"status": "ready",
"space_url": "https://tencent-hy3.hf.space",
"gradio_version": "5.29.0",
"flavor": "Blocks (Chat)",
"protocol": "call",
"tool_call_mode": "native_slot",
"models": ["hy3", "hunyuan3", "tencent/Hy3"]
}
```
## Installation