fix(tools): recover tool calls from tool_use_failed errors and update default space to digital-twin
This commit is contained in:
@@ -1,13 +1,13 @@
|
||||
# gr2gw: Universal Gradio to OpenAI LLM gateway
|
||||
# gr2gw: universal Gradio to OpenAI LLM gateway
|
||||
|
||||
A zero-dependency, high-performance Go proxy server that introspects any Gradio chat space (such as Hugging Face Spaces or custom deployments) and exposes a standards-compliant OpenAI `/v1/chat/completions` and `/v1/models` HTTP API.
|
||||
|
||||
Default demo space: `https://ghost2513-openai-gpt-oss-120b.hf.space`
|
||||
Default demo space: `https://lucasmarchettidelima-digital-twin.hf.space`
|
||||
|
||||
## Features
|
||||
|
||||
- **Zero external dependencies**: Pure Go standard library (`net/http`, `encoding/json`, `bufio`, etc.).
|
||||
- **Automatic space introspection**: Dynamically queries `/gradio_api/info`, `/config`, and Hugging Face space metadata to discover models, endpoints, and input parameter mappings.
|
||||
- **Zero external dependencies**: pure Go standard library (`net/http`, `encoding/json`, `bufio`, etc.).
|
||||
- **Automatic space introspection**: dynamically queries `/gradio_api/info`, `/config`, and Hugging Face space metadata to discover models, endpoints, and input parameter mappings.
|
||||
- **Native Tencent Hunyuan 3 (`tencent-hy3`) support**:
|
||||
- Full native zero-degradation handling for official spaces like `https://tencent-hy3.hf.space`.
|
||||
- Maps `functions_json_str` natively without polluting the system prompt.
|
||||
@@ -27,7 +27,8 @@ Default demo space: `https://ghost2513-openai-gpt-oss-120b.hf.space`
|
||||
- Keeps `content` clean without tag leakage.
|
||||
- **Full tool calling & function interception**:
|
||||
- Formats schemas into native `functions_json_str` (Hy3) or system prompts (standard spaces).
|
||||
- **`StreamToolCallFilter`**: Stateful sliding-window filter that prevents `<tool_call>` tags from leaking into `delta.content`. Emits structured OpenAI `delta.tool_calls` chunks and sets `finish_reason: "tool_calls"`.
|
||||
- Intercepts and recovers tool calls from upstream `tool_use_failed` errors containing `failed_generation`.
|
||||
- **`StreamToolCallFilter`**: stateful sliding-window filter that prevents `<tool_call>` tags from leaking into `delta.content`. Emits structured OpenAI `delta.tool_calls` chunks and sets `finish_reason: "tool_calls"`.
|
||||
- Seamlessly maintains multi-turn context when tool results are submitted back via `role: "tool"`.
|
||||
- **Built-in SOCKS5 proxy client**:
|
||||
- Full RFC 1928 / RFC 1929 implementation with domain resolution (`socks5h://`), IPv4, IPv6, and username/password auth.
|
||||
@@ -59,7 +60,7 @@ make test
|
||||
|
||||
### Quick start
|
||||
|
||||
Run with the default space (`https://ghost2513-openai-gpt-oss-120b.hf.space`):
|
||||
Run with the default space (`https://lucasmarchettidelima-digital-twin.hf.space`):
|
||||
```bash
|
||||
./bin/gr2gw -port 8080
|
||||
```
|
||||
@@ -71,14 +72,14 @@ Target any other Gradio space:
|
||||
|
||||
With SOCKS5 proxy:
|
||||
```bash
|
||||
./bin/gr2gw -space https://ghost2513-openai-gpt-oss-120b.hf.space -socks socks5://127.0.0.1:1080
|
||||
./bin/gr2gw -space https://lucasmarchettidelima-digital-twin.hf.space -socks socks5://127.0.0.1:1080
|
||||
```
|
||||
|
||||
### CLI flags
|
||||
|
||||
| Flag | Default | Description |
|
||||
|------|---------|-------------|
|
||||
| `-space`, `-url` | `https://ghost2513-openai-gpt-oss-120b.hf.space` | Target Gradio space URL |
|
||||
| `-space`, `-url` | `https://lucasmarchettidelima-digital-twin.hf.space` | Target Gradio space URL |
|
||||
| `-port` | `8080` | Port to listen on |
|
||||
| `-host` | `0.0.0.0` | Host interface to bind to |
|
||||
| `-socks`, `-proxy`, `-socks5` | `""` | SOCKS5 proxy URL (`socks5://user:pass@host:port`) |
|
||||
@@ -104,13 +105,7 @@ Response:
|
||||
"object": "list",
|
||||
"data": [
|
||||
{
|
||||
"id": "openai/gpt-oss-120b",
|
||||
"object": "model",
|
||||
"created": 1788756307,
|
||||
"owned_by": "gradio"
|
||||
},
|
||||
{
|
||||
"id": "gpt-oss-120b",
|
||||
"id": "digital-twin",
|
||||
"object": "model",
|
||||
"created": 1788756307,
|
||||
"owned_by": "gradio"
|
||||
@@ -125,7 +120,7 @@ Response:
|
||||
curl http://localhost:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "openai/gpt-oss-120b",
|
||||
"model": "digital-twin",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is the capital of France?"}
|
||||
]
|
||||
@@ -138,7 +133,7 @@ Response:
|
||||
"id": "chatcmpl-16425f9d-c350-47a1-9a6d-e9ce10871545",
|
||||
"object": "chat.completion",
|
||||
"created": 1788756310,
|
||||
"model": "openai/gpt-oss-120b",
|
||||
"model": "digital-twin",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
@@ -163,7 +158,7 @@ Response:
|
||||
curl -N http://localhost:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "openai/gpt-oss-120b",
|
||||
"model": "digital-twin",
|
||||
"messages": [
|
||||
{"role": "user", "content": "Count from 1 to 5."}
|
||||
],
|
||||
@@ -177,7 +172,7 @@ curl -N http://localhost:8080/v1/chat/completions \
|
||||
curl http://localhost:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "openai/gpt-oss-120b",
|
||||
"model": "digital-twin",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is the weather in Tokyo?"}
|
||||
],
|
||||
@@ -204,7 +199,7 @@ Response:
|
||||
"id": "chatcmpl-32727c62-ef2a-4866-855b-f1c7ec2b8023",
|
||||
"object": "chat.completion",
|
||||
"created": 1788756322,
|
||||
"model": "openai/gpt-oss-120b",
|
||||
"model": "digital-twin",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
@@ -239,7 +234,7 @@ Response:
|
||||
curl http://localhost:8080/v1/chat/completions \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"model": "openai/gpt-oss-120b",
|
||||
"model": "digital-twin",
|
||||
"messages": [
|
||||
{"role": "user", "content": "What is the weather in Tokyo?"},
|
||||
{
|
||||
@@ -284,7 +279,7 @@ Response:
|
||||
"id": "chatcmpl-4903ba12-f12b-4cd3-a801-7290bc91a421",
|
||||
"object": "chat.completion",
|
||||
"created": 1788756335,
|
||||
"model": "openai/gpt-oss-120b",
|
||||
"model": "digital-twin",
|
||||
"choices": [
|
||||
{
|
||||
"index": 0,
|
||||
|
||||
Reference in New Issue
Block a user