luxferre/bantam
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

improved compacting algo and token visibility

Browse Source
This commit is contained in:
Luxferre
2026-08-16 08:12:49 +03:00
parent 3c55ce14d7
commit d1597444d0
3 changed files with 429 additions and 110 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+34 -15
View File
@@ -2,7 +2,7 @@
## About
Bantam is a minimalist, dependency-free AI agent specification with reference implementations in **Go** (`main.go` + `term_*.go`, module `code.luxferre.top/luxferre/bantam`) and **Perl 5** as **MicroBantam** (`mb`, under 100 SLOC). It provides an agentic loop capable of autonomous tool execution, shell interaction, real-time response streaming, markdown terminal rendering with box-drawing tables, Fibonacci backoff network resilience, and subagent delegation using any OpenAI-compatible completions API.
Bantam is a minimalist, dependency-free AI agent specification with reference implementations in **Go** (`main.go` + `term_*.go`, module `code.luxferre.top/luxferre/bantam`) and **Perl 5** as **MicroBantam** (`mb`, under 100 SLOC). It provides an agentic loop capable of autonomous tool execution, direct shell interaction, real-time response streaming, markdown terminal rendering with box-drawing tables, Fibonacci backoff network resilience, context window auto-discovery, token usage tracking with prompt cache breakdowns, prefix-cache-friendly conversation compaction, and subagent delegation using any OpenAI-compatible completions API.
The entire philosophy of Bantam is built upon two principles:
@@ -44,6 +44,7 @@ All implementations read the same `model.cfg` and `system.txt` from the current
temperature=0.7
api_key=your_api_key_here
stream=true
context_window=200000
```
2. Interactive mode:
@@ -52,19 +53,31 @@ All implementations read the same `model.cfg` and `system.txt` from the current
./mb # MicroBantam (Perl 5)
```
In interactive mode, prompts can span multiple lines: press **Ctrl+J** to insert a real line break (the cursor moves to the next line), then **Enter** to submit the whole multi-line prompt. The Go port ships its own raw-mode line editor (arrow keys move the cursor, Up/Down browse history, Backspace edits, Ctrl+C/Ctrl+D exit), so this works everywhere without dependencies.
In interactive mode, prompts can span multiple lines: press **Ctrl+J** to insert a real line break (the cursor moves to the next line), then **Enter** to submit the whole multi-line prompt. The Go port ships its own raw-mode line editor (arrow keys move the cursor, Up/Down browse history, Backspace edits, Ctrl+C clears line / interrupts in-flight run, Ctrl+D exits), working everywhere without third-party dependencies.
After every interaction, Bantam displays token usage (prompt tokens, cached/uncached breakdown when supported by the provider, completion tokens, and context window utilization):
```text
[tokens: 1420 prompt (1000 cached, 420 uncached) + 85 completion | context: 1420/200000 (0.7%)]
```
Sessions are saved under `~/.bantam/sessions/` and can be managed with these commands:
- `/save` — save the entire conversation to a new session file (auto-id like `20260808-190038`) and generate its summary
- `/list` — list saved sessions (newest first) with their ids, timestamps, message counts and summaries
- `/load <id>` — load a saved session (exact id or unique prefix) and continue from there
- `/compact` — summarize the conversation with the LLM and compact the context down to just the system message plus the summary
- `/compact` — compact context down to the system message and a concise summary using the LLM; the compaction prompt is appended directly to the existing message prefix to guarantee a 100% prompt cache hit
- `/cfg <param> [val]` — inspect or update a configuration parameter in `model.cfg` live
- `!<cmd>` — execute a shell command directly through `shell_exec` without adding the result to the conversation context (Go port)
- `/help` — show all supported commands
- `/clear` — reset the conversation to just the system prompt
- `/quit` — exit
When context window usage reaches **60% or higher**, Bantam automatically offers to compact the conversation:
```text
Context usage is at 62.4% (124800 / 200000 tokens). Compact conversation? [Y/n]:
```
Pressing **Ctrl+C** during an active inference run or long-running shell execution cleanly interrupts the turn without appending partial or malformed responses to the conversation context.
The current conversation is also **auto-saved** to `~/.bantam/sessions/autosave.json` after every turn, on `/clear`, `/load`, `/compact`, and on exit — so you can always `/load autosave` to resume where you left off.
3. File input mode:
@@ -79,27 +92,32 @@ Using these rules, everyone can build their own copy of Bantam from scratch in l
### High-Level Overview
1. **Initialization**: Read `system.txt` and `model.cfg`. Prepare an array of messages starting with the system prompt `{"role": "system", "content": system_prompt}`.
2. **Input Processing**: Take user prompt (via command-line file parameter or interactive stdin), append `{"role": "user", "content": prompt}`, and invoke `AL(cfg, messages)`.
1. **Initialization**: Read `system.txt` and `model.cfg`. Discover context window size from the `/models` endpoint (or fallback to `context_window` from `model.cfg` or 200,000 tokens). Prepare an array of messages starting with the system prompt `{"role": "system", "content": system_prompt}`.
2. **Input Processing**: Take user prompt (via command-line file parameter or interactive stdin). If prefixed with `!`, execute the command directly via `shell_exec` without appending to conversation context. Otherwise, append `{"role": "user", "content": prompt}`, and invoke `AL(cfg, messages)`.
3. **Agentic Loop (`AL`)**:
- Send `messages` and tool definitions to the OpenAI-compatible `/chat/completions` API endpoint with custom `User-Agent` headers.
- Send `messages` and tool definitions to the OpenAI-compatible `/chat/completions` API endpoint with custom `User-Agent` headers and `stream_options: {"include_usage": true}`.
- Support context cancellation (e.g. on `SIGINT` / Ctrl+C) to cleanly abort in-flight requests without appending incomplete messages.
- On network or HTTP failure, retry using Fibonacci backoff delays (`1s, 1s, 2s, 3s, 5s, 8s, 13s, 21s, 34s`).
- If `stream=true`, parse SSE data chunks (`data: {...}`) in real-time to stream reasoning content (`reasoning_content`) and response text directly to stdout, bracketing the reasoning block with `--- reasoning start ---` / `--- reasoning end ---` markers.
- Reconstruct the assistant message. If `tool_calls` exist, trace the call (`[tool call: name(args)]`), validate JSON arguments, execute the requested tool (`shell_exec` or `run_subagent`), trace the result (`[tool result: name]`), append the tool response `{"role": "tool", "tool_call_id": id, "content": result}`, and repeat the loop.
- If no tool calls remain or `max_al_iterations` is reached, return the updated messages list.
- If `stream=true`, parse SSE data chunks (`data: {...}`) in real-time to stream reasoning content (`reasoning_content`) and response text directly to stdout, bracketing the reasoning block with `--- reasoning start ---` / `--- reasoning end ---` markers, rendering Markdown and tables constrained to terminal width.
- Reconstruct the assistant message and track usage tokens (`prompt_tokens`, `completion_tokens`, cached tokens). If `tool_calls` exist, trace the call (`[tool call: name(args)]`), validate JSON arguments, execute the requested tool (`shell_exec` or `run_subagent`), trace the result (`[tool result: name]`), append the tool response `{"role": "tool", "tool_call_id": id, "content": result}`, and repeat the loop.
- If no tool calls remain or `max_al_iterations` is reached, return the updated messages list and turn usage stats.
4. **Post-Turn Reporting & Compaction**:
- Display token usage and context window percentage.
- If context usage is >= 60%, prompt user to compact.
- Compaction appends `"You are now acting as a compaction engine. Summarize the preceding conversation concisely but completely..."` as a user message to the conversation, invokes the LLM (ensuring zero prompt cache misses), and resets the conversation to the system prompt and the resulting summary.
### Main program
1. Read system prompt from `system.txt` (default if missing).
2. Read model parameters from `model.cfg` (`key=value` format).
2. Read model parameters from `model.cfg` (`key=value` format) and discover context window size.
3. Prepare a new message list with the system prompt (`role: "system"`).
4. Read the first command-line parameter. If non-empty, read user prompt from the specified file. If prefixed with `!`, execute the shell command directly via `shell_exec` and exit. Otherwise, append to `messages` (`role: "user"`), run `AL(cfg, messages)`, and exit.
5. Read user prompt from standard input (with `readline` line editing and history in `~/.bantam_history`; **Ctrl+J** inserts a real newline into the line being edited). If equal to `/quit` or EOF, exit. If equal to `/clear`, reset `messages` to step 3 and return to step 5. If equal to `/save`, write the whole `messages` array to `~/.bantam/sessions/<id>.json` (with an auto-generated summary) and return to step 5. If equal to `/list`, print saved sessions and their summaries and return to step 5. If starting with `/load`, replace `messages` with the saved session's messages (by exact id or unique prefix) and return to step 5. If equal to `/compact`, ask the LLM to summarize the conversation, replace `messages` with `[system, summary-user-message]`, and return to step 5. If starting with `/cfg`, display the current value (`/cfg <param>`) or update `model.cfg` live (`/cfg <param> <val>`) and return to step 5. If starting with `!`, execute the command directly via `shell_exec` without adding the result to `messages` and return to step 5. If equal to `/help`, print the command list and return to step 5. After every user turn and on exit, auto-save `messages` to `~/.bantam/sessions/autosave.json`.
6. Append user prompt to `messages` (`role: "user"`), run `AL(cfg, messages)`, and go to step 5.
4. Read the first command-line parameter. If non-empty, read user prompt from the specified file. If prefixed with `!`, execute the shell command directly via `shell_exec` and exit. Otherwise, append to `messages` (`role: "user"`), run `AL(cfg, messages)`, display token usage, and exit.
5. Read user prompt from standard input (with `readline` line editing and history in `~/.bantam_history`; **Ctrl+J** inserts a real newline into the line being edited). If equal to `/quit` or EOF, exit. If equal to `/clear`, reset `messages` to step 3 and return to step 5. If equal to `/save`, write the whole `messages` array to `~/.bantam/sessions/<id>.json` (with an auto-generated summary) and return to step 5. If equal to `/list`, print saved sessions and their summaries and return to step 5. If starting with `/load`, replace `messages` with the saved session's messages (by exact id or unique prefix) and return to step 5. If equal to `/compact`, ask the LLM to summarize the conversation by appending the compaction prompt to preserve KV cache, replace `messages` with `[system, summary-user-message]`, and return to step 5. If starting with `/cfg`, display the current value (`/cfg <param>`) or update `model.cfg` live (`/cfg <param> <val>`) and return to step 5. If starting with `!`, execute the command directly via `shell_exec` without adding the result to `messages` and return to step 5. If equal to `/help`, print the command list and return to step 5. After every user turn and on exit, auto-save `messages` to `~/.bantam/sessions/autosave.json`.
6. Append user prompt to `messages` (`role: "user"`), run `AL(cfg, messages)`, display token usage, check 60% context threshold for auto-compaction, and go to step 5.
### Agentic loop (`AL(cfg, messages)`) function
1. Call OpenAI-compatible Completions API (`POST {endpoint}/chat/completions`) forwarding relevant parameters from `cfg` (`model`, `temperature`, `stream`, `reasoning_effort`, etc., excluding internal agent configs like `endpoint`, `api_key`, `timeout`, `shell_timeout`, `max_al_iterations`, `color`) and optional `api_key` bearer header.
1. Call OpenAI-compatible Completions API (`POST {endpoint}/chat/completions`) forwarding relevant parameters from `cfg` (`model`, `temperature`, `stream`, `reasoning_effort`, etc., excluding internal agent configs like `endpoint`, `api_key`, `timeout`, `shell_timeout`, `max_al_iterations`, `color`, `context_window`) and optional `api_key` bearer header.
- Set custom `User-Agent` header (`Mozilla/5.0 (compatible; Bantam/1.0)`) to avoid gateway 403 blocks.
- Retry network/HTTP errors with Fibonacci backoff delays (`1s, 1s, 2s, 3s, 5s, 8s, 13s, 21s, 34s`).
- If `stream=true`, parse SSE stream (`data: {...}`) for real-time reasoning and text output, bracketing reasoning with `--- reasoning start ---` / `--- reasoning end ---` markers.
@@ -111,7 +129,7 @@ Using these rules, everyone can build their own copy of Bantam from scratch in l
- Output a trace log of the result (`[tool result: name]`).
- Append tool result message (`role: "tool"`, `tool_call_id`, `content`: result string) to `messages`.
- Loop back to step 1.
4. If no pending tool calls (or if `max_al_iterations` is reached), stop and return `messages`.
4. If no pending tool calls (or if `max_al_iterations` is reached), stop and return `messages` and accumulated usage.
If the API rejects the request with an `Invalid assistant message: content or tool_calls must be set` error (usually caused by a previously cut-off stream that left an empty assistant message in the session), all implementations strip the last `assistant`-role message from the session and retry the call.
@@ -128,6 +146,7 @@ If the API rejects the request with an `Invalid assistant message: content or to
- `timeout` (HTTP timeout in seconds for LLM API calls, default 300; in the Go port it bounds connection setup and time-to-first-byte, so long streaming responses are not cut off mid-stream)
- `shell_timeout` (timeout in seconds for `shell_exec` commands, default 120)
- `max_al_iterations` (max tool-call loop iterations per `AL()` invocation, default 1000)
- `context_window` (context window size in tokens, auto-discovered from `/models` API if available, fallback to this setting, default 200000)
The Go port's built-in editor tracks the cursor with its own column math (terminal auto-wrap aware) and redraws from the first line of the buffer, so wrapped input stays clean at any terminal width.