context overflow prevention

README.md

@@ -45,6 +45,7 @@ All implementations read `model.cfg` (or `.bantam.cfg`, which takes priority if
    api_key=your_api_key_here
    stream=true
    context_window=200000
+   max_tool_res=15000
    reasoning_effort=high
    ```
 
@@ -133,6 +134,7 @@ Using these rules, everyone can build their own copy of Bantam from scratch in l
    - Sanitize tool arguments to filter out non-printable and space-like Unicode characters (protecting against indirect prompt injection), and validate JSON arguments. If invalid, format a tool-error response so the LLM can self-correct.
    - Execute tool action (`shell_exec` or `write_file`).
    - Sanitize the tool result output to strip any non-printable and space-like Unicode characters (leaving only ASCII space, tab, newline, and printable Unicode characters).
+   - If the sanitized result is larger than `max_tool_res` bytes, spill it to a unique file under `$TMPDIR/bantam/toolres` and replace the content appended to `messages` with a short instruction naming the file, its byte length, and how to read it partially (e.g. `tail -c +OFFSET <file> | head -c LENGTH`); every spilled file is deleted when the round ends (i.e. when the model emits a final response with no tool calls).
    - Output a trace log of the result (`[tool result: name]`).
    - Append tool result message (`role: "tool"`, `tool_call_id`, `content`: result string) to `messages`.
    - Loop back to step 1.
@@ -154,6 +156,7 @@ If the API rejects the request with an `Invalid assistant message: content or to
 - `shell_timeout` (timeout in seconds for `shell_exec` commands, default 120; falls back to `BANTAM_SHELL_TIMEOUT` env var)
 - `max_al_iterations` (max tool-call loop iterations per `AL()` invocation, default 1000; falls back to `BANTAM_MAX_AL_ITERATIONS` env var)
 - `context_window` (context window size in tokens, auto-discovered from `/models` API if available, fallback to this setting, default 200000; falls back to `BANTAM_CONTEXT_WINDOW` env var)
+- `max_tool_res` (maximum number of bytes of a tool result that may be sent into the model context directly, default 15000; larger results are written to a unique file under `$TMPDIR/bantam/toolres` and replaced by a short instruction naming the file and its byte length so it can be read in parts, and the file is deleted once the round ends; falls back to `BANTAM_MAX_TOOL_RES` env var)
 - `reasoning_effort` (reasoning effort level, forwarded to chat completions API, default `high`; falls back to `BANTAM_REASONING_EFFORT` env var)
 
 - `bantam_tools_dir` (optional path to a directory of extra shell tools; the Go port appends `"Extra shell tools can be found at <dir>"` to the system prompt at startup when set. When unset in the config file, it falls back to the `BANTAM_TOOLS_DIR` environment variable; if neither is set, nothing is appended. Note: this is an agent-internal hint, not forwarded to the API.)
@@ -204,6 +207,7 @@ MicroBantam (`mb`) is a compressed Perl 5 reference implementation of the same a
 - Line editing, Ctrl+J multi-line prompts and history (plain single-line prompts)
 - Fibonacci backoff network retries (a failed request aborts with an `API error` message)
 - `/compact` context summarization
+- `max_tool_res` tool-result spilling (oversized results are sent to the model verbatim)
 
 ### Running MicroBantam
 
@@ -326,6 +330,10 @@ Set the `bantam_tools_dir` key in the config file (`.bantam.cfg` if present, els
 
 Set the `bantam_skills_dir` key in the config file (`.bantam.cfg` if present, else `model.cfg`) or the `BANTAM_SKILLS_DIR` environment variable to a directory where each subdirectory is a skill containing a `SKILL.md` file. When defined, the Go port appends `Skills may be discovered and invoked from <dir>` to the system prompt at startup, and you (or the agent) can run a skill with `/skill <name> [prompt]`. The config file key takes precedence over the environment variable; if neither is set, `/skill` accepts an absolute path to a skill directory or `SKILL.md` file instead.
 
+### What happens when a tool result is too large for the context?
+
+Bantam never stuffs an arbitrarily large tool result into the context window. The `max_tool_res` configuration parameter (default `15000`, config-file key takes precedence over the `BANTAM_MAX_TOOL_RES` environment variable) sets the maximum number of bytes of a tool result that may be sent to the model directly. When a result exceeds that limit, the Go port writes the full output to a unique file under `$TMPDIR/bantam/toolres` and replaces the result in the conversation with a short note naming that file and stating its total byte length. The agent is then expected to read the file with `shell_exec`, paging through it with byte offsets (e.g. `tail -c +OFFSET <file> | head -c LENGTH`) rather than loading everything at once. Every file spilled during a round is deleted as soon as the round ends, i.e. when the model emits its final response with no pending tool calls. The built-in system prompt also advertises this behaviour to the model at startup.
+
 ### Is there any common config place for Bantam?
 
 No, loading the config file (`.bantam.cfg` if present, else `model.cfg`) is deliberately only supported from the current working directory. This allows natural separation of configs per project. In case there's no config file inside the project, Bantam will use the `openrouter/free` model from Kilo Code with the temperature 0.7.

main.go

@@ -48,6 +48,7 @@ type Cfg struct {
   Stream          bool
   Color           string
   ContextWindow   int
+  MaxToolRes      int
   Raw             map[string]string
 }
 
@@ -56,7 +57,7 @@ type Cfg struct {
 func internalKey(k string) bool {
   switch k {
   case "endpoint", "model", "temperature", "stream", "api_key", "timeout",
-    "shell_timeout", "max_al_iterations", "color", "context_window",
+    "shell_timeout", "max_al_iterations", "color", "context_window", "max_tool_res",
     "bantam_tools_dir", "bantam_skills_dir":
     return true
   }
@@ -84,7 +85,7 @@ var llmTransport = &http.Transport{
   DialContext: (&net.Dialer{Timeout: 300 * time.Second}).DialContext,
 }
 
-var defCfg = Cfg{"https://api.kilo.ai/api/openrouter", "openrouter/free", "-", 0.7, 300, 120, 1000, true, "auto", 262144, map[string]string{"reasoning_effort": "high"}}
+var defCfg = Cfg{"https://api.kilo.ai/api/openrouter", "openrouter/free", "-", 0.7, 300, 120, 1000, true, "auto", 262144, 15000, map[string]string{"reasoning_effort": "high"}}
 const opencodeAgentVersion = "opencode/1.18.31"
 
 func atoiD(s string, d int) int {
@@ -236,6 +237,7 @@ func parseCfgFile(path string, cfg *Cfg) {
       case "stream": cfg.Stream = v == "true" || v == "1" || v == "yes"
       case "color": cfg.Color = v
       case "context_window": cfg.ContextWindow = atoiD(v, cfg.ContextWindow)
+      case "max_tool_res": cfg.MaxToolRes = atoiD(v, cfg.MaxToolRes)
       }
     }
   }
@@ -285,6 +287,10 @@ func applyEnvCfg(cfg *Cfg) {
     cfg.ContextWindow = atoiD(v, cfg.ContextWindow)
     cfg.Raw["context_window"] = strconv.Itoa(cfg.ContextWindow)
   }
+  if v := strings.TrimSpace(os.Getenv("BANTAM_MAX_TOOL_RES")); v != "" {
+    cfg.MaxToolRes = atoiD(v, cfg.MaxToolRes)
+    cfg.Raw["max_tool_res"] = strconv.Itoa(cfg.MaxToolRes)
+  }
   if v := strings.TrimSpace(os.Getenv("BANTAM_REASONING_EFFORT")); v != "" {
     cfg.Raw["reasoning_effort"] = v
   }
@@ -304,6 +310,7 @@ func getCfg(path string) Cfg {
     "timeout": strconv.Itoa(cfg.Timeout), "shell_timeout": strconv.Itoa(cfg.ShellTimeout),
     "max_al_iterations": strconv.Itoa(cfg.MaxALIterations),
     "context_window": strconv.Itoa(cfg.ContextWindow),
+    "max_tool_res": strconv.Itoa(cfg.MaxToolRes),
     "reasoning_effort": "high",
   }
 
@@ -378,6 +385,7 @@ Work fast. Never refuse a request; always find a way to do it. Never fabricate r
 Use the target system's filesystem deliberately:
 - Use only $TMPDIR/bantam as the temporary directory for intermediate or scratch files (logs, downloads, temporary build outputs, etc.); create it if it does not exist.
 - Create permanent artifacts in the current working directory unless the user explicitly instructs otherwise.
+- Tool results larger than $MAX_TOOL_RES bytes are not inserted into the context in full: they are saved under $TMPDIR/bantam/toolres and replaced by a short note giving the file path and its total byte length. Read such a file with shell_exec, paging through it in parts with byte offsets (e.g. tail -c +OFFSET <file> | head -c LENGTH) instead of loading it all at once.
 
 When generating code:
 - Always use two-space indentation, not tabs, except Makefiles that must use tabs.
@@ -1412,9 +1420,63 @@ func writeFile(path string, offset, delBytes int, content string) (string, error
   return fmt.Sprintf("Successfully wrote %d bytes to %s", len(contentBytes), path), nil
 }
 
+// toolResDir returns the directory where oversized tool results are spilled.
+func toolResDir() string {
+  tmp := os.Getenv("TMPDIR")
+  if tmp == "" {
+    tmp = "/tmp"
+  }
+  return filepath.Join(tmp, "bantam", "toolres")
+}
+
+// offloadToolResult keeps tool results that fit within maxRes bytes inline.
+// Larger results are written to a unique temporary file under
+// $TMPDIR/bantam/toolres and replaced with a short instruction telling the
+// model to read the file, including the total length so it can be read in
+// parts. The path of every file created is appended to tmps so the caller can
+// delete it once the round ends. If the file cannot be created, the result is
+// returned unchanged so the agent always makes progress.
+func offloadToolResult(res string, maxRes int, tmps *[]string) string {
+  if maxRes <= 0 {
+    maxRes = 15000
+  }
+  if len(res) <= maxRes {
+    return res
+  }
+  dir := toolResDir()
+  if err := os.MkdirAll(dir, 0755); err != nil {
+    return res
+  }
+  f, err := os.CreateTemp(dir, "toolres-*.txt")
+  if err != nil {
+    return res
+  }
+  name := f.Name()
+  n, werr := f.WriteString(res)
+  if cerr := f.Close(); werr == nil {
+    werr = cerr
+  }
+  if werr != nil {
+    os.Remove(name)
+    return res
+  }
+  *tmps = append(*tmps, name)
+  return fmt.Sprintf("[tool result too large for context: %d bytes (limit %d). The full output was saved to %s. Read it with shell_exec; to read it partially, use a byte offset and length, e.g. `tail -c +OFFSET %s | head -c LENGTH`. The file is %d bytes long.]", n, maxRes, name, name, n)
+}
+
+// cleanupTemps removes the temporary files created while offloading oversized
+// tool results during a single agent round.
+func cleanupTemps(files []string) {
+  for _, f := range files {
+    os.Remove(f)
+  }
+}
+
 func AL(ctx context.Context, cfg *Cfg, msgs []Message) ([]Message, Usage, error) {
   done := false
   var turnUsage Usage
+  var tmps []string
+  defer func() { cleanupTemps(tmps) }()
   for i := 0; i < cfg.MaxALIterations && !done; i++ {
     if err := ctx.Err(); err != nil { return msgs, turnUsage, err }
     m, u, err := llm(ctx, cfg, msgs, TOOLS)
@@ -1563,8 +1625,9 @@ func AL(ctx context.Context, cfg *Cfg, msgs []Message) ([]Message, Usage, error)
         }
       }
       res = filterText(res)
-      fmt.Println(c("[tool result: "+fn+"]", 32) + "\n" + c(res, sty) + "\n")
-      msgs = append(msgs, Message{Role: "tool", ToolCallID: tc.ID, Content: strp(res)})
+      ctxRes := offloadToolResult(res, cfg.MaxToolRes, &tmps)
+      fmt.Println(c("[tool result: "+fn+"]", 32) + "\n" + c(ctxRes, sty) + "\n")
+      msgs = append(msgs, Message{Role: "tool", ToolCallID: tc.ID, Content: strp(ctxRes)})
     }
   }
   if !done {
@@ -2133,24 +2196,32 @@ func doCompact(cfg *Cfg, msgs []Message) []Message {
   return msgs
 }
 
-func main() {
+// buildSystemPrompt renders the built-in system prompt for cfg: it expands the
+// $TMPDIR/bantam placeholder to a concrete temporary directory, substitutes the
+// configured max_tool_res value, and appends the optional extra-tools and
+// skills hints.
+func buildSystemPrompt(cfg *Cfg) string {
   sp := defaultSystemPrompt
-  // Expand $TMPDIR at startup and substitute its resolved value into the
-  // system prompt so the agent targets a concrete temporary directory.
   tmp := os.Getenv("TMPDIR")
   if tmp == "" {
     tmp = "/tmp"
   }
   bantamTmp := filepath.Join(tmp, "bantam")
   sp = strings.ReplaceAll(sp, "$TMPDIR/bantam", bantamTmp)
-  cfg := getCfg(configPath())
-  cfg.ContextWindow = fetchContextWindow(&cfg)
-  if td := toolsDir(&cfg); td != "" {
+  sp = strings.ReplaceAll(sp, "$MAX_TOOL_RES", strconv.Itoa(cfg.MaxToolRes))
+  if td := toolsDir(cfg); td != "" {
     sp += "\n\nExtra shell tools can be found at " + td
   }
-  if sd := skillsDir(&cfg); sd != "" {
+  if sd := skillsDir(cfg); sd != "" {
     sp += "\n\nSkills may be discovered and invoked from " + sd
   }
+  return sp
+}
+
+func main() {
+  cfg := getCfg(configPath())
+  cfg.ContextWindow = fetchContextWindow(&cfg)
+  sp := buildSystemPrompt(&cfg)
   COL = col(cfg)
   stdin = bufio.NewReader(os.Stdin)
   msgs := []Message{{Role: "system", Content: strp(sp)}}

main_test.go

@@ -268,7 +268,7 @@ func clearBantamEnv(t *testing.T) {
     "BANTAM_API_KEY", "BANTAM_STREAM", "BANTAM_COLOR", "BANTAM_NO_COLOR",
     "BANTAM_TIMEOUT", "BANTAM_SHELL_TIMEOUT", "BANTAM_MAX_AL_ITERATIONS",
     "BANTAM_CONTEXT_WINDOW", "BANTAM_REASONING_EFFORT", "BANTAM_TOOLS_DIR",
-    "BANTAM_SKILLS_DIR",
+    "BANTAM_SKILLS_DIR", "BANTAM_MAX_TOOL_RES",
   } {
     t.Setenv(k, "")
   }
@@ -2945,3 +2945,207 @@ func TestLLMContextTokensIncludesReasoning(t *testing.T) {
     t.Errorf("ctxTokens=%d should equal ContextTokens=%d when prompt_tokens is absent", ctxTokens(u), u.ContextTokens)
   }
 }
+
+
+// ---------- context window overflow prevention (max_tool_res) ----------
+
+func TestMaxToolResDefault(t *testing.T) {
+  clearBantamEnv(t)
+  cfg := getCfg(filepath.Join(t.TempDir(), "missing.cfg"))
+  if cfg.MaxToolRes != 15000 {
+    t.Errorf("default MaxToolRes = %d, want 15000", cfg.MaxToolRes)
+  }
+  if cfg.Raw["max_tool_res"] != "15000" {
+    t.Errorf("default Raw[max_tool_res] = %q, want 15000", cfg.Raw["max_tool_res"])
+  }
+}
+
+func TestMaxToolResFileParse(t *testing.T) {
+  clearBantamEnv(t)
+  p := writeCfg(t, "max_tool_res=4096\n")
+  if got := getCfg(p).MaxToolRes; got != 4096 {
+    t.Errorf("max_tool_res from file = %d, want 4096", got)
+  }
+}
+
+func TestMaxToolResEnvFallback(t *testing.T) {
+  clearBantamEnv(t)
+  t.Setenv("BANTAM_MAX_TOOL_RES", "2048")
+  if got := getCfg(filepath.Join(t.TempDir(), "missing.cfg")).MaxToolRes; got != 2048 {
+    t.Errorf("max_tool_res from env = %d, want 2048", got)
+  }
+}
+
+// The .bantam.cfg file takes precedence over the env var (as for every other
+// tunable), matching the precedence rule stated for max_tool_res.
+func TestMaxToolResFileOverridesEnv(t *testing.T) {
+  clearBantamEnv(t)
+  t.Setenv("BANTAM_MAX_TOOL_RES", "1111")
+  p := filepath.Join(t.TempDir(), ".bantam.cfg")
+  if err := os.WriteFile(p, []byte("max_tool_res=2222\n"), 0644); err != nil {
+    t.Fatal(err)
+  }
+  if got := getCfg(p).MaxToolRes; got != 2222 {
+    t.Errorf("file should override env: got %d, want 2222", got)
+  }
+}
+
+func TestMaxToolResIsInternalKey(t *testing.T) {
+  if !internalKey("max_tool_res") {
+    t.Errorf("internalKey(\"max_tool_res\") = false, want true (must not be forwarded to the API)")
+  }
+}
+
+func TestToolResDir(t *testing.T) {
+  t.Setenv("TMPDIR", "/custom/tmp")
+  if got := toolResDir(); got != filepath.Join("/custom/tmp", "bantam", "toolres") {
+    t.Errorf("toolResDir = %q, want /custom/tmp/bantam/toolres", got)
+  }
+  t.Setenv("TMPDIR", "")
+  if got := toolResDir(); got != filepath.Join("/tmp", "bantam", "toolres") {
+    t.Errorf("toolResDir (no TMPDIR) = %q, want /tmp/bantam/toolres", got)
+  }
+}
+
+// Small results pass through untouched and create no files.
+func TestOffloadToolResultSmallPassthrough(t *testing.T) {
+  t.Setenv("TMPDIR", t.TempDir())
+  var tmps []string
+  res := "hello\n\nexit: 0"
+  if got := offloadToolResult(res, 15000, &tmps); got != res {
+    t.Errorf("small result altered: got %q, want %q", got, res)
+  }
+  if len(tmps) != 0 {
+    t.Errorf("small result should not create temp files, got %v", tmps)
+  }
+}
+
+// Large results are spilled to $TMPDIR/bantam/toolres and replaced by an
+// instruction that names the file and its total length; cleanupTemps then
+// removes the file.
+func TestOffloadToolResultLargeSpillsAndCleans(t *testing.T) {
+  tmp := t.TempDir()
+  t.Setenv("TMPDIR", tmp)
+  var tmps []string
+  big := strings.Repeat("x", 20000)
+  got := offloadToolResult(big, 15000, &tmps)
+  if len(tmps) != 1 {
+    t.Fatalf("expected 1 temp file, got %d (%v)", len(tmps), tmps)
+  }
+  path := tmps[0]
+  wantDir := filepath.Join(tmp, "bantam", "toolres")
+  if filepath.Dir(path) != wantDir {
+    t.Errorf("temp file dir = %q, want %q", filepath.Dir(path), wantDir)
+  }
+  // The full result must be recoverable from the file.
+  data, err := os.ReadFile(path)
+  if err != nil {
+    t.Fatalf("read spilled file: %v", err)
+  }
+  if string(data) != big {
+    t.Errorf("spilled file content mismatch (len %d vs %d)", len(data), len(big))
+  }
+  // The replacement must tell the model the path and the byte length.
+  if strings.Contains(got, big) {
+    t.Errorf("replacement still contains the full result")
+  }
+  if !strings.Contains(got, path) {
+    t.Errorf("replacement does not reference the file path: %q", got)
+  }
+  if !strings.Contains(got, "20000") {
+    t.Errorf("replacement does not state the file length: %q", got)
+  }
+  cleanupTemps(tmps)
+  if _, err := os.Stat(path); !os.IsNotExist(err) {
+    t.Errorf("cleanupTemps did not remove %q (err=%v)", path, err)
+  }
+}
+
+// A non-positive limit must fall back to the 15000 default rather than
+// spilling every non-empty result.
+func TestOffloadToolResultZeroLimitUsesDefault(t *testing.T) {
+  t.Setenv("TMPDIR", t.TempDir())
+  var tmps []string
+  res := strings.Repeat("y", 15000) // exactly the default limit: stays inline
+  if got := offloadToolResult(res, 0, &tmps); got != res {
+    t.Errorf("result at default limit should stay inline")
+  }
+  if len(tmps) != 0 {
+    t.Errorf("no spill expected at the limit, got %v", tmps)
+  }
+}
+
+// End-to-end: AL() offloads an oversized shell result, hands the model the
+// instruction instead of the raw bytes, and deletes the spilled file once the
+// round finishes.
+func TestALOffloadsLargeToolResultAndCleans(t *testing.T) {
+  tmp := t.TempDir()
+  t.Setenv("TMPDIR", tmp)
+  seen := ""
+  srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+    var req struct {
+      Messages []Message `json:"messages"`
+    }
+    json.NewDecoder(r.Body).Decode(&req)
+    for _, m := range req.Messages {
+      if m.Role == "tool" && m.Content != nil {
+        seen = *m.Content
+        w.Write([]byte(`{"choices":[{"message":{"role":"assistant","content":"done"}}]}`))
+        return
+      }
+    }
+    args, _ := json.Marshal(map[string]any{"command": "yes x | head -c 20000"})
+    w.Write([]byte(fmt.Sprintf(`{"choices":[{"message":{"role":"assistant","content":null,"tool_calls":[{"id":"c1","type":"function","function":{"name":"shell_exec","arguments":%s}}]}}]}`, strconv.Quote(string(args)))))
+  }))
+  defer srv.Close()
+
+  cfg := defCfg
+  cfg.Endpoint = srv.URL
+  cfg.Stream = false
+  cfg.APIKey = "-"
+  cfg.MaxToolRes = 15000
+  _, _, err := AL(context.Background(), &cfg, []Message{{Role: "system", Content: strp("sys")}, {Role: "user", Content: strp("run")}})
+  if err != nil {
+    t.Fatalf("AL: %v", err)
+  }
+  if seen == "" {
+    t.Fatalf("no tool message reached the model")
+  }
+  if len(seen) > cfg.MaxToolRes {
+    t.Errorf("tool message sent to the model is %d bytes, exceeds limit %d", len(seen), cfg.MaxToolRes)
+  }
+  if !strings.Contains(seen, "too large for context") {
+    t.Errorf("expected an overflow instruction, got %q", seen)
+  }
+  // The spilled file must have been deleted when the round ended.
+  dir := filepath.Join(tmp, "bantam", "toolres")
+  entries, err := os.ReadDir(dir)
+  if err != nil && !os.IsNotExist(err) {
+    t.Fatalf("read toolres dir: %v", err)
+  }
+  if len(entries) != 0 {
+    t.Errorf("expected no leftover temp files, found %d", len(entries))
+  }
+}
+
+
+// buildSystemPrompt must substitute the configured max_tool_res value and the
+// concrete temporary directory into the built-in prompt.
+func TestBuildSystemPromptMaxToolResHint(t *testing.T) {
+  t.Setenv("TMPDIR", "/custom/tmp")
+  cfg := defCfg
+  cfg.MaxToolRes = 12345
+  sp := buildSystemPrompt(&cfg)
+  if !strings.Contains(sp, "12345") {
+    t.Errorf("prompt should embed the configured max_tool_res (12345): %q", sp)
+  }
+  if strings.Contains(sp, "$MAX_TOOL_RES") {
+    t.Errorf("prompt still contains the unexpanded $MAX_TOOL_RES placeholder: %q", sp)
+  }
+  if !strings.Contains(sp, filepath.Join("/custom/tmp", "bantam", "toolres")) {
+    t.Errorf("prompt should reference the concrete toolres dir: %q", sp)
+  }
+  if strings.Contains(sp, "$TMPDIR/bantam") {
+    t.Errorf("prompt still contains the unexpanded $TMPDIR/bantam placeholder: %q", sp)
+  }
+}