brep -> gg

README.md

@@ -0,0 +1,242 @@
+# Grokkin' Grep (`gg`)
+
+GG is a minimal, fast, and **portable** grep-like utility that finds files
+containing a literal substring. `gg` prints the paths of every file that contains
+the search term — one path per line — and is designed to be competitive with
+The Silver Searcher and ripgrep on large source trees while depending on nothing
+but POSIX and the C standard library.
+
+```
+$ gg "public domain" ~/proj
+/home/user/proj/foo/license.txt
+/home/user/proj/bar/COPYING
+...
+```
+
+## About
+
+GG answers one question well: *which files contain this exact text?*
+It is intentionally narrow:
+
+- **Literal, case-sensitive substring search.** No regex, no flags, no
+  surprises. The term you pass is matched byte-for-byte.
+- **File-finding, not line-finding.** Output is a list of matching file paths,
+  not the matching lines. (Pipe to `grep -n` if you want line numbers.)
+- **Binary-aware.** Files containing a NUL byte are treated as binary and
+  skipped. A literal match that occurs *before* the first NUL in an
+  otherwise-text file is still reported.
+- **Recursive by default.** Given a directory it walks the whole tree.
+- **Self-contained.** No third-party libraries, no shelling out to other
+  programs. Just `libc` + `libpthread`.
+
+`gg` was built to be a clean, readable reference implementation that is also
+fast: on a 16 GB / ~209 000-file tree it completes the equivalent of
+`rg --no-ignore --hidden -l` in roughly **0.4 s**, about **1.5× faster than
+`rg`** on the same machine and query, using only portable POSIX interfaces.
+
+## Installation
+
+### From source
+
+You need a C compiler (`gcc` or `clang`) and `make`. No other dependencies.
+
+```sh
+git clone <repo-url> gg && cd gg
+make
+# optional, system-wide:
+sudo make install PREFIX=/usr/local
+```
+
+This produces a single executable named `gg`.
+
+### Build options
+
+- **Portable build (no x86-specific flags):**
+
+  ```sh
+  make MARCH=""
+  ```
+
+  `gg` uses POSIX `memmem()` for matching; glibc (and other libcs) already
+  accelerate that with SIMD on whatever CPU you run on. The default `-mavx2`
+  flag is therefore optional and can be dropped for maximum portability.
+
+- **Cross-compiling for ARM / other targets:**
+
+  ```sh
+  aarch64-linux-gnu-gcc -O2 -std=c11 -Wall -Wextra \
+    -D_POSIX_C_SOURCE=200809L -D_DEFAULT_SOURCE -o gg gg.c -lpthread
+  ```
+
+  `gg` contains no SIMD intrinsics and no platform-specific code, so it builds
+  and runs unchanged on ARM64 and other POSIX platforms.
+
+### Uninstall
+
+```sh
+sudo make uninstall PREFIX=/usr/local
+```
+
+## Usage
+
+```
+gg SEARCH_TERM [FILE_OR_DIRECTORY]
+```
+
+- `SEARCH_TERM` — required. A case-sensitive literal substring to find. Must be
+  non-empty.
+- `FILE_OR_DIRECTORY` — optional. Defaults to the current working directory.
+  May be a single regular file or a directory (which is walked recursively).
+
+### Examples
+
+```sh
+# Find every file under the current directory containing "TODO"
+gg TODO
+
+# Search a specific tree for a license header
+gg "public domain" ~/proj
+
+# Check a single file
+gg "def main" src/main.py
+```
+
+### Exit codes
+
+| Code | Meaning |
+|------|---------|
+| `0`  | At least one file matched. |
+| `1`  | No files matched. |
+| `2`  | Usage or input error (missing/empty term, bad path, etc.). |
+
+### Behaviour notes
+
+- **Symlinks are not followed.** Symbolic links to files or directories are
+  skipped, which prevents infinite loops on symlink cycles.
+- **Hidden files and directories are searched.** Unlike `rg`'s default, `gg`
+  does not consult `.gitignore` or skip hidden paths.
+- **Binary files are skipped.** A file containing a NUL byte is never reported,
+  even if it also contains the search term (except when the match occurs before
+  the first NUL, in which case it is reported — see FAQ).
+
+## How it works
+
+`gg` is a small, readable program. The full algorithm, in high-level terms:
+
+1. **Parse arguments.** Read the search term (must be non-empty) and the optional
+   target (a file or directory, defaulting to the current directory).
+
+2. **Parallel walk + scan (the core idea).** Rather than walking the whole tree
+   first and *then* scanning, `gg` overlaps the two. A shared, mutex-protected
+   **directory work-queue** holds directories still to be processed. `N` worker
+   threads (one per online CPU) each:
+
+   a. **Pop a directory** from the queue.
+   b. **Open it** with `opendir`/`readdir` and iterate its entries.
+   c. For each entry, use `d_type` from `readdir` to classify it cheaply:
+      - `DT_REG` (regular file) → scan it immediately (see step 3).
+      - `DT_DIR` (sub-directory) → **push it onto the work-queue** for another
+        worker to pick up.
+      - `DT_LNK` (symlink) → skip it (no following, no cycles).
+      - `DT_UNKNOWN` → fall back to `lstat` to decide file vs. directory, then
+        act accordingly.
+   d. **Free the directory** and loop back to pop the next one.
+
+   Because directories are handed out one at a time and sub-directories are
+   re-enqueued, the metadata walk and the data scan proceed concurrently across
+   all cores, and no single deep directory becomes a bottleneck for any one
+   thread.
+
+3. **Scan a file (early binary detection + streaming match).** For each regular
+   file the worker:
+
+   a. Opens it with `open(O_RDONLY | O_CLOEXEC)`.
+   b. Reads it in **fixed-size chunks** (32 KB) into a small per-thread buffer.
+      Chunks intentionally **overlap by `len(term) - 1` bytes**, so a match that
+      straddles a chunk boundary is never missed.
+   c. On each chunk, scans for a NUL byte with `memchr`:
+      - If a NUL is found, the file is **binary**: `gg` stops reading
+        immediately (no need to scan the rest) and reports nothing for it.
+      - Otherwise it searches the text portion of the chunk for the term with
+        `memmem`. If found, the file is a match and scanning stops.
+   d. If the term is not in this chunk and no NUL was seen, it carries the
+      overlap into the next read and continues.
+
+   Stopping at the first NUL is what makes `gg` fast on trees full of binary
+   artifacts (archives, object files, images): those files are abandoned after a
+   single tiny read instead of being fully scanned.
+
+4. **Report matches.** The first time any worker finds a match, it records a
+   global "found" flag and prints the file path. Reporting is guarded by a mutex
+   so output stays coherent; because matching files are comparatively rare, this
+   lock is almost never contended.
+
+5. **Finish.** When the work-queue is empty and all workers have exited, `gg`
+   returns `0` if anything matched, otherwise `1`.
+
+In short: **per-directory work units + inline file scanning + stop-at-NUL binary
+skipping + POSIX `memmem` matching**, all parallelised across the online CPUs.
+No `mmap`, no regex engine, no external processes — just portable syscalls and
+library calls.
+
+## FAQ
+
+### Why is it called Grokkin' Grep?
+
+A playful name for a tool that "groks" your files to find text. `gg` is also
+nicely short to type.
+
+### Is the search case-sensitive?
+
+Yes. `gg` matches the term exactly as given. There is no case-insensitive mode.
+
+### Does it support regular expressions?
+
+No. `gg` is a literal substring finder. For regex, use `grep`, `rg`, or `ag`.
+
+### Why does it print file paths instead of matching lines?
+
+`gg` is a *file finder* — it answers "which files contain this?". Pipe its output
+through `xargs grep -n` (or `rg`) if you need the lines and line numbers.
+
+### How is a "binary file" decided, and why skip it?
+
+A file is considered binary if it contains a NUL (`\0`) byte. This matches
+`ripgrep`'s default heuristic. Such files are skipped because text search over
+them is usually meaningless. If your term appears *before* the first NUL in an
+otherwise-readable file, `gg` still reports it (it only abandons the file once it
+reaches the NUL).
+
+### Does it follow symlinks?
+
+No. Symbolic links to files or directories are skipped, which makes `gg` safe
+against symlink cycles and avoids double-counting.
+
+### Does it respect `.gitignore`?
+
+No. `gg` searches everything it can read, including hidden files and directories.
+This is why its results match `rg --no-ignore --hidden`.
+
+### Why not `mmap` the files?
+
+`mmap` was prototyped and rejected: across hundreds of thousands of files the
+`munmap` syscall storm dominated, and `read()` into a small per-thread buffer
+scaled better. `read()` also avoids faulting the entire file into memory.
+
+### Is it really portable / dependency-free?
+
+Yes. The only external links are `libc` and `libpthread`. There are no SIMD
+intrinsics in the source; matching uses `memmem()`, which your libc already
+accelerates with SIMD on the host CPU. It builds and runs on x86-64 and ARM64
+alike.
+
+### How fast is it?
+
+On a warm cache over a 16 GB / ~209 000-file tree, `gg "public domain" ~/proj`
+completes in roughly **0.4 s** — about **1.5× faster** than the equivalent
+`rg --no-ignore --hidden -l` on the same hardware, using only portable POSIX
+interfaces.
+
+## Credits
+
+Created by Luxferre in 2026, released into the public domain with no warranties.