# Grokkin' Grep (`gg`) GG is a minimal, fast, and **portable** grep-like utility that finds files containing a literal substring. `gg` prints the paths of every file that contains the search term — one path per line — and is designed to be competitive with The Silver Searcher and ripgrep on large source trees while depending on nothing but POSIX and the C standard library. ``` $ gg "public domain" ~/proj /home/user/proj/foo/license.txt /home/user/proj/bar/COPYING ... ``` ## About GG answers one question well: *which files contain this exact text?* It is intentionally narrow: - **Literal, case-sensitive substring search.** No regex, no flags, no surprises. The term you pass is matched byte-for-byte. - **File-finding, not line-finding.** Output is a list of matching file paths, not the matching lines. (Pipe to `grep -n` if you want line numbers.) - **Binary-aware.** Files containing a NUL byte are treated as binary and skipped. A literal match that occurs *before* the first NUL in an otherwise-text file is still reported. - **Recursive by default.** Given a directory it walks the whole tree. - **Git-aware by default.** When walking a tree, `gg` honours every `.gitignore` it encounters and always skips `.git` directories — just like `git` and `rg`. Set `GG_GIT_ALL=1` to search everything instead (equivalent to `rg --no-ignore`). - **Self-contained.** No third-party libraries, no shelling out to other programs. Just `libc` + `libpthread`. `gg` was built to be a clean, readable reference implementation that is also fast: on a 16 GB / ~209 000-file tree it completes the equivalent of `rg --no-ignore --hidden -l` in roughly **0.4 s**, about **1.5× faster than `rg`** on the same machine and query, using only portable POSIX interfaces. (With `GG_GIT_ALL` unset, the equivalent is `rg --hidden -l`.) ## Installation ### From source You need a C compiler (`gcc` or `clang`) and `make`. No other dependencies. ```sh git clone gg && cd gg make # optional, system-wide: sudo make install PREFIX=/usr/local ``` This produces a single executable named `gg`. ### Build options - **Portable build (no x86-specific flags):** ```sh make MARCH="" ``` `gg` uses `memmem()` for matching; glibc (and other libcs) already accelerate that with SIMD on whatever CPU you run on. The default `-mavx2` flag was removed from the Makefile because it is x86-only and unnecessary (matching is done by the libc, not by hand-written SIMD), so the default build is already maximally portable. - **Building on BSDs and macOS:** `gg` builds and runs unchanged on FreeBSD, OpenBSD, NetBSD, DragonFly BSD, and macOS/Darwin — just use the system compiler (`cc`/`clang`): ```sh make # CC defaults to cc on systems without gcc # or directly: cc -O2 -std=c11 -Wall -Wextra -D_POSIX_C_SOURCE=200809L -D_DEFAULT_SOURCE -o gg gg.c -lpthread ``` To keep `gg` portable, the source no longer relies on any single libc's conformance quirks: the `d_type` constants (`DT_DIR`, `DT_REG`, `DT_LNK`, …) have portable fallbacks, `memmem()` is declared/provided where the libc hides it, and the online-CPU count is obtained via `sysconf` on Linux or `sysctlbyname("hw.ncpu")` on the BSDs and macOS. - **Cross-compiling for ARM / other targets:** ```sh aarch64-linux-gnu-gcc -O2 -std=c11 -Wall -Wextra \ -D_POSIX_C_SOURCE=200809L -D_DEFAULT_SOURCE -o gg gg.c -lpthread ``` `gg` contains no SIMD intrinsics and no platform-specific code, so it builds and runs unchanged on ARM64 and other POSIX platforms. ### Uninstall ```sh sudo make uninstall PREFIX=/usr/local ``` ## Usage ``` gg SEARCH_TERM [FILE_OR_DIRECTORY] ``` - `SEARCH_TERM` — required. A case-sensitive literal substring to find. Must be non-empty. - `FILE_OR_DIRECTORY` — optional. Defaults to the current working directory. May be a single regular file or a directory (which is walked recursively). ### Examples ```sh # Find every file under the current directory containing "TODO" gg TODO # Search a specific tree for a license header gg "public domain" ~/proj # Check a single file gg "def main" src/main.py ``` ### Exit codes | Code | Meaning | |------|---------| | `0` | At least one file matched. | | `1` | No files matched. | | `2` | Usage or input error (missing/empty term, bad path, etc.). | ### Behaviour notes - **Symlinks are not followed.** Symbolic links to files or directories are skipped, which prevents infinite loops on symlink cycles. - **Hidden files and directories are searched.** `gg` does not skip hidden paths. It *does* consult `.gitignore` files and skips `.git` directories by default; set `GG_GIT_ALL=1` to disable that and search everything (including ignored and `.git` paths). - **Binary files are skipped.** A file containing a NUL byte is never reported, even if it also contains the search term (except when the match occurs before the first NUL, in which case it is reported — see FAQ). ## How it works `gg` is a small, readable program. The full algorithm, in high-level terms: 1. **Parse arguments.** Read the search term (must be non-empty) and the optional target (a file or directory, defaulting to the current directory). 2. **Parallel walk + scan (the core idea).** Rather than walking the whole tree first and *then* scanning, `gg` overlaps the two. A shared, mutex-protected **directory work-queue** holds directories still to be processed. `N` worker threads (one per online CPU) each: a. **Pop a directory** from the queue. b. **Open it** with `opendir`/`readdir` and iterate its entries. c. For each entry, use `d_type` from `readdir` to classify it cheaply: - `DT_REG` (regular file) → scan it immediately (see step 3). - `DT_DIR` (sub-directory) → **push it onto the work-queue** for another worker to pick up. - `DT_LNK` (symlink) → skip it (no following, no cycles). - `DT_UNKNOWN` → fall back to `lstat` to decide file vs. directory, then act accordingly. d. **Free the directory** and loop back to pop the next one. Because directories are handed out one at a time and sub-directories are re-enqueued, the metadata walk and the data scan proceed concurrently across all cores, and no single deep directory becomes a bottleneck for any one thread. 3. **Scan a file (early binary detection + streaming match).** For each regular file the worker: a. Opens it with `open(O_RDONLY | O_CLOEXEC)`. b. Reads it in **fixed-size chunks** (32 KB) into a small per-thread buffer. Chunks intentionally **overlap by `len(term) - 1` bytes**, so a match that straddles a chunk boundary is never missed. c. On each chunk, scans for a NUL byte with `memchr`: - If a NUL is found, the file is **binary**: `gg` stops reading immediately (no need to scan the rest) and reports nothing for it. - Otherwise it searches the text portion of the chunk for the term with `memmem`. If found, the file is a match and scanning stops. d. If the term is not in this chunk and no NUL was seen, it carries the overlap into the next read and continues. Stopping at the first NUL is what makes `gg` fast on trees full of binary artifacts (archives, object files, images): those files are abandoned after a single tiny read instead of being fully scanned. 4. **Report matches.** The first time any worker finds a match, it records a global "found" flag and prints the file path. Reporting is guarded by a mutex so output stays coherent; because matching files are comparatively rare, this lock is almost never contended. 5. **Finish.** When the work-queue is empty and all workers have exited, `gg` returns `0` if anything matched, otherwise `1`. In short: **per-directory work units + inline file scanning + stop-at-NUL binary skipping + POSIX `memmem` matching**, all parallelised across the online CPUs. No `mmap`, no regex engine, no external processes — just portable syscalls and library calls. ## FAQ ### Why is it called Grokkin' Grep? A playful name for a tool that "groks" your files to find text. `gg` is also nicely short to type. ### Is the search case-sensitive? Yes. `gg` matches the term exactly as given. There is no case-insensitive mode. ### Does it support regular expressions? No. `gg` is a literal substring finder. For regex, use `grep`, `rg`, or `ag`. ### Why does it print file paths instead of matching lines? `gg` is a *file finder* — it answers "which files contain this?". Pipe its output through `xargs grep -n` (or `rg`) if you need the lines and line numbers. ### How is a "binary file" decided, and why skip it? A file is considered binary if it contains a NUL (`\0`) byte. This matches `ripgrep`'s default heuristic. Such files are skipped because text search over them is usually meaningless. If your term appears *before* the first NUL in an otherwise-readable file, `gg` still reports it (it only abandons the file once it reaches the NUL). ### Does it follow symlinks? No. Symbolic links to files or directories are skipped, which makes `gg` safe against symlink cycles and avoids double-counting. ### Does it respect `.gitignore`? Yes, by default. `gg` reads every `.gitignore` found while walking and skips matching files and directories, and it always skips `.git` directories — the same rules `git` itself uses (including `!` negation, `**` globs, and trailing-`/` directory-only patterns). Hidden files and directories are still searched; only `.gitignore`-matched paths and `.git` are excluded. To search everything — ignored files, `.git`, the lot — set the `GG_GIT_ALL` environment variable (to any value) before running `gg`: ```sh GG_GIT_ALL=1 gg "needle" . ``` With `GG_GIT_ALL` set, `gg`'s results match `rg --no-ignore --hidden`. ### Why not `mmap` the files? `mmap` was prototyped and rejected: across hundreds of thousands of files the `munmap` syscall storm dominated, and `read()` into a small per-thread buffer scaled better. `read()` also avoids faulting the entire file into memory. ### Is it really portable / dependency-free? Yes. The only external links are `libc` and `libpthread`. There are no SIMD intrinsics in the source; matching uses `memmem()`, which your libc already accelerates with SIMD on the host CPU. It builds and runs unchanged on x86-64, ARM64, the BSD family (FreeBSD, OpenBSD, NetBSD, DragonFly) and macOS/Darwin — on any POSIX platform with a C11 compiler, `cc`/`gcc`/`clang`, and a `pthread` library. ### How fast is it? On a warm cache over a 16 GB / ~209 000-file tree, `gg "public domain" ~/proj` completes in roughly **0.4 s** — about **1.5× faster** than the equivalent `rg --no-ignore --hidden -l` on the same hardware, using only portable POSIX interfaces. ## Credits Created by Luxferre in 2026, released into the public domain with no warranties.