gg/README.md

Grokkin' Grep (gg)

GG is a minimal, fast, and portable grep-like utility that finds files containing a literal substring. gg prints the paths of every file that contains the search term — one path per line — and is designed to be competitive with The Silver Searcher and ripgrep on large source trees while depending on nothing but POSIX and the C standard library.

$ gg "public domain" ~/proj
/home/user/proj/foo/license.txt
/home/user/proj/bar/COPYING
...

About

GG answers one question well: which files contain this exact text? It is intentionally narrow:

• Literal, case-sensitive substring search. No regex, no flags, no surprises. The term you pass is matched byte-for-byte.
• File-finding, not line-finding. Output is a list of matching file paths, not the matching lines. (Pipe to grep -n if you want line numbers.)
• Binary-aware. Files containing a NUL byte are treated as binary and skipped. A literal match that occurs before the first NUL in an otherwise-text file is still reported.
• Recursive by default. Given a directory it walks the whole tree.
• Git-aware by default. When walking a tree, gg honours every .gitignore it encounters and always skips .git directories — just like git and rg. Set GG_GIT_ALL=1 to search everything instead (equivalent to rg --no-ignore).
• Self-contained. No third-party libraries, no shelling out to other programs. Just libc + libpthread.

gg was built to be a clean, readable reference implementation that is also fast: on a 16 GB / ~209 000-file tree it completes the equivalent of rg --no-ignore --hidden -l in roughly 0.4 s, about 1.5× faster than rg on the same machine and query, using only portable POSIX interfaces. (With GG_GIT_ALL unset, the equivalent is rg --hidden -l.)

Installation

From source

You need a C compiler (gcc or clang) and make. No other dependencies.

git clone <repo-url> gg && cd gg
make
# optional, system-wide:
sudo make install PREFIX=/usr/local

This produces a single executable named gg.

Build options

• Portable build (no x86-specific flags):

  make MARCH=""
  

  gg uses POSIX memmem() for matching; glibc (and other libcs) already accelerate that with SIMD on whatever CPU you run on. The default -mavx2 flag is therefore optional and can be dropped for maximum portability.

• Cross-compiling for ARM / other targets:

  aarch64-linux-gnu-gcc -O2 -std=c11 -Wall -Wextra \
    -D_POSIX_C_SOURCE=200809L -D_DEFAULT_SOURCE -o gg gg.c -lpthread
  

  gg contains no SIMD intrinsics and no platform-specific code, so it builds and runs unchanged on ARM64 and other POSIX platforms.

Uninstall

sudo make uninstall PREFIX=/usr/local

Usage

gg SEARCH_TERM [FILE_OR_DIRECTORY]

• SEARCH_TERM — required. A case-sensitive literal substring to find. Must be non-empty.
• FILE_OR_DIRECTORY — optional. Defaults to the current working directory. May be a single regular file or a directory (which is walked recursively).

Examples

# Find every file under the current directory containing "TODO"
gg TODO

# Search a specific tree for a license header
gg "public domain" ~/proj

# Check a single file
gg "def main" src/main.py

Exit codes

Code	Meaning
0	At least one file matched.
1	No files matched.
2	Usage or input error (missing/empty term, bad path, etc.).

Behaviour notes

• Symlinks are not followed. Symbolic links to files or directories are skipped, which prevents infinite loops on symlink cycles.
• Hidden files and directories are searched. gg does not skip hidden paths. It does consult .gitignore files and skips .git directories by default; set GG_GIT_ALL=1 to disable that and search everything (including ignored and .git paths).
• Binary files are skipped. A file containing a NUL byte is never reported, even if it also contains the search term (except when the match occurs before the first NUL, in which case it is reported — see FAQ).

How it works

gg is a small, readable program. The full algorithm, in high-level terms:

1. Parse arguments. Read the search term (must be non-empty) and the optional target (a file or directory, defaulting to the current directory).

2. Parallel walk + scan (the core idea). Rather than walking the whole tree first and then scanning, gg overlaps the two. A shared, mutex-protected directory work-queue holds directories still to be processed. N worker threads (one per online CPU) each:

   a. Pop a directory from the queue. b. Open it with opendir/readdir and iterate its entries. c. For each entry, use d_type from readdir to classify it cheaply:

   • DT_REG (regular file) → scan it immediately (see step 3).
   • DT_DIR (sub-directory) → push it onto the work-queue for another worker to pick up.
   • DT_LNK (symlink) → skip it (no following, no cycles).
   • DT_UNKNOWN → fall back to lstat to decide file vs. directory, then act accordingly. d. Free the directory and loop back to pop the next one.

   Because directories are handed out one at a time and sub-directories are re-enqueued, the metadata walk and the data scan proceed concurrently across all cores, and no single deep directory becomes a bottleneck for any one thread.

3. Scan a file (early binary detection + streaming match). For each regular file the worker:

   a. Opens it with open(O_RDONLY | O_CLOEXEC). b. Reads it in fixed-size chunks (32 KB) into a small per-thread buffer. Chunks intentionally overlap by len(term) - 1 bytes, so a match that straddles a chunk boundary is never missed. c. On each chunk, scans for a NUL byte with memchr:

   • If a NUL is found, the file is binary: gg stops reading immediately (no need to scan the rest) and reports nothing for it.
   • Otherwise it searches the text portion of the chunk for the term with memmem. If found, the file is a match and scanning stops. d. If the term is not in this chunk and no NUL was seen, it carries the overlap into the next read and continues.

   Stopping at the first NUL is what makes gg fast on trees full of binary artifacts (archives, object files, images): those files are abandoned after a single tiny read instead of being fully scanned.

4. Report matches. The first time any worker finds a match, it records a global "found" flag and prints the file path. Reporting is guarded by a mutex so output stays coherent; because matching files are comparatively rare, this lock is almost never contended.

5. Finish. When the work-queue is empty and all workers have exited, gg returns 0 if anything matched, otherwise 1.

In short: per-directory work units + inline file scanning + stop-at-NUL binary skipping + POSIX memmem matching, all parallelised across the online CPUs. No mmap, no regex engine, no external processes — just portable syscalls and library calls.

FAQ

Why is it called Grokkin' Grep?

A playful name for a tool that "groks" your files to find text. gg is also nicely short to type.

Is the search case-sensitive?

Yes. gg matches the term exactly as given. There is no case-insensitive mode.

Does it support regular expressions?

No. gg is a literal substring finder. For regex, use grep, rg, or ag.

Why does it print file paths instead of matching lines?

gg is a file finder — it answers "which files contain this?". Pipe its output through xargs grep -n (or rg) if you need the lines and line numbers.

How is a "binary file" decided, and why skip it?

A file is considered binary if it contains a NUL (\0) byte. This matches ripgrep's default heuristic. Such files are skipped because text search over them is usually meaningless. If your term appears before the first NUL in an otherwise-readable file, gg still reports it (it only abandons the file once it reaches the NUL).

Does it follow symlinks?

No. Symbolic links to files or directories are skipped, which makes gg safe against symlink cycles and avoids double-counting.

Does it respect .gitignore?

Yes, by default. gg reads every .gitignore found while walking and skips matching files and directories, and it always skips .git directories — the same rules git itself uses (including ! negation, ** globs, and trailing-/ directory-only patterns). Hidden files and directories are still searched; only .gitignore-matched paths and .git are excluded.

To search everything — ignored files, .git, the lot — set the GG_GIT_ALL environment variable (to any value) before running gg:

GG_GIT_ALL=1 gg "needle" .

With GG_GIT_ALL set, gg's results match rg --no-ignore --hidden.

Why not mmap the files?

mmap was prototyped and rejected: across hundreds of thousands of files the munmap syscall storm dominated, and read() into a small per-thread buffer scaled better. read() also avoids faulting the entire file into memory.

Is it really portable / dependency-free?

Yes. The only external links are libc and libpthread. There are no SIMD intrinsics in the source; matching uses memmem(), which your libc already accelerates with SIMD on the host CPU. It builds and runs on x86-64 and ARM64 alike.

How fast is it?

On a warm cache over a 16 GB / ~209 000-file tree, gg "public domain" ~/proj completes in roughly 0.4 s — about 1.5× faster than the equivalent rg --no-ignore --hidden -l on the same hardware, using only portable POSIX interfaces.

Credits

Created by Luxferre in 2026, released into the public domain with no warranties.