codeindex

Self-contained, deterministic repo-indexing engine: file walking, language detection, symbol/import extraction (tree-sitter AST with a regex fallback), import resolution, a typed cross-file link-graph, and graph analytics — shipped as a single zero-dependency engine.mjs that consumer tools vendor (copy into their repo) instead of installing.

Designed for downstream tools — agent skills, CLIs, CI gates — that vendor the engine as a single file instead of taking an npm dependency. How it stacks up against universal-ctags, Serena and Graphify: How it compares.

What it does

• Walk a repo deterministically: ignore lists, binary/lockfile skips, a size cap, symlink-cycle guard. No file-count cap unless you ask for one (--max-files), and asking sets the capped flag — never a silent truncation.
• Scan every file into a FileRecord: classification, language, symbols, imports, headings, hashes — with an incremental cache fastpath. Extraction runs across worker threads by default (--workers, CODEINDEX_WORKERS); artifacts are byte-identical either way, and anything that would make a worker's result differ falls back to the single-threaded path.
• Extract symbols via tree-sitter (15 committed grammars, plus 6 more via grammars pull) or per-language regex rules (16 languages, always available). Each symbol carries its complete signature (parameters and return type, not the first physical line), its own doc comment, its qualified parent, and its line span — including the members a declaration-only walk misses: interface members, class fields, enum members, every declare/.d.ts declaration, Rust trait method signatures, Go interface method sets, record components and constructor val parameters.
• Resolve imports across languages: tsconfig paths, package exports, go.mod, Cargo, Java packages, PSR-4, C# namespaces.
• Build a typed link-graph: import / call / extends / implements / use / doc-link / mention edges at file and module level, plus Louvain communities, PageRank/betweenness centrality, a tests→code map, and surprise-edge detection. Inheritance also yields a type hierarchy (what a type extends and implements, and what extends and implements IT) and a symbol-level graph for bounded "what does this reach" neighborhoods.
• Render byte-stable graph.json / symbols.json (two builds of an unchanged repo are byte-identical), plus a SCIP code-intelligence index (index.scip) via a hand-rolled zero-dependency protobuf encoder — validated by the official scip CLI (stats/lint).

Measured against other indexers

"Finds better" is a claim, so the checks that count are the ones this project did not author. Four oracles score extraction against outside authorities — a real compiler, a mature indexer, and the grammars' own published queries and vocabulary:

oracle	what makes it independent	result
TypeScript compiler index (scip-typescript 0.4.0)	an index built by the real TypeScript compiler — authoritative where every other check here is syntactic	100% of its 93 named declarations, against ctags' 94.6% on the same files
universal-ctags differential (Universal Ctags 6.2.1)	an independent, mature indexer covering ~40 languages	reports 2,014 declarations ctags does not over 6 real repositories, and reproduces 61.7%–98.8% of ctags' names — what is left bucketed by kind, per repo below
Official tags.scm queries	the code-navigation patterns each grammar's own authors publish, and GitHub uses	1 adjudicated difference, over the 14 of 17 languages that publish one
Grammar vocabulary	each tree-sitter grammar's own declared node types, read at runtime from the parser	21 grammar