What nci index actually does
Indexing is one fixed pipeline driven by nci index. Stage names below match the engine’s internal modules (scanner, filter, parser, crawler, resolver, dedupe, storage). The CLI is the public surface — the modules are documented for orientation, not as configuration knobs.
The shape
A grep over node_modules returns lines. NCI returns symbols — merged graph rows with signature, js_doc, kind_name, parent_symbol_id, source_package_name, is_internal, plus a stable id. That difference is what makes the index queryable in milliseconds and what lets agents call nci query without re-parsing.
Scanner — discover install roots
project_root plus each workspaces entry — every directory that contributes a node_modules.
One package row per discovered install: name, version, install path, source dir, manifest path.
The configuration that drives this stage is project_root, workspaces, and index_root_workspace in nci.config.json — overridable with -r, --skip-root-workspace, and --include-root-workspace.
Filter — apply package_scope
Every discovered install.
The packages whose names appear in the union of every consumer manifest’s dependencies / dev_dependencies — or every install when package_scope is "all_installed".
--package-scope dependencies,dev-dependencies matches package_scope: ["dependencies", "dev_dependencies"]. packages.include / packages.exclude globs filter on top.
Parser + Crawler — read .d.ts
Each filtered package’s entry .d.ts resolved from exports / types / typings, falling back to index.d.ts.
Parsed declarations + parsed imports for every reached file.
Parsing produces declarations; the crawler decides which other files to fetch. Splitting them lets --max-hops change without re-parsing already-loaded files.
Resolver — promote re-exports
The parsed declarations plus the per-package module graph.
Every public symbol promoted to its terminal declaration. import X from "./x"; export { X } becomes X, never a placeholder. Dependency edges follow nci-dep-v1.
Result10 → default; covers every common library, including barrel-heavy ones.
Dedupe + Storage — write to SQLite
Resolved declarations from every package.
Rows in packages, symbols, symbol_dependencies, plus FTS5 (symbols_fts). Each symbols row gets signature, kind_name, parent_symbol_id, and either fused or distinct overloads, recorded in merge_provenance_json.
Everything lands in one nci.sqlite file under the OS cache dir (override with NCI_CACHE_DIR). nci query and nci sql both read this file with no further parsing — that is why responses are sub-millisecond once the index exists.
Re-running
--dry-run walks scan + filter only and stops. Use it as a CI sanity gate: if the count drops, your package_scope or filters regressed.
With progress on (see Configuration), each indexed package prints in … — per-package wall time from start through persist (queue wait included). Use --index-timing-detail to add crawl / wait / save splits. The closing summary includes wall for the whole nci index run.
Packages whose index_cache_key still matches are skipped (no re-crawl). Pending symbol backfills run on the writer connection before those cache probes.
--package accepts globs and is repeatable. Combine with --dependency-stub-package to skip walking into noisy SDKs — every consumer import of those packages collapses into an npm::… stub edge.