fix(scanner): index dot-directories instead of skipping them by default

[nicknsheth-beep]

Problem

codemap --deps / --importers never index any file under a directory whose name starts with . (other than .git), with no config option to opt in. A project that keeps source under .agents/, .config/, etc. is silently invisible to codemap.

Repro

repro/
├── .git/
└── .agents/
    ├── a.js   # const b = require('./b');
    └── b.js

codemap --deps .
# before: "No source files found."

The identical files in a non-dot directory index correctly.

Root cause

The scanner shells out to ast-grep scan, which skips hidden files/dirs by default (--no-ignore hidden is ast-grep's own opt-out flag). codemap never passed that flag and had no config knob for it.

Fix

scanner/astgrep.go: pass --no-ignore hidden to ast-grep, paired with explicit --globs excludes for codemap's existing IgnoredDirs (.git, node_modules, vendor, testdata, build, dist, target, .venv, .idea, .next, ...) plus .codemap itself — so relaxing the hidden-file default doesn't start indexing .git internals or slow down large repos.

Second-order bug I found while testing this: --globs match the literal path handed to ast-grep with no notion of where the scan root begins, so a bare **/testdata/** exclude also matches when testdata is an ancestor of the scan root — exactly how this repo's own scanner tests are laid out (fixtures under ../testdata/...). Fixed by running ast-grep with cmd.Dir = root and "." as the scan path, so every walked path is root-relative. The existing filepath.Rel(root, m.File) fallback logic in astgrep.go already handles ast-grep now reporting root-relative paths, so nothing else needed to change.

Tests

scanner/astgrep_test.go:

• TestScanDirectoryInvokesAstGrepWithHiddenDirsEnabled — locks in the exact args and cmd.Dir, via a fake sg script
• TestScanDirectoryIndexesHiddenDirectories — end-to-end: a file under .agents/ indexes and its importer resolves
• TestScanDirectoryDoesNotExcludeRootsNestedUnderAnIgnoredDirName — regression test for the ancestor-name false-exclusion bug above
• TestScanDirectoryStillExcludesKnownNeverSourceDirs — .git, node_modules, vendor stay excluded with hidden-dir scanning on

go test ./... passes (all 17 packages), gofmt -l . and go vet ./... are clean.

Related: filing alongside two other independent fixes found during the same investigation (-C scoping, and coverage-honesty for unindexed files) — happy to merge in any order, this one has no dependency on the others.

[JordanCoin]

Thanks for this — a clear write-up, and the ancestor-name bug you found in your own first draft (**/testdata/** matching a testdata above the scan root) is exactly the kind of second-order problem that's easy to miss. Fixing it with cmd.Dir = root + "." rather than patching the globs is the right call.

What I ran

CI hasn't run on this PR (GitHub holds workflows from first-time fork contributors until a maintainer approves them), so I verified locally rather than assuming:

• Your four new tests with a real ast-grep installed. Three of them t.Skip when ast-grep is absent, so a plain go test on a machine without it shows green while proving nothing. I ran them against both the pinned 0.42.1 (what the release bundles, via scripts/download-bundled-astgrep.sh) and the latest 0.45.3 (what CI installs). All four pass on both.
• Full go test ./...: only the three known root-permission failures that also fail on main. gofmt, go vet and staticcheck (the three things CI's Lint job runs) are clean.
• Merges cleanly with fix(cmd): -C operates on the given directory, not the Git root #194 and fix(scanner): distinguish never-indexed from zero-importers, and resolve gitignored import targets #195; the three combined build and sit at the same baseline.
• Cost: 9,000 generated files under .svelte-kit/, .vercel/ and .turbo/ added ~0.3s (0.09s → 0.41s), with the answer and Coverage: complete unchanged.

One observation (not a blocker)

This is an opt-out model: anything not in IgnoredDirs is now scanned. In a scratch repo, an un-gitignored .cache/tmp.js doing require('../src/util') now shows up as an importer of src/util.js. That edge is real — the file does import it — but it's a cache directory appearing in a dependency answer. In practice ast-grep still honors .gitignore (only the hidden rule was lifted), and most tool caches are gitignored, so I'd expect this to be rare. Worth a line in the release notes that dot-directories are now scanned, since it's a default-behavior change.

Not yet verified

The macOS and Windows legs. TestScanDirectoryInvokesAstGrepWithHiddenDirsEnabled skips on Windows (it needs a shell script), so the --globs and cmd.Dir behavior there is covered only by the real-ast-grep tests once CI runs.

From my side this looks good to land once CI is approved and green.

---

Generated by Claude Code