# How to Index a Large Codebase for Semantic Code Search Entirely Locally > A hands-on guide to pointing an AI agent at a huge monorepo when grep stops being enough. How Octocode chunks code with tree-sitter, runs local embeddings with no GPU and no cloud, fuses semantic and keyword search, and exposes it all over MCP — plus the gotchas that bit us. The repo was about 11,000 files. The agent had a `bash` tool and a vague instruction: "find where we validate webhook signatures." It ran `grep -rn "signature" .`, got 600 hits across vendored JS, test fixtures, a `CHANGELOG`, and three unrelated crypto helpers, then confidently pointed at the wrong file. We watched it do this four times in a row with different phrasings before someone muttered the obvious thing: grep matches strings, and the agent was asking a question about _meaning_. That's the whole problem in one sentence. `grep` and `rg` are excellent at "where does this exact token appear." They are useless at "where is the thing that does X," because the code that does X almost never contains the words you'd use to describe X. The function is called `verify_hmac`. You searched for "webhook signature." Zero overlap, and the one tool the agent reaches for first returns nothing useful. The fix is semantic search: embed the code, embed the query, find the chunks whose meaning is closest. The catch is that most "just use embeddings" advice assumes you'll ship your source to a cloud API and rent a GPU. For a private monorepo, neither is acceptable — legal won't sign off on uploading proprietary code to a third party, and you don't want a per-token bill that scales with every reindex. So this is the setup we actually run: [Octocode](https://github.com/Muvon/octocode), indexing a large repo for semantic search **entirely locally** — no GPU, no cloud embeddings, nothing leaving the machine. Here's exactly how it works, the config that matters, and the four things that bit us. --- ## Why grep isn't enough, concretely Before reaching for embeddings, it's worth being precise about _why_ text search fails for agents, because it tells you what the replacement has to do. - **Vocabulary mismatch.** The query is intent ("rate limiting"), the code is mechanism (`Semaphore`, `token_bucket`, `RetryAfter`). No shared tokens. - **No ranking.** `rg` gives you every match with equal weight. 600 hits is not an answer; it's a second search problem. - **No structure.** A text match in a comment, a test, and a core function all look identical to `grep`. It can't tell you the third one is the definition. What you want instead is: rank by _semantic_ closeness, but don't lose the thing `grep` is genuinely good at — exact identifier matches. Octocode's answer is to do both and fuse them, which I'll get to. First, how the code gets into a searchable form at all. --- ## How Octocode chunks and indexes code Standard RAG slices files into fixed-size text windows — every N characters, overlap a bit, embed. For prose that's fine. For code it's actively bad: it splits a function across two chunks, embeds half a `match` arm, and loses the fact that a method belongs to a type. Octocode parses each file with **tree-sitter** first, walks the AST, and chunks along real symbol boundaries — functions, methods, classes, modules — instead of arbitrary byte offsets. The architecture is a parser per language extracting functions, classes, imports and exports, then chunk-based processing for anything too large to embed in one piece. A method pulled from inside an `impl` block or a class carries its parent type's name in its symbol list, so searching for `Suppression.mark_set` or `Foo.bar` hits the method chunk directly. The language coverage is real tree-sitter grammars, not regex heuristics. From the repo, indexed as **code** with full AST parsing: | Language | Extensions | | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | Rust | `.rs` | | Python | `.py` | | TypeScript / JavaScript | `.ts`, `.tsx`, `.js`, `.jsx` | | Go | `.go` | | PHP | `.php` | | C / C++ | `.c`, `.h`, `.cpp`, `.hpp`, `.cc`, `.cxx`, plus C++20 module extensions `.cppm`, `.ixx`, `.mxx`, `.ccm`, `.cxxm` | | Ruby | `.rb` | | Java | `.java` | | Swift | `.swift` | | JSON, Bash, CSS, Lua, Svelte, Markdown | dedicated tree-sitter paths | Everything else useful for context — `yaml`, `toml`, `dockerfile`, `makefile`, `ini`, `conf`, `env`, `xml`, `html`, `sql`, `rst`, and friends — is indexed as **text blocks** so it's still searchable, just without semantic symbol extraction. The chunking knobs live under `[index]`: ```toml [index] chunk_size = 2000 # max characters per code chunk chunk_overlap = 100 # overlap between adjacent chunks quantization = true # RaBitQ vector compression, ~32x, minimal quality loss require_git = true # only index inside a git repo by default ``` `chunk_size` is a ceiling, not a target — a 40-line function smaller than the limit stays whole. The limit only kicks in for genuinely large bodies, and Octocode already indexes large classes in Python, TypeScript, C++, and Ruby method-by-method rather than dumping the whole class into one chunk. The point of all this: when you later search for "the function that handles remote pull setup," the unit that gets embedded is _the function_, with its name and its parent type attached — not a window that happens to straddle it. --- ## Local embeddings: no GPU, no cloud, no API key The part people assume requires a cloud: turning code into vectors. It doesn't. Octocode ships a [local-first stack](/blog/octocode-0-15-0-local-first-hybrid-search) built on `fastembed` (ONNX Runtime under the hood). You point the config at a local model with the `fastembed:` prefix and embeddings run on CPU, on your machine: ```toml [embedding] code_model = "fastembed:jinaai/jina-embeddings-v2-base-code" # 768-d, purpose-built for code text_model = "fastembed:nomic-ai/nomic-embed-text-v1.5" # text and docs ``` A model string is always `provider:model`. The provider prefix is the whole routing decision — `fastembed:` and `huggingface:` are local (no key, no network); `voyage:`, `jina:`, `google:`, `openai:`, `together:`, `octohub:` are cloud and need the corresponding API key in the environment. Mix them if you want — local code embeddings, cloud text embeddings — but for a private repo the all-local path is the point. There's no GPU requirement and no GPU code path here: this is CPU ONNX inference. On a modern multi-core machine with AVX2, that's fast enough that the embedder is rarely the bottleneck — disk I/O and tree-sitter parsing usually are. The repo's own performance notes put local FastEmbed indexing around 45 seconds for 1,000 files as a rough order of magnitude; your mileage varies with file size and core count, so treat that as illustrative, not a promise. Two honest caveats: 1. **First run downloads the model.** The first `octocode index` after pointing at a `fastembed:` model pulls the ONNX weights — a few hundred MB — into a cache directory (`~/.local/share/octocode/fastembed/` on Linux/macOS). One-time cost, then it's offline forever. 2. **Local providers are feature-gated and platform-sensitive.** FastEmbed and HuggingFace require the `fastembed`/`huggingface` build features. Default builds ship them where they're well-supported; on platforms where they aren't, you fall back to a cloud provider. Check `octocode models list fastembed` to see what your binary actually exposes. The recent [local embedding provider work in 0.17.0](/blog/octocode-0-17-1-local-embedding-providers) is what made this listing honest about what's available where. If you've never set up semantic search over code before, the [Octocode semantic code search intro](/blog/octocode-semantic-code-search) covers the conceptual basics; this post assumes you're past that and want the production setup. --- ## Indexing the repo With the config in place, indexing is one command from the repo root: ```bash cd /path/to/big-monorepo octocode index # → Indexed 12,847 blocks across 342 files ``` What actually happens, in order: discover files (respecting ignore rules), parse each with tree-sitter, extract symbols, chunk along AST boundaries, embed each chunk locally, and write everything to a **LanceDB** columnar store. The database does _not_ live in your repo — it's keyed per-project under the system data dir (`~/.local/share/octocode//storage` on Linux/macOS). That keeps your working tree clean and means a `git clean -fdx` won't nuke your index. Useful flags: ```bash octocode index --force # ignore the incremental cache, rebuild from scratch octocode index --verbose # see per-file progress octocode stats # how many code/text/doc blocks, graph nodes, index staleness ``` `octocode stats` is the one to run when something feels off — it tells you whether your `HEAD` matches the last indexed commit, so you can see at a glance whether the index is stale. ### Incremental vs. full reindex Subsequent `octocode index` runs are incremental — only changed files get re-parsed and re-embedded. Each indexing run ends with a table optimization pass so the query path stays fast as the database grows (repeated incremental indexing used to leave an unindexed tail that searches had to brute-force scan; that's handled automatically now). When you change _how_ things are indexed — a new chunking rule, a different embedding model — incremental won't retroactively re-chunk files it already has. Switching the embedding model is the big one: vectors from one model are not comparable with another's. After changing `code_model` or `text_model`, do a clean rebuild: ```bash octocode clear # drop the indexed tables octocode index # rebuild with the new model ``` --- ## Watch mode: keep the index live A one-shot index goes stale the moment someone commits. For an agent that's supposed to answer questions about the _current_ state of the repo, run the watcher: ```bash octocode watch # auto-reindex on file changes octocode watch --debounce 5 # wait 5s after the last change before reindexing octocode watch --quiet ``` The watcher debounces (configurable, 1–30 seconds) so a `git checkout` that touches 400 files triggers one reindex, not 400. It honors the same ignore rules as a full index, so editing a `node_modules` file doesn't wake it up. If you're running the MCP server (next section), there's an even cleaner option: set `mcp_index = true` under `[index]` and the MCP server indexes and watches **in-process** — one running thing instead of two. The default is `false`, which serves an existing index read-only and assumes you index separately; flip it on when you want the server to own the whole lifecycle. --- ## Hybrid search: semantic _and_ exact, fused Here's the part that fixes the original grep failure without throwing away grep's one strength. Pure vector search loses on identifier-heavy queries. Search for `parse_remote` and dense embeddings will happily return things that are _semantically near_ `parse_remote` while missing the function literally named `parse_remote`. BM25 keyword search has the opposite failure: it nails exact identifiers and whiffs on paraphrased intent like "function that handles remote pull setup." Octocode runs both on every query and fuses them with Weighted Reciprocal Rank Fusion, inside LanceDB. The weights are exposed: ```toml [search.hybrid] enabled = true default_vector_weight = 0.7 default_keyword_weight = 0.3 ``` For a code-heavy repo where exact identifiers carry a lot of signal, tilt toward keyword. The repo's own retrieval benchmark — 127 code-search queries with line-range ground truth, run fully local with `jina-embeddings-v2-base-code` and _no_ reranker — shows the effect clearly: moving from the default `0.7/0.3` split to a keyword-tuned `0.3/0.7` lifted Hit@5 from 0.598 to 0.732 and Recall@10 from 0.671 to 0.807, at zero added cost. That's a +22% Hit@5 from one config line. For long-form docs, tilt the other way (`0.8/0.2`) since intent dominates there. On top of fusion, an optional reranker reorders the top candidates: ```toml [search.reranker] enabled = true model = "fastembed:jina-reranker-v2-base-multilingual" # local reranker, no key top_k_candidates = 50 # fetch this many from vector search final_top_k = 10 # return this many after reranking ``` The reranker runs locally too if you point it at a `fastembed:` model. It's a cross-encoder pass over the top 50 candidates, which catches cases where the first-stage ranking got the order wrong. Cost is real but bounded — it only sees `top_k_candidates`, not the whole index. From the CLI, the search surface is straightforward: ```bash octocode search "webhook signature verification" octocode search "auth" "middleware" "session" # multi-query, up to 10 octocode search "database connection pool" --mode code # restrict to code blocks octocode search "auth" --detail-level signatures # signatures only, compact octocode search "authentication refactor" --mode commits # search git history ``` The `--mode commits` path is worth knowing about: commit history is indexed lazily on first commit-search (not during `octocode index`), so you can ask "when did we change the auth flow" and get semantically-ranked commits, not just a `git log --grep`. --- ## Structural search: when you need AST patterns, not meaning Sometimes you don't want semantics at all — you want "every `.unwrap()` in Rust" or "every `new Foo()` in JS." That's a structural query, and text search gets it wrong constantly (a comment mentioning `.unwrap()` is not a call). Octocode wraps ast-grep: ```bash octocode grep '$FUNC.unwrap()' --lang rust octocode grep 'new $CLASS($$$ARGS)' --lang javascript octocode grep 'console.log($ARG)' --lang javascript --rewrite 'logger.info($ARG)' --update-all ``` `$VAR` matches one AST node, `$$$ARGS` matches a run of arguments, and `--rewrite` does AST-aware refactoring in place. The genuinely useful bit for agents: LLMs reliably get the node _kind_ wrong — they'll write `function_declaration` for Python, which is actually `function_definition`. When a pattern matches nothing, Octocode tries progressively looser interpretations and knows the right kind per language, so the agent's near-miss still returns results instead of a silent zero. --- ## GraphRAG: relationships, not just similarity Semantic search finds _similar_ code. It can't tell you that `auth_middleware.rs` imports `jwt.rs`, calls `user_store.rs`, and is wired into `router.rs`. That's a graph question, and Octocode builds the graph during indexing. ```toml [graphrag] enabled = true use_llm = true ``` It extracts `imports`, `calls`, `extends`, and `implements` edges across nine languages from the AST, and with `use_llm = true` adds higher-level architectural relationships (`configures`, `uses`, factory/observer/strategy patterns) discovered by an LLM. From the CLI: ```bash octocode graphrag get-relationships --node-id "src/auth/middleware.rs" octocode graphrag find-path --source-id "src/auth/mod.rs" --target-id "src/database/mod.rs" octocode graphrag overview ``` This is the difference between "show me code that looks like auth" and "show me everything that _depends on_ the auth module." For an agent doing a refactor, the second question is the one that prevents breakage. There's a newer, optional twist: **graph-aware retrieval expansion**. With `[search] graph_expansion = true` (and GraphRAG enabled), search pulls in code blocks from files that are structurally related to your top hits _before_ reranking — so a query that lands on the auth middleware can surface the JWT helper it calls even if that helper didn't match the query text. It's off by default and the code comments are blunt about it: A/B it on your own eval before trusting it, because expansion can add noise as easily as signal. --- ## Wiring it to an agent over MCP All of the above is exposed to AI assistants through a built-in **MCP server**. This is the primary way to use Octocode — the agent gets tools, not a shell. ```bash octocode mcp --path /path/to/your/project ``` Or in a client config (Claude Desktop, Cursor, Windsurf, Claude Code): ```json { "mcpServers": { "octocode": { "command": "octocode", "args": ["mcp", "--path", "/path/to/your/project"] } } } ``` The tools the agent sees, verified against the server: | MCP tool | What it does | | ------------------- | ---------------------------------------------------------------------------------------- | | `semantic_search` | Hybrid semantic + keyword search, multi-query, all modes including `commits` | | `view_signatures` | File structure — signatures, classes, imports — by glob, without reading whole files | | `structural_search` | AST pattern matching via ast-grep, with the kind-fallback recovery | | `graphrag` | Relationship queries: `search`, `get-node`, `get-relationships`, `find-path`, `overview` | `view_signatures` is the unsung one. Instead of an agent burning context reading three 800-line files to find a function signature, it asks for signatures by glob and gets the skeleton. (As of recent versions you can pass a single glob string, not just an array — a small thing that stopped a lot of malformed tool calls.) For a monorepo with many sub-projects, there's a multi-repo mode — `octocode mcp --multi --path /workspace` scans the immediate subdirectories for git repos and serves all of them from one endpoint, with each tool gaining a `project` argument to pick the target. One MCP server, every repo in the workspace. --- ## The four things that bit us Grounded, not hypothetical — these are what cost us time. **1. The index lives outside the repo, and that confused everyone.** First time someone ran the agent on a fresh clone, it had no index — the database is per-project under `~/.local/share/octocode/`, not in the working tree. Clone ≠ indexed. The fix is a one-liner in onboarding: after cloning, run `octocode index` once. Obvious in hindsight; not obvious at 2am. **2. Switching embedding models silently degraded results.** Someone changed `code_model` to try a different local model and _didn't_ clear the index. New queries embedded with the new model, old chunks embedded with the old one, vectors not comparable, garbage rankings. Vectors from different models do not live in the same space. **Always `octocode clear && octocode index` after changing an embedding model.** There's no warning that will save you here — it just quietly gets worse. **3. The default vector/keyword split was wrong for our repo.** We ran with `0.7/0.3` (vector-heavy) for a week and kept missing exact-identifier queries. Code is identifier-dense; the benchmark above shows keyword-tuned weights are dramatically better for code search. Flipping to `0.3/0.7` was the single highest-leverage config change we made. **4. Binary and generated files don't get indexed — and that's correct, but check it.** Octocode skips binary files (it checks for null bytes and a printable-character ratio before treating content as text) and respects `.gitignore`, `.git/info/exclude`, and `.noindex` files. That's exactly what you want — you don't want embeddings of minified vendor bundles. But if a directory you _care_ about is gitignored (some teams gitignore generated API clients), it won't be indexed and you'll wonder why search can't find it. Drop a `.noindex` to _exclude_ extra paths; check your `.gitignore` if something you expected is missing. `octocode stats` will show you the block counts so you can sanity-check coverage. --- ## A minimal end-to-end setup Putting it together, this is the whole thing for a private monorepo, all local: ```toml # ~/.local/share/octocode/config.toml (or a project-local override) [embedding] code_model = "fastembed:jinaai/jina-embeddings-v2-base-code" text_model = "fastembed:nomic-ai/nomic-embed-text-v1.5" [search.hybrid] enabled = true default_vector_weight = 0.3 # identifier-heavy code → favor keyword default_keyword_weight = 0.7 [search.reranker] enabled = true model = "fastembed:jina-reranker-v2-base-multilingual" [graphrag] enabled = true use_llm = false # AST-only relationships, no LLM calls, fully offline ``` ```bash cd /path/to/monorepo octocode index # first run downloads models, then indexes octocode search "webhook signature verification" # sanity check octocode watch --quiet & # keep it live claude mcp add octocode -- octocode mcp --path . # wire to the agent ``` Nothing in that flow touches the network after the one-time model download. No GPU. No per-token bill. The agent that was grepping for "signature" and pointing at the wrong file now calls `semantic_search`, gets `verify_hmac` ranked first, and `view_signatures` to confirm before it touches anything. --- ## FAQ **Do I need a GPU?** No. Local embeddings run on CPU via ONNX Runtime. A modern multi-core CPU is fine; the embedder is rarely the bottleneck. **Does any code leave my machine?** With `fastembed:`/`huggingface:` models and `graphrag.use_llm = false`, no — indexing, search, and reranking are all local. Cloud embedding providers and the LLM-powered GraphRAG/commit-message features are opt-in and clearly need keys. **Where does the index live, and is it in my repo?** Under the system data dir, keyed per project (`~/.local/share/octocode//` on Linux/macOS), not in your working tree. A `git clean` won't touch it; a fresh clone won't have it. **How big does the database get?** Roughly on the order of ~10KB per file, with RaBitQ quantization giving ~32x vector compression. A 10k-file repo is tens to low-hundreds of MB, not gigabytes. **Incremental or full reindex?** Incremental by default — only changed files are re-embedded. Force a full rebuild with `octocode index --force`, or `octocode clear && octocode index` after changing the embedding model. **Can it search git history?** Yes, `octocode search "..." --mode commits`. Commits are indexed lazily on first commit-search, so the initial `octocode index` stays fast. --- — Don --- _[Octocode](https://github.com/Muvon/octocode) is open source under Apache-2.0. The local-first stack landed in [0.15.0](/blog/octocode-0-15-0-local-first-hybrid-search) and the local embedding provider listing in [0.17.x](/blog/octocode-0-17-1-local-embedding-providers). If your setup hits a gotcha this post didn't cover, [open an issue](https://github.com/Muvon/octocode/issues) — that's how most of these tips got written down in the first place._