Skip to content

Commit 08b81c8

Browse files
HumanBean17claude
andauthored
fix: set COCOINDEX_SOURCE_MAX_INFLIGHT_ROWS=256 to prevent "too many open files" error (#300)
Fixes #293. Users experienced "Too many open files (os error 24)" during indexing because CocoIndex's default concurrency (1024 inflight rows) opens more file handles than OS limits allow (typically 256-1024). Set COCOINDEX_SOURCE_MAX_INFLIGHT_ROWS=256 in both code paths that invoke cocoindex (pipeline.run_cocoindex_update and server._cocoindex_subprocess_env). Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent 5bf66fb commit 08b81c8

2 files changed

Lines changed: 7 additions & 0 deletions

File tree

java_codebase_rag/pipeline.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -128,6 +128,10 @@ def run_cocoindex_update(
128128
stdout="",
129129
stderr=f"java_index_flow_lancedb.py not found under {bd}",
130130
)
131+
# Set CocoIndex concurrency limits to prevent "too many open files" error
132+
# See: https://github.com/HumanBean17/java-codebase-rag/issues/293
133+
env = env.copy()
134+
env.setdefault("COCOINDEX_SOURCE_MAX_INFLIGHT_ROWS", "256")
131135
cmd: list[str] = [str(exe), "update", COCOINDEX_TARGET]
132136
if full_reprocess:
133137
cmd.extend(["--full-reprocess", "-f"])

server.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -162,6 +162,9 @@ def _cocoindex_subprocess_env(project_root: Path) -> dict[str, str]:
162162
idx = os.environ.get("JAVA_CODEBASE_RAG_INDEX_DIR", "").strip()
163163
if idx:
164164
sub_env["JAVA_CODEBASE_RAG_INDEX_DIR"] = str(Path(idx).expanduser().resolve())
165+
# Set CocoIndex concurrency limits to prevent "too many open files" error
166+
# See: https://github.com/HumanBean17/java-codebase-rag/issues/293
167+
sub_env.setdefault("COCOINDEX_SOURCE_MAX_INFLIGHT_ROWS", "256")
165168
return sub_env
166169

167170

0 commit comments

Comments
 (0)