Make Finder ID presence/absence caching opt-in

[Copilot]

The Finder class accumulates boolean flags tracking document presence/absence in a dictionary. This can consume significant RAM in long-running library contexts.

Changes

• Finder.__init__: Added enable_id_caching: bool = False parameter. Defaults to disabled for library usage.

• Cache methods: Modified _get_cached_id_presence_in_collection() and _set_cached_id_presence_in_collection() to short-circuit when caching is disabled.

• CLI integration: Enabled caching in scanner.py by passing enable_id_caching=True when instantiating Finder. CLI scans trade memory for performance.

Usage

from refscan.lib.Finder import Finder

# Library usage - minimal memory footprint
finder = Finder(database=db)  # Caching disabled by default

# CLI/performance-critical usage
finder = Finder(database=db, enable_id_caching=True)  # Explicit opt-in

When disabled, the cached_id_presence_by_collection dictionary remains empty regardless of query volume.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>Make cacheing of presence/absence be optional</issue_title>
<issue_description>In v0.2.2, instances of the Finder class began accumulating boolean flags in a dictionary, to keep track of which documents it succeeded or failed to find in a given collection.

Some users may not want to sacrifice the extra RAM that that consumes. For example, maybe CLI users will be OK with it, but library importers will not.

The task here is to make that caching optional. It can be enabled via a parameter to the Finder constructor.

We can disable it by default there so that the feature is opt-in. In our CLI code, we can enable it.

This way, it is enabled for CLI scans and optional for importers.

Later, we could add a CLI option so that even CLI users can choose.</issue_description>

<agent_instructions>also include tests</agent_instructions>

Comments on the Issue (you are @copilot in this section)

• Fixes Make cacheing of presence/absence be optional #42

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.