MacKV-Opt helps your Apple Silicon Mac manage large amounts of text data for AI models. It organizes memory so you can talk to local models for longer periods. You gain access to tools that make your Mac models faster and more efficient.
Large language models require memory to track conversations. When a conversation grows, these models often run out of space. MacKV-Opt manages your KV cache. This is the memory area where the model stores conversation history. By optimizing this space, the application allows for extended chat sessions. It works with GGUF format models and stays compatible with tools like Ollama.
Your computer needs specific hardware to run this tool well:
- An Apple Silicon chip (M1, M2, or M3 series).
- At least 16GB of unified memory. 8GB models may handle smaller tasks but lack room for long context windows.
- macOS 13.0 or newer.
- Ollama installed on your system if you plan to run Ollama-compatible benchmarks.
- Approximately 200MB of free disk space for the installation.
You need to download the installer from our release page. Visit this link to find the most recent version:
On this page, look for the file ending in .dmg. Click the file to start the download. Once the process completes, open the file to begin your installation.
Follow these steps to set up the software on your machine:
- Open the downloaded
.dmgfile. - Drag the MacKV-Opt icon into your Applications folder.
- Open the Applications folder and double-click the MacKV-Opt icon.
- If a security prompt appears, click Open to confirm you trust the application.
- Wait for the initial configuration window to load.
The software checks for Ollama during the first launch. If it finds Ollama, it maps your model paths automatically. You do not need to move your files or change deep settings.
Benchmarks help you understand how much extra context your Mac handles with MacKV-Opt.
- Open the main window of the MacKV-Opt app.
- Select the Benchmark tab from the top menu.
- Choose your model from the dropdown list.
- Pick a context window size, such as 32k tokens.
- Click Run Test.
The test takes several minutes. The app writes the results to a local file. You can see your tokens-per-second count and the memory usage throughout the test. Lower memory peaks confirm that the tool manages your cache efficiently.
You can adjust how the app interacts with your AI models:
- Cache Strategy: Change how the app reuses memory from previous prompts. The Standard setting works for most users. Use Aggressive mode for very large text documents.
- Model Path: Set this to the folder where you store your GGUF files.
- Auto-Update: Keep this enabled to get performance patches and new compatibility updates for Ollama.
Do I need an internet connection to use this? No. Once you install the models, the software runs locally on your Mac. You do not need the internet to process prompts.
Does this work with all models? It works best with GGUF files. These are common files used for local AI. If a model uses a different format, the app will alert you.
Is my data private? Yes. Since the software runs on your hardware, your data never leaves your computer. No servers process your text.
The app runs slowly. What can I do? Close other applications that use heavy memory. Large AI models use a lot of RAM. Closing your web browser often frees up significant space.
If you see an error during startup, check these items:
- Permissions: Go to System Settings, click Privacy & Security, and check for notifications about the app. You may need to grant permission to access your files.
- Ollama Path: If the app cannot find your models, go to Settings and point the Model Path manually to the folder inside your user directory where Ollama stores its data.
- Updates: Ensure your macOS version is current. Updates often fix small memory bugs that affect AI software.
You now have a tool to manage long AI contexts on your Mac. Experiment with different model sizes and context windows to find what works for your hardware. If you find errors or have ideas for performance, report them to the repository for review. This tool continues to grow based on how users interact with their local models on the latest Apple hardware.