Skip to content

feat: add Nix flake for reproducible dev environment#76

Open
davidlghellin wants to merge 10 commits intomicrosoft:mainfrom
davidlghellin:feat/nix
Open

feat: add Nix flake for reproducible dev environment#76
davidlghellin wants to merge 10 commits intomicrosoft:mainfrom
davidlghellin:feat/nix

Conversation

@davidlghellin
Copy link

@davidlghellin davidlghellin commented Feb 28, 2026

Why this change is needed

One-command dev environment with everything ready to go. nix develop sets up Python, all dependencies (including optional extras), and dev tools automatically.

How

Add a flake.nix that provides a single-command dev setup via nix develop:

  • Python 3.13 with uv for fast dependency management and ruff for linting
  • Editable install of lakebench
  • All optional extras (duckdb, polars, daft, spark, sail, etc.) installed individually so one failure doesn't block the rest
  • Dev dependency group (pytest, pytest-cov) via uv pip install --group dev
  • spark and sail conflict with each other — installing them separately in a loop handles this gracefully

Test

  • Verified nix develop creates the venv and installs all dependencies
  • Ran TPC-DS SF=1 data generation and power test with DuckDB (99 queries)
  • Ran TPC-DS SF=1 power test with Sail (99 queries)
  • Ran pytest successfully

@davidlghellin
Copy link
Author

davidlghellin commented Feb 28, 2026

image

@davidlghellin davidlghellin marked this pull request as draft February 28, 2026 14:12
@davidlghellin davidlghellin marked this pull request as ready for review February 28, 2026 14:16
@mwc360
Copy link
Contributor

mwc360 commented Mar 12, 2026

@davidlghellin - thx for the PR. I'm not familiar with nix, what is the value over uv? Currently the project is set up to do the save via uv... install uv, then run uv sync, uv run pytest`, etc.

@davidlghellin
Copy link
Author

thx for the PR. I'm not familiar with nix, what is the value over uv? Currently the project is set up to do the save via uv... install uv, then run uv sync, uv run pytest`, etc.

Nix doesn’t replace uv — it just wraps the existing workflow.

The flake mainly provides:

Pinned system dependencies (e.g., Python 3.13, ruff). uv manages Python packages but not the Python interpreter or system tools.

One-command setup: nix develop gives you Python + uv ready to use and creates the virtualenv automatically (uv venv, uv sync, etc.).

Reproducibility across machines: anyone using Nix gets the same environment regardless of their local Python version or system state.

Under the hood it still uses uv exactly as the project already does — Nix just ensures the environment is consistent.

So you can think of it as: Nix manages the environment, uv manages the Python dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants