feat: Firecrawl package and module by DaRacci · Pull Request #591 · DaRacci/nix-config

DaRacci · 2026-06-21T13:40:42Z

No description provided.

…nRouter, and Playwright support

…tMQUrl option

coderabbitai · 2026-06-21T13:40:49Z

Warning

Review limit reached

@DaRacci, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 56 minutes and 19 seconds. Learn how PR review limits work.

Your organization has used up its prepaid credits, and credit purchases are no longer available. Enable the review add-on in the billing tab to keep reviews running — you're only billed for reviews past your plan's rate limits ($0.25/file).

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: c47a2c63-f855-4232-a142-dc4711bc28dc

📥 Commits

Reviewing files that changed from the base of the PR and between 0c41dcb and 2d24dc7.

⛔ Files ignored due to path filters (1)

pkgs/firecrawl/Cargo.lock is excluded by !**/*.lock

📒 Files selected for processing (13)

docs/src/modules/nixos/server/database.md
docs/src/modules/nixos/services/ai_agent.md
docs/src/modules/nixos/services/firecrawl.md
hosts/server/nixai/ai-agent.nix
hosts/server/nixai/secrets.yaml
hosts/server/nixai/web.nix
hosts/server/nixio/redis-mappings.json
modules/nixos/services/ai-agent.nix
modules/nixos/services/default.nix
modules/nixos/services/firecrawl.nix
pkgs/default.nix
pkgs/firecrawl/default.nix
pkgs/firecrawl/smoke-test.nix

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch push-vrnqnmosswwq

✨ Simplify code

Create PR with simplified code
Commit simplified code in branch push-vrnqnmosswwq

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

mergify · 2026-06-21T13:41:07Z

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🔴 Do not merge outdated PRs

Waiting for

#commits-behind <= 10

This rule is failing.

Make sure PRs are almost up to date before merging

#commits-behind <= 10

🟢 Enforce conventional commit

Wonderful, this rule succeeded.

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:$.+$)?:

🟢 🚦 Auto-queue

Wonderful, this rule succeeded.

When all merge protections are satisfied, this pull request will be queued automatically.

RacciDev-Bot

Hermes Code Review -- PR #591

Verdict: REQUEST_CHANGES

Summary: 17 findings (6 critical, 6 major, 5 minor)

Critical (won't compile or runtime crash)

firecrawl.nix:44-76 -- effectiveRedisUrl/databaseVars block is malformed Nix (double else, dangling in, orphaned attrset with undefined dsn)
firecrawl.nix:80 -- effectiveRedisRateLimitUrl referenced but never defined
firecrawl.nix:82 -- databaseVars merged but never defined
firecrawl.nix:72 -- USE_DB_AUTHENTICATION=*** -- invalid Nix in toShellVars
firecrawl.nix:161-168 -- Broken shell syntax in script (=*** pattern, stray )")
ai-agent.nix:377 -- FIRECRAWL_API_KEY=*** -- broken Nix attr, won't parse
smoke-test.nix:260-340 -- Auth bypass section has broken shell throughout
smoke-test.nix:119 -- REDIS_URL uses RabbitMQ port instead of Redis
smoke-test.nix:123 -- export USE_DB_AUTHENTICATION=*** -- invalid shell

Major (wrong behavior)

default.nix:170 -- GOFLAGS set twice, second overwrites first
default.nix:246 -- --start-docker contradicts FIRECRAWL_DISABLE_CONTAINER_MANAGEMENT=1
firecrawl.nix:128-146 -- Missing semicolons after after/wants list values

Minor

firecrawl.nix:15 -- toUpper unused
web.nix:7 -- sopsPlaceholder unused

See inline comments for detailed findings and suggestions.

RacciDev-Bot · 2026-06-21T13:59:30Z

+  redisPasswordPlaceholder =
+    config.sops.placeholder."REDIS_PASSWORD" or (config.sops.placeholder."REDIS/PASSWORD" or "");
+
+  # Resolve effective connection strings:
+  #   explicit option override > repo DB module > safe localhost default
+  hasRedisOverride = cfg.redisUrl != null;
+  hasRedisRateLimitOverride = cfg.redisRateLimitUrl != null;
+  hasDatabaseOverride = cfg.databaseUrl != null;
+
+  effectiveRedisUrl =
+    if hasRedisOverride then
+      cfg.redisUrl
+    else if redis != null then
+      "redis://:${redisPasswordPlaceholder}@${redis.host}:${toString redis.port}/${toString redis.database_id}"
+    else
+      "redis://127.0.0.1:6379";
+
+  effectiveRedisRateLimitUrl =
+    if hasRedisRateLimitOverride then
+      cfg.redisRateLimitUrl
+    else if redisRateLimit != null then
+      "redis://:${redisPasswordPlaceholder}@${redisRateLimit.host}:${toString redisRateLimit.port}/${toString redisRateLimit.database_id}"
+    else
+      "redis://127.0.0.1:6379";
+
+  # Build DATABASE_URL and NUQ_DATABASE_URL from repo postgres module when available
+  databaseVars =
+    if postgres != null && !hasDatabaseOverride then
+      let
+        dbNameUpper = toUpper (builtins.replaceStrings [ "-" ] [ "_" ] postgres.database);
+        dsn = "postgresql://${postgres.user}:${
+          config.sops.placeholder."POSTGRES/${dbNameUpper}_PASSWORD"
+        }@${postgres.host}:${toString postgres.port}/${postgres.database}";


⚠️ Potential issue | 🔴 Critical | ✅ High confidence

effectiveRedisUrl / databaseVars block is syntactically malformed Nix -- won't compile.

The expression has: (1) two else branches with only 3 conditions, (2) a dangling in keyword with no preceding let, (3) an orphaned attrset referencing undefined dsn, (4) a bare else if after in. The intent is three separate let bindings (effectiveRedisUrl, effectiveRedisRateLimitUrl, databaseVars) but they're all smashed into one broken expression.

Fix: split into separate let bindings per the suggestion below.

RacciDev-Bot · 2026-06-21T13:59:30Z

+      in
+      {
+        DATABASE_URL = dsn;
+        NUQ_DATABASE_URL = dsn;


⚠️ Potential issue | 🔴 Critical | ✅ High confidence

effectiveRedisRateLimitUrl referenced but never defined.

Used as REDIS_RATE_LIMIT_URL = effectiveRedisRateLimitUrl; in the sops template but never defined in let. Will eval error.

RacciDev-Bot · 2026-06-21T13:59:30Z

+        DATABASE_URL = dsn;
+        NUQ_DATABASE_URL = dsn;
+      }
+    else if hasDatabaseOverride then


⚠️ Potential issue | 🔴 Critical | ✅ High confidence

databaseVars referenced but never defined.

Merged into the sops template via // databaseVars but never defined in let. Will eval error.

RacciDev-Bot · 2026-06-21T13:59:30Z

+  # Build DATABASE_URL and NUQ_DATABASE_URL from repo postgres module when available
+  databaseVars =
+    if postgres != null && !hasDatabaseOverride then
+      let


⚠️ Potential issue | 🟠 Major | ✅ High confidence

USE_DB_AUTHENTICATION=*** cfg.useDbAuthentication; is not valid Nix.

toShellVars expects name = value; pairs. The =*** is a copy-paste artifact from a shell context.

Should be: USE_DB_AUTHENTICATION = if cfg.useDbAuthentication then \"true\" else \"false\";

RacciDev-Bot · 2026-06-21T13:59:30Z

+
+    redisUrl = mkOption {
+      type = nullOr str;
+      default = null;
+      description = ''
+        Override Redis URL. When null (default), uses repo DB module
+        (config.server.database.redis.firecrawl) or "redis://127.0.0.1:6379".
+      '';


⚠️ Potential issue | 🔴 Critical | ✅ High confidence

Broken shell syntax in script block -- =*** pattern and stray )\".

export BULL_AUTH_KEY=*** \"$CREDENTIALS_DIRECTORY/bullAuthKey\")\" -- =*** is not valid shell assignment. Should use $(cat ...). The trailing )\" has unmatched closing paren and quote.

RacciDev-Bot · 2026-06-21T13:59:30Z

+      mkdir -p $out
+      cp -r $GOMODCACHE/cache $out/cache


⚠️ Potential issue | 🟠 Major | ✅ High confidence

GOFLAGS set twice -- second overwrites first.

GOFLAGS=\"-mod=mod\" ... GOFLAGS=\"$GOFLAGS -modcacherw\" -- the second assignment replaces the first. -mod=mod is lost.

RacciDev-Bot · 2026-06-21T13:59:30Z

+
+        cd apps/api
+


⚠️ Potential issue | 🟠 Major | ✅ High confidence

--start-docker flag contradicts FIRECRAWL_DISABLE_CONTAINER_MANAGEMENT=1.

--start-docker signals to start Docker containers, but FIRECRAWL_DISABLE_CONTAINER_MANAGEMENT=1 suppresses them. Remove --add-flags \"--start-docker\".

RacciDev-Bot · 2026-06-21T13:59:30Z

+
+    environment = mkOption {
+      type = attrsOf str;
+      default = { };
+      description = "Extra environment variables passed to Firecrawl.";
+    };
+
+    extraArgs = mkOption {
+      type = listOf str;
+      default = [ ];
+      description = "Extra args passed to Firecrawl binary.";
+    };
+
+    numWorkersPerQueue = mkOption {
+      type = int;
+      default = 8;
+      description = "Worker count per queue.";
+    };
+


⚠️ Potential issue | 🟠 Major | ⚠️ Medium confidence

Missing semicolons after after and wants list values.

Both after = [ ... ] and wants = [ ... ] are missing semicolons before the ++ optional concat. While Nix can parse this, it's non-standard and may trip up strict parsers.

RacciDev-Bot · 2026-06-21T13:59:30Z

+    optional
+    optionalAttrs
+    toShellVars
+    mkIf


🧹 Nitpick | 🔵 Trivial

toUpper imported from lib but never used.

RacciDev-Bot · 2026-06-21T13:59:30Z

+let
+  db = config.server.database.postgres.open_webui;


🧹 Nitpick | 🔵 Trivial

sopsPlaceholder = config.sops.placeholder; defined but never used.

DaRacci added 4 commits June 21, 2026 23:39

feat(hosts/server/nixai): firecrawl web extractor

f897722

feat(nixos/services): add Firecrawl service module with database, Ope…

b83465a

…nRouter, and Playwright support

feat(pkgs/firecrawl): add native build with Rust and Go, new nuqRabbi…

4f0aaae

…tMQUrl option

test(pkgs/firecrawl): add smoke test and improve Playwright integration

2d24dc7

RacciDev-Bot suggested changes Jun 21, 2026

View reviewed changes

mergify Bot added the conflict label Jun 23, 2026

Conversation

DaRacci commented Jun 21, 2026

Uh oh!

coderabbitai Bot commented Jun 21, 2026

Review limit reached

Uh oh!

mergify Bot commented Jun 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merge Protections

🔴 Do not merge outdated PRs

🟢 Enforce conventional commit

🟢 🚦 Auto-queue

Uh oh!

RacciDev-Bot left a comment

Choose a reason for hiding this comment

Hermes Code Review -- PR #591

Summary: 17 findings (6 critical, 6 major, 5 minor)

Critical (won't compile or runtime crash)

Major (wrong behavior)

Minor

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mergify Bot commented Jun 21, 2026 •

edited

Loading