Skip to content

Better nested queries API#125

Merged
AivanF merged 1 commit into
mainfrom
aivan/2026-05-14-Nested-Queries-API
May 14, 2026
Merged

Better nested queries API#125
AivanF merged 1 commit into
mainfrom
aivan/2026-05-14-Nested-Queries-API

Conversation

@AivanF
Copy link
Copy Markdown
Collaborator

@AivanF AivanF commented May 14, 2026

Summary by CodeRabbit

  • New Features

    • Runtime multi-stage query lists are now auto-sorted by dependencies; the last stage is the returned entry point.
    • Added a MCP tool to run multi-stage/DAG queries with JSON/CSV/Markdown output.
  • Documentation

    • Expanded guidance on query-list validation (cycles, self/forward references, root usage), accepted/omitted utility stages, and which APIs support multi-stage lists.
  • Tests

    • Added and updated tests covering DAG sorting, validation errors, empty lists, and formatting.
  • Chores

    • Version bumped to 0.6.5.

Review Change Stack

@AivanF AivanF requested a review from ZmeiGorynych May 14, 2026 10:13
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 14, 2026

📝 Walkthrough

Walkthrough

Runtime-provided multi-stage query lists are validated and auto-topologically ordered before execution (final entry remains the returned root). A new MCP query_nested tool invokes engine.execute with staged lists. Tests and docs updated; package version bumped.

Changes

Multi-Stage DAG Query Execution with Topological Auto-Sorting

Layer / File(s) Summary
DAG Topological Ordering and Validation
slayer/engine/query_engine.py
_topologically_order_queries() added: extracts sibling deps from source_model and joins.target_model, enforces non-final-name requirement, detects duplicates/self-refs/root-as-sink violations and cycles, and returns reordered stages with final stage last. _scope_named_queries_to_prior() doc updated.
Query Engine Execute Path Integration
slayer/engine/query_engine.py
execute() now rejects empty runtime lists, normalizes dict items, calls _topologically_order_queries() to reorder stages, sets the reordered final stage as the execution entry, and builds named_queries from reordered non-final stages.
MCP Tool for Multi-Stage Query Execution
slayer/mcp/server.py
New query_nested FastMCP tool accepts non-empty list of query-stage dicts plus optional variables, show_sql, dry_run, explain, and format; calls engine.execute(query=list(...)) and formats results with DB error mapping.
Test Coverage for DAG Execution and Validation
tests/test_mcp_server.py, tests/test_query_backed_models.py, tests/integration/test_integration.py
Adds TestQueryNested MCP tests (dry-run SQL, empty-list rejection, out-of-order auto-sorting, cycle detection, invalid format) and expands runtime-list tests to assert auto-sorting, cycle errors, unused utility-stage acceptance, root-reference rejection, empty-list rejection, and unnamed non-final-stage rejection. Integration test error-message regex updated for cycle wording.
Documentation and Version Updates
.claude/skills/slayer-query.md, CLAUDE.md, docs/concepts/queries.md, pyproject.toml
Docs updated to describe runtime auto-sorting (Kahn-style), DAG root as final entry, validation rules (cycles, self-refs, root refs), omission of unreachable utility stages from emitted SQL, and which surfaces accept multi-stage lists (Python SDK engine.execute(query=[...]), CLI slayer query @file.json``, MCP query_nested vs REST single-query). `pyproject.toml` version bumped 0.6.4 → 0.6.5.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • MotleyAI/slayer#97: Related sibling-stage resolution and _scope_named_queries_to_prior logic that this PR builds on for topological ordering.
  • MotleyAI/slayer#9: Earlier changes to execute() multi-stage handling that relate to this runtime reordering surface.

Suggested reviewers

  • ZmeiGorynych

Poem

🐰 I hop through DAGs with a tidy sort,
Stages aligned in dependency court.
No cycles to tangle, no roots out of place—
Queries now dance to a topological pace.
🌿🥕

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 73.08% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Better nested queries API' accurately describes the main objective of the PR, which introduces a new nested queries API through the query_nested MCP tool, improved DAG handling for multi-stage query lists, and comprehensive documentation of the multi-stage query execution flow.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch aivan/2026-05-14-Nested-Queries-API

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
tests/test_mcp_server.py (1)

2932-2932: ⚡ Quick win

Move ToolError imports to module scope.

These three function-local imports at lines 2932, 2972, and 2985 should be added to the file import block at the top for consistency with the repository guideline that imports must be placed at the top of files.

As per coding guidelines **/*.py: Place imports at the top of files.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_mcp_server.py` at line 2932, Three local imports of ToolError
inside test functions must be moved to module scope; add a single top-level
import "from mcp.server.fastmcp.exceptions import ToolError" to the file's
import block and remove the three function-local import lines (the occurrences
that currently import ToolError inside functions), so all tests reference the
module-level ToolError instead of importing it locally.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@slayer/engine/query_engine.py`:
- Around line 274-410: The _topologically_order_queries function is too complex;
split its logic into three private helpers: implement _validate_queries(rest:
List[SlayerQuery], root: SlayerQuery, rest_by_name: Dict[str, SlayerQuery]) to
perform non-final name checks, duplicates, self-reference (using the existing
_refs) and root-referrer validation; implement
_build_dependency_graph(rest_by_name: Dict[str, SlayerQuery], sibling_names:
set) to produce and return (in_degree: Dict[str,int], dependents:
Dict[str,List[str]]) by iterating over rest_by_name and calling _refs; and
implement _kahn_sort(in_degree, dependents) to run the Kahn traversal and return
sorted_names or raise the same cycle ValueError; then simplify
_topologically_order_queries to compute rest/rest_by_name/root/sibling_names,
call _validate_queries, call _build_dependency_graph, call _kahn_sort, and
finally return [rest_by_name[n] for n in sorted_names] + [root].
- Around line 435-443: The code assumes queries list is non-empty and does
queries[-1], which raises IndexError for empty staged-query lists; add a guard
after building/ordering queries (after calls to SlayerQuery.model_validate and
_topologically_order_queries) to detect empty queries and raise a user-facing
validation error (e.g., ValueError or a custom ValidationError) with a clear
message before accessing queries[-1]; ensure the same guard applies before
constructing named_queries and that any callers of this block still receive the
validation error rather than an IndexError.

---

Nitpick comments:
In `@tests/test_mcp_server.py`:
- Line 2932: Three local imports of ToolError inside test functions must be
moved to module scope; add a single top-level import "from
mcp.server.fastmcp.exceptions import ToolError" to the file's import block and
remove the three function-local import lines (the occurrences that currently
import ToolError inside functions), so all tests reference the module-level
ToolError instead of importing it locally.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 04e12485-f602-4624-a033-04fd69a544f3

📥 Commits

Reviewing files that changed from the base of the PR and between 83784af and 8cd32b6.

📒 Files selected for processing (8)
  • .claude/skills/slayer-query.md
  • CLAUDE.md
  • docs/concepts/queries.md
  • pyproject.toml
  • slayer/engine/query_engine.py
  • slayer/mcp/server.py
  • tests/test_mcp_server.py
  • tests/test_query_backed_models.py

Comment thread slayer/engine/query_engine.py
Comment thread slayer/engine/query_engine.py
@AivanF AivanF force-pushed the aivan/2026-05-14-Nested-Queries-API branch from 8cd32b6 to 301c940 Compare May 14, 2026 10:43
@sonarqubecloud
Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
4.1% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
slayer/engine/query_engine.py (1)

399-409: 💤 Low value

Consider deque for O(1) pop from front.

frontier.pop(0) is O(n) on a list. For small DAGs this is negligible, but collections.deque provides O(1) popleft() and aligns better with canonical Kahn's implementations.

♻️ Optional optimization
+from collections import deque
+
     `@staticmethod`
     def _kahn_sort(
         in_degree: Dict[str, int],
         dependents: Dict[str, List[str]],
     ) -> List[str]:
-        frontier: List[str] = sorted(n for n, d in in_degree.items() if d == 0)
+        frontier: deque = deque(sorted(n for n, d in in_degree.items() if d == 0))
         sorted_names: List[str] = []
         while frontier:
-            n = frontier.pop(0)
+            n = frontier.popleft()
             sorted_names.append(n)
             unlocked: List[str] = []
             for dep in dependents[n]:
                 in_degree[dep] -= 1
                 if in_degree[dep] == 0:
                     unlocked.append(dep)
             frontier.extend(sorted(unlocked))
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@slayer/engine/query_engine.py` around lines 399 - 409, Replace the list-based
FIFO used in the Kahn topological sort with a deque to avoid O(n) pops: import
collections.deque and change the frontier initialization to a
deque(sorted(...)), replace frontier.pop(0) with frontier.popleft(), and keep
the rest of the logic (appending to sorted_names and extending frontier with
sorted(unlocked)) but use frontier.extend(sorted(unlocked)); this touches the
variables/function block using frontier, sorted_names, unlocked, in_degree, and
dependents in the topological-sort loop.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@slayer/engine/query_engine.py`:
- Around line 274-311: The _extract_sibling_refs function is too complex due to
repeated branching for the three shapes of query.source_model; refactor by
normalizing source_model into a unified representation (e.g., derive a string
source_name and an iterable of join objects regardless of whether sm is str,
dict, or typed) and then run a single shared extraction loop that checks
source_name and iterates joins to collect target_model values; specifically,
inside _extract_sibling_refs pull out src = ... (handle str -> src=sm, dict ->
src=sm.get("source_name"), typed -> getattr(sm,"source_name",None)) and joins =
... (handle dict -> sm.get("joins") or [], typed -> getattr(sm,"joins", None) or
[], str -> empty), then run one set of checks that adds src if in against and
loops joins to add j.target_model / j.get("target_model") when a string in
against — this removes duplicated logic and lowers cognitive complexity.

---

Nitpick comments:
In `@slayer/engine/query_engine.py`:
- Around line 399-409: Replace the list-based FIFO used in the Kahn topological
sort with a deque to avoid O(n) pops: import collections.deque and change the
frontier initialization to a deque(sorted(...)), replace frontier.pop(0) with
frontier.popleft(), and keep the rest of the logic (appending to sorted_names
and extending frontier with sorted(unlocked)) but use
frontier.extend(sorted(unlocked)); this touches the variables/function block
using frontier, sorted_names, unlocked, in_degree, and dependents in the
topological-sort loop.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 2acc8ac0-e507-4931-be0c-dd74aa6c70f5

📥 Commits

Reviewing files that changed from the base of the PR and between 8cd32b6 and 301c940.

📒 Files selected for processing (9)
  • .claude/skills/slayer-query.md
  • CLAUDE.md
  • docs/concepts/queries.md
  • pyproject.toml
  • slayer/engine/query_engine.py
  • slayer/mcp/server.py
  • tests/integration/test_integration.py
  • tests/test_mcp_server.py
  • tests/test_query_backed_models.py
✅ Files skipped from review due to trivial changes (5)
  • .claude/skills/slayer-query.md
  • pyproject.toml
  • tests/integration/test_integration.py
  • CLAUDE.md
  • docs/concepts/queries.md
🚧 Files skipped from review as they are similar to previous changes (3)
  • tests/test_mcp_server.py
  • tests/test_query_backed_models.py
  • slayer/mcp/server.py

Comment thread slayer/engine/query_engine.py
@AivanF AivanF merged commit b4fe108 into main May 14, 2026
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant