diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index 8a1416f..a090362 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -29,7 +29,7 @@ What actually happens? - Elixir version: (run `elixir --version`) - Erlang/OTP version: (run `erl -eval 'erlang:halt(0).'`) - PostgreSQL version: (run `psql --version`) -- QuantumFlow version: 0.1.0 +- Singularity.Workflow version: 1.0.x - OS: (e.g., macOS, Ubuntu, etc.) ## Minimal Code Example diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 20d82d4..164be44 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -1,6 +1,6 @@ --- name: Feature Request -about: Suggest an idea for improving QuantumFlow +about: Suggest an idea for improving Singularity.Workflow title: "[FEATURE] " labels: enhancement --- diff --git a/.github/ISSUE_TEMPLATE/question.md b/.github/ISSUE_TEMPLATE/question.md index f86a34c..8c7f303 100644 --- a/.github/ISSUE_TEMPLATE/question.md +++ b/.github/ISSUE_TEMPLATE/question.md @@ -1,13 +1,13 @@ --- name: Question -about: Ask a question about QuantumFlow +about: Ask a question about Singularity.Workflow title: "[QUESTION] " labels: question --- ## Question -What would you like to know about QuantumFlow? +What would you like to know about Singularity.Workflow? ## Context @@ -18,10 +18,10 @@ Provide any relevant context about your question: ## Documentation Checked -- [ ] [GETTING_STARTED.md](https://github.com/mikkihugo/quantum_flow/blob/main/GETTING_STARTED.md) -- [ ] [ARCHITECTURE.md](https://github.com/mikkihugo/quantum_flow/blob/main/ARCHITECTURE.md) -- [ ] Generated API docs (https://hexdocs.pm/quantum_flow) -- [ ] [docs/ reference materials](https://github.com/mikkihugo/quantum_flow/tree/main/docs) +- [ ] [GETTING_STARTED.md](https://github.com/Singularity-ng/singularity-workflows/blob/main/GETTING_STARTED.md) +- [ ] [ARCHITECTURE.md](https://github.com/Singularity-ng/singularity-workflows/blob/main/ARCHITECTURE.md) +- [ ] Generated API docs (https://hexdocs.pm/singularity_workflow) +- [ ] [docs/ reference materials](https://github.com/Singularity-ng/singularity-workflows/tree/main/docs) ## Code Example (if applicable) diff --git a/.github/workflows/docker-build.yml b/.github/workflows/docker-build.yml index 88cf536..400a5b0 100644 --- a/.github/workflows/docker-build.yml +++ b/.github/workflows/docker-build.yml @@ -10,7 +10,7 @@ on: env: REGISTRY: ghcr.io - IMAGE_NAME: mikkihugo/quantum_flow-postgres + IMAGE_NAME: singularity-ng/singularity_workflow-postgres jobs: build-postgres: diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 51750df..d8906d2 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -18,7 +18,7 @@ jobs: env: POSTGRES_USER: postgres POSTGRES_PASSWORD: postgres - POSTGRES_DB: quantum_flow_test + POSTGRES_DB: singularity_workflow_test options: >- --health-cmd pg_isready --health-interval 10s @@ -41,9 +41,9 @@ jobs: # Wait for PostgreSQL to be ready until pg_isready -h localhost -U postgres; do sleep 1; done # Create the database if it doesn't exist - PGPASSWORD=postgres psql -h localhost -U postgres -c "CREATE DATABASE IF NOT EXISTS quantum_flow_test;" + PGPASSWORD=postgres psql -h localhost -U postgres -c "CREATE DATABASE IF NOT EXISTS singularity_workflow_test;" # pgmq extension is pre-installed in tembo/pgmq image - PGPASSWORD=postgres psql -h localhost -U postgres -d quantum_flow_test -c "CREATE EXTENSION IF NOT EXISTS pgmq;" + PGPASSWORD=postgres psql -h localhost -U postgres -d singularity_workflow_test -c "CREATE EXTENSION IF NOT EXISTS pgmq;" - name: Restore dependencies cache uses: actions/cache@v4 @@ -61,7 +61,7 @@ jobs: MIX_ENV: test POSTGRES_USER: postgres POSTGRES_PASSWORD: postgres - POSTGRES_DB: quantum_flow_test + POSTGRES_DB: singularity_workflow_test POSTGRES_HOST: localhost - name: Check formatting diff --git a/CHANGELOG.md b/CHANGELOG.md index 33a6255..966aa9a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -25,14 +25,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added -- Initial release of QuantumFlow - Elixir implementation of QuantumFlow's database-driven DAG execution -- Complete feature parity with QuantumFlow including: +- Initial release of Singularity.Workflow - Elixir implementation of database-driven DAG execution +- Complete feature parity including: - **DAG Workflow Support**: Define workflows with explicit dependencies between steps - **Parallel Execution**: Automatically execute independent steps in parallel - **Map Steps**: Execute the same step across multiple items (map/reduce pattern) - **Dependency Merging**: Steps can depend on multiple other steps - **Database-First Coordination**: PostgreSQL + pgmq for reliable task coordination - - **Multi-Instance Scaling**: Multiple QuantumFlow instances can safely execute the same workflows + - **Multi-Instance Scaling**: Multiple Singularity.Workflow instances can safely execute the same workflows - **Visibility Timeout Pattern**: Automatic retry if task executor crashes - **Comprehensive Testing**: 160+ tests covering all execution paths @@ -87,7 +87,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Known Limitations (v0.1.0) -- Step definitions use legacy Elixir module functions (not JSON-based like QuantumFlow) +- Step definitions use Elixir module functions - Plan to support JSON workflow definitions in v0.2.0 - Conditional step execution not yet supported - Plan for v0.2.0 diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ce2b87e..5e4260c 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,6 +1,6 @@ -# Contributing to QuantumFlow +# Contributing to Singularity.Workflow -Thank you for your interest in QuantumFlow! This document provides guidelines for contributing to the project. +Thank you for your interest in Singularity.Workflow! This document provides guidelines for contributing to the project. ## Getting Started @@ -48,7 +48,7 @@ mix test mix test # Run specific test file -mix test test/QuantumFlow/executor_test.exs +mix test test/singularity_workflow/executor_test.exs # Run tests with coverage report mix test.coverage @@ -59,7 +59,7 @@ mix test.watch ### Code Quality -QuantumFlow enforces high code quality standards: +Singularity.Workflow enforces high code quality standards: ```bash # Run all quality checks (recommended before committing) @@ -75,7 +75,7 @@ mix deps.audit # Check dependencies for vulnerabilities ### Code Style -QuantumFlow follows standard Elixir conventions: +Singularity.Workflow follows standard Elixir conventions: 1. **Formatting**: Run `mix format` before committing 2. **Line Length**: Maximum 100 characters (enforced in .formatter.exs) @@ -138,7 +138,7 @@ All code contributions must include: ### Database-First Design -QuantumFlow uses PostgreSQL as the source of truth. When adding features: +Singularity.Workflow uses PostgreSQL as the source of truth. When adding features: 1. **Schema Changes**: Create migrations for all schema additions 2. **SQL Functions**: Complex logic lives in PostgreSQL functions (for atomicity) @@ -186,7 +186,7 @@ When working with workflows: ``` singularity_workflow/ -├── lib/QuantumFlow/ +├── lib/singularity_workflow/ │ ├── executor.ex # Main entry point │ ├── flow_builder.ex # Dynamic workflow API │ ├── repo.ex # Ecto repository @@ -200,7 +200,7 @@ singularity_workflow/ ├── priv/repo/ │ └── migrations/ # Database migrations ├── test/ -│ ├── QuantumFlow/ +│ ├── singularity_workflow/ │ │ └── *_test.exs # Unit/integration tests │ └── support/ │ └── sql_case.ex # Test helpers @@ -324,7 +324,7 @@ Reviewers may request changes. Please: ## Release Process -QuantumFlow follows [Semantic Versioning](https://semver.org/): +Singularity.Workflow follows [Semantic Versioning](https://semver.org/): - **0.1.0** (current): Initial release, API may change - **0.x.0**: Minor versions (new features, API additions) @@ -348,4 +348,4 @@ Only maintainers can publish releases: - **Architecture questions**: See [ARCHITECTURE.md](docs/ARCHITECTURE.md) - **Issue or PR**: Open a GitHub issue -Thank you for contributing to QuantumFlow! +Thank you for contributing to Singularity.Workflow! diff --git a/GETTING_STARTED.md b/GETTING_STARTED.md index d17748e..b9405d2 100644 --- a/GETTING_STARTED.md +++ b/GETTING_STARTED.md @@ -1,6 +1,6 @@ -# Getting Started with QuantumFlow +# Getting Started with Singularity.Workflow -QuantumFlow is an Elixir implementation of [QuantumFlow](https://github.com/singularity_workflow-dev/QuantumFlow), a database-driven DAG execution engine. This guide walks you through installation, basic setup, and running your first workflow. +Singularity.Workflow is an Elixir implementation of [Singularity.Workflow](https://github.com/singularity_workflow-dev/Singularity.Workflow), a database-driven DAG execution engine. This guide walks you through installation, basic setup, and running your first workflow. ## Installation @@ -22,7 +22,7 @@ mix deps.get ## Database Setup -QuantumFlow requires PostgreSQL 14+ with the `pgmq` extension: +Singularity.Workflow requires PostgreSQL 14+ with the `pgmq` extension: ### 1. Create a PostgreSQL Database @@ -30,7 +30,7 @@ QuantumFlow requires PostgreSQL 14+ with the `pgmq` extension: createdb my_app ``` -### 2. Add QuantumFlow Repository +### 2. Add Singularity.Workflow Repository Configure Ecto in your app to include the Singularity.Workflow.Repo: @@ -57,7 +57,7 @@ psql my_app -c "CREATE EXTENSION IF NOT EXISTS pgmq" ### 4. Run Migrations ```bash -# Generate migrations for QuantumFlow tables +# Generate migrations for Singularity.Workflow tables mix ecto.gen.migration init_singularity_workflow # Run all migrations @@ -155,7 +155,7 @@ IO.inspect(run.status) # => "completed" ## DAG Workflows with Dependencies -QuantumFlow supports complex DAG workflows with parallel execution and dependency management: +Singularity.Workflow supports complex DAG workflows with parallel execution and dependency management: ```elixir defmodule MyApp.Workflows.DataPipeline do @@ -394,7 +394,7 @@ For the complete guide, see [HTDAG_ORCHESTRATOR_GUIDE.md](docs/HTDAG_ORCHESTRATO ## Configuration -QuantumFlow respects these environment variables: +Singularity.Workflow respects these environment variables: ```bash # PostgreSQL connection @@ -448,7 +448,7 @@ Executor.execute_pending_tasks() ### Type errors in custom workflows -QuantumFlow uses Dialyzer for type checking. Run: +Singularity.Workflow uses Dialyzer for type checking. Run: ```bash mix dialyzer diff --git a/SECURITY.md b/SECURITY.md index 5d73154..0ff24f3 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -22,7 +22,7 @@ Include in your report: ## Security Considerations -QuantumFlow is designed for workflow orchestration in trusted environments. Here are important security practices: +Singularity.Workflow is designed for workflow orchestration in trusted environments. Here are important security practices: ### Database Security @@ -107,11 +107,11 @@ QuantumFlow is designed for workflow orchestration in trusted environments. Here ## Security Audit -QuantumFlow has been reviewed for common vulnerabilities. See [SECURITY_AUDIT.md](docs/SECURITY_AUDIT.md) for detailed findings. +Singularity.Workflow has been reviewed for common vulnerabilities. See [SECURITY_AUDIT.md](docs/SECURITY_AUDIT.md) for detailed findings. ## Dependencies -QuantumFlow uses well-maintained Elixir packages. Dependency security is monitored with: +Singularity.Workflow uses well-maintained Elixir packages. Dependency security is monitored with: - `mix deps.audit` - Check for known vulnerabilities - Regular dependency updates - GitHub Dependabot alerts @@ -150,4 +150,4 @@ For security-related questions that aren't vulnerabilities, open an issue with t --- Last Updated: 2025-01-10 -QuantumFlow Version: 0.1.0 +Singularity.Workflow Version: 1.0.x diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 294631a..58fe2b9 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,6 +1,6 @@ -# QuantumFlow Architecture +# Singularity.Workflow Architecture -QuantumFlow is an Elixir implementation of QuantumFlow's database-driven DAG execution engine. This document explains the internal architecture, design decisions, and how components interact. +Singularity.Workflow is an Elixir implementation of Singularity.Workflow's database-driven DAG execution engine. This document explains the internal architecture, design decisions, and how components interact. ## Architecture Overview @@ -29,7 +29,7 @@ graph TB ### Layer Stack -QuantumFlow is organized into three layers: +Singularity.Workflow is organized into three layers: 1. **Application Layer** - Workflow definitions, user code entry points 2. **Orchestration Layer** - DAG parsing/validation and task execution coordination @@ -41,7 +41,7 @@ The DAG layer handles workflow definition parsing, validation, and graph analysi ### Key Modules -**Singularity.Workflow.DAG.WorkflowDefinition** (`lib/QuantumFlow/dag/workflow_definition.ex`) +**Singularity.Workflow.DAG.WorkflowDefinition** (`lib/Singularity.Workflow/dag/workflow_definition.ex`) - Parses JSON workflow definitions - Validates step structure and dependencies - Detects cycles to prevent infinite loops @@ -62,7 +62,7 @@ definition = %{ {:ok, workflow} = WorkflowDefinition.parse(definition) ``` -**Singularity.Workflow.DAG.DynamicWorkflowLoader** (`lib/QuantumFlow/dag/dynamic_workflow_loader.ex`) +**Singularity.Workflow.DAG.DynamicWorkflowLoader** (`lib/Singularity.Workflow/dag/dynamic_workflow_loader.ex`) - Loads workflow definitions from modules at runtime - Implements dynamic behavior callbacks - Bridges workflow code and engine execution @@ -104,7 +104,7 @@ The execution layer orchestrates task processing, state management, and completi ### Key Modules -**Singularity.Workflow.Executor** (`lib/QuantumFlow/executor.ex`) +**Singularity.Workflow.Executor** (`lib/Singularity.Workflow/executor.ex`) - Main entry point for starting and monitoring workflows - Delegates to DAG for definition parsing - Initializes workflow state in database @@ -117,13 +117,13 @@ The execution layer orchestrates task processing, state management, and completi {:ok, run} = Executor.status(run_id) ``` -**Singularity.Workflow.DAG.RunInitializer** (`lib/QuantumFlow/dag/run_initializer.ex`) +**Singularity.Workflow.DAG.RunInitializer** (`lib/Singularity.Workflow/dag/run_initializer.ex`) - Creates workflow_runs record - Initializes step_states for all workflow steps - Sets up step_dependencies edges - For map steps, creates one task per item -**Singularity.Workflow.DAG.TaskExecutor** (`lib/QuantumFlow/dag/task_executor.ex`) +**Singularity.Workflow.DAG.TaskExecutor** (`lib/Singularity.Workflow/dag/task_executor.ex`) - Polls pgmq queue for pending tasks - Executes tasks via workflow callback functions - Handles retries and error handling @@ -198,7 +198,7 @@ sequenceDiagram ### Counter-Based Coordination -QuantumFlow uses two counters to track completion: +Singularity.Workflow uses two counters to track completion: ```mermaid %%{init: {'theme':'dark'}}%% @@ -414,7 +414,7 @@ graph LR ## Layer 3: HTDAG Orchestration (Goal-Driven Workflows) -QuantumFlow includes an optional **Hierarchical Task DAG (HTDAG)** layer for goal-driven workflow composition: +Singularity.Workflow includes an optional **Hierarchical Task DAG (HTDAG)** layer for goal-driven workflow composition: ``` ┌─────────────────────────────────────────────────────────┐ @@ -457,7 +457,7 @@ QuantumFlow includes an optional **Hierarchical Task DAG (HTDAG)** layer for goa | Component | Purpose | Input | Output | |-----------|---------|-------|--------| | **Decomposer Function** | Custom goal → task graph logic | Goal (string) | Task Graph (JSON) | -| **Orchestrator** | Converts task graphs to executable workflows | Task Graph | QuantumFlow Workflow | +| **Orchestrator** | Converts task graphs to executable workflows | Task Graph | Singularity.Workflow Workflow | | **OrchestratorOptimizer** | Learns from execution patterns | Workflow + Metrics | Optimized Workflow | | **WorkflowComposer** | Unified high-level API | Goal + Decomposer + Steps | Execution Result | @@ -729,21 +729,21 @@ Workflows implement `Singularity.Workflow.Executor.Workflow` behavior with callb # Called when any step fails ``` -## Comparison with QuantumFlow (Python) +## Comparison with Singularity.Workflow (Python) -QuantumFlow is a faithful Elixir port of QuantumFlow with identical execution semantics: +Singularity.Workflow is a faithful Elixir port of Singularity.Workflow with identical execution semantics: ```mermaid %%{init: {'theme':'dark'}}%% graph TB - subgraph "QuantumFlow (Python)" + subgraph "Singularity.Workflow (Python)" PL["Python
asyncio runtime"] PDAG["JSON parsing
DFS validation"] PQ["pgmq-python
driver"] PDB["PostgreSQL
same schema"] end - subgraph "QuantumFlow (Elixir)" + subgraph "Singularity.Workflow (Elixir)" EL["Elixir
BEAM VM"] EDAG["Module parsing
DFS validation"] EQ["Postgrex
driver"] @@ -777,7 +777,7 @@ graph TB ### Feature Comparison -| Feature | QuantumFlow | QuantumFlow | +| Feature | Singularity.Workflow | Singularity.Workflow | |---------|--------|----------| | **DAG Parsing** | JSON parsing | Module parsing + JSON | | **Cycle Detection** | DFS | DFS (identical algorithm) | @@ -789,7 +789,7 @@ graph TB | **Error Handling** | try/except | {:ok, result} \| {:error, reason} | | **Type Safety** | Type hints | Dialyzer/specs | -### Why QuantumFlow Over QuantumFlow? +### Why Singularity.Workflow Over Singularity.Workflow? ✅ **BEAM advantages:** - Lightweight processes (millions possible) @@ -818,7 +818,7 @@ graph TB - **Bottleneck**: PostgreSQL connection pool, pgmq queue throughput ### Scalability -- **Horizontal**: Add more QuantumFlow instances polling same pgmq queue +- **Horizontal**: Add more Singularity.Workflow instances polling same pgmq queue - **Vertical**: Increase PostgreSQL resources (CPU, RAM, I/O) - **Maximum**: Limited by PostgreSQL capacity (10K+ tasks/second possible) @@ -836,7 +836,7 @@ Possible enhancements without breaking the core architecture: ## Testing Strategy -QuantumFlow uses: +Singularity.Workflow uses: - **Unit tests**: SQL logic, cycle detection algorithm - **Integration tests**: Full workflow execution end-to-end - **Mock workflows**: Deterministic testing without external dependencies diff --git a/docs/GITHUB_REPOSITORY_SETUP.md b/docs/GITHUB_REPOSITORY_SETUP.md index 44a00a2..05626f9 100644 --- a/docs/GITHUB_REPOSITORY_SETUP.md +++ b/docs/GITHUB_REPOSITORY_SETUP.md @@ -1,6 +1,6 @@ -# GitHub Repository Setup for QuantumFlow +# GitHub Repository Setup for Singularity.Workflow -This guide covers configuring the GitHub repository for QuantumFlow v0.1.0 release. +This guide covers configuring the GitHub repository for Singularity.Workflow v0.1.0 release. ## Repository Description @@ -8,7 +8,7 @@ Update the repository description to help potential users understand the project ### Current Description ``` -Elixir implementation of QuantumFlow's database-driven DAG execution engine +Elixir implementation of Singularity.Workflow's database-driven DAG execution engine ``` ### Setup Steps @@ -18,7 +18,7 @@ Elixir implementation of QuantumFlow's database-driven DAG execution engine 3. In the "General" section at the top, find the **Description** field 4. Update to: ``` - Elixir implementation of QuantumFlow - database-driven DAG execution engine with 100% feature parity. + Elixir implementation of Singularity.Workflow - database-driven DAG execution engine with 100% feature parity. Parallel execution, map steps, dependency merging, multi-instance scaling via PostgreSQL + pgmq. ``` 5. Add a **Website** URL (optional): @@ -51,7 +51,7 @@ Discussions provide a space for Q&A and community discussion: - Q&A - Questions about usage and best practices - Announcements - Release notes and updates - Ideas - Feature requests and suggestions -- Show and tell - Community projects using QuantumFlow +- Show and tell - Community projects using Singularity.Workflow ## Repository Topics @@ -65,7 +65,7 @@ Add topics to help discoverability: - `workflow` - `dag` - `task-execution` - - `QuantumFlow` + - `Singularity.Workflow` - `distributed-systems` ## Branch Protection Rules (Optional) @@ -84,7 +84,7 @@ Protect the `main` branch to enforce quality standards: ## Labels for Issues -GitHub creates default labels. Customize them for QuantumFlow: +GitHub creates default labels. Customize them for Singularity.Workflow: 1. Go to **Issues → Labels** 2. Keep/customize these labels: @@ -96,7 +96,7 @@ GitHub creates default labels. Customize them for QuantumFlow: - `question` - User questions (purple) - `test` - Test-related (yellow) -3. Add QuantumFlow-specific labels: +3. Add Singularity.Workflow-specific labels: - `migration` - Related to database migrations - `performance` - Performance improvements/issues - `security` - Security concerns @@ -180,7 +180,7 @@ When publishing v0.1.0: - Go to **Code → Releases** - Click **Draft a new release** - Tag: `v0.1.0` - - Title: `QuantumFlow v0.1.0` + - Title: `Singularity.Workflow v0.1.0` - Description: Copy from CHANGELOG.md 2. **Publish Release**: @@ -244,7 +244,7 @@ What should happen instead. - Elixir version: `elixir --version` - PostgreSQL version: `psql --version` -- QuantumFlow version: 0.1.0 +- Singularity.Workflow version: 0.1.0 ## Additional context @@ -294,7 +294,7 @@ We will respond within 48 hours and work on a fix in a private security advisory ## Security Considerations -QuantumFlow is designed for internal use cases. Key security aspects: +Singularity.Workflow is designed for internal use cases. Key security aspects: - Database connections should use strong credentials - pgmq queue should not be publicly accessible diff --git a/docs/HTDAG_ORCHESTRATOR_GUIDE.md b/docs/HTDAG_ORCHESTRATOR_GUIDE.md index a6e72ed..cea1f86 100644 --- a/docs/HTDAG_ORCHESTRATOR_GUIDE.md +++ b/docs/HTDAG_ORCHESTRATOR_GUIDE.md @@ -1,6 +1,6 @@ # HTDAG Orchestrator Guide -Comprehensive guide to QuantumFlow's Hierarchical Task Directed Acyclic Graph (HTDAG) orchestration system for goal-driven workflow execution. +Comprehensive guide to Singularity.Workflow's Hierarchical Task Directed Acyclic Graph (HTDAG) orchestration system for goal-driven workflow execution. ## Table of Contents @@ -816,5 +816,5 @@ end - [README.md](../README.md) - Project overview - [ARCHITECTURE.md](./ARCHITECTURE.md) - System architecture -- [SINGULARITY_WORKFLOW_REFERENCE.md](./SINGULARITY_WORKFLOW_REFERENCE.md) - QuantumFlow comparison +- [SINGULARITY_WORKFLOW_REFERENCE.md](./SINGULARITY_WORKFLOW_REFERENCE.md) - Singularity.Workflow comparison - [DYNAMIC_WORKFLOWS_GUIDE.md](./DYNAMIC_WORKFLOWS_GUIDE.md) - FlowBuilder examples diff --git a/docs/QUANTUM_FLOW_REFERENCE.md b/docs/QUANTUM_FLOW_REFERENCE.md index 8726033..e7d08b0 100644 --- a/docs/QUANTUM_FLOW_REFERENCE.md +++ b/docs/QUANTUM_FLOW_REFERENCE.md @@ -1,8 +1,8 @@ -# QuantumFlow Reference - What's in /tmp/QuantumFlow +# Singularity.Workflow Reference - What's in /tmp/Singularity.Workflow -Complete overview of the official QuantumFlow TypeScript implementation. +Complete overview of the official Singularity.Workflow TypeScript implementation. -**Location:** `/tmp/QuantumFlow/` +**Location:** `/tmp/Singularity.Workflow/` **What it is:** Official TypeScript workflow orchestration using PostgreSQL + pgmq **Our achievement:** singularity_workflow = 100% feature parity with this @@ -11,9 +11,9 @@ Complete overview of the official QuantumFlow TypeScript implementation. ## Directory Structure ``` -/tmp/QuantumFlow/ +/tmp/Singularity.Workflow/ ├── pkgs/ -│ ├── cli/ # Command-line tool for QuantumFlow +│ ├── cli/ # Command-line tool for Singularity.Workflow │ ├── client/ # TypeScript client library │ ├── core/ # ⭐ Core SQL schemas (what we matched!) │ ├── dsl/ # TypeScript DSL for workflow definitions @@ -29,12 +29,12 @@ Complete overview of the official QuantumFlow TypeScript implementation. ## 1. Core SQL Schemas (`pkgs/core/schemas/`) -**The heart of QuantumFlow** - All the SQL we matched in singularity_workflow: +**The heart of Singularity.Workflow** - All the SQL we matched in singularity_workflow: | File | Size | What It Does | Our Migration | |------|------|--------------|---------------| | `0010_extensions.sql` | 89B | Installs pgmq extension | `20251025150000_add_pgmq_extension.exs` | -| `0020_schemas.sql` | 54B | Creates QuantumFlow schema | Implicit in migrations | +| `0020_schemas.sql` | 54B | Creates Singularity.Workflow schema | Implicit in migrations | | `0030_utilities.sql` | 626B | Utility functions (is_valid_slug) | `20251025160000_add_is_valid_slug_function.exs` | | `0040_types.sql` | 164B | Custom types | Embedded in migrations | | `0050_tables_definitions.sql` | 2.4K | workflows, workflow_steps, deps tables | `20251025160001_create_workflow_definition_tables.exs` | @@ -59,7 +59,7 @@ Complete overview of the official QuantumFlow TypeScript implementation. ### Simple Flow (`example-flow.ts`) ```typescript -import { Flow } from '@QuantumFlow/dsl'; +import { Flow } from '@Singularity.Workflow/dsl'; export const ExampleFlow = new Flow<{ value: number }>({ slug: 'example_flow', @@ -185,7 +185,7 @@ export class EdgeWorker { ### Key Features -| Feature | QuantumFlow Edge Worker | singularity_workflow TaskExecutor | +| Feature | Singularity.Workflow Edge Worker | singularity_workflow TaskExecutor | |---------|-------------------|----------------------| | **Runtime** | Deno (Supabase Edge Function) | BEAM/Erlang | | **Polling** | `read_with_poll()` (5s default) | `read_with_poll()` (5s configurable) | @@ -196,10 +196,10 @@ export class EdgeWorker { ### Example Usage -**QuantumFlow (TypeScript):** +**Singularity.Workflow (TypeScript):** ```typescript -import { EdgeWorker } from '@QuantumFlow/edge-worker'; +import { EdgeWorker } from '@Singularity.Workflow/edge-worker'; import { MyFlow } from './flows.js'; EdgeWorker.start(MyFlow, { @@ -229,7 +229,7 @@ EdgeWorker.start(MyFlow, { ### Documentation Structure ``` -/tmp/QuantumFlow/pkgs/website/src/content/docs/ +/tmp/Singularity.Workflow/pkgs/website/src/content/docs/ ├── index.mdx # Homepage ├── get-started/ # Getting started guides ├── concepts/ # Core concepts (DAGs, map steps, etc.) @@ -258,7 +258,7 @@ EdgeWorker.start(MyFlow, { **What it is:** TypeScript DSL for defining workflows with full type safety ```typescript -import { Flow } from '@QuantumFlow/dsl'; +import { Flow } from '@Singularity.Workflow/dsl'; // Type-safe workflow definition const MyFlow = new Flow<{ userId: string }>({ @@ -285,7 +285,7 @@ const MyFlow = new Flow<{ userId: string }>({ **What it is:** TypeScript client for starting workflows and querying status ```typescript -import { createClient } from '@QuantumFlow/client'; +import { createClient } from '@Singularity.Workflow/client'; const client = createClient(supabase); @@ -302,12 +302,12 @@ const status = await client.getStatus(runId); ## 7. CLI (`pkgs/cli/`) -**What it is:** Command-line tool for QuantumFlow operations +**What it is:** Command-line tool for Singularity.Workflow operations ```bash -npx QuantumFlow install # Install SQL schemas -npx QuantumFlow migrate # Run migrations -npx QuantumFlow compile # Compile DSL to SQL +npx Singularity.Workflow install # Install SQL schemas +npx Singularity.Workflow migrate # Run migrations +npx Singularity.Workflow compile # Compile DSL to SQL ``` **singularity_workflow Equivalent:** Mix tasks @@ -319,9 +319,9 @@ mix test # Run tests --- -## Key Differences: QuantumFlow vs singularity_workflow +## Key Differences: Singularity.Workflow vs singularity_workflow -| Aspect | QuantumFlow | singularity_workflow | +| Aspect | Singularity.Workflow | singularity_workflow | |--------|--------|-----------| | **Language** | TypeScript | Elixir | | **Runtime** | Deno/Node.js | BEAM/Erlang | @@ -338,7 +338,7 @@ mix test # Run tests ## Summary -**What we learned from /tmp/QuantumFlow:** +**What we learned from /tmp/Singularity.Workflow:** 1. ✅ **SQL Core** - Matched all 22 SQL schema files 2. ✅ **Example Flows** - Understood patterns (simple, map, wide) @@ -348,12 +348,12 @@ mix test # Run tests 6. ✅ **Client** - Created Executor + Ecto query API 7. ✅ **CLI** - Created Mix tasks -**Result:** singularity_workflow = 100% feature parity with QuantumFlow! 🎯 +**Result:** singularity_workflow = 100% feature parity with Singularity.Workflow! 🎯 --- **References:** -- QuantumFlow GitHub: https://github.com/QuantumFlow/QuantumFlow -- QuantumFlow Website: https://Singularity.Workflow.dev +- Singularity.Workflow GitHub: https://github.com/Singularity.Workflow/Singularity.Workflow +- Singularity.Workflow Website: https://Singularity.Workflow.dev - singularity_workflow: Our standalone Elixir implementation diff --git a/docs/SCHEMA_MIGRATION_GUIDE.md b/docs/SCHEMA_MIGRATION_GUIDE.md new file mode 100644 index 0000000..d2f0b4e --- /dev/null +++ b/docs/SCHEMA_MIGRATION_GUIDE.md @@ -0,0 +1,141 @@ +# Schema Migration Guide: QuantumFlow → singularity_workflow + +This guide explains how to safely migrate existing databases from the old `QuantumFlow` PostgreSQL schema to the new `singularity_workflow` schema. + +## For Fresh Installations + +If you're installing Singularity.Workflow on a fresh database, you don't need to do anything special. Just run: + +```bash +mix ecto.migrate +``` + +The system will automatically create the `singularity_workflow` schema with all functions. + +## For Existing Databases (Upgrade Path) + +If you have an existing database that was created with the `QuantumFlow` schema, follow these steps: + +### Step 1: Backup Your Database + +**IMPORTANT**: Always backup your database before running migrations. + +```bash +pg_dump -Fc your_database_name > backup_before_schema_rename.dump +``` + +### Step 2: Run the Migration + +The migration is designed to be safe and idempotent: + +```bash +mix ecto.migrate +``` + +The migration (`20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs`) will: +- Check if the `QuantumFlow` schema exists +- If it exists, rename it to `singularity_workflow` +- If it doesn't exist, do nothing (assumes fresh install) + +### Step 3: Verify the Migration + +After running the migration, verify that: + +```sql +-- Check that the new schema exists +SELECT schema_name FROM information_schema.schemata WHERE schema_name = 'singularity_workflow'; + +-- Check that all functions are in the new schema +SELECT routine_name +FROM information_schema.routines +WHERE routine_schema = 'singularity_workflow'; +``` + +You should see all 10 functions: +- `read_with_poll` +- `ensure_workflow_queue` +- `create_flow` +- `add_step` +- `fail_task` +- `calculate_retry_delay` +- `maybe_complete_run` +- `set_vt_batch` +- `is_valid_slug` +- `cascade_complete_taskless_steps` + +### Step 4: (Optional) Rollback + +If you need to rollback the migration: + +```bash +mix ecto.rollback --step 1 +``` + +This will rename the schema back from `singularity_workflow` to `QuantumFlow`. + +## Migration Details + +### What Gets Renamed + +- **PostgreSQL Schema**: `QuantumFlow` → `singularity_workflow` +- **All Functions**: Automatically moved to the new schema +- **All Data**: Preserved (schema rename doesn't affect data) + +### What Doesn't Change + +- Table structures remain the same +- Data in tables is preserved +- Workflow execution logic is unchanged + +### Idempotency + +The migration can be run multiple times safely. If the `QuantumFlow` schema doesn't exist, the migration does nothing. + +## Troubleshooting + +### Error: "schema 'QuantumFlow' does not exist" + +This is normal if you're on a fresh installation. The migration will skip the rename and log a notice. + +### Error: "schema 'singularity_workflow' already exists" + +This could happen if you manually created the schema. The migration will fail to prevent data loss. Options: + +1. Drop the manually-created `singularity_workflow` schema first (if it's empty) +2. Or manually rename `QuantumFlow` to `singularity_workflow` before running migrations + +### Verification Failed + +If functions are missing after migration: + +```sql +-- Check which schema they're in +SELECT routine_schema, routine_name +FROM information_schema.routines +WHERE routine_name IN ('read_with_poll', 'create_flow', 'add_step'); +``` + +If they're still in `QuantumFlow`, the migration didn't run. Check migration status: + +```bash +mix ecto.migrations +``` + +## Production Deployment Checklist + +- [ ] Database backup completed +- [ ] Reviewed migration plan with team +- [ ] Tested migration in staging environment +- [ ] Scheduled maintenance window (if required) +- [ ] Run `mix ecto.migrate` +- [ ] Verify all functions are in `singularity_workflow` schema +- [ ] Verify application still works +- [ ] Monitor logs for any schema-related errors + +## Support + +If you encounter issues during migration, please open an issue on GitHub with: +- PostgreSQL version +- Output of `mix ecto.migrations` +- Any error messages +- Result of the verification queries above diff --git a/docs/SECURITY_AUDIT.md b/docs/SECURITY_AUDIT.md index aebfb10..fc50cda 100644 --- a/docs/SECURITY_AUDIT.md +++ b/docs/SECURITY_AUDIT.md @@ -140,9 +140,9 @@ Task.async_stream(tasks, fn task -> ... end, --- -## Comparison to QuantumFlow (TypeScript) +## Comparison to Singularity.Workflow (TypeScript) -| Security Aspect | QuantumFlow | singularity_workflow | Notes | +| Security Aspect | Singularity.Workflow | singularity_workflow | Notes | |----------------|--------|-----------|-------| | **SQL Injection** | ✅ Safe (pg parameterization) | ✅ Safe (Ecto parameterization) | Both use driver-level protection | | **Type Safety** | ✅ TypeScript | ✅ Dialyzer + @spec | Compile-time checks | @@ -150,7 +150,7 @@ Task.async_stream(tasks, fn task -> ... end, | **Error Handling** | ✅ try/catch | ✅ {:ok, _} / {:error, _} | BEAM supervision more robust | | **Process Isolation** | ❌ Single-threaded JS | ✅ BEAM process isolation | singularity_workflow has better fault isolation | -**Verdict:** singularity_workflow has EQUAL or BETTER security than QuantumFlow! +**Verdict:** singularity_workflow has EQUAL or BETTER security than Singularity.Workflow! --- diff --git a/docs/TESTING_GUIDE.md b/docs/TESTING_GUIDE.md index a913721..cc66782 100644 --- a/docs/TESTING_GUIDE.md +++ b/docs/TESTING_GUIDE.md @@ -12,7 +12,7 @@ mix test mix test --exclude flow_builder_test # Run specific test file -mix test test/QuantumFlow/complete_task_test.exs +mix test test/Singularity.Workflow/complete_task_test.exs # Run with verbose output mix test --trace @@ -120,7 +120,7 @@ This creates: - `workflow_step_tasks` - Individual tasks - `workflow_runs` - Run instances - `workflow_step_dependencies` - Step dependencies -- QuantumFlow SQL functions +- Singularity.Workflow SQL functions - pgmq queue tables ## Running Tests @@ -151,16 +151,16 @@ This runs 336/336 tests and passes on all PostgreSQL versions. ```bash # Test complete_task functionality -mix test test/QuantumFlow/complete_task_test.exs +mix test test/Singularity.Workflow/complete_task_test.exs # Test clock abstraction -mix test test/QuantumFlow/clock_test.exs +mix test test/Singularity.Workflow/clock_test.exs # Test step task logic -mix test test/QuantumFlow/step_task_test.exs +mix test test/Singularity.Workflow/step_task_test.exs # Test workflow execution -mix test test/QuantumFlow/workflow_run_test.exs +mix test test/Singularity.Workflow/workflow_run_test.exs ``` ### Run with Verbose Output @@ -212,10 +212,10 @@ mix test --statistics ```bash # Run with debug SQL output -ECTO_LOG_LEVEL=debug mix test test/QuantumFlow/complete_task_test.exs +ECTO_LOG_LEVEL=debug mix test test/Singularity.Workflow/complete_task_test.exs # Run with very verbose output -mix test test/QuantumFlow/complete_task_test.exs --trace +mix test test/Singularity.Workflow/complete_task_test.exs --trace ``` ### Check Test Helpers diff --git a/docs/architecture_diagrams.md b/docs/architecture_diagrams.md index 2d9d973..9be2bfb 100644 --- a/docs/architecture_diagrams.md +++ b/docs/architecture_diagrams.md @@ -1,4 +1,4 @@ -# QuantumFlow Architecture Diagrams +# Singularity.Workflow Architecture Diagrams This document contains comprehensive Mermaid diagrams showing the PGMQ + NOTIFY architecture and data flow. @@ -254,9 +254,9 @@ graph TB end subgraph "Application Tier" - B[QuantumFlow App 1] - C[QuantumFlow App 2] - D[QuantumFlow App 3] + B[Singularity.Workflow App 1] + C[Singularity.Workflow App 2] + D[Singularity.Workflow App 3] end subgraph "Database Tier" @@ -291,8 +291,8 @@ graph TB ```mermaid graph TB subgraph "Kubernetes Cluster" - subgraph "Namespace: QuantumFlow" - A[QuantumFlow Deployment] + subgraph "Namespace: Singularity.Workflow" + A[Singularity.Workflow Deployment] B[PostgreSQL StatefulSet] C[pgmq Extension] end @@ -414,17 +414,17 @@ sequenceDiagram ```mermaid graph TB subgraph "Instance 1" - A1[QuantumFlow App 1] + A1[Singularity.Workflow App 1] B1[Local Tasks] end subgraph "Instance 2" - A2[QuantumFlow App 2] + A2[Singularity.Workflow App 2] B2[Local Tasks] end subgraph "Instance 3" - A3[QuantumFlow App 3] + A3[Singularity.Workflow App 3] B3[Local Tasks] end @@ -444,4 +444,4 @@ graph TB E --> A3 ``` -These diagrams provide a comprehensive view of the QuantumFlow architecture, showing how PGMQ + NOTIFY integration works across all layers of the system. \ No newline at end of file +These diagrams provide a comprehensive view of the Singularity.Workflow architecture, showing how PGMQ + NOTIFY integration works across all layers of the system. \ No newline at end of file diff --git a/lib/singularity_workflow.ex b/lib/singularity_workflow.ex index 5e60062..6614c61 100644 --- a/lib/singularity_workflow.ex +++ b/lib/singularity_workflow.ex @@ -77,7 +77,7 @@ defmodule Singularity.Workflow do ## Architecture - singularity_workflow uses the same architecture as QuantumFlow (TypeScript): + singularity_workflow provides complete workflow orchestration capabilities: - **pgmq Extension** - PostgreSQL Message Queue for task coordination - **Database-Driven** - Task state persisted in PostgreSQL tables @@ -161,20 +161,6 @@ defmodule Singularity.Workflow do - **pgmq extension 1.4.4+** - `CREATE EXTENSION pgmq` - **Ecto & Postgrex** - For database access - ## Comparison with QuantumFlow - - | Feature | QuantumFlow (TypeScript) | singularity_workflow (Elixir) | - |---------|---------------------|---------------------| - | DAG Syntax | ✅ | ✅ | - | pgmq Integration | ✅ | ✅ | - | Parallel Execution | ✅ | ✅ | - | Map Steps | ✅ | ✅ | - | Dependency Merging | ✅ | ✅ | - | Multi-Instance | ✅ | ✅ | - | Database-Driven | ✅ | ✅ | - - **Result: 100% Feature Parity** 🎉 - See `Singularity.Workflow.Executor` for execution options and `Singularity.Workflow.DAG.WorkflowDefinition` for workflow syntax details. diff --git a/lib/singularity_workflow/dag/run_initializer.ex b/lib/singularity_workflow/dag/run_initializer.ex index b6cf365..6482d7a 100644 --- a/lib/singularity_workflow/dag/run_initializer.ex +++ b/lib/singularity_workflow/dag/run_initializer.ex @@ -14,7 +14,7 @@ defmodule Singularity.Workflow.DAG.RunInitializer do 5. Call start_ready_steps() to: - Mark root steps as 'started' - Create workflow_step_tasks records - - Send messages to pgmq queue (matches QuantumFlow architecture) + - Send messages to pgmq queue (matches Singularity.Workflow architecture) ## Example diff --git a/lib/singularity_workflow/dag/task_executor.ex b/lib/singularity_workflow/dag/task_executor.ex index bc9ac4d..f89e3aa 100644 --- a/lib/singularity_workflow/dag/task_executor.ex +++ b/lib/singularity_workflow/dag/task_executor.ex @@ -2,7 +2,7 @@ defmodule Singularity.Workflow.DAG.TaskExecutor do @moduledoc """ Executes workflow step tasks from the database. - Implements the QuantumFlow execution model: + Implements the Singularity.Workflow execution model: 1. Poll for queued tasks 2. Claim a task (mark as 'started') 3. Execute the step function @@ -47,7 +47,7 @@ defmodule Singularity.Workflow.DAG.TaskExecutor do @doc """ Execute all tasks for a workflow run until completion or failure. - Uses pgmq for task coordination (matches QuantumFlow architecture): + Uses pgmq for task coordination (matches Singularity.Workflow architecture): 1. Poll messages from pgmq queue 2. Call start_tasks() to claim tasks 3. Execute step functions @@ -93,9 +93,9 @@ defmodule Singularity.Workflow.DAG.TaskExecutor do # 200ms between polls worker_id = Keyword.get(opts, :worker_id, Ecto.UUID.generate()) batch_size = Keyword.get(opts, :batch_size, 10) - # Poll up to 10 messages at once (matches QuantumFlow default) + # Poll up to 10 messages at once (matches Singularity.Workflow default) max_poll_seconds = Keyword.get(opts, :max_poll_seconds, 5) - # Max time to wait for messages (matches QuantumFlow default) + # Max time to wait for messages (matches Singularity.Workflow default) task_timeout_ms = Keyword.get(opts, :task_timeout_ms, 30_000) # Task execution timeout in milliseconds (default: 30 seconds) @@ -234,7 +234,7 @@ defmodule Singularity.Workflow.DAG.TaskExecutor do end end - # Poll pgmq for messages and execute tasks (matches QuantumFlow architecture) + # Poll pgmq for messages and execute tasks (matches Singularity.Workflow architecture) @spec poll_and_execute_batch( String.t(), WorkflowDefinition.t(), @@ -444,7 +444,7 @@ defmodule Singularity.Workflow.DAG.TaskExecutor do end end - # Complete task successfully (using QuantumFlow complete_task function) + # Complete task successfully (using Singularity.Workflow complete_task function) # # Calls the PostgreSQL complete_task stored function to mark the task as successfully # executed and store the output. This function handles dependency counters, task queue @@ -484,7 +484,7 @@ defmodule Singularity.Workflow.DAG.TaskExecutor do end end - # Complete task with failure (using QuantumFlow fail_task function) + # Complete task with failure (using Singularity.Workflow fail_task function) # # Calls the PostgreSQL fail_task stored function to mark a task as failed with an error # message. This function may increment retry counters or mark the step/workflow as failed diff --git a/lib/singularity_workflow/executor.ex b/lib/singularity_workflow/executor.ex index 12de537..d898ee8 100644 --- a/lib/singularity_workflow/executor.ex +++ b/lib/singularity_workflow/executor.ex @@ -1,6 +1,6 @@ defmodule Singularity.Workflow.Executor do @moduledoc """ - Database-driven DAG workflow executor matching QuantumFlow's architecture. + Database-driven DAG workflow executor matching Singularity.Workflow's architecture. ## Overview diff --git a/lib/singularity_workflow/flow_builder.ex b/lib/singularity_workflow/flow_builder.ex index 03f092c..0ae42b7 100644 --- a/lib/singularity_workflow/flow_builder.ex +++ b/lib/singularity_workflow/flow_builder.ex @@ -71,7 +71,7 @@ defmodule Singularity.Workflow.FlowBuilder do - `repo` - Ecto repo module - `opts` - Options: - `:max_attempts` - Default retry count for all steps (default: 3) - - `:timeout` - Default timeout in seconds (default: 60, matches QuantumFlow) + - `:timeout` - Default timeout in seconds (default: 60, matches Singularity.Workflow) ## Returns diff --git a/lib/singularity_workflow/messaging.ex b/lib/singularity_workflow/messaging.ex index 828f01c..5aeeead 100644 --- a/lib/singularity_workflow/messaging.ex +++ b/lib/singularity_workflow/messaging.ex @@ -1,6 +1,6 @@ defmodule Singularity.Workflow.Messaging do @moduledoc """ - Messaging helpers for publishing QuantumFlow events via PostgreSQL + pgmq. + Messaging helpers for publishing Singularity.Workflow events via PostgreSQL + pgmq. Provides convenience wrappers that resolve the appropriate Ecto repo and delegate to `Singularity.Workflow.Notifications` for durable delivery with NOTIFY. diff --git a/lib/singularity_workflow/notifications.ex b/lib/singularity_workflow/notifications.ex index 4b7cbdf..6022d19 100644 --- a/lib/singularity_workflow/notifications.ex +++ b/lib/singularity_workflow/notifications.ex @@ -1,6 +1,6 @@ defmodule Singularity.Workflow.Notifications.Behaviour do @moduledoc """ - Behaviour definition for QuantumFlow notifications, used for testing and mocking. + Behaviour definition for Singularity.Workflow notifications, used for testing and mocking. """ @callback send_with_notify(String.t(), map(), Ecto.Repo.t()) :: diff --git a/lib/singularity_workflow/orchestrator/README.md b/lib/singularity_workflow/orchestrator/README.md index f7d85f0..1fb0298 100644 --- a/lib/singularity_workflow/orchestrator/README.md +++ b/lib/singularity_workflow/orchestrator/README.md @@ -1,23 +1,23 @@ -# Orchestrator Integration for quantum_flow +# Orchestrator Integration for Singularity.Workflow -This directory contains the Orchestrator (formerly HTDAG: Hierarchical Task Directed Acyclic Graph) integration for quantum_flow, enabling goal-driven workflow creation and execution. +This directory contains the Orchestrator (formerly HTDAG: Hierarchical Task Directed Acyclic Graph) integration for Singularity.Workflow, enabling goal-driven workflow creation and execution. ## Overview -Orchestrator allows you to describe what you want to achieve (goals) rather than how to achieve it (workflow steps). The system automatically decomposes complex goals into hierarchical task graphs and converts them into executable quantum_flow workflows. +Orchestrator allows you to describe what you want to achieve (goals) rather than how to achieve it (workflow steps). The system automatically decomposes complex goals into hierarchical task graphs and converts them into executable Singularity.Workflow workflows. ## Key Components ### Core Modules -- **`QuantumFlow.Orchestrator`** - Main HTDAG functionality for goal decomposition and workflow creation -- **`QuantumFlow.OrchestratorNotifications`** - Real-time event broadcasting for HTDAG workflows -- **`QuantumFlow.WorkflowComposer`** - High-level API for goal-driven workflow composition -- **`QuantumFlow.OrchestratorOptimizer`** - Workflow optimization based on historical performance data +- **`Singularity.Workflow.Orchestrator`** - Main HTDAG functionality for goal decomposition and workflow creation +- **`Singularity.Workflow.OrchestratorNotifications`** - Real-time event broadcasting for HTDAG workflows +- **`Singularity.Workflow.WorkflowComposer`** - High-level API for goal-driven workflow composition +- **`Singularity.Workflow.OrchestratorOptimizer`** - Workflow optimization based on historical performance data ### Example Implementations -- **`QuantumFlow.Orchestrator.ExampleDecomposer`** - Sample decomposer implementations for common workflow types +- **`Singularity.Workflow.Orchestrator.ExampleDecomposer`** - Sample decomposer implementations for common workflow types ## Quick Start @@ -52,7 +52,7 @@ step_functions = %{ ### 3. Compose and Execute Workflow ```elixir -{:ok, result} = QuantumFlow.WorkflowComposer.compose_from_goal( +{:ok, result} = Singularity.Workflow.WorkflowComposer.compose_from_goal( "Build user authentication system", &MyApp.GoalDecomposer.decompose/1, step_functions, @@ -66,7 +66,7 @@ step_functions = %{ ```elixir # Listen for HTDAG events -{:ok, pid} = QuantumFlow.OrchestratorNotifications.listen("my_workflow", MyApp.Repo) +{:ok, pid} = Singularity.Workflow.OrchestratorNotifications.listen("my_workflow", MyApp.Repo) # Handle events receive do @@ -79,7 +79,7 @@ end ```elixir # Optimize workflow based on historical data -{:ok, optimized_workflow} = QuantumFlow.OrchestratorOptimizer.optimize_workflow( +{:ok, optimized_workflow} = Singularity.Workflow.OrchestratorOptimizer.optimize_workflow( workflow, MyApp.Repo, optimization_level: :advanced @@ -90,7 +90,7 @@ end ```elixir # Compose multiple related workflows -{:ok, results} = QuantumFlow.WorkflowComposer.compose_multiple_workflows( +{:ok, results} = Singularity.Workflow.WorkflowComposer.compose_multiple_workflows( "Build complete microservices platform", &MyApp.GoalDecomposer.decompose_complex/1, step_functions, @@ -124,14 +124,14 @@ Events ← Notifications ← Task Events ← Workflow Events ← Execution Event 5. **Flexible**: Works with any decomposer function 6. **Scalable**: Supports complex hierarchical workflows -## Integration with quantum_flow +## Integration with Singularity.Workflow -HTDAG seamlessly integrates with quantum_flow's existing features: +HTDAG seamlessly integrates with Singularity.Workflow's existing features: -- **Workflow Execution**: Uses `QuantumFlow.Executor` for workflow execution -- **Dynamic Workflows**: Uses `QuantumFlow.FlowBuilder` for workflow creation -- **Real-time Notifications**: Uses `QuantumFlow.Notifications` for event broadcasting -- **Multi-instance Support**: Works with quantum_flow's distributed architecture +- **Workflow Execution**: Uses `Singularity.Workflow.Executor` for workflow execution +- **Dynamic Workflows**: Uses `Singularity.Workflow.FlowBuilder` for workflow creation +- **Real-time Notifications**: Uses `Singularity.Workflow.Notifications` for event broadcasting +- **Multi-instance Support**: Works with Singularity.Workflow's distributed architecture ## Best Practices diff --git a/lib/singularity_workflow/orchestrator/schemas.ex b/lib/singularity_workflow/orchestrator/schemas.ex index a15abfc..fec70a8 100644 --- a/lib/singularity_workflow/orchestrator/schemas.ex +++ b/lib/singularity_workflow/orchestrator/schemas.ex @@ -24,7 +24,8 @@ defmodule Singularity.Workflow.Orchestrator.Schemas do field :task_graph, :map field :max_depth, :integer, default: 5 - has_many :workflows, Singularity.Workflow.Orchestrator.Schemas.Workflow, foreign_key: :task_graph_id + has_many :workflows, Singularity.Workflow.Orchestrator.Schemas.Workflow, + foreign_key: :task_graph_id timestamps() end @@ -60,7 +61,8 @@ defmodule Singularity.Workflow.Orchestrator.Schemas do belongs_to :task_graph, Singularity.Workflow.Orchestrator.Schemas.TaskGraph, foreign_key: :task_graph_id - has_many :executions, Singularity.Workflow.Orchestrator.Schemas.Execution, foreign_key: :workflow_id + has_many :executions, Singularity.Workflow.Orchestrator.Schemas.Execution, + foreign_key: :workflow_id has_many :performance_metrics, Singularity.Workflow.Orchestrator.Schemas.PerformanceMetric, foreign_key: :workflow_id @@ -107,7 +109,8 @@ defmodule Singularity.Workflow.Orchestrator.Schemas do field :result, :map field :error_message, :string - belongs_to :workflow, Singularity.Workflow.Orchestrator.Schemas.Workflow, foreign_key: :workflow_id + belongs_to :workflow, Singularity.Workflow.Orchestrator.Schemas.Workflow, + foreign_key: :workflow_id has_many :task_executions, Singularity.Workflow.Orchestrator.Schemas.TaskExecution, foreign_key: :execution_id @@ -157,8 +160,11 @@ defmodule Singularity.Workflow.Orchestrator.Schemas do field :error_message, :string field :retry_count, :integer, default: 0 - belongs_to :execution, Singularity.Workflow.Orchestrator.Schemas.Execution, foreign_key: :execution_id - has_many :events, Singularity.Workflow.Orchestrator.Schemas.Event, foreign_key: :task_execution_id + belongs_to :execution, Singularity.Workflow.Orchestrator.Schemas.Execution, + foreign_key: :execution_id + + has_many :events, Singularity.Workflow.Orchestrator.Schemas.Event, + foreign_key: :task_execution_id timestamps() end @@ -199,7 +205,8 @@ defmodule Singularity.Workflow.Orchestrator.Schemas do field :event_data, :map field :timestamp, :utc_datetime - belongs_to :execution, Singularity.Workflow.Orchestrator.Schemas.Execution, foreign_key: :execution_id + belongs_to :execution, Singularity.Workflow.Orchestrator.Schemas.Execution, + foreign_key: :execution_id belongs_to :task_execution, Singularity.Workflow.Orchestrator.Schemas.TaskExecution, foreign_key: :task_execution_id @@ -245,7 +252,8 @@ defmodule Singularity.Workflow.Orchestrator.Schemas do field :context, :map field :timestamp, :utc_datetime - belongs_to :workflow, Singularity.Workflow.Orchestrator.Schemas.Workflow, foreign_key: :workflow_id + belongs_to :workflow, Singularity.Workflow.Orchestrator.Schemas.Workflow, + foreign_key: :workflow_id timestamps() end diff --git a/lib/singularity_workflow/orchestrator_notifications.ex b/lib/singularity_workflow/orchestrator_notifications.ex index 7e5a6e1..5e1472f 100644 --- a/lib/singularity_workflow/orchestrator_notifications.ex +++ b/lib/singularity_workflow/orchestrator_notifications.ex @@ -68,7 +68,11 @@ defmodule Singularity.Workflow.OrchestratorNotifications do @notifications_env_key :notifications_impl defp notifications_impl do - Application.get_env(:singularity_workflow, @notifications_env_key, Singularity.Workflow.Notifications) + Application.get_env( + :singularity_workflow, + @notifications_env_key, + Singularity.Workflow.Notifications + ) end @doc """ diff --git a/lib/singularity_workflow/step_state.ex b/lib/singularity_workflow/step_state.ex index aac5726..14fe282 100644 --- a/lib/singularity_workflow/step_state.ex +++ b/lib/singularity_workflow/step_state.ex @@ -3,7 +3,7 @@ defmodule Singularity.Workflow.StepState do Ecto schema for workflow_step_states table. Tracks step progress within a workflow run - the coordination layer for DAG execution. - Matches QuantumFlow's step_states table design. + Matches Singularity.Workflow's step_states table design. ## Counter-Based Coordination Flow @@ -34,7 +34,7 @@ defmodule Singularity.Workflow.StepState do ## Counter-Based Coordination - The key innovation from QuantumFlow: + The key innovation from Singularity.Workflow: - `remaining_deps` - How many dependency steps haven't completed yet - `remaining_tasks` - How many tasks in this step are still executing diff --git a/lib/singularity_workflow/step_task.ex b/lib/singularity_workflow/step_task.ex index a0e63f0..c4b0885 100644 --- a/lib/singularity_workflow/step_task.ex +++ b/lib/singularity_workflow/step_task.ex @@ -3,7 +3,7 @@ defmodule Singularity.Workflow.StepTask do Ecto schema for workflow_step_tasks table. Tracks individual task executions within a step - the execution layer for DAG workflows. - Matches QuantumFlow's step_tasks table design. + Matches Singularity.Workflow's step_tasks table design. ## Task Lifecycle with Retry Logic diff --git a/lib/singularity_workflow/worker.ex b/lib/singularity_workflow/worker.ex index 364731c..2f92aa7 100644 --- a/lib/singularity_workflow/worker.ex +++ b/lib/singularity_workflow/worker.ex @@ -1,8 +1,8 @@ defmodule Singularity.Workflow.Worker do @moduledoc """ - Compatibility layer that plugs QuantumFlow workflows into Oban workers. + Compatibility layer that plugs Singularity.Workflow workflows into Oban workers. - The original implementation lives alongside the QuantumFlow Elixir wrappers. + The original implementation lives alongside the Singularity.Workflow Elixir wrappers. For Observer we only need the minimal behaviour that delegates to `Oban.Worker` while exposing the convenience `new/2` function expected by Singularity job modules. diff --git a/lib/singularity_workflow/workflow.ex b/lib/singularity_workflow/workflow.ex index 1f7732b..87e6e6a 100644 --- a/lib/singularity_workflow/workflow.ex +++ b/lib/singularity_workflow/workflow.ex @@ -1,9 +1,9 @@ defmodule Singularity.Workflow.Workflow do @moduledoc """ - Public API for orchestrating QuantumFlow workflows. + Public API for orchestrating Singularity.Workflow workflows. Provides functions to start, execute, monitor, and gather metrics for - database-backed workflows executed by the QuantumFlow runtime. Repositories + database-backed workflows executed by the Singularity.Workflow runtime. Repositories are resolved automatically from the workflow module's owning OTP application, but can also be supplied explicitly via the `:repo` option. """ @@ -50,7 +50,7 @@ defmodule Singularity.Workflow.Workflow do defdelegate get_parent(workflow_pid), to: RuntimeWorkflow @doc """ - Provides the workflow macro for defining QuantumFlow workflows. + Provides the workflow macro for defining Singularity.Workflow workflows. """ defmacro __using__(opts \\ []) do quote do diff --git a/lib/singularity_workflow/workflow/context.ex b/lib/singularity_workflow/workflow/context.ex index e38318b..f367603 100644 --- a/lib/singularity_workflow/workflow/context.ex +++ b/lib/singularity_workflow/workflow/context.ex @@ -1,9 +1,9 @@ defmodule Singularity.Workflow.Workflow.Context do @moduledoc """ - Lightweight workflow context struct used when invoking QuantumFlow workflow + Lightweight workflow context struct used when invoking Singularity.Workflow workflow steps outside of the orchestration runtime. - The production QuantumFlow executor builds workflow contexts dynamically as maps + The production Singularity.Workflow executor builds workflow contexts dynamically as maps that include the original `:input` plus per-step results. Certain parts of the umbrella (for example, synchronous code paths that call workflow steps directly) still construct a struct to satisfy compile-time references. @@ -20,7 +20,7 @@ end defmodule Singularity.Workflow.WorkflowContext do @moduledoc """ CamelCase alias of `Singularity.Workflow.Workflow.Context` provided for backwards - compatibility with modules that still reference the legacy `QuantumFlow` + compatibility with modules that still reference the legacy `Singularity.Workflow` namespace. """ diff --git a/lib/singularity_workflow/workflow_composer.ex b/lib/singularity_workflow/workflow_composer.ex index 238e439..31cdc1e 100644 --- a/lib/singularity_workflow/workflow_composer.ex +++ b/lib/singularity_workflow/workflow_composer.ex @@ -169,10 +169,18 @@ defmodule Singularity.Workflow.WorkflowComposer do _workflow_name = Keyword.get(opts, :workflow_name, generate_workflow_name(goal)) optimize = - Keyword.get(opts, :optimize, Singularity.Workflow.Orchestrator.Config.feature_enabled?(:optimization)) + Keyword.get( + opts, + :optimize, + Singularity.Workflow.Orchestrator.Config.feature_enabled?(:optimization) + ) monitor = - Keyword.get(opts, :monitor, Singularity.Workflow.Orchestrator.Config.feature_enabled?(:monitoring)) + Keyword.get( + opts, + :monitor, + Singularity.Workflow.Orchestrator.Config.feature_enabled?(:monitoring) + ) Logger.info("Composing workflow from goal: #{inspect(goal)}") @@ -436,7 +444,9 @@ defmodule Singularity.Workflow.WorkflowComposer do Logger.info("Executing workflow with monitoring: #{workflow.name}") # Use HTDAG executor for enhanced monitoring - Singularity.Workflow.Orchestrator.Executor.execute_workflow(workflow, %{goal: goal}, repo, monitor: true) + Singularity.Workflow.Orchestrator.Executor.execute_workflow(workflow, %{goal: goal}, repo, + monitor: true + ) end defp execute_workflow(workflow, goal, false, repo) do diff --git a/lib/singularity_workflow/workflow_run.ex b/lib/singularity_workflow/workflow_run.ex index b5c5f76..832be1b 100644 --- a/lib/singularity_workflow/workflow_run.ex +++ b/lib/singularity_workflow/workflow_run.ex @@ -3,7 +3,7 @@ defmodule Singularity.Workflow.WorkflowRun do Ecto schema for workflow_runs table. Tracks workflow execution instances - one record per workflow invocation. - Matches QuantumFlow's runs table design. + Matches Singularity.Workflow's runs table design. ## State Transition Diagram diff --git a/priv/repo/migrations/20250110000001_create_orchestrator_tables.exs b/priv/repo/migrations/20250110000001_create_orchestrator_tables.exs index 4b0bfd2..951057c 100644 --- a/priv/repo/migrations/20250110000001_create_orchestrator_tables.exs +++ b/priv/repo/migrations/20250110000001_create_orchestrator_tables.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.CreateHtdagTables do +defmodule Singularity.Workflow.Repo.Migrations.CreateHtdagTables do use Ecto.Migration def change do diff --git a/priv/repo/migrations/20251025140000_create_workflow_runs.exs b/priv/repo/migrations/20251025140000_create_workflow_runs.exs index 545cbb3..55448ee 100644 --- a/priv/repo/migrations/20251025140000_create_workflow_runs.exs +++ b/priv/repo/migrations/20251025140000_create_workflow_runs.exs @@ -1,8 +1,8 @@ -defmodule QuantumFlow.Repo.Migrations.CreateWorkflowRuns do +defmodule Singularity.Workflow.Repo.Migrations.CreateWorkflowRuns do @moduledoc """ Creates workflow_runs table for tracking workflow execution instances. - Matches QuantumFlow's runs table design - one record per workflow execution. + Matches Singularity.Workflow's runs table design - one record per workflow execution. """ use Ecto.Migration @@ -36,7 +36,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateWorkflowRuns do execute """ COMMENT ON TABLE workflow_runs IS - 'Tracks workflow execution instances (quantum_flow-compatible design)' + 'Tracks workflow execution instances (Singularity.Workflow-compatible design)' """ execute """ diff --git a/priv/repo/migrations/20251025140001_create_workflow_step_states.exs b/priv/repo/migrations/20251025140001_create_workflow_step_states.exs index 4565e91..52b6b5c 100644 --- a/priv/repo/migrations/20251025140001_create_workflow_step_states.exs +++ b/priv/repo/migrations/20251025140001_create_workflow_step_states.exs @@ -1,8 +1,8 @@ -defmodule QuantumFlow.Repo.Migrations.CreateWorkflowStepStates do +defmodule Singularity.Workflow.Repo.Migrations.CreateWorkflowStepStates do @moduledoc """ Creates workflow_step_states table for tracking step progress within a run. - Matches QuantumFlow's step_states table design - the coordination layer for DAG execution. + Matches Singularity.Workflow's step_states table design - the coordination layer for DAG execution. """ use Ecto.Migration @@ -18,7 +18,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateWorkflowStepStates do # Step execution status add :status, :string, null: false, default: "created" - # Counter-based coordination (QuantumFlow's key innovation) + # Counter-based coordination (key innovation) add :remaining_deps, :integer, null: false, default: 0 add :remaining_tasks, :integer add :initial_tasks, :integer diff --git a/priv/repo/migrations/20251025140002_create_workflow_step_tasks.exs b/priv/repo/migrations/20251025140002_create_workflow_step_tasks.exs index 1b82c80..2ea0587 100644 --- a/priv/repo/migrations/20251025140002_create_workflow_step_tasks.exs +++ b/priv/repo/migrations/20251025140002_create_workflow_step_tasks.exs @@ -1,8 +1,8 @@ -defmodule QuantumFlow.Repo.Migrations.CreateWorkflowStepTasks do +defmodule Singularity.Workflow.Repo.Migrations.CreateWorkflowStepTasks do @moduledoc """ Creates workflow_step_tasks table for tracking individual task executions. - Matches QuantumFlow's step_tasks table design - the execution layer for DAG workflows. + Matches Singularity.Workflow's step_tasks table design - the execution layer for DAG workflows. Each step can have multiple tasks (e.g., map steps with arrays). """ use Ecto.Migration diff --git a/priv/repo/migrations/20251025140003_create_start_ready_steps_function.exs b/priv/repo/migrations/20251025140003_create_start_ready_steps_function.exs index 4753e48..90ccff0 100644 --- a/priv/repo/migrations/20251025140003_create_start_ready_steps_function.exs +++ b/priv/repo/migrations/20251025140003_create_start_ready_steps_function.exs @@ -1,11 +1,11 @@ -defmodule QuantumFlow.Repo.Migrations.CreateStartReadyStepsFunction do +defmodule Singularity.Workflow.Repo.Migrations.CreateStartReadyStepsFunction do @moduledoc """ Creates start_ready_steps() PostgreSQL function for DAG coordination. This function finds steps with all dependencies completed (remaining_deps = 0) and marks them as 'started', making their tasks available for execution. - Matches QuantumFlow's dependency awakening mechanism. + Matches Singularity.Workflow's dependency awakening mechanism. """ use Ecto.Migration diff --git a/priv/repo/migrations/20251025140004_create_complete_task_function.exs b/priv/repo/migrations/20251025140004_create_complete_task_function.exs index c0c87a8..0170284 100644 --- a/priv/repo/migrations/20251025140004_create_complete_task_function.exs +++ b/priv/repo/migrations/20251025140004_create_complete_task_function.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.CreateCompleteTaskFunction do +defmodule Singularity.Workflow.Repo.Migrations.CreateCompleteTaskFunction do @moduledoc """ Creates complete_task() PostgreSQL function for DAG coordination. @@ -9,7 +9,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateCompleteTaskFunction do 4. Cascades to dependent steps by decrementing their remaining_deps 5. Triggers start_ready_steps to awaken newly ready steps - Matches QuantumFlow's cascading completion mechanism. + Matches Singularity.Workflow's cascading completion mechanism. """ use Ecto.Migration diff --git a/priv/repo/migrations/20251025140005_create_workflow_step_dependencies.exs b/priv/repo/migrations/20251025140005_create_workflow_step_dependencies.exs index df6d451..552c3fd 100644 --- a/priv/repo/migrations/20251025140005_create_workflow_step_dependencies.exs +++ b/priv/repo/migrations/20251025140005_create_workflow_step_dependencies.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.CreateWorkflowStepDependencies do +defmodule Singularity.Workflow.Repo.Migrations.CreateWorkflowStepDependencies do @moduledoc """ Creates workflow_step_dependencies table to explicitly track step dependencies. diff --git a/priv/repo/migrations/20251025150000_add_pgmq_extension.exs b/priv/repo/migrations/20251025150000_add_pgmq_extension.exs index 1fefb36..7d8b061 100644 --- a/priv/repo/migrations/20251025150000_add_pgmq_extension.exs +++ b/priv/repo/migrations/20251025150000_add_pgmq_extension.exs @@ -1,9 +1,9 @@ -defmodule QuantumFlow.Repo.Migrations.AddPgmqExtension do +defmodule Singularity.Workflow.Repo.Migrations.AddPgmqExtension do use Ecto.Migration def up do # Create pgmq extension (PostgreSQL Message Queue) - REQUIRED - # Matches QuantumFlow's architecture: https://github.com/tembo-io/pgmq + # Matches Singularity.Workflow's architecture: https://github.com/tembo-io/pgmq # pgmq is REQUIRED for task coordination and queue management # If this fails, ensure PostgreSQL instance has pgmq extension installed execute("CREATE EXTENSION IF NOT EXISTS pgmq") diff --git a/priv/repo/migrations/20251025150001_create_pgmq_queue_functions.exs b/priv/repo/migrations/20251025150001_create_pgmq_queue_functions.exs index 9020c70..f5c9501 100644 --- a/priv/repo/migrations/20251025150001_create_pgmq_queue_functions.exs +++ b/priv/repo/migrations/20251025150001_create_pgmq_queue_functions.exs @@ -1,14 +1,14 @@ -defmodule QuantumFlow.Repo.Migrations.CreatePgmqQueueFunctions do +defmodule Singularity.Workflow.Repo.Migrations.CreatePgmqQueueFunctions do use Ecto.Migration def up do - # Create QuantumFlow schema for custom functions - execute("CREATE SCHEMA IF NOT EXISTS QuantumFlow;") + # Create singularity_workflow PostgreSQL schema for custom functions + execute("CREATE SCHEMA IF NOT EXISTS singularity_workflow;") # Create read_with_poll function (backport from pgmq 1.5.0) - # This matches QuantumFlow's implementation for task polling + # This matches Singularity.Workflow's implementation for task polling execute(""" - CREATE FUNCTION QuantumFlow.read_with_poll( + CREATE FUNCTION singularity_workflow.read_with_poll( queue_name TEXT, vt INTEGER, qty INTEGER, @@ -73,7 +73,7 @@ defmodule QuantumFlow.Repo.Migrations.CreatePgmqQueueFunctions do # Create function to initialize workflow queue execute(""" - CREATE FUNCTION QuantumFlow.ensure_workflow_queue(workflow_slug TEXT) + CREATE FUNCTION singularity_workflow.ensure_workflow_queue(workflow_slug TEXT) RETURNS TEXT LANGUAGE SQL SET search_path TO '' @@ -88,7 +88,7 @@ defmodule QuantumFlow.Repo.Migrations.CreatePgmqQueueFunctions do end def down do - execute("DROP FUNCTION IF EXISTS QuantumFlow.read_with_poll") - execute("DROP FUNCTION IF EXISTS QuantumFlow.ensure_workflow_queue") + execute("DROP FUNCTION IF EXISTS singularity_workflow.read_with_poll") + execute("DROP FUNCTION IF EXISTS singularity_workflow.ensure_workflow_queue") end end diff --git a/priv/repo/migrations/20251025150002_add_message_id_to_step_tasks.exs b/priv/repo/migrations/20251025150002_add_message_id_to_step_tasks.exs index 5398eb1..a75711c 100644 --- a/priv/repo/migrations/20251025150002_add_message_id_to_step_tasks.exs +++ b/priv/repo/migrations/20251025150002_add_message_id_to_step_tasks.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.AddMessageIdToStepTasks do +defmodule Singularity.Workflow.Repo.Migrations.AddMessageIdToStepTasks do use Ecto.Migration def change do diff --git a/priv/repo/migrations/20251025150003_rewrite_start_ready_steps_with_pgmq.exs b/priv/repo/migrations/20251025150003_rewrite_start_ready_steps_with_pgmq.exs index 2e876aa..473568a 100644 --- a/priv/repo/migrations/20251025150003_rewrite_start_ready_steps_with_pgmq.exs +++ b/priv/repo/migrations/20251025150003_rewrite_start_ready_steps_with_pgmq.exs @@ -1,14 +1,14 @@ -defmodule QuantumFlow.Repo.Migrations.RewriteStartReadyStepsWithPgmq do +defmodule Singularity.Workflow.Repo.Migrations.RewriteStartReadyStepsWithPgmq do @moduledoc """ Rewrites start_ready_steps() to use pgmq for task coordination. - Matches QuantumFlow's architecture: + Matches Singularity.Workflow's architecture: 1. Find ready steps (remaining_deps = 0) 2. Mark as started 3. Create step_tasks records 4. Send messages to pgmq queue - This is the KEY function for QuantumFlow parity - uses pgmq for work distribution! + This is the KEY function for workflow parity - uses pgmq for work distribution! """ use Ecto.Migration @@ -31,7 +31,7 @@ defmodule QuantumFlow.Repo.Migrations.RewriteStartReadyStepsWithPgmq do WHERE id = p_run_id; -- Ensure queue exists for this workflow - PERFORM QuantumFlow.ensure_workflow_queue(v_workflow_slug); + PERFORM singularity_workflow.ensure_workflow_queue(v_workflow_slug); -- Mark ready steps as started and send tasks to pgmq WITH ready_steps AS ( @@ -142,7 +142,7 @@ defmodule QuantumFlow.Repo.Migrations.RewriteStartReadyStepsWithPgmq do execute(""" COMMENT ON FUNCTION start_ready_steps(UUID) IS - 'Finds steps with all dependencies completed, marks as started, creates task records, and sends messages to pgmq queue. Matches QuantumFlow architecture.' + 'Finds steps with all dependencies completed, marks as started, creates task records, and sends messages to pgmq queue. Matches Singularity.Workflow architecture.' """) end diff --git a/priv/repo/migrations/20251025150004_create_start_tasks_function.exs b/priv/repo/migrations/20251025150004_create_start_tasks_function.exs index 2e7f748..5dbae93 100644 --- a/priv/repo/migrations/20251025150004_create_start_tasks_function.exs +++ b/priv/repo/migrations/20251025150004_create_start_tasks_function.exs @@ -1,8 +1,8 @@ -defmodule QuantumFlow.Repo.Migrations.CreateStartTasksFunction do +defmodule Singularity.Workflow.Repo.Migrations.CreateStartTasksFunction do @moduledoc """ Creates start_tasks() PostgreSQL function for claiming tasks after polling pgmq. - Matches QuantumFlow's architecture: + Matches Singularity.Workflow's architecture: 1. Receive message IDs from pgmq.read_with_poll() 2. Mark corresponding tasks as 'started' 3. Build task input from run input + dependency outputs @@ -105,7 +105,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateStartTasksFunction do execute(""" COMMENT ON FUNCTION start_tasks(TEXT, BIGINT[], TEXT) IS - 'Claims tasks from pgmq messages, marks as started, builds input from run + dependencies. Matches QuantumFlow architecture.' + 'Claims tasks from pgmq messages, marks as started, builds input from run + dependencies. Matches Singularity.Workflow architecture.' """) end diff --git a/priv/repo/migrations/20251025150005_create_fail_task_function.exs b/priv/repo/migrations/20251025150005_create_fail_task_function.exs index f887986..0b34878 100644 --- a/priv/repo/migrations/20251025150005_create_fail_task_function.exs +++ b/priv/repo/migrations/20251025150005_create_fail_task_function.exs @@ -1,8 +1,8 @@ -defmodule QuantumFlow.Repo.Migrations.CreateFailTaskFunction do +defmodule Singularity.Workflow.Repo.Migrations.CreateFailTaskFunction do @moduledoc """ - Creates QuantumFlow.fail_task() function for handling task failures. + Creates singularity_workflow.fail_task() function for handling task failures. - Matches QuantumFlow's architecture: + Matches Singularity.Workflow's architecture: 1. Check if run is already failed (no retry allowed) 2. Determine if task should retry based on attempts_count vs max_attempts 3. Update task status (queued for retry, or failed permanently) @@ -14,7 +14,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateFailTaskFunction do def up do # First create helper function for retry delay calculation execute(""" - CREATE OR REPLACE FUNCTION QuantumFlow.calculate_retry_delay( + CREATE OR REPLACE FUNCTION singularity_workflow.calculate_retry_delay( base_delay INTEGER, attempts_count INTEGER ) @@ -27,7 +27,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateFailTaskFunction do """) execute(""" - CREATE OR REPLACE FUNCTION QuantumFlow.fail_task( + CREATE OR REPLACE FUNCTION singularity_workflow.fail_task( p_run_id UUID, p_step_slug TEXT, p_task_index INTEGER, @@ -116,7 +116,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateFailTaskFunction do PERFORM pgmq.set_vt( v_workflow_slug, v_message_id, - QuantumFlow.calculate_retry_delay(5, v_attempts_count) + singularity_workflow.calculate_retry_delay(5, v_attempts_count) ); END IF; END IF; @@ -125,13 +125,13 @@ defmodule QuantumFlow.Repo.Migrations.CreateFailTaskFunction do """) execute(""" - COMMENT ON FUNCTION QuantumFlow.fail_task(UUID, TEXT, INTEGER, TEXT) IS - 'Handles task failure with retry logic. Either requeues for retry or marks as permanently failed. Matches QuantumFlow architecture.' + COMMENT ON FUNCTION singularity_workflow.fail_task(UUID, TEXT, INTEGER, TEXT) IS + 'Handles task failure with retry logic. Either requeues for retry or marks as permanently failed. Matches Singularity.Workflow architecture.' """) end def down do - execute("DROP FUNCTION IF EXISTS QuantumFlow.fail_task(UUID, TEXT, INTEGER, TEXT)") - execute("DROP FUNCTION IF EXISTS QuantumFlow.calculate_retry_delay(INTEGER, INTEGER)") + execute("DROP FUNCTION IF EXISTS singularity_workflow.fail_task(UUID, TEXT, INTEGER, TEXT)") + execute("DROP FUNCTION IF EXISTS singularity_workflow.calculate_retry_delay(INTEGER, INTEGER)") end end diff --git a/priv/repo/migrations/20251025150006_create_set_vt_batch_function.exs b/priv/repo/migrations/20251025150006_create_set_vt_batch_function.exs index 78ac6c5..611cae7 100644 --- a/priv/repo/migrations/20251025150006_create_set_vt_batch_function.exs +++ b/priv/repo/migrations/20251025150006_create_set_vt_batch_function.exs @@ -1,17 +1,17 @@ -defmodule QuantumFlow.Repo.Migrations.CreateSetVtBatchFunction do +defmodule Singularity.Workflow.Repo.Migrations.CreateSetVtBatchFunction do @moduledoc """ - Creates QuantumFlow.set_vt_batch() for batch visibility timeout updates. + Creates singularity_workflow.set_vt_batch() for batch visibility timeout updates. - Matches QuantumFlow's implementation - batch-update visibility timeouts for multiple + Matches Singularity.Workflow's implementation - batch-update visibility timeouts for multiple messages in one operation. Used by start_tasks() to set task timeouts efficiently. - Note: pgmq extension doesn't provide set_vt_batch natively, so QuantumFlow implements it. + Note: pgmq extension doesn't provide set_vt_batch natively, so Singularity.Workflow implements it. """ use Ecto.Migration def up do execute(""" - CREATE OR REPLACE FUNCTION QuantumFlow.set_vt_batch( + CREATE OR REPLACE FUNCTION singularity_workflow.set_vt_batch( queue_name TEXT, msg_ids BIGINT[], vt_offsets INTEGER[] @@ -62,12 +62,12 @@ defmodule QuantumFlow.Repo.Migrations.CreateSetVtBatchFunction do """) execute(""" - COMMENT ON FUNCTION QuantumFlow.set_vt_batch(TEXT, BIGINT[], INTEGER[]) IS - 'Batch-update visibility timeouts for multiple pgmq messages. Matches QuantumFlow implementation.' + COMMENT ON FUNCTION singularity_workflow.set_vt_batch(TEXT, BIGINT[], INTEGER[]) IS + 'Batch-update visibility timeouts for multiple pgmq messages. Matches Singularity.Workflow implementation.' """) end def down do - execute("DROP FUNCTION IF EXISTS QuantumFlow.set_vt_batch(TEXT, BIGINT[], INTEGER[])") + execute("DROP FUNCTION IF EXISTS singularity_workflow.set_vt_batch(TEXT, BIGINT[], INTEGER[])") end end diff --git a/priv/repo/migrations/20251025150007_create_maybe_complete_run_function.exs b/priv/repo/migrations/20251025150007_create_maybe_complete_run_function.exs index 76895e0..497582b 100644 --- a/priv/repo/migrations/20251025150007_create_maybe_complete_run_function.exs +++ b/priv/repo/migrations/20251025150007_create_maybe_complete_run_function.exs @@ -1,8 +1,8 @@ -defmodule QuantumFlow.Repo.Migrations.CreateMaybeCompleteRunFunction do +defmodule Singularity.Workflow.Repo.Migrations.CreateMaybeCompleteRunFunction do @moduledoc """ - Creates QuantumFlow.maybe_complete_run() for checking and completing workflow runs. + Creates singularity_workflow.maybe_complete_run() for checking and completing workflow runs. - Matches QuantumFlow's implementation: + Matches Singularity.Workflow's implementation: 1. Check if all steps are completed (remaining_steps = 0) 2. If yes, mark run as completed 3. Aggregate outputs from leaf steps (steps with no dependents) into a flat array @@ -14,7 +14,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateMaybeCompleteRunFunction do def up do execute(""" - CREATE OR REPLACE FUNCTION QuantumFlow.maybe_complete_run(p_run_id UUID) + CREATE OR REPLACE FUNCTION singularity_workflow.maybe_complete_run(p_run_id UUID) RETURNS void LANGUAGE plpgsql AS $$ @@ -56,7 +56,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateMaybeCompleteRunFunction do AND workflow_runs.status != 'completed' RETURNING * INTO v_completed_run; - -- Log completion (optional: would broadcast event in QuantumFlow) + -- Log completion (optional: would broadcast event in Singularity.Workflow) IF v_completed_run.id IS NOT NULL THEN RAISE NOTICE 'Run completed: run_id=%, output=%', v_completed_run.id, v_completed_run.output; @@ -66,12 +66,12 @@ defmodule QuantumFlow.Repo.Migrations.CreateMaybeCompleteRunFunction do """) execute(""" - COMMENT ON FUNCTION QuantumFlow.maybe_complete_run(UUID) IS - 'Checks if run is complete (all steps done), marks as completed, and aggregates leaf step outputs into a flat array. Matches QuantumFlow implementation.' + COMMENT ON FUNCTION singularity_workflow.maybe_complete_run(UUID) IS + 'Checks if run is complete (all steps done), marks as completed, and aggregates leaf step outputs into a flat array. Matches Singularity.Workflow implementation.' """) end def down do - execute("DROP FUNCTION IF EXISTS QuantumFlow.maybe_complete_run(UUID)") + execute("DROP FUNCTION IF EXISTS singularity_workflow.maybe_complete_run(UUID)") end end diff --git a/priv/repo/migrations/20251025150008_update_complete_task_with_pgmq.exs b/priv/repo/migrations/20251025150008_update_complete_task_with_pgmq.exs index f68404c..f6b0986 100644 --- a/priv/repo/migrations/20251025150008_update_complete_task_with_pgmq.exs +++ b/priv/repo/migrations/20251025150008_update_complete_task_with_pgmq.exs @@ -1,6 +1,6 @@ -defmodule QuantumFlow.Repo.Migrations.UpdateCompleteTaskWithPgmq do +defmodule Singularity.Workflow.Repo.Migrations.UpdateCompleteTaskWithPgmq do @moduledoc """ - Updates complete_task() to match QuantumFlow's pgmq-integrated implementation. + Updates complete_task() to match Singularity.Workflow's pgmq-integrated implementation. Changes: 1. Archive pgmq message when task completes @@ -149,13 +149,13 @@ defmodule QuantumFlow.Repo.Migrations.UpdateCompleteTaskWithPgmq do WHERE id = p_run_id; -- After step completion, cascade any taskless steps (empty-array propagation) - PERFORM QuantumFlow.cascade_complete_taskless_steps(p_run_id); + PERFORM singularity_workflow.cascade_complete_taskless_steps(p_run_id); -- Trigger start_ready_steps to awaken newly ready steps PERFORM start_ready_steps(p_run_id); -- Then check if run is complete and aggregate leaf outputs - PERFORM QuantumFlow.maybe_complete_run(p_run_id); + PERFORM singularity_workflow.maybe_complete_run(p_run_id); END IF; END; $$; @@ -163,7 +163,7 @@ defmodule QuantumFlow.Repo.Migrations.UpdateCompleteTaskWithPgmq do execute(""" COMMENT ON FUNCTION complete_task(UUID, TEXT, INTEGER, JSONB) IS - 'Marks task as completed, archives pgmq message, cascades to dependencies, and checks run completion. Matches QuantumFlow architecture.' + 'Marks task as completed, archives pgmq message, cascades to dependencies, and checks run completion. Matches Singularity.Workflow architecture.' """) end diff --git a/priv/repo/migrations/20251025150009_create_workers_table.exs b/priv/repo/migrations/20251025150009_create_workers_table.exs index 972d2c5..df22b93 100644 --- a/priv/repo/migrations/20251025150009_create_workers_table.exs +++ b/priv/repo/migrations/20251025150009_create_workers_table.exs @@ -1,8 +1,8 @@ -defmodule QuantumFlow.Repo.Migrations.CreateWorkersTable do +defmodule Singularity.Workflow.Repo.Migrations.CreateWorkersTable do @moduledoc """ Creates workers table for worker heartbeat tracking. - Matches QuantumFlow's worker registration and heartbeat system. Workers register + Matches Singularity.Workflow's worker registration and heartbeat system. Workers register when they start polling and update last_heartbeat_at periodically. Used for monitoring and debugging worker activity. diff --git a/priv/repo/migrations/20251025150010_update_start_tasks_with_worker_and_timeout.exs b/priv/repo/migrations/20251025150010_update_start_tasks_with_worker_and_timeout.exs index 3453f97..b7ba22d 100644 --- a/priv/repo/migrations/20251025150010_update_start_tasks_with_worker_and_timeout.exs +++ b/priv/repo/migrations/20251025150010_update_start_tasks_with_worker_and_timeout.exs @@ -1,11 +1,11 @@ -defmodule QuantumFlow.Repo.Migrations.UpdateStartTasksWithWorkerAndTimeout do +defmodule Singularity.Workflow.Repo.Migrations.UpdateStartTasksWithWorkerAndTimeout do @moduledoc """ Updates start_tasks() to: 1. Set last_worker_id when claiming tasks 2. Use set_vt_batch() to set visibility timeouts for all messages 3. Calculate timeout based on step/workflow configuration - Matches QuantumFlow's complete start_tasks implementation. + Matches Singularity.Workflow's complete start_tasks implementation. """ use Ecto.Migration @@ -121,7 +121,7 @@ defmodule QuantumFlow.Repo.Migrations.UpdateStartTasksWithWorkerAndTimeout do execute(""" COMMENT ON FUNCTION start_tasks(TEXT, BIGINT[], TEXT) IS - 'Claims tasks from pgmq messages, sets worker tracking, configures timeouts via set_vt_batch, builds input. Matches QuantumFlow implementation.' + 'Claims tasks from pgmq messages, sets worker tracking, configures timeouts via set_vt_batch, builds input. Matches Singularity.Workflow implementation.' """) end diff --git a/priv/repo/migrations/20251025150011_add_type_violation_detection.exs b/priv/repo/migrations/20251025150011_add_type_violation_detection.exs index 5654806..b0584b7 100644 --- a/priv/repo/migrations/20251025150011_add_type_violation_detection.exs +++ b/priv/repo/migrations/20251025150011_add_type_violation_detection.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.AddTypeViolationDetection do +defmodule Singularity.Workflow.Repo.Migrations.AddTypeViolationDetection do @moduledoc """ Adds type violation detection to complete_task() for map step validation. @@ -9,7 +9,7 @@ defmodule QuantumFlow.Repo.Migrations.AddTypeViolationDetection do 3. Store violation details in error messages 4. Prevent further execution - Matches QuantumFlow's type safety for map steps. + Matches Singularity.Workflow's type safety for map steps. """ use Ecto.Migration @@ -222,13 +222,13 @@ defmodule QuantumFlow.Repo.Migrations.AddTypeViolationDetection do WHERE id = p_run_id; -- After step completion, cascade any taskless steps (empty-array propagation) - PERFORM QuantumFlow.cascade_complete_taskless_steps(p_run_id); + PERFORM singularity_workflow.cascade_complete_taskless_steps(p_run_id); -- Trigger start_ready_steps to awaken newly ready steps PERFORM start_ready_steps(p_run_id); -- Then check if run is complete and aggregate leaf outputs - PERFORM QuantumFlow.maybe_complete_run(p_run_id); + PERFORM singularity_workflow.maybe_complete_run(p_run_id); END IF; END; $$; @@ -236,7 +236,7 @@ defmodule QuantumFlow.Repo.Migrations.AddTypeViolationDetection do execute(""" COMMENT ON FUNCTION complete_task(UUID, TEXT, INTEGER, JSONB) IS - 'Completes task with type violation detection for map steps. Archives pgmq message, cascades to dependencies. Matches QuantumFlow implementation.' + 'Completes task with type violation detection for map steps. Archives pgmq message, cascades to dependencies. Matches Singularity.Workflow implementation.' """) end @@ -321,7 +321,7 @@ defmodule QuantumFlow.Repo.Migrations.AddTypeViolationDetection do SET remaining_steps = remaining_steps - 1 WHERE id = p_run_id; - PERFORM QuantumFlow.maybe_complete_run(p_run_id); + PERFORM singularity_workflow.maybe_complete_run(p_run_id); PERFORM start_ready_steps(p_run_id); END IF; END; diff --git a/priv/repo/migrations/20251025160000_add_is_valid_slug_function.exs b/priv/repo/migrations/20251025160000_add_is_valid_slug_function.exs index c7bcd0e..031cca2 100644 --- a/priv/repo/migrations/20251025160000_add_is_valid_slug_function.exs +++ b/priv/repo/migrations/20251025160000_add_is_valid_slug_function.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.AddIsValidSlugFunction do +defmodule Singularity.Workflow.Repo.Migrations.AddIsValidSlugFunction do @moduledoc """ Adds is_valid_slug() utility function for slug validation. @@ -8,13 +8,13 @@ defmodule QuantumFlow.Repo.Migrations.AddIsValidSlugFunction do - Pattern: ^[a-zA-Z_][a-zA-Z0-9_]*$ - Not reserved words ('run') - Matches QuantumFlow's validation logic. + Matches Singularity.Workflow's validation logic. """ use Ecto.Migration def up do execute(""" - CREATE OR REPLACE FUNCTION QuantumFlow.is_valid_slug(slug TEXT) + CREATE OR REPLACE FUNCTION singularity_workflow.is_valid_slug(slug TEXT) RETURNS BOOLEAN LANGUAGE plpgsql IMMUTABLE @@ -31,12 +31,12 @@ defmodule QuantumFlow.Repo.Migrations.AddIsValidSlugFunction do """) execute(""" - COMMENT ON FUNCTION QuantumFlow.is_valid_slug(TEXT) IS - 'Validates slug format: not null, 1-128 chars, alphanumeric with underscores, no reserved words. Matches QuantumFlow implementation.' + COMMENT ON FUNCTION singularity_workflow.is_valid_slug(TEXT) IS + 'Validates slug format: not null, 1-128 chars, alphanumeric with underscores, no reserved words. Matches Singularity.Workflow implementation.' """) end def down do - execute("DROP FUNCTION IF EXISTS QuantumFlow.is_valid_slug(TEXT)") + execute("DROP FUNCTION IF EXISTS singularity_workflow.is_valid_slug(TEXT)") end end diff --git a/priv/repo/migrations/20251025160001_create_workflow_definition_tables.exs b/priv/repo/migrations/20251025160001_create_workflow_definition_tables.exs index bec46b9..2d8750d 100644 --- a/priv/repo/migrations/20251025160001_create_workflow_definition_tables.exs +++ b/priv/repo/migrations/20251025160001_create_workflow_definition_tables.exs @@ -1,9 +1,9 @@ -defmodule QuantumFlow.Repo.Migrations.CreateWorkflowDefinitionTables do +defmodule Singularity.Workflow.Repo.Migrations.CreateWorkflowDefinitionTables do @moduledoc """ Creates tables for storing dynamic workflow definitions. Enables runtime workflow creation via API (for AI/LLM workflow generation). - Matches QuantumFlow's flows, steps, and deps tables. + Matches Singularity.Workflow's flows, steps, and deps tables. Usage: - Code-based workflows: Use __workflow_steps__/0 callback (existing) @@ -18,11 +18,11 @@ defmodule QuantumFlow.Repo.Migrations.CreateWorkflowDefinitionTables do create table(:workflows, primary_key: false) do add :workflow_slug, :text, primary_key: true, null: false add :max_attempts, :integer, null: false, default: 3 - add :timeout, :integer, null: false, default: 60 # Matches QuantumFlow opt_timeout default + add :timeout, :integer, null: false, default: 60 # Matches Singularity.Workflow opt_timeout default add :created_at, :utc_datetime, null: false, default: fragment("NOW()") end - create constraint(:workflows, :workflow_slug_is_valid, check: "QuantumFlow.is_valid_slug(workflow_slug)") + create constraint(:workflows, :workflow_slug_is_valid, check: "singularity_workflow.is_valid_slug(workflow_slug)") create constraint(:workflows, :max_attempts_is_nonnegative, check: "max_attempts >= 0") create constraint(:workflows, :timeout_is_positive, check: "timeout > 0") @@ -41,7 +41,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateWorkflowDefinitionTables do create unique_index(:workflow_steps, [:workflow_slug, :step_slug], name: :workflow_steps_pkey) create unique_index(:workflow_steps, [:workflow_slug, :step_index]) - create constraint(:workflow_steps, :step_slug_is_valid, check: "QuantumFlow.is_valid_slug(step_slug)") + create constraint(:workflow_steps, :step_slug_is_valid, check: "singularity_workflow.is_valid_slug(step_slug)") create constraint(:workflow_steps, :step_type_is_valid, check: "step_type IN ('single', 'map')") create constraint(:workflow_steps, :deps_count_nonnegative, check: "deps_count >= 0") create constraint(:workflow_steps, :initial_tasks_nonnegative, check: "initial_tasks IS NULL OR initial_tasks >= 0") diff --git a/priv/repo/migrations/20251025160002_create_create_flow_function.exs b/priv/repo/migrations/20251025160002_create_create_flow_function.exs index 989d20c..39aefdf 100644 --- a/priv/repo/migrations/20251025160002_create_create_flow_function.exs +++ b/priv/repo/migrations/20251025160002_create_create_flow_function.exs @@ -1,11 +1,11 @@ -defmodule QuantumFlow.Repo.Migrations.CreateCreateFlowFunction do +defmodule Singularity.Workflow.Repo.Migrations.CreateCreateFlowFunction do @moduledoc """ Creates create_flow() function for dynamic workflow initialization. Creates workflow record + ensures pgmq queue exists. Idempotent - can be called multiple times safely. - Matches QuantumFlow's create_flow implementation. + Matches Singularity.Workflow's create_flow implementation. CRITICAL: This implementation works around a PostgreSQL 17.x parser regression that incorrectly flags column references as ambiguous. The bug occurs even @@ -14,12 +14,12 @@ defmodule QuantumFlow.Repo.Migrations.CreateCreateFlowFunction do use Ecto.Migration def up do - execute("DROP FUNCTION IF EXISTS QuantumFlow.create_flow(TEXT, INTEGER, INTEGER) CASCADE") + execute("DROP FUNCTION IF EXISTS singularity_workflow.create_flow(TEXT, INTEGER, INTEGER) CASCADE") # Use explicit column numbering to avoid parser bug # This bypasses PostgreSQL's column name resolution entirely execute(""" - CREATE FUNCTION QuantumFlow.create_flow( + CREATE FUNCTION singularity_workflow.create_flow( arg1 TEXT, arg2 INTEGER DEFAULT 3, arg3 INTEGER DEFAULT 60 @@ -35,7 +35,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateCreateFlowFunction do BEGIN DELETE FROM workflows WHERE workflows.workflow_slug = arg1; INSERT INTO workflows (workflow_slug, max_attempts, timeout) VALUES (arg1, arg2, arg3); - PERFORM QuantumFlow.ensure_workflow_queue(arg1); + PERFORM singularity_workflow.ensure_workflow_queue(arg1); RETURN QUERY SELECT (SELECT workflow_slug FROM workflows WHERE workflow_slug = arg1), (SELECT max_attempts FROM workflows WHERE workflow_slug = arg1), (SELECT timeout FROM workflows WHERE workflow_slug = arg1), @@ -45,12 +45,12 @@ defmodule QuantumFlow.Repo.Migrations.CreateCreateFlowFunction do """) execute(""" - COMMENT ON FUNCTION QuantumFlow.create_flow(TEXT, INTEGER, INTEGER) IS - 'Creates workflow definition and ensures pgmq queue exists. Idempotent. Matches QuantumFlow create_flow().' + COMMENT ON FUNCTION singularity_workflow.create_flow(TEXT, INTEGER, INTEGER) IS + 'Creates workflow definition and ensures pgmq queue exists. Idempotent. Matches Singularity.Workflow create_flow().' """) end def down do - execute("DROP FUNCTION IF EXISTS QuantumFlow.create_flow(TEXT, INTEGER, INTEGER)") + execute("DROP FUNCTION IF EXISTS singularity_workflow.create_flow(TEXT, INTEGER, INTEGER)") end end diff --git a/priv/repo/migrations/20251025160003_create_add_step_function.exs b/priv/repo/migrations/20251025160003_create_add_step_function.exs index 8a6077f..66e2cee 100644 --- a/priv/repo/migrations/20251025160003_create_add_step_function.exs +++ b/priv/repo/migrations/20251025160003_create_add_step_function.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.CreateAddStepFunction do +defmodule Singularity.Workflow.Repo.Migrations.CreateAddStepFunction do @moduledoc """ Creates add_step() function for dynamic step creation. @@ -8,15 +8,15 @@ defmodule QuantumFlow.Repo.Migrations.CreateAddStepFunction do - Dependencies exist Auto-increments step_index for ordering. - Matches QuantumFlow's add_step implementation. + Matches Singularity.Workflow's add_step implementation. """ use Ecto.Migration def up do - execute("DROP FUNCTION IF EXISTS QuantumFlow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER) CASCADE") + execute("DROP FUNCTION IF EXISTS singularity_workflow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER) CASCADE") execute(""" - CREATE FUNCTION QuantumFlow.add_step( + CREATE FUNCTION singularity_workflow.add_step( p_workflow_slug TEXT, p_step_slug TEXT, p_depends_on TEXT[] DEFAULT '{}', @@ -44,11 +44,11 @@ defmodule QuantumFlow.Repo.Migrations.CreateAddStepFunction do v_deps_count INTEGER; BEGIN -- Validate slugs - IF NOT QuantumFlow.is_valid_slug(p_workflow_slug) THEN + IF NOT singularity_workflow.is_valid_slug(p_workflow_slug) THEN RAISE EXCEPTION 'Invalid workflow_slug: %', p_workflow_slug; END IF; - IF NOT QuantumFlow.is_valid_slug(p_step_slug) THEN + IF NOT singularity_workflow.is_valid_slug(p_step_slug) THEN RAISE EXCEPTION 'Invalid step_slug: %', p_step_slug; END IF; @@ -155,12 +155,12 @@ defmodule QuantumFlow.Repo.Migrations.CreateAddStepFunction do """) execute(""" - COMMENT ON FUNCTION QuantumFlow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER) IS - 'Adds step to workflow definition. Validates dependencies and map step constraints. Idempotent. Matches QuantumFlow add_step().' + COMMENT ON FUNCTION singularity_workflow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER) IS + 'Adds step to workflow definition. Validates dependencies and map step constraints. Idempotent. Matches Singularity.Workflow add_step().' """) end def down do - execute("DROP FUNCTION IF EXISTS QuantumFlow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER)") + execute("DROP FUNCTION IF EXISTS singularity_workflow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER)") end end diff --git a/priv/repo/migrations/20251025160723_enable_uuid_v7_support.exs b/priv/repo/migrations/20251025160723_enable_uuid_v7_support.exs index d75bfcb..067b04d 100644 --- a/priv/repo/migrations/20251025160723_enable_uuid_v7_support.exs +++ b/priv/repo/migrations/20251025160723_enable_uuid_v7_support.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.EnableUuidV7Support do +defmodule Singularity.Workflow.Repo.Migrations.EnableUuidV7Support do @moduledoc """ Placeholder migration for UUID v7 support. diff --git a/priv/repo/migrations/20251025160724_fix_start_tasks_ambiguous_column.exs b/priv/repo/migrations/20251025160724_fix_start_tasks_ambiguous_column.exs index c3cfaf8..2f26df9 100644 --- a/priv/repo/migrations/20251025160724_fix_start_tasks_ambiguous_column.exs +++ b/priv/repo/migrations/20251025160724_fix_start_tasks_ambiguous_column.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.FixStartTasksAmbiguousColumn do +defmodule Singularity.Workflow.Repo.Migrations.FixStartTasksAmbiguousColumn do @moduledoc """ Fixes ambiguous column reference in start_tasks() function. diff --git a/priv/repo/migrations/20251025170000_create_test_wrapper_functions.exs b/priv/repo/migrations/20251025170000_create_test_wrapper_functions.exs index 44fea39..f78b0e2 100644 --- a/priv/repo/migrations/20251025170000_create_test_wrapper_functions.exs +++ b/priv/repo/migrations/20251025170000_create_test_wrapper_functions.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.CreateTestWrapperFunctions do +defmodule Singularity.Workflow.Repo.Migrations.CreateTestWrapperFunctions do @moduledoc """ Creates test wrapper functions for void-returning functions. diff --git a/priv/repo/migrations/20251025203100_force_recreate_start_tasks.exs b/priv/repo/migrations/20251025203100_force_recreate_start_tasks.exs index 13a20f7..c4dcc91 100644 --- a/priv/repo/migrations/20251025203100_force_recreate_start_tasks.exs +++ b/priv/repo/migrations/20251025203100_force_recreate_start_tasks.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.ForceRecreateStartTasks do +defmodule Singularity.Workflow.Repo.Migrations.ForceRecreateStartTasks do @moduledoc """ Force recreation of start_tasks function by dropping all versions first. diff --git a/priv/repo/migrations/20251025210000_add_smart_uuid_generation.exs b/priv/repo/migrations/20251025210000_add_smart_uuid_generation.exs index c6da40e..0f459a9 100644 --- a/priv/repo/migrations/20251025210000_add_smart_uuid_generation.exs +++ b/priv/repo/migrations/20251025210000_add_smart_uuid_generation.exs @@ -1,11 +1,11 @@ -defmodule QuantumFlow.Repo.Migrations.AddSmartUuidGeneration do +defmodule Singularity.Workflow.Repo.Migrations.AddSmartUuidGeneration do @moduledoc """ Creates a smart UUID generation function that uses the best available method. - PostgreSQL 18+: Uses uuidv7() (time-ordered, built-in) - PostgreSQL 13-17: Falls back to gen_random_uuid() (random v4) - This ensures quantum_flow works on both old and new PostgreSQL versions. + This ensures Singularity.Workflow works on both old and new PostgreSQL versions. """ use Ecto.Migration diff --git a/priv/repo/migrations/20251025210500_change_complete_task_return_type.exs b/priv/repo/migrations/20251025210500_change_complete_task_return_type.exs index 347eb0c..67ca4f9 100644 --- a/priv/repo/migrations/20251025210500_change_complete_task_return_type.exs +++ b/priv/repo/migrations/20251025210500_change_complete_task_return_type.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.ChangeCompleteTaskReturnType do +defmodule Singularity.Workflow.Repo.Migrations.ChangeCompleteTaskReturnType do @moduledoc """ Changes complete_task() return type from void to INTEGER. @@ -17,7 +17,7 @@ defmodule QuantumFlow.Repo.Migrations.ChangeCompleteTaskReturnType do ## See Also - - test/QuantumFlow/complete_task_test.exs (lines 14-45) - Detailed explanation + - test/singularity_workflow/complete_task_test.exs (lines 14-45) - Detailed explanation - https://github.com/elixir-ecto/postgrex/issues/ (Postgrex void return issues) """ use Ecto.Migration @@ -231,13 +231,13 @@ defmodule QuantumFlow.Repo.Migrations.ChangeCompleteTaskReturnType do WHERE id = p_run_id; -- After step completion, cascade any taskless steps (empty-array propagation) - PERFORM QuantumFlow.cascade_complete_taskless_steps(p_run_id); + PERFORM singularity_workflow.cascade_complete_taskless_steps(p_run_id); -- Trigger start_ready_steps to awaken newly ready steps PERFORM start_ready_steps(p_run_id); -- Then check if run is complete and aggregate leaf outputs - PERFORM QuantumFlow.maybe_complete_run(p_run_id); + PERFORM singularity_workflow.maybe_complete_run(p_run_id); END IF; RETURN 1; -- Success: 1 task completed diff --git a/priv/repo/migrations/20251025213000_fix_complete_task_select.exs b/priv/repo/migrations/20251025213000_fix_complete_task_select.exs index 91b93d4..71c1d83 100644 --- a/priv/repo/migrations/20251025213000_fix_complete_task_select.exs +++ b/priv/repo/migrations/20251025213000_fix_complete_task_select.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.FixCompleteTaskSelect do +defmodule Singularity.Workflow.Repo.Migrations.FixCompleteTaskSelect do @moduledoc """ Fixes "query has no destination for result data" error in complete_task(). @@ -216,9 +216,9 @@ defmodule QuantumFlow.Repo.Migrations.FixCompleteTaskSelect do SET remaining_steps = remaining_steps - 1 WHERE id = p_run_id; - PERFORM QuantumFlow.cascade_complete_taskless_steps(p_run_id); + PERFORM singularity_workflow.cascade_complete_taskless_steps(p_run_id); PERFORM start_ready_steps(p_run_id); - PERFORM QuantumFlow.maybe_complete_run(p_run_id); + PERFORM singularity_workflow.maybe_complete_run(p_run_id); END IF; RETURN 1; diff --git a/priv/repo/migrations/20251026165801_create_cascade_complete_taskless_steps_function.exs b/priv/repo/migrations/20251026165801_create_cascade_complete_taskless_steps_function.exs index ac9a4d9..d4e7b61 100644 --- a/priv/repo/migrations/20251026165801_create_cascade_complete_taskless_steps_function.exs +++ b/priv/repo/migrations/20251026165801_create_cascade_complete_taskless_steps_function.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.CreateCascadeCompleteTasklessStepsFunction do +defmodule Singularity.Workflow.Repo.Migrations.CreateCascadeCompleteTasklessStepsFunction do @moduledoc """ Creates the missing cascade_complete_taskless_steps function. @@ -12,7 +12,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateCascadeCompleteTasklessStepsFunction def up do execute(""" - CREATE OR REPLACE FUNCTION QuantumFlow.cascade_complete_taskless_steps(p_run_id UUID) + CREATE OR REPLACE FUNCTION singularity_workflow.cascade_complete_taskless_steps(p_run_id UUID) RETURNS INTEGER LANGUAGE plpgsql AS $$ @@ -54,7 +54,7 @@ defmodule QuantumFlow.Repo.Migrations.CreateCascadeCompleteTasklessStepsFunction PERFORM start_ready_steps(p_run_id); -- Check if the run is now complete - PERFORM QuantumFlow.maybe_complete_run(p_run_id); + PERFORM singularity_workflow.maybe_complete_run(p_run_id); END LOOP; RETURN v_completed_count; @@ -64,6 +64,6 @@ defmodule QuantumFlow.Repo.Migrations.CreateCascadeCompleteTasklessStepsFunction end def down do - execute("DROP FUNCTION IF EXISTS QuantumFlow.cascade_complete_taskless_steps(UUID)") + execute("DROP FUNCTION IF EXISTS singularity_workflow.cascade_complete_taskless_steps(UUID)") end end diff --git a/priv/repo/migrations/20251026193331_fix_maybe_complete_run_output_aggregation.exs b/priv/repo/migrations/20251026193331_fix_maybe_complete_run_output_aggregation.exs index c8e9319..158f43a 100644 --- a/priv/repo/migrations/20251026193331_fix_maybe_complete_run_output_aggregation.exs +++ b/priv/repo/migrations/20251026193331_fix_maybe_complete_run_output_aggregation.exs @@ -1,6 +1,6 @@ -defmodule QuantumFlow.Repo.Migrations.FixMaybeCompleteRunOutputAggregation do +defmodule Singularity.Workflow.Repo.Migrations.FixMaybeCompleteRunOutputAggregation do @moduledoc """ - Fixes QuantumFlow.maybe_complete_run() to aggregate leaf step outputs into a flat array + Fixes singularity_workflow.maybe_complete_run() to aggregate leaf step outputs into a flat array instead of nested objects. Previous version used jsonb_object_agg(step_slug, output) which created: @@ -13,7 +13,7 @@ defmodule QuantumFlow.Repo.Migrations.FixMaybeCompleteRunOutputAggregation do def up do execute(""" - CREATE OR REPLACE FUNCTION QuantumFlow.maybe_complete_run(p_run_id UUID) + CREATE OR REPLACE FUNCTION singularity_workflow.maybe_complete_run(p_run_id UUID) RETURNS void LANGUAGE plpgsql AS $$ @@ -81,15 +81,15 @@ defmodule QuantumFlow.Repo.Migrations.FixMaybeCompleteRunOutputAggregation do """) execute(""" - COMMENT ON FUNCTION QuantumFlow.maybe_complete_run(UUID) IS - 'Checks if run is complete (all steps done), marks as completed, and aggregates leaf step outputs into a flat array. Matches QuantumFlow implementation.' + COMMENT ON FUNCTION singularity_workflow.maybe_complete_run(UUID) IS + 'Checks if run is complete (all steps done), marks as completed, and aggregates leaf step outputs into a flat array. Matches Singularity.Workflow implementation.' """) end def down do # Revert to the previous buggy version that creates nested objects execute(""" - CREATE OR REPLACE FUNCTION QuantumFlow.maybe_complete_run(p_run_id UUID) + CREATE OR REPLACE FUNCTION singularity_workflow.maybe_complete_run(p_run_id UUID) RETURNS void LANGUAGE plpgsql AS $$ @@ -134,7 +134,7 @@ defmodule QuantumFlow.Repo.Migrations.FixMaybeCompleteRunOutputAggregation do AND workflow_runs.status != 'completed' RETURNING * INTO v_completed_run; - -- Log completion (optional: would broadcast event in QuantumFlow) + -- Log completion (optional: would broadcast event in Singularity.Workflow) IF v_completed_run.id IS NOT NULL THEN RAISE NOTICE 'Run completed: run_id=%, output=%', v_completed_run.id, v_completed_run.output; @@ -144,8 +144,8 @@ defmodule QuantumFlow.Repo.Migrations.FixMaybeCompleteRunOutputAggregation do """) execute(""" - COMMENT ON FUNCTION QuantumFlow.maybe_complete_run(UUID) IS - 'Checks if run is complete (all steps done), marks as completed, and aggregates leaf step outputs. Matches QuantumFlow implementation.' + COMMENT ON FUNCTION singularity_workflow.maybe_complete_run(UUID) IS + 'Checks if run is complete (all steps done), marks as completed, and aggregates leaf step outputs. Matches Singularity.Workflow implementation.' """) end end diff --git a/priv/repo/migrations/20251026200000_add_idempotency_key_to_step_tasks.exs b/priv/repo/migrations/20251026200000_add_idempotency_key_to_step_tasks.exs index 88f5c5a..46e9491 100644 --- a/priv/repo/migrations/20251026200000_add_idempotency_key_to_step_tasks.exs +++ b/priv/repo/migrations/20251026200000_add_idempotency_key_to_step_tasks.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.AddIdempotencyKeyToStepTasks do +defmodule Singularity.Workflow.Repo.Migrations.AddIdempotencyKeyToStepTasks do @moduledoc """ Adds idempotency_key column to workflow_step_tasks table to prevent duplicate task execution. diff --git a/priv/repo/migrations/20251026200100_update_start_ready_steps_with_idempotency.exs b/priv/repo/migrations/20251026200100_update_start_ready_steps_with_idempotency.exs index f83365e..b58361b 100644 --- a/priv/repo/migrations/20251026200100_update_start_ready_steps_with_idempotency.exs +++ b/priv/repo/migrations/20251026200100_update_start_ready_steps_with_idempotency.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.UpdateStartReadyStepsWithIdempotency do +defmodule Singularity.Workflow.Repo.Migrations.UpdateStartReadyStepsWithIdempotency do @moduledoc """ Updates start_ready_steps() to use idempotency_key for exactly-once task execution. @@ -34,7 +34,7 @@ defmodule QuantumFlow.Repo.Migrations.UpdateStartReadyStepsWithIdempotency do WHERE id = p_run_id; -- Ensure queue exists for this workflow - PERFORM QuantumFlow.ensure_workflow_queue(v_workflow_slug); + PERFORM singularity_workflow.ensure_workflow_queue(v_workflow_slug); -- Mark ready steps as started and send tasks to pgmq WITH ready_steps AS ( @@ -182,7 +182,7 @@ defmodule QuantumFlow.Repo.Migrations.UpdateStartReadyStepsWithIdempotency do WHERE id = p_run_id; -- Ensure queue exists for this workflow - PERFORM QuantumFlow.ensure_workflow_queue(v_workflow_slug); + PERFORM singularity_workflow.ensure_workflow_queue(v_workflow_slug); -- Mark ready steps as started and send tasks to pgmq WITH ready_steps AS ( @@ -293,7 +293,7 @@ defmodule QuantumFlow.Repo.Migrations.UpdateStartReadyStepsWithIdempotency do execute(""" COMMENT ON FUNCTION start_ready_steps(UUID) IS - 'Finds steps with all dependencies completed, marks as started, creates task records, and sends messages to pgmq queue. Matches QuantumFlow architecture.' + 'Finds steps with all dependencies completed, marks as started, creates task records, and sends messages to pgmq queue. Matches Singularity.Workflow architecture.' """) end end diff --git a/priv/repo/migrations/20251026200200_add_compute_idempotency_key_function.exs b/priv/repo/migrations/20251026200200_add_compute_idempotency_key_function.exs index 323586a..9b924f6 100644 --- a/priv/repo/migrations/20251026200200_add_compute_idempotency_key_function.exs +++ b/priv/repo/migrations/20251026200200_add_compute_idempotency_key_function.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.AddComputeIdempotencyKeyFunction do +defmodule Singularity.Workflow.Repo.Migrations.AddComputeIdempotencyKeyFunction do @moduledoc """ Adds a SQL helper function to compute idempotency_key. diff --git a/priv/repo/migrations/20251027000000_rename_all_function_parameters.exs b/priv/repo/migrations/20251027000000_rename_all_function_parameters.exs index 3a0dadf..2fcdcc3 100644 --- a/priv/repo/migrations/20251027000000_rename_all_function_parameters.exs +++ b/priv/repo/migrations/20251027000000_rename_all_function_parameters.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.RenameAllFunctionParameters do +defmodule Singularity.Workflow.Repo.Migrations.RenameAllFunctionParameters do @moduledoc """ Comprehensive fix: Rename ALL parameters in PL/pgSQL functions to use _arg prefix. @@ -18,10 +18,10 @@ defmodule QuantumFlow.Repo.Migrations.RenameAllFunctionParameters do def up do # Fix create_flow - revert to last known working version but qualify all column references - execute("DROP FUNCTION IF EXISTS QuantumFlow.create_flow(TEXT, INTEGER, INTEGER) CASCADE") + execute("DROP FUNCTION IF EXISTS singularity_workflow.create_flow(TEXT, INTEGER, INTEGER) CASCADE") execute(""" - CREATE FUNCTION QuantumFlow.create_flow( + CREATE FUNCTION singularity_workflow.create_flow( arg_workflow_slug TEXT, arg_max_attempts INTEGER DEFAULT 3, arg_timeout INTEGER DEFAULT 60 @@ -44,8 +44,8 @@ defmodule QuantumFlow.Repo.Migrations.RenameAllFunctionParameters do v_max_attempts := arg_max_attempts; v_timeout := arg_timeout; - -- Validate slug using QuantumFlow schema prefix - IF NOT QuantumFlow.is_valid_slug(v_workflow_slug) THEN + -- Validate slug using singularity_workflow PostgreSQL schema prefix + IF NOT singularity_workflow.is_valid_slug(v_workflow_slug) THEN RAISE EXCEPTION 'Invalid workflow_slug: %', v_workflow_slug; END IF; @@ -66,10 +66,10 @@ defmodule QuantumFlow.Repo.Migrations.RenameAllFunctionParameters do """) # Fix add_step - similar qualified column approach - execute("DROP FUNCTION IF EXISTS QuantumFlow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER) CASCADE") + execute("DROP FUNCTION IF EXISTS singularity_workflow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER) CASCADE") execute(""" - CREATE FUNCTION QuantumFlow.add_step( + CREATE FUNCTION singularity_workflow.add_step( arg_workflow_slug TEXT, arg_step_slug TEXT, arg_depends_on TEXT[] DEFAULT '{}', @@ -111,11 +111,11 @@ defmodule QuantumFlow.Repo.Migrations.RenameAllFunctionParameters do v_max_attempts := arg_max_attempts; v_timeout := arg_timeout; - IF NOT QuantumFlow.is_valid_slug(v_workflow_slug) THEN + IF NOT singularity_workflow.is_valid_slug(v_workflow_slug) THEN RAISE EXCEPTION 'Invalid workflow_slug: %', v_workflow_slug; END IF; - IF NOT QuantumFlow.is_valid_slug(v_step_slug) THEN + IF NOT singularity_workflow.is_valid_slug(v_step_slug) THEN RAISE EXCEPTION 'Invalid step_slug: %', v_step_slug; END IF; @@ -166,10 +166,10 @@ defmodule QuantumFlow.Repo.Migrations.RenameAllFunctionParameters do def down do # Restore original create_flow - execute("DROP FUNCTION IF EXISTS QuantumFlow.create_flow(TEXT, INTEGER, INTEGER) CASCADE") + execute("DROP FUNCTION IF EXISTS singularity_workflow.create_flow(TEXT, INTEGER, INTEGER) CASCADE") execute(""" - CREATE FUNCTION QuantumFlow.create_flow( + CREATE FUNCTION singularity_workflow.create_flow( p_workflow_slug TEXT, p_max_attempts INTEGER DEFAULT 3, p_timeout INTEGER DEFAULT 60 @@ -183,7 +183,7 @@ defmodule QuantumFlow.Repo.Migrations.RenameAllFunctionParameters do LANGUAGE plpgsql AS $$ BEGIN - IF NOT QuantumFlow.is_valid_slug(p_workflow_slug) THEN + IF NOT singularity_workflow.is_valid_slug(p_workflow_slug) THEN RAISE EXCEPTION 'Invalid workflow_slug: %', p_workflow_slug; END IF; @@ -201,10 +201,10 @@ defmodule QuantumFlow.Repo.Migrations.RenameAllFunctionParameters do """) # Restore original add_step - execute("DROP FUNCTION IF EXISTS QuantumFlow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER) CASCADE") + execute("DROP FUNCTION IF EXISTS singularity_workflow.add_step(TEXT, TEXT, TEXT[], TEXT, INTEGER, INTEGER, INTEGER) CASCADE") execute(""" - CREATE FUNCTION QuantumFlow.add_step( + CREATE FUNCTION singularity_workflow.add_step( p_workflow_slug TEXT, p_step_slug TEXT, p_depends_on TEXT[] DEFAULT '{}', @@ -231,11 +231,11 @@ defmodule QuantumFlow.Repo.Migrations.RenameAllFunctionParameters do v_next_index INTEGER; v_deps_count INTEGER; BEGIN - IF NOT QuantumFlow.is_valid_slug(p_workflow_slug) THEN + IF NOT singularity_workflow.is_valid_slug(p_workflow_slug) THEN RAISE EXCEPTION 'Invalid workflow_slug: %', p_workflow_slug; END IF; - IF NOT QuantumFlow.is_valid_slug(p_step_slug) THEN + IF NOT singularity_workflow.is_valid_slug(p_step_slug) THEN RAISE EXCEPTION 'Invalid step_slug: %', p_step_slug; END IF; diff --git a/priv/repo/migrations/20251027030000_add_performance_indexes.exs b/priv/repo/migrations/20251027030000_add_performance_indexes.exs index 8e72858..ea0ef83 100644 --- a/priv/repo/migrations/20251027030000_add_performance_indexes.exs +++ b/priv/repo/migrations/20251027030000_add_performance_indexes.exs @@ -1,4 +1,4 @@ -defmodule QuantumFlow.Repo.Migrations.AddPerformanceIndexes do +defmodule Singularity.Workflow.Repo.Migrations.AddPerformanceIndexes do use Ecto.Migration def change do diff --git a/priv/repo/migrations/20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs b/priv/repo/migrations/20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs new file mode 100644 index 0000000..ff8c183 --- /dev/null +++ b/priv/repo/migrations/20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs @@ -0,0 +1,60 @@ +defmodule Singularity.Workflow.Repo.Migrations.RenameQuantumflowSchemaToSingularityWorkflow do + @moduledoc """ + Renames the PostgreSQL schema from QuantumFlow to singularity_workflow. + + This migration provides a safe upgrade path for existing databases that were + created with the old QuantumFlow schema name. + + ## What This Does + + - Checks if the QuantumFlow schema exists + - If it exists, renames it to singularity_workflow + - If it doesn't exist, does nothing (assumes fresh install with new schema name) + + ## For Fresh Installs + + If you're installing on a fresh database, this migration will do nothing since + the QuantumFlow schema never existed. The singularity_workflow schema will be + created by the earlier migration (20251025150001_create_pgmq_queue_functions.exs). + + ## For Existing Databases + + If you have an existing database with the QuantumFlow schema, this migration + will rename it to singularity_workflow, preserving all functions and data. + """ + use Ecto.Migration + + def up do + # Check if QuantumFlow schema exists and rename it + execute(""" + DO $$ + BEGIN + IF EXISTS ( + SELECT 1 FROM information_schema.schemata WHERE schema_name = 'QuantumFlow' + ) THEN + ALTER SCHEMA "QuantumFlow" RENAME TO singularity_workflow; + RAISE NOTICE 'Renamed schema QuantumFlow to singularity_workflow'; + ELSE + RAISE NOTICE 'Schema QuantumFlow does not exist, skipping rename'; + END IF; + END $$; + """) + end + + def down do + # Rename back to QuantumFlow for rollback + execute(""" + DO $$ + BEGIN + IF EXISTS ( + SELECT 1 FROM information_schema.schemata WHERE schema_name = 'singularity_workflow' + ) THEN + ALTER SCHEMA singularity_workflow RENAME TO "QuantumFlow"; + RAISE NOTICE 'Renamed schema singularity_workflow back to QuantumFlow'; + ELSE + RAISE NOTICE 'Schema singularity_workflow does not exist, skipping rename'; + END IF; + END $$; + """) + end +end diff --git a/test/SNAPSHOT_TESTING.md b/test/SNAPSHOT_TESTING.md index 4a28f3e..02e1f20 100644 --- a/test/SNAPSHOT_TESTING.md +++ b/test/SNAPSHOT_TESTING.md @@ -1,8 +1,8 @@ -# Snapshot Testing in QuantumFlow +# Snapshot Testing in Singularity.Workflow ## Overview -QuantumFlow uses **hybrid snapshot testing** to combine the benefits of: +Singularity.Workflow uses **hybrid snapshot testing** to combine the benefits of: - **Focused assertions** - Test critical business logic with explicit assertions - **Snapshot regression detection** - Catch unintended structural changes @@ -28,7 +28,7 @@ test "complex operation produces expected structure" do assert length(result.tasks) == 5 # Snapshot for structure regression detection - QuantumFlow.Test.Snapshot.assert_snapshot(result, "operation_structure") + Singularity.Workflow.Test.Snapshot.assert_snapshot(result, "operation_structure") end ``` @@ -43,7 +43,7 @@ SNAPSHOT_UPDATE=1 mix test Or update a specific snapshot: ```bash -SNAPSHOT_UPDATE=1 mix test test/quantum_flow/orchestrator_test.exs +SNAPSHOT_UPDATE=1 mix test test/singularity_workflow/orchestrator_test.exs ``` ## File Organization @@ -86,7 +86,7 @@ test "orchestrator decomposes complex goal" do assert task_graph.tasks[:finalize].depends_on == [:validate, :merge] # Structure regression detection: full snapshot - QuantumFlow.Test.Snapshot.assert_snapshot(task_graph, "complex_goal_decomposition") + Singularity.Workflow.Test.Snapshot.assert_snapshot(task_graph, "complex_goal_decomposition") end ``` @@ -136,10 +136,10 @@ Compare data with stored snapshot. ```elixir # Create or compare snapshot -QuantumFlow.Test.Snapshot.assert_snapshot(result, "operation_result") +Singularity.Workflow.Test.Snapshot.assert_snapshot(result, "operation_result") # Force update even if it matches -QuantumFlow.Test.Snapshot.assert_snapshot(result, "operation_result", update: true) +Singularity.Workflow.Test.Snapshot.assert_snapshot(result, "operation_result", update: true) ``` ### `assert_json_equal(actual, expected, message)` @@ -148,7 +148,7 @@ Compare two structures as JSON without snapshots. ```elixir # Useful for dynamic comparisons -QuantumFlow.Test.Snapshot.assert_json_equal(actual_dag, expected_dag, "DAG structure") +Singularity.Workflow.Test.Snapshot.assert_json_equal(actual_dag, expected_dag, "DAG structure") ``` ## See Also diff --git a/test/singularity_workflow/complete_task_test.exs b/test/singularity_workflow/complete_task_test.exs index aacc21f..56fa731 100644 --- a/test/singularity_workflow/complete_task_test.exs +++ b/test/singularity_workflow/complete_task_test.exs @@ -7,7 +7,7 @@ defmodule Singularity.Workflow.CompleteTaskTest do @moduledoc """ Integration tests for complete_task() SQL function. - These tests require a running Postgres with the QuantumFlow schema/migrations applied. + These tests require a running Postgres with the Singularity.Workflow schema/migrations applied. Set DATABASE_URL to point to the DB before running `mix test` to enable them. NOTE: These tests have database state management issues with simultaneous test execution. diff --git a/test/singularity_workflow/dag/run_initializer_test.exs b/test/singularity_workflow/dag/run_initializer_test.exs index 586dd89..d2c9729 100644 --- a/test/singularity_workflow/dag/run_initializer_test.exs +++ b/test/singularity_workflow/dag/run_initializer_test.exs @@ -90,9 +90,9 @@ defmodule Singularity.Workflow.DAG.RunInitializerTest do - Dependency graph setup - Counter initialization - NOTE: These tests require PostgreSQL with QuantumFlow SQL functions. + NOTE: These tests require PostgreSQL with Singularity.Workflow SQL functions. Set DATABASE_URL or start database with migrations. - Run with: mix test test/QuantumFlow/dag/run_initializer_test.exs + Run with: mix test test/Singularity.Workflow/dag/run_initializer_test.exs Tests are tagged :integration and can be skipped if database is unavailable. """ diff --git a/test/singularity_workflow/dag/task_executor_test.exs b/test/singularity_workflow/dag/task_executor_test.exs index 5d63a6b..922bd05 100644 --- a/test/singularity_workflow/dag/task_executor_test.exs +++ b/test/singularity_workflow/dag/task_executor_test.exs @@ -54,7 +54,7 @@ defmodule Singularity.Workflow.DAG.TaskExecutorTest do Comprehensive TaskExecutor tests covering: - Chicago-style TDD (state-based testing) - Task polling and claiming via pgmq - - Execution loop with QuantumFlow PostgreSQL functions + - Execution loop with Singularity.Workflow PostgreSQL functions - Error handling and retries - Timeout management """ diff --git a/test/singularity_workflow/dag/workflow_definition_test.exs b/test/singularity_workflow/dag/workflow_definition_test.exs index 2d16e8c..cc1dc2f 100644 --- a/test/singularity_workflow/dag/workflow_definition_test.exs +++ b/test/singularity_workflow/dag/workflow_definition_test.exs @@ -159,7 +159,10 @@ defmodule Singularity.Workflow.DAG.WorkflowDefinitionTest do dependencies: definition.dependencies } - Singularity.Workflow.Test.Snapshot.assert_snapshot(snapshot_data, "workflow_definition_parallel_dag") + Singularity.Workflow.Test.Snapshot.assert_snapshot( + snapshot_data, + "workflow_definition_parallel_dag" + ) end end diff --git a/test/singularity_workflow/executor_test.exs b/test/singularity_workflow/executor_test.exs index 83ffced..d32fc6b 100644 --- a/test/singularity_workflow/executor_test.exs +++ b/test/singularity_workflow/executor_test.exs @@ -67,7 +67,7 @@ defmodule Singularity.Workflow.ExecutorTest do - Error handling and status queries - Database-driven DAG coordination - NOTE: These are integration tests requiring PostgreSQL with QuantumFlow schema. + NOTE: These are integration tests requiring PostgreSQL with Singularity.Workflow schema. Tests run against real database with migrations applied. """ @@ -306,7 +306,9 @@ defmodule Singularity.Workflow.ExecutorTest do # Add steps {:ok, _} = Singularity.Workflow.FlowBuilder.add_step("test_dynamic_simple", "step1", [], Repo) - {:ok, _} = Singularity.Workflow.FlowBuilder.add_step("test_dynamic_simple", "step2", ["step1"], Repo) + + {:ok, _} = + Singularity.Workflow.FlowBuilder.add_step("test_dynamic_simple", "step2", ["step1"], Repo) # Define step functions step_functions = %{ @@ -332,10 +334,17 @@ defmodule Singularity.Workflow.ExecutorTest do test "maps step functions correctly" do # Create workflow {:ok, _} = Singularity.Workflow.FlowBuilder.create_flow("test_dynamic_mapping", Repo) - {:ok, _} = Singularity.Workflow.FlowBuilder.add_step("test_dynamic_mapping", "transform", [], Repo) {:ok, _} = - Singularity.Workflow.FlowBuilder.add_step("test_dynamic_mapping", "validate", ["transform"], Repo) + Singularity.Workflow.FlowBuilder.add_step("test_dynamic_mapping", "transform", [], Repo) + + {:ok, _} = + Singularity.Workflow.FlowBuilder.add_step( + "test_dynamic_mapping", + "validate", + ["transform"], + Repo + ) # Define step functions with specific behavior step_functions = %{ @@ -365,10 +374,17 @@ defmodule Singularity.Workflow.ExecutorTest do test "handles missing step functions" do # Create workflow with more steps than functions provided {:ok, _} = Singularity.Workflow.FlowBuilder.create_flow("test_dynamic_missing", Repo) - {:ok, _} = Singularity.Workflow.FlowBuilder.add_step("test_dynamic_missing", "step_a", [], Repo) {:ok, _} = - Singularity.Workflow.FlowBuilder.add_step("test_dynamic_missing", "step_b", ["step_a"], Repo) + Singularity.Workflow.FlowBuilder.add_step("test_dynamic_missing", "step_a", [], Repo) + + {:ok, _} = + Singularity.Workflow.FlowBuilder.add_step( + "test_dynamic_missing", + "step_b", + ["step_a"], + Repo + ) # Only provide step_a function, missing step_b step_functions = %{ diff --git a/test/singularity_workflow/idempotency_test.exs b/test/singularity_workflow/idempotency_test.exs index 033632e..cc95997 100644 --- a/test/singularity_workflow/idempotency_test.exs +++ b/test/singularity_workflow/idempotency_test.exs @@ -39,7 +39,10 @@ defmodule Singularity.Workflow.IdempotencyTest do started_at: DateTime.utc_now() } - Repo.insert!(%Singularity.Workflow.WorkflowRun{} |> Singularity.Workflow.WorkflowRun.changeset(run_attrs)) + Repo.insert!( + %Singularity.Workflow.WorkflowRun{} + |> Singularity.Workflow.WorkflowRun.changeset(run_attrs) + ) end describe "StepTask.compute_idempotency_key/4" do @@ -166,7 +169,11 @@ defmodule Singularity.Workflow.IdempotencyTest do remaining_steps: 1 } - run = Repo.insert!(%Singularity.Workflow.WorkflowRun{} |> Singularity.Workflow.WorkflowRun.changeset(run_attrs)) + run = + Repo.insert!( + %Singularity.Workflow.WorkflowRun{} + |> Singularity.Workflow.WorkflowRun.changeset(run_attrs) + ) # Insert first task task1 = @@ -207,7 +214,11 @@ defmodule Singularity.Workflow.IdempotencyTest do remaining_steps: 1 } - run = Repo.insert!(%Singularity.Workflow.WorkflowRun{} |> Singularity.Workflow.WorkflowRun.changeset(run_attrs)) + run = + Repo.insert!( + %Singularity.Workflow.WorkflowRun{} + |> Singularity.Workflow.WorkflowRun.changeset(run_attrs) + ) # Insert first task task1 = diff --git a/test/singularity_workflow/orchestrator/executor_test.exs b/test/singularity_workflow/orchestrator/executor_test.exs index 3c42dc4..31dcdcd 100644 --- a/test/singularity_workflow/orchestrator/executor_test.exs +++ b/test/singularity_workflow/orchestrator/executor_test.exs @@ -1,4 +1,4 @@ -if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do +if System.get_env("SINGULARITY_WORKFLOW_SKIP_DB") != "1" do defmodule Singularity.Workflow.Orchestrator.ExecutorTest do use ExUnit.Case, async: true @@ -30,9 +30,9 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do end) Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :update_execution_status, fn _, - _, - _, - _ -> + _, + _, + _ -> {:ok, %{id: "exec_123", status: "completed"}} end) @@ -65,13 +65,14 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do end) Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :update_execution_status, fn _, - _, - _, - _ -> + _, + _, + _ -> {:ok, %{id: "exec_123", status: "failed"}} end) - {:error, :execution_failed} = Executor.execute_workflow(workflow, context, Singularity.Workflow.Repo) + {:error, :execution_failed} = + Executor.execute_workflow(workflow, context, Singularity.Workflow.Repo) end test "handles execution record creation failure" do @@ -88,7 +89,8 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do {:error, :database_error} end) - {:error, :database_error} = Executor.execute_workflow(workflow, context, Singularity.Workflow.Repo) + {:error, :database_error} = + Executor.execute_workflow(workflow, context, Singularity.Workflow.Repo) end end @@ -105,18 +107,21 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do context = %{goal: "Build auth system"} execution = %{id: "exec_123", execution_id: "exec_123"} - Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :create_task_execution, fn _, _ -> + Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :create_task_execution, fn _, + _ -> {:ok, %{id: "task_exec_123", task_id: "task1", status: "pending"}} end) - Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :update_task_execution_status, fn _, - _, - _, - _ -> - {:ok, %{id: "task_exec_123", status: "completed"}} - end) + Mox.stub( + Singularity.Workflow.Orchestrator.Repository.Mock, + :update_task_execution_status, + fn _, _, _, _ -> + {:ok, %{id: "task_exec_123", status: "completed"}} + end + ) - {:ok, result} = Executor.execute_task(task_config, context, execution, Singularity.Workflow.Repo) + {:ok, result} = + Executor.execute_task(task_config, context, execution, Singularity.Workflow.Repo) assert result == "result1" end @@ -133,16 +138,18 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do context = %{goal: "Build auth system"} execution = %{id: "exec_123", execution_id: "exec_123"} - Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :create_task_execution, fn _, _ -> + Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :create_task_execution, fn _, + _ -> {:ok, %{id: "task_exec_123", task_id: "task1", status: "pending"}} end) - Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :update_task_execution_status, fn _, - _, - _, - _ -> - {:ok, %{id: "task_exec_123", status: "failed"}} - end) + Mox.stub( + Singularity.Workflow.Orchestrator.Repository.Mock, + :update_task_execution_status, + fn _, _, _, _ -> + {:ok, %{id: "task_exec_123", status: "failed"}} + end + ) {:error, :task_failed} = Executor.execute_task(task_config, context, execution, Singularity.Workflow.Repo) @@ -161,16 +168,18 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do context = %{goal: "Build auth system"} execution = %{id: "exec_123", execution_id: "exec_123"} - Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :create_task_execution, fn _, _ -> + Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :create_task_execution, fn _, + _ -> {:ok, %{id: "task_exec_123", task_id: "task1", status: "pending"}} end) - Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :update_task_execution_status, fn _, - _, - _, - _ -> - {:ok, %{id: "task_exec_123", status: "failed"}} - end) + Mox.stub( + Singularity.Workflow.Orchestrator.Repository.Mock, + :update_task_execution_status, + fn _, _, _, _ -> + {:ok, %{id: "task_exec_123", status: "failed"}} + end + ) {:error, :max_retries_exceeded} = Executor.execute_task(task_config, context, execution, Singularity.Workflow.Repo) @@ -217,7 +226,8 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do {:error, :not_found} end) - {:error, :not_found} = Executor.get_execution_status("nonexistent", Singularity.Workflow.Repo) + {:error, :not_found} = + Executor.get_execution_status("nonexistent", Singularity.Workflow.Repo) end end @@ -234,13 +244,14 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do end) Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :update_execution_status, fn _, - _, - _, - _ -> + _, + _, + _ -> {:ok, %{id: "exec_123", status: "cancelled"}} end) - :ok = Executor.cancel_execution("exec_123", Singularity.Workflow.Repo, reason: "User requested") + :ok = + Executor.cancel_execution("exec_123", Singularity.Workflow.Repo, reason: "User requested") end test "handles execution not found" do @@ -262,7 +273,8 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do {:ok, execution} end) - {:error, :execution_not_running} = Executor.cancel_execution("exec_123", Singularity.Workflow.Repo) + {:error, :execution_not_running} = + Executor.cancel_execution("exec_123", Singularity.Workflow.Repo) end test "force cancels non-running execution" do @@ -277,9 +289,9 @@ if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do end) Mox.stub(Singularity.Workflow.Orchestrator.Repository.Mock, :update_execution_status, fn _, - _, - _, - _ -> + _, + _, + _ -> {:ok, %{id: "exec_123", status: "cancelled"}} end) diff --git a/test/singularity_workflow/orchestrator_notifications_test.exs b/test/singularity_workflow/orchestrator_notifications_test.exs index 68e2ce0..a63b455 100644 --- a/test/singularity_workflow/orchestrator_notifications_test.exs +++ b/test/singularity_workflow/orchestrator_notifications_test.exs @@ -5,7 +5,13 @@ defmodule Singularity.Workflow.OrchestratorNotificationsTest do setup do Singularity.Workflow.Test.MoxHelper.setup_mox() - Application.put_env(:singularity_workflow, :notifications_impl, Singularity.Workflow.Notifications.Mock) + + Application.put_env( + :singularity_workflow, + :notifications_impl, + Singularity.Workflow.Notifications.Mock + ) + Mox.set_mox_global() :ok end @@ -101,7 +107,10 @@ defmodule Singularity.Workflow.OrchestratorNotificationsTest do assert data.metrics.execution_time == 1500 # Snapshot the notification payload for structure regression detection - Singularity.Workflow.Test.Snapshot.assert_snapshot(data, "orchestrator_notifications_performance") + Singularity.Workflow.Test.Snapshot.assert_snapshot( + data, + "orchestrator_notifications_performance" + ) {:ok, "message_perf"} end) diff --git a/test/singularity_workflow/orchestrator_test.exs b/test/singularity_workflow/orchestrator_test.exs index 278e9f9..66a448d 100644 --- a/test/singularity_workflow/orchestrator_test.exs +++ b/test/singularity_workflow/orchestrator_test.exs @@ -31,7 +31,10 @@ defmodule Singularity.Workflow.OrchestratorTest do assert tasks["task3"].depends_on == ["task2"] # Snapshot for complete structure regression detection - Singularity.Workflow.Test.Snapshot.assert_snapshot(task_graph, "orchestrator_decompose_goal_linear") + Singularity.Workflow.Test.Snapshot.assert_snapshot( + task_graph, + "orchestrator_decompose_goal_linear" + ) end test "handles decomposer errors" do @@ -158,7 +161,12 @@ defmodule Singularity.Workflow.OrchestratorTest do end) {:ok, result} = - Orchestrator.execute_goal("Test goal", decomposer, step_functions, Singularity.Workflow.Repo) + Orchestrator.execute_goal( + "Test goal", + decomposer, + step_functions, + Singularity.Workflow.Repo + ) assert result.success == true assert result.results["task1"] == "result1" @@ -170,7 +178,12 @@ defmodule Singularity.Workflow.OrchestratorTest do step_functions = %{} {:error, :decomposition_failed} = - Orchestrator.execute_goal("Invalid goal", decomposer, step_functions, Singularity.Workflow.Repo) + Orchestrator.execute_goal( + "Invalid goal", + decomposer, + step_functions, + Singularity.Workflow.Repo + ) end test "handles workflow creation failure" do @@ -182,7 +195,12 @@ defmodule Singularity.Workflow.OrchestratorTest do step_functions = %{} {:error, %RuntimeError{}} = - Orchestrator.execute_goal("Test goal", decomposer, step_functions, Singularity.Workflow.Repo) + Orchestrator.execute_goal( + "Test goal", + decomposer, + step_functions, + Singularity.Workflow.Repo + ) end end diff --git a/test/singularity_workflow/schema_rename_migration_test.exs b/test/singularity_workflow/schema_rename_migration_test.exs new file mode 100644 index 0000000..60f242d --- /dev/null +++ b/test/singularity_workflow/schema_rename_migration_test.exs @@ -0,0 +1,410 @@ +defmodule Singularity.Workflow.SchemaRenameMigrationTest do + use ExUnit.Case, async: true + + @moduletag :migration_test + + # Load the migration module + Code.require_file( + "priv/repo/migrations/20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs", + File.cwd!() + ) + + alias Singularity.Workflow.Repo.Migrations.RenameQuantumflowSchemaToSingularityWorkflow + + describe "migration module structure" do + test "migration module is properly defined" do + assert Code.ensure_loaded?(RenameQuantumflowSchemaToSingularityWorkflow) + end + + test "migration module has up/0 function" do + assert function_exported?(RenameQuantumflowSchemaToSingularityWorkflow, :up, 0) + end + + test "migration module has down/0 function" do + assert function_exported?(RenameQuantumflowSchemaToSingularityWorkflow, :down, 0) + end + + test "migration module uses Ecto.Migration" do + migration_file = + Path.join([ + File.cwd!(), + "priv", + "repo", + "migrations", + "20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs" + ]) + + content = File.read!(migration_file) + assert content =~ "use Ecto.Migration" + end + + test "migration file exists in correct location" do + migration_path = + Path.join([ + File.cwd!(), + "priv", + "repo", + "migrations", + "20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs" + ]) + + assert File.exists?(migration_path) + end + + test "migration filename follows Ecto naming convention" do + migration_path = + Path.join([ + File.cwd!(), + "priv", + "repo", + "migrations", + "20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs" + ]) + + assert Path.basename(migration_path) =~ + ~r/^\d{14}_rename_quantumflow_schema_to_singularity_workflow\.exs$/ + end + end + + describe "migration documentation" do + setup do + migration_file = + Path.join([ + File.cwd!(), + "priv", + "repo", + "migrations", + "20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs" + ]) + + {:ok, content: File.read!(migration_file)} + end + + test "migration has @moduledoc", %{content: content} do + assert content =~ "@moduledoc" + end + + test "documentation mentions schema rename", %{content: content} do + assert content =~ "Renames the PostgreSQL schema" + end + + test "documentation mentions QuantumFlow", %{content: content} do + assert content =~ "QuantumFlow" + end + + test "documentation mentions singularity_workflow", %{content: content} do + assert content =~ "singularity_workflow" + end + + test "documentation mentions safe upgrade path", %{content: content} do + assert content =~ "safe upgrade path" + end + + test "documentation mentions existing databases", %{content: content} do + assert content =~ "existing databases" or content =~ "Existing Databases" + end + + test "documentation mentions fresh installs", %{content: content} do + assert content =~ "fresh install" or content =~ "Fresh Install" + end + end + + describe "migration up SQL validation" do + setup do + migration_file = + Path.join([ + File.cwd!(), + "priv", + "repo", + "migrations", + "20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs" + ]) + + {:ok, content: File.read!(migration_file)} + end + + test "up migration uses execute/1", %{content: content} do + assert content =~ "execute(" + end + + test "up migration checks information_schema.schemata", %{content: content} do + assert content =~ "information_schema.schemata" + end + + test "up migration checks for QuantumFlow schema existence", %{content: content} do + assert content =~ "schema_name = 'QuantumFlow'" + end + + test "up migration uses ALTER SCHEMA", %{content: content} do + assert content =~ "ALTER SCHEMA" + end + + test "up migration renames to singularity_workflow", %{content: content} do + assert content =~ "RENAME TO singularity_workflow" + end + + test "up migration uses PL/pgSQL", %{content: content} do + assert content =~ "DO $$" + end + + test "up migration uses IF EXISTS conditional", %{content: content} do + assert content =~ "IF EXISTS" + end + + test "up migration uses RAISE NOTICE for renamed message", %{content: content} do + assert content =~ "RAISE NOTICE 'Renamed schema QuantumFlow to singularity_workflow'" + end + + test "up migration uses RAISE NOTICE for skip message", %{content: content} do + assert content =~ "RAISE NOTICE 'Schema QuantumFlow does not exist, skipping rename'" + end + end + + describe "migration down SQL validation" do + setup do + migration_file = + Path.join([ + File.cwd!(), + "priv", + "repo", + "migrations", + "20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs" + ]) + + {:ok, content: File.read!(migration_file)} + end + + test "down migration uses execute/1", %{content: content} do + [_before_down, down_section] = String.split(content, "def down do") + assert down_section =~ "execute(" + end + + test "down migration checks for singularity_workflow schema", %{content: content} do + assert content =~ "schema_name = 'singularity_workflow'" + end + + test "down migration renames back to QuantumFlow", %{content: content} do + assert content =~ ~s(RENAME TO "QuantumFlow") + end + + test "down migration uses RAISE NOTICE for rollback message", %{content: content} do + assert content =~ "Renamed schema singularity_workflow back to QuantumFlow" + end + + test "down migration uses RAISE NOTICE for skip message", %{content: content} do + assert content =~ "Schema singularity_workflow does not exist, skipping rename" + end + end + + describe "migration SQL structure" do + setup do + migration_file = + Path.join([ + File.cwd!(), + "priv", + "repo", + "migrations", + "20251103234710_rename_quantumflow_schema_to_singularity_workflow.exs" + ]) + + {:ok, content: File.read!(migration_file)} + end + + test "migration uses proper BEGIN/END blocks", %{content: content} do + assert content =~ "BEGIN" + assert content =~ "END" + end + + test "migration SQL uses proper THEN clause", %{content: content} do + assert content =~ "THEN" + end + + test "migration SQL uses ELSE clause for non-existence case", %{content: content} do + assert content =~ "ELSE" + end + + test "migration ends with END IF", %{content: content} do + assert content =~ "END IF" + end + end + + describe "all migration files naming validation" do + test "no migration files reference QuantumFlow in module name" do + migrations_dir = Path.join([File.cwd!(), "priv", "repo", "migrations"]) + migration_files = File.ls!(migrations_dir) + + for file <- migration_files do + content = File.read!(Path.join(migrations_dir, file)) + + if content =~ "defmodule" and content =~ "Migrations" do + assert content =~ "Singularity.Workflow.Repo.Migrations", + "Migration #{file} should use Singularity.Workflow.Repo.Migrations namespace" + + refute content =~ "QuantumFlow.Repo.Migrations", + "Migration #{file} should not use QuantumFlow.Repo.Migrations namespace" + end + end + end + + test "all PostgreSQL functions use singularity_workflow schema" do + migrations_dir = Path.join([File.cwd!(), "priv", "repo", "migrations"]) + migration_files = File.ls!(migrations_dir) + + for file <- migration_files do + content = File.read!(Path.join(migrations_dir, file)) + + if content =~ "CREATE FUNCTION" or content =~ "CREATE OR REPLACE FUNCTION" do + if content =~ ~r/CREATE.*FUNCTION\s+\w+\./ do + refute content =~ ~r/CREATE.*FUNCTION\s+QuantumFlow\./, + "Migration #{file} should not create functions in QuantumFlow schema" + end + end + end + end + end + + describe "codebase QuantumFlow reference validation" do + test "lib/ files use Singularity.Workflow not QuantumFlow in module names" do + lib_dir = Path.join([File.cwd!(), "lib"]) + lib_files = Path.wildcard(Path.join(lib_dir, "**/*.ex")) + + for file <- lib_files do + content = File.read!(file) + + if content =~ "defmodule" do + refute content =~ "defmodule QuantumFlow", + "#{file} should not define QuantumFlow modules" + + refute content =~ "defmodule Quantum", + "#{file} should not define Quantum modules" + end + end + end + + test "README mentions Singularity.Workflow" do + readme_path = Path.join([File.cwd!(), "README.md"]) + + if File.exists?(readme_path) do + content = File.read!(readme_path) + assert content =~ "Singularity.Workflow" or content =~ "singularity_workflow" + end + end + + test "mix.exs uses singularity_workflow as app name" do + mix_file = Path.join([File.cwd!(), "mix.exs"]) + content = File.read!(mix_file) + assert content =~ "app: :singularity_workflow" + refute content =~ "app: :quantum_flow" + end + end + + describe "production deployment guide validation" do + setup do + guide_path = Path.join([File.cwd!(), "docs", "SCHEMA_MIGRATION_GUIDE.md"]) + {:ok, guide_path: guide_path, content: File.read!(guide_path)} + end + + test "SCHEMA_MIGRATION_GUIDE.md exists", %{guide_path: guide_path} do + assert File.exists?(guide_path) + end + + test "guide mentions backup procedures", %{content: content} do + assert content =~ "backup" or content =~ "Backup" + end + + test "guide mentions rollback procedures", %{content: content} do + assert content =~ "rollback" or content =~ "Rollback" + end + + test "guide mentions verification steps", %{content: content} do + assert content =~ "verify" or content =~ "Verify" + end + + test "guide mentions troubleshooting", %{content: content} do + assert content =~ "troubleshooting" or content =~ "Troubleshooting" + end + + test "guide provides pg_dump command", %{content: content} do + assert content =~ "pg_dump" + end + + test "guide mentions mix ecto.migrate", %{content: content} do + assert content =~ "mix ecto.migrate" + end + + test "guide mentions mix ecto.rollback", %{content: content} do + assert content =~ "mix ecto.rollback" + end + + test "guide lists read_with_poll function", %{content: content} do + assert content =~ "read_with_poll" + end + + test "guide lists create_flow function", %{content: content} do + assert content =~ "create_flow" + end + + test "guide lists add_step function", %{content: content} do + assert content =~ "add_step" + end + + test "guide lists fail_task function", %{content: content} do + assert content =~ "fail_task" + end + + test "guide lists maybe_complete_run function", %{content: content} do + assert content =~ "maybe_complete_run" + end + + test "guide lists set_vt_batch function", %{content: content} do + assert content =~ "set_vt_batch" + end + + test "guide lists is_valid_slug function", %{content: content} do + assert content =~ "is_valid_slug" + end + + test "guide lists cascade_complete_taskless_steps function", %{content: content} do + assert content =~ "cascade_complete_taskless_steps" + end + + test "guide lists ensure_workflow_queue function", %{content: content} do + assert content =~ "ensure_workflow_queue" + end + + test "guide lists calculate_retry_delay function", %{content: content} do + assert content =~ "calculate_retry_delay" + end + + test "guide has production deployment checklist", %{content: content} do + assert content =~ "checklist" or content =~ "Checklist" + end + + test "guide mentions schema rename operation", %{content: content} do + # The guide explains the rename happens automatically via the migration + assert content =~ "rename" + end + end + + describe "GitHub workflows validation" do + test "docker-build.yml uses singularity_workflow naming" do + workflow_path = Path.join([File.cwd!(), ".github", "workflows", "docker-build.yml"]) + + if File.exists?(workflow_path) do + content = File.read!(workflow_path) + refute content =~ "quantum_flow-postgres" + assert content =~ "singularity" or content =~ "workflow" + end + end + + test "publish.yml uses singularity_workflow_test database" do + workflow_path = Path.join([File.cwd!(), ".github", "workflows", "publish.yml"]) + + if File.exists?(workflow_path) do + content = File.read!(workflow_path) + assert content =~ "singularity_workflow_test" + refute content =~ "quantum_flow_test" + end + end + end +end diff --git a/test/singularity_workflow/step_dependency_test.exs b/test/singularity_workflow/step_dependency_test.exs index 5764209..5b3b4ad 100644 --- a/test/singularity_workflow/step_dependency_test.exs +++ b/test/singularity_workflow/step_dependency_test.exs @@ -95,7 +95,7 @@ defmodule Singularity.Workflow.StepDependencyTest do # NOTE: find_dependents/3 and find_dependencies/3 are better tested # as integration tests with a real database connection. - # See test/QuantumFlow/complete_task_test.exs for integration testing. + # See test/Singularity.Workflow/complete_task_test.exs for integration testing. describe "schema properties" do test "has no primary key" do diff --git a/test/singularity_workflow/step_state_test.exs b/test/singularity_workflow/step_state_test.exs index 9655a5b..865cb28 100644 --- a/test/singularity_workflow/step_state_test.exs +++ b/test/singularity_workflow/step_state_test.exs @@ -7,7 +7,7 @@ defmodule Singularity.Workflow.StepStateTest do @moduledoc """ Chicago-style TDD: State-based testing for StepState schema. - Focus on counter-based coordination logic - the heart of QuantumFlow's DAG execution. + Focus on counter-based coordination logic - the heart of Singularity.Workflow's DAG execution. Tests verify final state after operations, not implementation details. """ diff --git a/test/singularity_workflow/workflow_composer_test.exs b/test/singularity_workflow/workflow_composer_test.exs index b18f366..222d14f 100644 --- a/test/singularity_workflow/workflow_composer_test.exs +++ b/test/singularity_workflow/workflow_composer_test.exs @@ -51,7 +51,10 @@ defmodule Singularity.Workflow.WorkflowComposerTest do assert result.results["task2"] == "result2" # Snapshot for execution result structure regression detection - Singularity.Workflow.Test.Snapshot.assert_snapshot(result, "workflow_composer_execution_result") + Singularity.Workflow.Test.Snapshot.assert_snapshot( + result, + "workflow_composer_execution_result" + ) end test "handles decomposition failure" do @@ -112,7 +115,10 @@ defmodule Singularity.Workflow.WorkflowComposerTest do {:ok, %{name: "test_workflow", steps: [], max_parallel: 10}} end) - Mox.stub(Singularity.Workflow.Orchestrator.Executor.Mock, :execute_workflow, fn _, _, _, opts -> + Mox.stub(Singularity.Workflow.Orchestrator.Executor.Mock, :execute_workflow, fn _, + _, + _, + opts -> assert Keyword.get(opts, :monitor) == true {:ok, %{success: true}} end) diff --git a/test/support/mox_helper.ex b/test/support/mox_helper.ex index df4d64e..546f8eb 100644 --- a/test/support/mox_helper.ex +++ b/test/support/mox_helper.ex @@ -10,7 +10,9 @@ defmodule Singularity.Workflow.Test.MoxHelper do # Define all required mocks using Mox.defmock # Mox is loaded as a dependency and doesn't need to be started as an application try do - Mox.defmock(Singularity.Workflow.Notifications.Mock, for: Singularity.Workflow.Notifications.Behaviour) + Mox.defmock(Singularity.Workflow.Notifications.Mock, + for: Singularity.Workflow.Notifications.Behaviour + ) rescue _ -> :ok end diff --git a/test/support/sql_case.ex b/test/support/sql_case.ex index 036ed98..83e6bba 100644 --- a/test/support/sql_case.ex +++ b/test/support/sql_case.ex @@ -3,7 +3,7 @@ defmodule Singularity.Workflow.SqlCase do Helper for SQL-based tests. Connects to Postgres using POSTGRES_URL or DATABASE_URL. Tests using this helper will be skipped if a DB is not reachable or the required - QuantumFlow tables are not present. This makes the migrated tests non-fatal in CI + Singularity.Workflow tables are not present. This makes the migrated tests non-fatal in CI environments where the developer hasn't prepared a DB. """ @@ -11,7 +11,7 @@ defmodule Singularity.Workflow.SqlCase do @doc """ Attempt to connect to the database and return a Postgrex connection. - If the DB is unavailable or the QuantumFlow tables are not present, call + If the DB is unavailable or the Singularity.Workflow tables are not present, call ExUnit.Callbacks.skip/1 to skip the test at runtime. """ def connect_or_skip do @@ -36,7 +36,7 @@ defmodule Singularity.Workflow.SqlCase do Process.exit(conn, :normal) {:skip, - "Database does not have QuantumFlow tables; run migrations before enabling SQL tests"} + "Database does not have Singularity.Workflow tables; run migrations before enabling SQL tests"} _ -> # register a stop on exit and return connection @@ -49,7 +49,7 @@ defmodule Singularity.Workflow.SqlCase do Process.exit(conn, :normal) {:skip, - "Database query failed; ensure DATABASE_URL points to a migrated QuantumFlow DB"} + "Database query failed; ensure DATABASE_URL points to a migrated Singularity.Workflow DB"} end {:error, reason} -> diff --git a/test/test_helper.exs b/test/test_helper.exs index 247ca16..091d540 100644 --- a/test/test_helper.exs +++ b/test/test_helper.exs @@ -10,7 +10,7 @@ Code.require_file("support/sql_case.ex", __DIR__) Code.require_file("support/snapshot.ex", __DIR__) # Start Ecto Repo for integration tests unless explicitly skipped -if System.get_env("QUANTUM_FLOW_SKIP_DB") != "1" do +if System.get_env("SINGULARITY_WORKFLOW_SKIP_DB") != "1" do {:ok, _} = Singularity.Workflow.Repo.start_link() Ecto.Adapters.SQL.Sandbox.mode(Singularity.Workflow.Repo, :manual) else