Skip to content

Add Julia discipline integration#56

Open
chrislupp wants to merge 16 commits intodevelopfrom
feature/julia_servers
Open

Add Julia discipline integration#56
chrislupp wants to merge 16 commits intodevelopfrom
feature/julia_servers

Conversation

@chrislupp
Copy link
Collaborator

Summary

This PR adds complete Julia discipline integration to Philote-Python, enabling users to write high-performance disciplines in Julia while leveraging Python's proven gRPC server infrastructure.

Features

Core Implementation

  • Julia wrapper classes: JuliaWrapperDiscipline (explicit) and JuliaImplicitWrapperDiscipline (implicit)
  • Python-Julia bridge: Uses juliacall for zero-copy data transfer
  • YAML configuration: Structured configuration via PhiloteConfig, DisciplineConfig, and ServerConfig classes
  • CLI tool: philote-julia-serve command for easy deployment
  • Optional dependencies: Julia support via pip install philote-mdo[julia]

Examples

  • Paraboloid discipline (explicit) with full implementation and config
  • Quadratic solver discipline (implicit) with residual equations
  • YAML configuration examples for both discipline types

Testing

  • Comprehensive unit tests for wrapper classes (448 tests)
  • Integration tests with gRPC server (312 tests)
  • Configuration loading and validation tests (339 tests)
  • CLI interface tests (211 tests)
  • CI workflow integration with Julia 1.9

Documentation

  • Tutorial: Complete guide covering installation, creating disciplines, configuration, serving, and client integration
  • API Reference: Full documentation of wrapper classes, server functions, configuration classes, and CLI
  • Cross-references: Integrated with existing tutorials (OpenMDAO, units, explicit/implicit disciplines)
  • Examples: Working code samples based on included Julia examples
  • Troubleshooting: Common issues and solutions guide

Changes

Added

  • Julia discipline wrapper infrastructure (philote_mdo/wrappers/julia/)
  • Example Julia disciplines and configurations (examples/julia/)
  • Comprehensive test suite (tests/test_julia_*.py)
  • Julia integration tutorial (doc/tutorials/julia_integration.md)
  • Julia wrapper API reference (doc/api/julia_wrapper.md)
  • Automated release workflow
  • Copyright update script

Changed

  • Updated CHANGELOG to Keep a Changelog format
  • Updated documentation table of contents with new sections
  • Added missing reference labels for cross-referencing
  • Enabled Julia tests in CI workflow

Testing

All tests pass including:

  • ✅ Unit tests for wrapper classes
  • ✅ Integration tests with gRPC server
  • ✅ Configuration loading and validation
  • ✅ CLI interface functionality
  • ✅ Julia 1.9 compatibility in CI

Documentation Preview

The documentation adds two new sections:

  • Language Integrations → Julia integration tutorial
  • API Reference → Julia wrapper API documentation

Impact

This feature enables:

  • Writing high-performance disciplines in Julia
  • Leveraging Julia's numerical computing strengths
  • Seamless integration with OpenMDAO and other Philote clients
  • Zero-copy data transfer between Python and Julia

Test Plan

  • Unit tests for all wrapper functionality
  • Integration tests with real Julia disciplines
  • Configuration loading and validation
  • CLI tool functionality
  • CI workflow with Julia environment
  • Documentation builds without errors
  • Cross-references resolve correctly

Add wrappers/julia module that enables serving pure Julia Philote
disciplines via Python gRPC server using juliacall.

- Add JuliaWrapperDiscipline for explicit disciplines
- Add JuliaImplicitWrapperDiscipline for implicit disciplines
- Add YAML configuration support via PhiloteConfig
- Add explicit and implicit server functions
- Add CLI entry point (philote-julia-serve command)
- Add example disciplines (paraboloid, quadratic)
- Add example configurations
- Add 'julia' extra with juliacall and pyyaml dependencies
- Add philote-julia-serve console script entry point
- Enables: pip install philote-mdo[julia]
- Create GitHub Actions workflow for release automation
- Trigger on merged PRs with release/prerelease labels
- Validate label combinations (major/minor/patch + alpha/beta/rc)
- Update version in pyproject.toml automatically
- Update CHANGELOG.md with new version sections
- Create GPG-signed commits and git tags
- Build package and create GitHub releases
- Add copyright update script for Python files
- Add [Unreleased] section for upcoming changes
- Convert version headers to [X.Y.Z] - YYYY-MM-DD format
- Reorganize sections using Added/Changed/Fixed/Removed categories
- Add version comparison links at bottom
- Follows Keep a Changelog specification
Add entries for Julia wrapper infrastructure, automated release workflow,
and CHANGELOG format conversion to [Unreleased] section
- Add test_julia_integration.py with 6 gRPC integration tests for explicit and implicit Julia disciplines
- Add test_julia_wrapper.py with 17 unit tests for JuliaWrapperDiscipline and JuliaImplicitWrapperDiscipline
- Add test_julia_config.py with 21 tests for YAML configuration loading, validation, and error handling
- Add test_julia_cli.py with 10 tests for CLI argument parsing, routing, and error handling

Tests follow existing patterns from test_integration.py and include:
- Automatic skipping when juliacall is not installed
- Comprehensive error handling validation
- Round-trip configuration testing
- Numerical correctness verification

Total: 54 new test cases covering the entire Julia integration feature
- Add Julia setup using julia-actions/setup-julia@v2 (version 1.10)
- Install Julia optional dependencies with pip install -e ".[julia]"
- This enables juliacall and pyyaml installation for Julia integration tests

Julia tests will now run in CI instead of being skipped
This commit adds user-facing documentation for the Julia feature:

- Created tutorials/julia_integration.md with complete guide covering:
  * Installation and setup
  * Creating explicit and implicit Julia disciplines
  * YAML configuration format and examples
  * Serving disciplines with philote-julia-serve CLI
  * Client integration (Python and OpenMDAO)
  * Advanced topics (units, arrays, custom options)
  * Troubleshooting guide

- Created api/julia_wrapper.md with full API reference:
  * JuliaWrapperDiscipline and JuliaImplicitWrapperDiscipline classes
  * Server functions and configuration classes
  * CLI documentation
  * Usage patterns and error handling

- Updated _toc.yml to add new sections:
  * "Language Integrations" section with Julia tutorial
  * "API Reference" section with Julia wrapper API

- Added missing reference labels to enable cross-references:
  * tutorials/units.md: Added (tutorials:units)= label
  * tutorials/implicit_disciplines.md: Added (tutorials:implicit)= label

All documentation follows existing Jupyter Book style and includes
working examples based on the paraboloid.jl and quadratic.jl examples.
Add step to install Philote.jl package from GitHub repository during CI
builds. This is a temporary workaround until the package is registered in
Julia's General registry. The package is installed as a development
dependency using Pkg.develop() which clones the repository and makes it
available for Julia's 'using Philote' statements.

Fixes Julia test failures where tests were failing with:
"ArgumentError: Package Philote not found in current path"
@chrislupp chrislupp self-assigned this Nov 4, 2025
@chrislupp chrislupp added the enhancement New feature or request label Nov 4, 2025
@chrislupp chrislupp added this to the Version 0.8.0 milestone Nov 4, 2025
Add dedicated precompilation and verification step after installing
Philote.jl from GitHub. This ensures the package is fully compiled
before tests run, preventing long compilation delays during test
execution that could cause timeouts.

The precompilation step:
- Runs Pkg.precompile() to compile all packages
- Verifies Philote.jl can be loaded with 'using Philote'
- Provides clear feedback if installation was successful

This should resolve test hanging issues where juliacall was compiling
the Philote package on first use during tests.
juliacall uses its own separate Julia environment (pyjuliapkg) that is
independent from the system Julia installation. This was causing tests
to hang for 7+ minutes as juliacall tried to install and compile
Philote.jl during the first test run.

This commit adds a dedicated step that initializes juliacall's Julia
environment and installs Philote.jl into it before tests run. This
ensures:
- Philote.jl is available in juliacall's environment
- All package compilation happens upfront, not during tests
- Tests can run immediately without waiting for package installation

This should resolve the test hanging issue completely.
Previous attempts were installing Philote.jl in the system Julia
environment, but juliacall uses its own separate pyjuliapkg environment.
This caused tests to hang as juliacall tried to install packages during
the first test run.

This commit:
- Removes separate system Julia installation steps
- Pre-warms juliacall's Julia environment by importing it from Python
- Installs Philote.jl directly into juliacall's environment
- Precompiles all packages before tests run
- Verifies Philote.jl loads correctly

This ensures all package installation and compilation happens in a
dedicated setup step with clear progress output, preventing test hangs.
Add timeout-minutes directives to prevent CI from hanging indefinitely:
- Pre-warm step: 5 minute timeout (package installation + compilation)
- Test execution: 3 minute timeout (tests should complete quickly)

Also added flush=True to all print statements in the pre-warm step to
ensure progress output is immediately visible in CI logs, making it
easier to diagnose where hangs occur.
Tests complete in ~2 seconds locally, so 30 seconds (15x buffer) is more
than sufficient. This will fail fast if Julia tests hang, making it
easier to debug the issue.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant