Skip to content

Integrating Polars DataFrame#674

Open
ahmedanadwc wants to merge 11 commits into
sassoftware:mainfrom
ahmedanadwc:main
Open

Integrating Polars DataFrame#674
ahmedanadwc wants to merge 11 commits into
sassoftware:mainfrom
ahmedanadwc:main

Conversation

@ahmedanadwc

Copy link
Copy Markdown

This fork includes a major feature set centered around Polars and Apache Arrow integration.

Feature Category

  • Full Polars Integration: Includes polars_stream.py, polars_types.py.
  • Expanded Testing: Includes test_polars.py, test_polars_unit.py, etc.
  • Extended Documentation: Contains local rst file updates detailing new functionality.
  • Enhanced Dependency: Includes polars in extras_require.

Summary of Enhancements

This is feature-rich branch that adds high-performance data processing capabilities to saspy.

  • Polars Support: The local fork enables high-performance, memory-efficient data streaming between SAS and Python using the Polars DataFrame library. This is a substantial performance improvement over the
    standard Pandas-based approach for large datasets.
  • Apache Arrow Integration: The implementation includes support for Arrow-based types, which is foundational for modern, cross-language data interchange and is likely utilized by the Polars integration.
  • Architectural Additions: The fork modifies core files like sasbase.py, sasdata.py, and the sasio (access method) modules to support these new data types, rather than just adding them as an external
    plugin.

Generated-by: OpenCode and Google Gemini

ahmedanadwc and others added 11 commits May 15, 2026 16:28
- Add 'Optional Dependencies' section to install.rst (pyarrow, polars)
- Add Polars DataFrame section to getting-started.rst with streaming examples
- Add 'Using Polars' and 'Using Apache Arrow' sections to advanced-topics.rst
- Add DataFrame library options overview to index.rst
- Add Polars feature entry to CHANGELOG.md (v5.105.0)

This addresses the documentation gap between code and docs for the
Polars integration (20+ code nodes) and Arrow features (32 nodes).
Allow users to install Polars support with: pip install saspy[polars]
Added Test scenarios to cover Polars dataframe support
added files to ignore
Added files to ignore
Updated the documentation to include Polars related contents
Add/Update code modules to support Polars Dataframe
Added/Modified code to enhance the Polars Dataframe support
Commented out lines 195-197 to match parent/origin SASPY repo changes
Minor char case change
@DMcGee-SAS

Copy link
Copy Markdown
Contributor

@ahmedanadwc, hope to review this pull request soon... We have seen it, but haven't had time to review it yet...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants