Add support for Python and improve zopen-generate by IgorTodorovskiIBM · Pull Request #1157 · zopencommunity/meta

IgorTodorovskiIBM · 2026-04-09T02:52:28Z

What type of PR is this? (check all applicable)

Description

Fixed zopen-generate so --build-system is no longer ignored.
Improved Python port generation, including support for C extensions, better build/check/install templates, and pytest-based test handling.
Added native Python build-system support directly in zopen-build, which simplifies the generated Python templates and centralizes the logic.
Added implicit Python-related dependencies such as check_python and grep where needed.
Improved Python test reliability by forcing reinstall of the freshly built wheel before running checks.
Added support in zopen-publish for publishing Python wheels to a Pulp PyPI repository, alongside existing GitHub publishing.
Improved stderr handling for Gemini/Bob shell environments by redirecting messages to stdout when needed.
Cleaned up zopen-build logging/output to use the common print helpers more consistently.
Fixed several zopen-generate logic issues, including setup checks, --list-* flag behavior with --json, and case-insensitive handling of --build-line.
Improved stable tag detection so only proper stable Git tags are selected, excluding prerelease-style tags.
Clarified help text for source URLs to better describe supported Git and archive formats.

Related Issues

Related Issue #
Closes #

[optional] Are there any post-deployment tasks or follow-up actions required?

The build_system variable was being reset to "" after argument parsing, causing the --build-system CLI option to always be overwritten.

- Add --c-extensions flag and interactive prompt for Python ports with C extensions (omits ZOPEN_COMP=skip so compiler is available) - Add ZOPEN_MAKE_MINIMAL=yes and ZOPEN_CHECK_MINIMAL=yes so compiler flags stay in env instead of being passed as make arguments - Improve build function: install setuptools/wheel/installer, rm stale dist - Improve install function: create dirs, symlink scripts into bin/ - Add pytest-based check function and pytest-aware zopen_check_results that parses passed/failed/error counts from pytest output - Fix zopen_append_to_env quoting (was expanding at generation time)

stderr output is garbled when running under GEMINI_CLI or BOBSHELL_CLI. Add ZOPEN_STDERR_TO_STDOUT flag set once at init, and use it in printError, printSoftError, printWarning, and other stderr messages to redirect output to stdout instead. Also fix redundant >&2 on printError calls that already handle stderr.

Replace raw echo/head >&2 calls with proper print functions: - printSoftError for error context - printWarning for diagnostic output - printInfo for informational messages Remove redundant >&2 on printError calls that already handle stderr.

- Add zopen_init() to merge LIBS into LDFLAGS for C extension projects (Python's build system reads LDFLAGS but ignores LIBS) - Install built wheel before pytest for C extensions so .so files are available during tests - Split custom_functions into separate C extensions and pure Python variants

zopen_check_results uses grep -oE to parse pytest output, which is not supported by z/OS system grep. Auto-add the zopen GNU grep package.

Move Python build/check/install logic from generated buildenv templates into zopen-build itself. When ZOPEN_BUILD_SYSTEM=Python is set: - Built-in functions handle venv creation, wheel building, pytest, pip install to target dir, bin symlinking, and wheel preservation - Built-in zopen_check_results parses pytest output - Built-in zopen_append_to_env sets up PYTHONPATH - Built-in zopen_init merges LIBS into LDFLAGS for C extension ports - check_python and grep added as implicit dependencies - All defaults use := so user overrides from buildenv are preserved Simplify zopen-generate Python template from 50+ lines of custom functions down to just ZOPEN_BUILD_SYSTEM="Python" (and optionally ZOPEN_COMP="skip" for pure Python ports).

New options for publishing Python wheels to a Pulp PyPI repository: --whl FILE Path to wheel file --pulp-url URL Pulp PyPI repository URL --pulp-user USER Pulp username (or PULP_USER env var) --pulp-password PASS Pulp password (or PULP_PASSWORD env var) Uploads via curl POST to the Pulp legacy endpoint. Polls async task status to confirm success/failure. GitHub and Pulp publishing can be used independently or together in a single invocation.

Ensures the freshly built wheel is used during tests, not a previously cached version from an earlier build.

- Fix setupMyself check (&&->||) so missing common.sh is caught - Fix --build-line to accept DEV/dev/Dev case-insensitively - Fix --list-* flags to respect --json regardless of argument order - Filter pre-release tags (rc, dev, extra suffixes) when detecting the latest stable release tag from a Git repo - Clarify --stable-url and --dev-url help text to list supported URL formats (Git HTTPS .git clone URLs and archive formats)

augmentcode · 2026-04-21T13:18:25Z

🤖 Augment PR Summary

Summary: This PR adds first-class Python support across the zopen toolchain and fixes several long-standing zopen-generate issues.

Changes:

Introduces a native Python build system in zopen-build (build/check/install/env + check results parsing).
Updates zopen-generate to produce improved Python port templates, including optional C-extension handling and pytest-oriented checks.
Fixes zopen-generate argument parsing so --build-system isn’t ignored and --list-* works correctly with --json.
Improves stable tag auto-detection by filtering to proper stable semver-style tags (excluding prerelease-like tags).
Adds implicit Python-related dependencies (e.g., check_python, grep) when needed.
Enhances wheel test reliability by force-reinstalling the freshly built wheel before running pytest.
Extends zopen-publish to optionally publish Python wheels to a Pulp PyPI repository in addition to GitHub releases.
Improves output reliability in Gemini/Bob shells by redirecting stderr output to stdout when required.
Aligns several zopen-build messages to use shared print helpers more consistently.

Technical Notes: The Python build system centralizes logic in zopen-build, simplifying generated templates and enabling consistent behavior across ports.

_{🤖 Was this summary useful? React with 👍 or 👎}

augmentcode

Review completed. 3 suggestions posted.

Comment augment review to trigger a new review at any time.

augmentcode · 2026-04-21T13:18:26Z

+}
+
+_zopen_python_check() {
+  . .venv/bin/activate


_zopen_python_check sources .venv/bin/activate directly, so running check without a prior successful _zopen_python_build may fail if the venv was never created. Consider ensuring the venv exists before activation (same concern applies to _zopen_python_install).

Severity: medium

Other Locations

bin/zopen-build:349

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

augmentcode · 2026-04-21T13:18:26Z

 export ZOPEN_CONFIGURE_OPTS="-B ../build -DCMAKE_INSTALL_PREFIX=\${ZOPEN_INSTALL_DIR}"
 export ZOPEN_MAKE="cmake"
-export ZOPEN_MAKE_OPTS="--build ../build"
+export ZOPEN_MAKE_OPTS="--build ../build --parallel \\\$ZOPEN_NUM_JOBS --config Release"


The generated CMake ZOPEN_MAKE_OPTS contains \$ZOPEN_NUM_JOBS, which will remain escaped even when runAndLog later evals the build command, so cmake --build may receive a literal $ZOPEN_NUM_JOBS for --parallel. That would likely break parallel builds for generated CMake ports.

Severity: high

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

augmentcode · 2026-04-21T13:18:26Z

-  if [ ! -f "$PAX_FILE" ]; then
-    printError "PAX file not found: $PAX_FILE"
-    exit 1
+  if [ -n "$WHL_FILE" ] || [ -n "$PULP_URL_OPT" ] || [ -n "$PULP_URL" ]; then


PUBLISH_PULP is enabled when $PULP_URL is set in the environment even if --whl wasn’t provided, which can unexpectedly force the script into Pulp validation and fail GitHub-only publishes. This seems likely to surprise users who have PULP_URL exported but only intend a GitHub release.

Severity: medium

_{🤖 Was this useful? React with 👍 or 👎, or 🚀 if it prevented an incident/outage.}

IgorTodorovskiIBM added 11 commits April 8, 2026 11:32

Fix --build-system flag being ignored in zopen-generate

c777ab7

The build_system variable was being reset to "" after argument parsing, causing the --build-system CLI option to always be overwritten.

Add grep as auto-dependency for Python ports

546044a

zopen_check_results uses grep -oE to parse pytest output, which is not supported by z/OS system grep. Auto-add the zopen GNU grep package.

Use --force-reinstall for wheel in Python check step

546febe

Ensures the freshly built wheel is used during tests, not a previously cached version from an earlier build.

Fix zopen-generate logic issues

64a07c7

IgorTodorovskiIBM marked this pull request as ready for review April 21, 2026 13:15

IgorTodorovskiIBM requested review from DevonianTeuchter, HarithaIBM, MikeFultonDev, sachintu47 and v1gnesh as code owners April 21, 2026 13:15

augmentcode Bot reviewed Apr 21, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Python and improve zopen-generate#1157

Add support for Python and improve zopen-generate#1157
IgorTodorovskiIBM wants to merge 11 commits intomainfrom
improve_generate

IgorTodorovskiIBM commented Apr 9, 2026 •

edited

Loading

Uh oh!

augmentcode Bot commented Apr 21, 2026

Uh oh!

augmentcode Bot left a comment

Uh oh!

augmentcode Bot Apr 21, 2026 •

edited

Loading

Uh oh!

augmentcode Bot Apr 21, 2026 •

edited

Loading

Uh oh!

augmentcode Bot Apr 21, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

IgorTodorovskiIBM commented Apr 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this? (check all applicable)

Category

Description

Related Issues

[optional] Are there any post-deployment tasks or follow-up actions required?

Uh oh!

augmentcode Bot commented Apr 21, 2026

Uh oh!

augmentcode Bot left a comment

Choose a reason for hiding this comment

Uh oh!

augmentcode Bot Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

augmentcode Bot Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

augmentcode Bot Apr 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

IgorTodorovskiIBM commented Apr 9, 2026 •

edited

Loading

augmentcode Bot Apr 21, 2026 •

edited

Loading

augmentcode Bot Apr 21, 2026 •

edited

Loading

augmentcode Bot Apr 21, 2026 •

edited

Loading