Skip to content

QDB-18913 - Add buildkite pipeline#120

Merged
igorniebylski merged 59 commits intomasterfrom
sc-18913/buildkite-add-python-api-pipeline
May 4, 2026
Merged

QDB-18913 - Add buildkite pipeline#120
igorniebylski merged 59 commits intomasterfrom
sc-18913/buildkite-add-python-api-pipeline

Conversation

@igorniebylski
Copy link
Copy Markdown
Contributor

igorniebylski and others added 30 commits April 23, 2026 10:39
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
igorniebylski and others added 8 commits April 27, 2026 11:22
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Copilot AI review requested due to automatic review settings April 27, 2026 10:28
@igorniebylski igorniebylski requested a review from solatis April 27, 2026 10:28
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds Buildkite CI support for qdb-api-python by introducing a dynamic pipeline generator, step templates, and a Docker-based Linux build environment (plus wiring in shared CI tooling via submodules).

Changes:

  • Introduce a Buildkite dynamic pipeline generator (.buildkite/pipeline.py) with platform/build-type/python-version expansion.
  • Add a Dockerfile and docker-compose config to run Linux builds/tests in a manylinux container.
  • Wire in shared Buildkite tooling as a git submodule and add Buildkite-specific Python requirements.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
scripts/teamcity/10.build.sh Removes original wheel after auditwheel repair to avoid keeping unrepaired artifacts.
docker/docker-compose.yml Adds a compose service to build/run the manylinux-based build container.
docker/Dockerfile Adds a manylinux-derived build image with a non-root builder user and Python version setup.
.gitmodules Adds .buildkite/tools submodule for shared CI/pipeline tooling.
.buildkite/steps/_build.yml Adds a Buildkite step template to start services, run tests, stop services, and build artifacts.
.buildkite/requirements.txt Adds Python dependencies needed to generate/validate the dynamic pipeline.
.buildkite/pipeline.py Generates grouped Buildkite steps across OS/build-type/Python version and applies docker-compose on Linux.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread .buildkite/pipeline.py
Comment thread .buildkite/steps/_build.yml Outdated
Comment thread docker/Dockerfile Outdated
Comment thread docker/Dockerfile
RUN useradd --comment "$COMMENT" --home-dir $HOME --create-home --system --uid $USER_ID --gid $GROUP_ID $USERNAME

USER builder

Copy link

Copilot AI Apr 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

set-python-version.sh only appends a PATH export to ~/.bashrc and sets PATH within its own RUN layer; that doesn’t reliably affect non-interactive bash scripts/... commands later when the container is used. If the intent is to make the selected /opt/python the default, set it via ENV PATH=... (or set PYTHON_CMD/PYTHON_EXECUTABLE explicitly) so the version is active for subsequent commands.

Suggested change
ENV BASH_ENV=/home/builder/.bashrc

Copilot uses AI. Check for mistakes.
Copy link
Copy Markdown
Contributor

@solatis solatis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-approved, but could we already use ~/scripts/cicd/ instead of ~/scripts/teamcity/? Maybe just symlink or something like that, so that we can write the buildkite templates consistently.

Not required, but nice to have.

@igorniebylski igorniebylski merged commit defc910 into master May 4, 2026
2 checks passed
@igorniebylski igorniebylski deleted the sc-18913/buildkite-add-python-api-pipeline branch May 4, 2026 07:04
igorniebylski added a commit that referenced this pull request May 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants