QDB-18913 - Add buildkite pipeline#120
Conversation
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
There was a problem hiding this comment.
Pull request overview
Adds Buildkite CI support for qdb-api-python by introducing a dynamic pipeline generator, step templates, and a Docker-based Linux build environment (plus wiring in shared CI tooling via submodules).
Changes:
- Introduce a Buildkite dynamic pipeline generator (
.buildkite/pipeline.py) with platform/build-type/python-version expansion. - Add a Dockerfile and docker-compose config to run Linux builds/tests in a manylinux container.
- Wire in shared Buildkite tooling as a git submodule and add Buildkite-specific Python requirements.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| scripts/teamcity/10.build.sh | Removes original wheel after auditwheel repair to avoid keeping unrepaired artifacts. |
| docker/docker-compose.yml | Adds a compose service to build/run the manylinux-based build container. |
| docker/Dockerfile | Adds a manylinux-derived build image with a non-root builder user and Python version setup. |
| .gitmodules | Adds .buildkite/tools submodule for shared CI/pipeline tooling. |
| .buildkite/steps/_build.yml | Adds a Buildkite step template to start services, run tests, stop services, and build artifacts. |
| .buildkite/requirements.txt | Adds Python dependencies needed to generate/validate the dynamic pipeline. |
| .buildkite/pipeline.py | Generates grouped Buildkite steps across OS/build-type/Python version and applies docker-compose on Linux. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| RUN useradd --comment "$COMMENT" --home-dir $HOME --create-home --system --uid $USER_ID --gid $GROUP_ID $USERNAME | ||
|
|
||
| USER builder | ||
|
|
There was a problem hiding this comment.
set-python-version.sh only appends a PATH export to ~/.bashrc and sets PATH within its own RUN layer; that doesn’t reliably affect non-interactive bash scripts/... commands later when the container is used. If the intent is to make the selected /opt/python the default, set it via ENV PATH=... (or set PYTHON_CMD/PYTHON_EXECUTABLE explicitly) so the version is active for subsequent commands.
| ENV BASH_ENV=/home/builder/.bashrc |
Co-authored-by: Copilot <copilot@github.com>
solatis
left a comment
There was a problem hiding this comment.
Pre-approved, but could we already use ~/scripts/cicd/ instead of ~/scripts/teamcity/? Maybe just symlink or something like that, so that we can write the buildkite templates consistently.
Not required, but nice to have.
Story details: https://app.shortcut.com/quasardb/story/18913