python2-to-3: port python to python3; auto-install a python3 gitdm for git-stats#1125
Closed
bhouse-nexthop wants to merge 3 commits into
Closed
Conversation
ONIE still shipped python2-only code throughout the tree. Port every
python asset to python3 so the build no longer needs a python2
interpreter (a prerequisite for moving to a modern Debian build host)
and so nothing in contrib/ or the machine tests is left on a dead
runtime. This change deliberately does NOT touch the CI Dockerfile, so
the cross-toolchain cache key is preserved (the Debian-13 build-host
bump is deferred to the toolchain-upgrade step, where it must land
alongside crosstool-NG/GCC -- the current GCC 8.3.0 toolchain cannot be
built on a GCC 14 host).
Build path:
- build-config/scripts/mk-part-table.py (the only python invoked by
the image build -- it writes the recovery ISO MBR partition table):
env python3 shebang, integer division (/ -> //) so CHS values stay
ints, and the ISO opened in binary mode ('r+b') so struct.pack()
bytes are written correctly. A shebang-only change is insufficient;
without the // and binary fixes the script silently writes a wrong
partition table under python3. Verified to emit a correct MBR
(0xaa55 signature, 0x80 boot flag, 0xEF EFI type) on a test image.
- build-config/Makefile: retarget $(PYTHON) from python2 to python3.
grub's autogen.sh invokes "${PYTHON:-python}", and a bare "python"
no longer exists once python2 is gone, so the build must export
PYTHON=python3 (grub's gentpl.py/import_gcry.py are python3-safe).
The old block only set PYTHON=python2 when "python" was absent;
replace it with an exported `PYTHON ?= python3`. Also switch
DEBIAN_BUILD_HOST_PACKAGES from python-all-dev/python-sphinx to
python3-all-dev/python3-sphinx.
Tooling and tests (mechanical 2to3, behaviour preserved):
- contrib/oce, contrib/onie-server, contrib/git-stats/gitdm (vendored),
and the machine/*/test DUT scripts: python3 shebangs; print()
functions; `except E, e` -> `except E as e`; iteritems/iterkeys/
itervalues -> items/keys/values; has_key -> `in`; raw_input -> input;
cStringIO/StringIO -> io; py2 stdlib renames (Queue->queue,
SimpleHTTPServer/SocketServer->http.server/socketserver, thread->
_thread); sort(cmp=...) -> key=cmp_to_key(...); list() wrapping of
views/maps that are indexed or sorted; onie-server.py spawns its
helpers with "python3" instead of "python".
- contrib/onie-server/proxy-server.py was cp1252-encoded; re-encode it
as UTF-8 and update the coding declaration so its (Portuguese)
comments survive the python3 port intact.
Every python file in the tree now passes `python3 -m py_compile`, all
shebang scripts use `#!/usr/bin/env python3`, and no python2-only syntax
remains.
Signed-off-by: Brad House <bhouse@nexthop.ai>
…n3 gitdm
The git-stats tooling vendored a full copy of gitdm (git.lwn.net) under
contrib/git-stats/gitdm. That copy is python2 and effectively
unmaintained, so it does not survive the python2-to-3 port.
Rather than carry (and port/maintain) a copy of gitdm in this tree,
remove the vendored gitdm and have onie-git-stats obtain a python3 port
of gitdm on demand:
- On first run, onie-git-stats clones a pinned python3 gitdm
(OSSystems/gitdm, which is a straight python3 port of Corbet's
gitdm) into a local, git-ignored cache directory
(contrib/git-stats/.gitdm) and reuses it thereafter. The source and
revision are pinned and overridable via GITDM_REPO / GITDM_REF.
- A user who already has a python3-capable gitdm can set GITDM to point
at it; onie-git-stats then uses it directly and skips the clone
entirely.
- gitdm is now invoked with python3 (was python2), and the pinned
upstream's harmless SyntaxWarnings are silenced.
The ONIE-specific gitdm configuration (gitdm-config/) and the
onie-git-stats wrapper are kept; only the vendored gitdm engine is
removed. README.md documents the auto-install, the pinning/override
knobs, and how to use a pre-existing gitdm.
Signed-off-by: Brad House <bhouse@nexthop.ai>
Add a path-scoped GitHub Action that exercises onie-git-stats end-to-end
so the git-stats change can be trusted without manual testing:
- Auto-install path: run onie-git-stats with no $GITDM set, asserting it
clones the pinned python3 gitdm and produces a non-empty report.
- Override path: run again with $GITDM pointing at a pre-existing gitdm,
asserting the auto-install cache is NOT re-created and a report is
still produced.
Triggers only when contrib/git-stats (or this workflow) changes, since
nothing else affects the result.
Signed-off-by: Brad House <bhouse@nexthop.ai>
Author
|
Added a small, path-scoped CI check ( |
This was referenced Jul 9, 2026
bhouse-nexthop
added a commit
to bhouse-nexthop/onie
that referenced
this pull request
Jul 9, 2026
Squashed baseline of PR opencomputeproject#1125, on which this PR is stacked. Drops out once PR opencomputeproject#1125 merges into onie-modernization-2026. Signed-off-by: Brad House <bhouse@nexthop.ai>
bhouse-nexthop
added a commit
to bhouse-nexthop/onie
that referenced
this pull request
Jul 9, 2026
…3 gitdm) Squashed content of PR opencomputeproject#1125, pulled in as a dependency for the toolchain milestone. Signed-off-by: Brad House <bhouse@nexthop.ai>
bhouse-nexthop
added a commit
to bhouse-nexthop/onie
that referenced
this pull request
Jul 10, 2026
…3 gitdm) Squashed content of PR opencomputeproject#1125, pulled in as a dependency for the toolchain milestone. Signed-off-by: Brad House <bhouse@nexthop.ai>
Collaborator
|
Thank you, pushed to onie-modernization-2026 branch. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebased onto
onie-modernization-2026(replaces #1116, which targetedmasterand stacked on the SBOM work; the SBOM content is not included here).What this does
1. Port all python from python2 to python3
python2-to-3: port all python from python2 to python3— ports the remaining python2 code (build-config Makefile/scripts,contrib/oce,contrib/onie-server, machine DUT test helpers) to python3.$(PYTHON)now defaults topython3. All ported files compile under python3.2. git-stats: drop the vendored python2 gitdm, auto-install a python3 gitdm
The
contrib/git-statstooling vendored a full copy of gitdm (fromgit.lwn.net), which is python2 and effectively unmaintained. Per maintainer request, rather than port/maintain a copy in-tree:contrib/git-stats/gitdm/engine.onie-git-statswrapper, thegitdm-config/maps, and the README.onie-git-statsnow auto-installs a python3 gitdm on demand: on first run it clones a pinned python3 port (OSSystems/gitdm, a straight python3 port of Corbet's gitdm) into a git-ignored cache (contrib/git-stats/.gitdm) and reuses it thereafter. Soonie-git-statsruns as-is with onlygit+python3present.GITDM_REPO/GITDM_REF.GITDMto point at it; the wrapper then uses it directly and skips the clone entirely.python3(waspython), and the pinned upstream's harmlessSyntaxWarnings are silenced.README.mddocuments the auto-install, the pinning/override knobs, and the pre-existing-gitdm path.Testing
onie-git-stats <range> <prefix>exercised end-to-end: fresh auto-clone → python3 gitdm → validresults.txt/.html.GITDMoverride skips the clone.python3 -m py_compilepasses on all ported.pyfiles.