Dependabot & Devendoring

[Symmetricity]

Hi all,

I've been working on general improvements for a couple of weeks now, and one big one that I thought warranted a discussion was de-vendoring libaries so that the repo can use upstream versions - and using dependabot to keep those versions up to date.

I've prepared some details here for your consideration:

Dependabot support for vendored C/C++ dependencies

Checked on 2026-06-06.

Context

Tilemaker vendors several third-party C/C++ libraries directly in the tree:

• include/external/
• include/protozero/
• include/vtzero/
• server/Simple-Web-Server/

Dependabot can update supported manifest ecosystems such as vcpkg, GitHub Actions, Docker, git submodules, and several language package managers. It does not update arbitrary copied C/C++ headers or source files. Any library we keep as plain vendored source will remain a manual update unless we move it into a supported manifest, use a git submodule, or create/package a vcpkg port.

Version comparison

Library	Vendored version or state	Upstream/current version	Dependabot path	Likely benefit
libdeflate	1.22 in include/external/libdeflate/libdeflate.h	1.25	Existing vcpkg port	Build fixes for newer GCC/Clang, checksum bug fixes from 1.23, CRC optimizations, better CMake package behavior.
protozero	1.7.1 in include/protozero/version.hpp	1.8.1	Existing vcpkg port	Fixes a get_bool() buffer overrun, modern C++14 cleanup, STL/toolchain compatibility.
libpopcnt	Exact match to upstream v2.5	3.1	Existing vcpkg port	Better AVX512 and ARM SVE paths, sanitizer-safe unaligned access handling, macro namespacing to avoid collisions.
streamvbyte	Old/custom snapshot, header roughly v0.5.3/v1.0.0 era	3.0.0	No current vcpkg port found	Adds stream validation APIs for disk/network data, ARM/SSE detection fixes, x86 decode changes. Potentially valuable for robust compact stores.
PMTiles C++ header	Unversioned vendored include/external/pmtiles.hpp	Current PMTiles main header	No clean release/port for the C++ header	Adds TILETYPE_MLT, comparator helper, and safer/faster tile ID conversion with overflow checks. Needs output determinism testing.
vtzero	1.1.0 in include/vtzero/version.hpp	1.2.0	No current vcpkg port found	Mostly C++14/build hygiene and stats-tool changes. Useful if paired with protozero 1.8.x, but not urgent by itself.
sqlite_modern_cpp	Patched local copy; include/mbtiles.h notes blob support and .init changes	Upstream 3.2; vcpkg date 2023-12-03	Existing vcpkg port	Better exceptions, extended result codes, integer-type support, statement fixes. De-vendoring requires porting local changes first.
Simple-Web-Server	Close to v3.1.1, but with local signal-handling changes	3.1.1	No vcpkg port found	Upstream has URL parsing and Boost.Asio compile fixes. Replacing it requires deciding whether to preserve local signal behavior.
kaguya	Patched amalgamated local copy	1.3.2 from 2017	No current vcpkg port found	Little upgrade value. Local Lua 5.4 compatibility matters more than upstream. Longer term, consider replacing it with sol2, which is available in vcpkg.
polylabel	Adapted local Boost.Geometry copy	2.0.1	Existing mapbox-polylabel vcpkg port	Upstream 2.0.1 is only a small bundle change. Local adaptation makes this low value to de-vendor.
visvalingam.cpp	Adapted algorithmic code from other projects	No clean package version	None	Not a normal dependency bump candidate.
minunit	Tiny test helper	No meaningful release stream	None	Not worth moving for Dependabot.

Recommendations

1. Move clear wins to vcpkg first

These are the best candidates because vcpkg already has ports and Dependabot can already monitor vcpkg.json:

• libdeflate
• protozero
• libpopcnt

This removes substantial vendored code, gives Dependabot a supported manifest to update, and captures real upstream fixes. protozero is especially relevant because tilemaker depends heavily on robust PBF parsing.

2. Treat streamvbyte as the next important dependency project

streamvbyte looks worth upgrading because newer upstream provides validation APIs and CPU-detection fixes. The blocker is packaging: Dependabot cannot bump the current copied source tree directly, and I did not find an existing vcpkg port.

Options:

• Add/request a vcpkg port for streamvbyte, then consume it through vcpkg.json.
• Use a git submodule and enable Dependabot's gitsubmodule ecosystem.
• Keep vendoring it, but document a manual refresh procedure and accept that Dependabot will not manage it.

The vcpkg-port route is the cleanest if we want regular Dependabot PRs.

3. Pair vtzero with a protozero migration

vtzero 1.2.0 is not a high-impact update by itself, but it follows protozero's C++14 direction. If we move protozero to vcpkg, it is worth either:

• creating/requesting a vtzero vcpkg port, or
• keeping vtzero vendored but refreshing it manually once after the protozero update.

4. Handle patched libraries separately

Some vendored libraries are not drop-in dependency swaps:

• sqlite_modern_cpp has local changes for blob support and .init.
• Simple-Web-Server has local signal-handling behavior.
• kaguya has local Lua compatibility changes and appears effectively stale upstream.
• polylabel has local Boost.Geometry adaptation and precision changes.

For these, the right first step is not simply "switch to dependency"; it is to isolate the local changes, add tests around the behavior we rely on, and then decide whether a dependency swap still makes sense.

5. Consider a bigger Lua binding migration

kaguya does not look like a good long-term dependency target. Upstream's latest release is from 2017, while tilemaker already carries local compatibility work. If we want a maintainable package-manager-backed Lua binding, sol2 is available in vcpkg (3.5.0 at the time of this check). That would be a larger migration, not a vendored-header refresh.

6. Expand Dependabot beyond vcpkg

Current Dependabot coverage can be improved independently of vendored C/C++ work:

• Keep github-actions updates enabled.
• Keep vcpkg updates enabled.
• Add docker updates if we want the root Dockerfile base image monitored.
• Add gitsubmodule only if we intentionally convert some vendored libraries to submodules.

CDN script URLs in server/static/index.html and manually managed workflow variables such as PMTILES_VERSION will not be updated by Dependabot without changing how they are represented.

Proposed path

1. Move libdeflate, protozero, and libpopcnt from vendored source to vcpkg dependencies.
2. Add focused build and runtime tests around PBF parsing, tile writing, and compression paths before merging those dependency changes.
3. Investigate a vcpkg port for streamvbyte; if not practical, decide between submodule or documented manual vendoring.
4. Refresh PMTiles header in a targeted branch and compare generated PMTiles output for determinism and compatibility.
5. Defer sqlite_modern_cpp, Simple-Web-Server, polylabel, and kaguya until their local patches/adaptations are explicitly isolated.
6. Consider a separate long-term proposal to replace kaguya with sol2.

Sources

• libdeflate 1.25 release notes: https://github.com/ebiggers/libdeflate/blob/v1.25/NEWS.md
• protozero 1.8.1 changelog: https://github.com/mapbox/protozero/blob/v1.8.1/CHANGELOG.md
• vtzero 1.2.0 changelog: https://github.com/mapbox/vtzero/blob/v1.2.0/CHANGELOG.md
• libpopcnt changelog: https://github.com/kimwalisch/libpopcnt/blob/master/ChangeLog
• streamvbyte 3.0.0 release: https://github.com/fast-pack/streamvbyte/releases/tag/v3.0.0
• sqlite_modern_cpp 3.2 release: https://github.com/SqliteModernCpp/sqlite_modern_cpp/releases/tag/v3.2
• Simple-Web-Server 3.1.1 release: https://gitlab.com/eidheim/Simple-Web-Server/-/releases/v3.1.1
• polylabel 2.0.1 release: https://github.com/mapbox/polylabel/releases/tag/v2.0.1
• PMTiles C++ header: https://github.com/protomaps/PMTiles/blob/main/cpp/pmtiles.hpp
• sol2 vcpkg port: https://github.com/microsoft/vcpkg/blob/master/ports/sol2/vcpkg.json