Unvendor `libmpdec` sources

[zware]

To facilitate cleaner updates of the externally-maintained libmpdec required by the _decimal module, we should migrate away from the bundled copy in Modules/_decimal/libmpdec and towards an "external" in cpython-source-deps for Windows and --with-system-libmpdec by default elsewhere.

My tentative plan is as follows:

• In 3.13:
  • Add mpdecimal-2.5.1 and mpdecimal-4.0.0 to cpython-source-deps:
    • Sort list of downloads, and add link to mpdecimal sources cpython-source-deps#32
    • https://github.com/python/cpython-source-deps/tree/mpdecimal
  • Switch Windows build to mpdecimal-2.5.1 external (gh-115119: Switch Windows build to mpdecimal external #115182)
  • gh-115119: Detect _decimal dependencies using pkg-config #115406
  • Update deps and build instructions wrt. libmpdecimal devguide#1313
  • gh-115119: Update macOS installer to build with libmpdec 4.0.0 #118382
  • gh-115119: Update Windows installer to mpdecimal 4.0.0 #118506
  • Update macOS build to mpdecimal-4.0.0
  • gh-115119: Default to --with-system-libmpdec=yes #118539
  • gh-115119: Bump CI to use Ubuntu 22.04 #118631
  • Post #118539: amend Ubuntu CI to use system installed mpdecimal
  • Update the docs; see Unvendor libmpdec sources #115119 (comment)
    • gh-115119: recommend upstream libmpdec in build requirements #136205
    • Update libmpdec-related instructions devguide#1598
    • gh-115119: Defer --with-system-libmpdec removal to 3.16 #139318
  • gh-115119: Fall back to bundled libmpdec if system libmpdec is not found #119196
• In 3.15:
  • Defer removal of bundled libmpdec in 3.15: gh-115119: defer removal of bundled libmpdec to 3.16 #133997
  • Remove the implicit fallback to the bundled copy, keep support for explicit --without-system-libmpdec (with warning): gh-115119: removed implicit fallback to the bundled libmpdec #134078
• In 3.16:
  • Remove --with[out]-system-libmpdec configure options and Modules/_decimal/libmpdec/: gh-115119: Removed bundled copy of the libmpdec #133964

Linked PRs

• gh-115119: Switch Windows build to mpdecimal external #115182
• gh-115119: Detect _decimal dependencies using pkg-config #115406
• gh-115119: Update macOS installer to build with libmpdec 4.0.0 #118382
• gh-115119: Update Windows installer to mpdecimal 4.0.0 #118506
• gh-115119: Default to --with-system-libmpdec=yes #118539
• gh-115119: Bump CI to use Ubuntu 22.04 #118631
• [3.12] gh-115119: Bump CI to use Ubuntu 22.04 (#118631) #118634
• gh-115119: Fall back to bundled libmpdec if system libmpdec is not found #119196
• [3.13] gh-115119: Fall back to bundled libmpdec if system libmpdec is not found (GH-119196) #119217
• gh-115119: Removed bundled copy of the libmpdec #133964
• gh-115119: defer removal of bundled libmpdec to 3.16 #133997
• gh-115119: removed implicit fallback to the bundled libmpdec #134078
• gh-115119: removed implicit fallback to the bundled libmpdec #135568
• [3.14] gh-115119: Defer removal of bundled libmpdec to 3.16 (GH-133997) #136199
• [3.13] gh-115119: Defer removal of bundled libmpdec to 3.16 (GH-133997) #136201
• gh-115119: recommend upstream libmpdec in build requirements #136205
• gh-115119: Document libmpdec dependency in the build doc #136239
• [3.14] gh-115119: Recommend upstream libmpdec in build requirements (GH-136205) #136244
• [3.13] gh-115119: Recommend upstream libmpdec in build requirements (#136205) #136245
• gh-115119: Defer --with-system-libmpdec removal to 3.16 #139318
• [3.14] gh-115119: Defer --with-system-libmpdec removal to 3.16 (GH-139318) #139347
• [3.13] gh-115119: Defer --with-system-libmpdec removal to 3.16 (GH-139318) #139348
• gh-115119: Update macOS installer to use libmpdecimal 4.0.1. #140077

[erlend-aasland]

[...] with a fallback to the bundled copy (maybe with a warning?)

+1 for an autoconf warning.

[ned-deily]

Adding an external libmpdec to the macOS installer build shouldn't be a big issue. Assign it to me if you proceed with this.

[erlend-aasland]

FTR: the source deps repo is now updated.

[erlend-aasland]

@ned-deily: for build-installer.py, is there more to do than adding a configuration for mpdecimal? I've used the following configure options: ['--disable-cxx', 'MACHINE=universal']

[ned-deily]

Likely yes, I'll take care of it. Thanks.

[ned-deily]

I took a quick look at this and be aware that this change proposal will likely impact everyone building Python 3.13 on macOS. macOS does not ship a copy of libmpdec so, if the bundled version is removed, anyone building Python on macOS will now need to provide a local copy, either building from source or using a version from one of the widely-used third-party package managers, like Homebrew or MacPorts. Both currently have mpdecimal packages at the 4.0.0 level. (Recall that, unlike Windows builds, we do not currently provide non-vendored third-party library binaries for macOS users, other than those built for and linked into the Pythons provided by python.org installers for macOS.) In a quick test, it appeared that with ./configure --with-system-libmpdec today, pkg-config info for libmpdec was not being used. If so, that should be corrected. And, in any case, the devguide instructions for installing dependencies will need to be updated to cover libmpdec. We should add all this and get this working before moving on to changing the installer.

[erlend-aasland]

In a quick test, it appeared that with ./configure --with-system-libmpdec today, pkg-config info for libmpdec was not being used. If so, that should be corrected.

Last time I checked, libmpdec did not provide pkgconfig files:

$ ls $(brew --prefix libmpdec)/lib/pkgconfig
ls: /opt/homebrew/opt/mpdecimal/lib/pkgconfig: No such file or directory
$ cd mpdecimal-2.5.1; find . | grep pkg
$

[ned-deily]

With the current 4.0.0 version, it appears both Homebrew and MacPorts ship them:

$ pkg-config libmpdec --cflags
-I/opt/homebrew/Cellar/mpdecimal/4.0.0/include
$ pkg-config libmpdec --libs
-L/opt/homebrew/Cellar/mpdecimal/4.0.0/lib -lmpdec -lm

...

$ pkg-config libmpdec --libs
-L/opt/macports/lib -lmpdec -lm