Update dependency lxml to v6 [SECURITY]#12424
Conversation
|
Thanks for this automated security update. 🤖 Copilot has been assigned for an initial review. A reviewer must first be assigned. There are currently 10 open PRs of equal or higher priority ahead of yours. PR triage checklist (maintainers / Pam)
Note This comment was automatically generated by Pam, Open Library's Project AI Manager, on behalf of @mekarpeles. Pam is designed to provide status visibility, perform basic project management functions and relevant codebase research, and provide actionable feedback so contributors aren't left waiting. |
renovate
Bot
changed the title
renovate
Bot
changed the title
renovate
Bot
force-pushed
the
renovate/pypi-lxml-vulnerability
branch
2 times, most recently
from
34dcab6 to
d2cb375
Compare
|
Consolidated into #12558 which upgrades all 6 Python security dependencies together, with Docker testing and full test suite verification (3524 tests passing). Closing in favor of that PR. |
Renovate Ignore NotificationBecause you closed this PR without merging, Renovate will ignore this update. You will not get PRs for any future If you accidentally closed this PR, or if you changed your mind: rename this PR to get a fresh replacement PR. |
…chive#12424) Fixes CVE-2026-41066 (XXE via iterparse/ETCompatXMLParser, CVSS 7.5). lxml 6.x changes the default resolve_entities from True to 'internal' for iterparse() and ETCompatXMLParser(). Impact analysis: - marc_xml.py uses iterparse() without explicit resolve_entities: behavior change is safe since LoC MARC21 slim XML has no external entities, and the stricter default is a security improvement. - All other callers (importapi, get_ia, scripts, tests) already pass resolve_entities=False explicitly — no change in behavior.
This PR contains the following updates:
==4.9.4→==6.1.0lxml: Default configuration of iterparse() and ETCompatXMLParser() allows XXE to local files
CVE-2026-41066 / GHSA-vfmq-68hx-4jfw
More information
Details
Impact
Using either of the two parsers in the default configuration (with
resolve_entities=True) allows untrusted XML input to read local files.Patches
lxml 6.1.0 changes the default to
resolve_entities='internal', thus disallowing local file access by default.Workarounds
Setting the
resolve_entitiesoption explicitly toresolve_entities='internal'orresolve_entities=Falsedisables the local file access.Resources
Original report: https://bugs.launchpad.net/lxml/+bug/2146291
The default option was changed to
resolve_entities='internal'for the normal XML and HTML parsers in lxml 5.0. The default was not changed foriterparse()andETCompatXMLParser()at the time. lxml 6.1 makes the safe option the default for all parsers.Severity
CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:NReferences
This data is provided by the GitHub Advisory Database (CC-BY 4.0).
Release Notes
lxml/lxml (lxml)
v6.1.0Compare Source
==================
This release fixes a possible external entity injection (XXE) vulnerability in
iterparse()and theETCompatXMLParser.Features added
GH#486: The HTML ARIA accessibility attributes were added to the set of safe attributes
in
lxml.html.defs. This allowslxml_html_cleanto pass them through.Patch by oomsveta.
The default chunk size for reading from file-likes in
iterparse()is now configurablewith a new
chunk_sizeargument.Bugs fixed
resolve_entitiesoption was still set toTrueforiterparseandETCompatXMLParser, allowing for external entity injection (XXE)when using these parsers without setting this option explicitly.
The default was now changed to
'internal'only (as for the normal XML and HTML parserssince lxml 5.0).
Issue found by Sihao Qiu as CVE-2026-41066.
v6.0.4Compare Source
==================
Bugs fixed
v6.0.3Compare Source
==================
Bugs fixed
Several out of memory error cases now raise
MemoryErrorthat were not handled before.Slicing with large step values (outside of
+/- sys.maxsize) could trigger undefined C behaviour.LP#2125399: Some failing tests were fixed or disabled in PyPy.
LP#2138421: Memory leak in error cases when setting the
public_idorsystem_urlof a document.Memory leak in case of a memory allocation failure when copying document subtrees.
When mapping an XPath result to Python failed, the result memory could leak.
When preparing an XSLT transform failed, the XSLT parameter memory could leak.
Other changes
Built using Cython 3.2.4.
Binary wheels use zlib 1.3.2.
v6.0.2Compare Source
==================
Bugs fixed
LP#2125278: Compilation with libxml2 2.15.0 failed.
Original patch by Xi Ruoyao.
Setting
decompress=Truein the parser had no effect in libxml2 2.15.Binary wheels on Linux and macOS use the library version libxml2 2.14.6.
See https://gitlab.gnome.org/GNOME/libxml2/-/releases/v2.14.6
Test failures in libxml2 2.15.0 were fixed.
Other changes
Binary wheels for Py3.9-3.11 on the
riscv64architecture were added.Error constants were updated to match libxml2 2.15.0.
Built using Cython 3.1.4.
v6.0.1Compare Source
==================
Bugs fixed
LP#2116333:
lxml.sax._getNsTag()could fail with an exception on malformed input.GH#467: Some test adaptations were made for libxml2 2.15.
Patch by Nick Wellnhofer.
LP2119510, GH#473: A Python compatibility test was fixed for Python 3.14+.
Patch by Lumír Balhar.
GH#471: Wheels for "riscv64" on recent Python versions were added.
Patch by ffgan.
GH#469: The wheel build no longer requires the
wheelpackage unconditionally.Patch by Miro Hrončok.
Binary wheels use the library version libxml2 2.14.5.
See https://gitlab.gnome.org/GNOME/libxml2/-/releases/v2.14.5
Windows binary wheels continue to use a security patched library version libxml2 2.11.9.
v6.0.0Compare Source
==================
Features added
GH#463:
lxml.html.diffis faster and provides structurally better diffs.Original patch by Steven Fernandez.
GH#405: The factories
ElementandElementTreecan now be used in type hints.GH#448: Parsing from
memoryviewand other buffers is supported to allow zero-copy parsing.GH#437:
lxml.html.builderwas missing several HTML5 tag names.Patch by Nick Tarleton.
GH#458:
CDATAcan now be written into the incrementalxmlfile()writer.Original patch by Lane Shaw.
A new parser option
decompress=Falsewas added that controls the automaticinput decompression when using libxml2 2.15.0 or later. Disabling this option
by default will effectively prevent decompression bombs when handling untrusted
input. Code that depends on automatic decompression must enable this option.
Note that libxml2 2.15.0 was not released yet, so this option currently has no
effect but can already be used.
The set of compile time / runtime supported libxml2 feature names is available as
etree.LIBXML_COMPILED_FEATURESandetree.LIBXML_FEATURES.This currently includes
catalog,ftp,html,http,iconv,icu,lzma,regexp,schematron,xmlschema,xpath,zlib.Bugs fixed
GH#353: Predicates in
.find*()could mishandle tag indices if a default namespace is provided.Original patch by Luise K.
GH#272: The
headandbodyproperties oflxml.htmlelements failed if no such elementwas found. They now return
Noneinstead.Original patch by FVolral.
Tag names provided by code (API, not data) that are longer than
INT_MAXcould be truncated or mishandled in other ways.
.text_content()onlxml.htmlelements accidentally returned a "smart string"without additional information. It now returns a plain string.
LP#2109931: When building lxml with coverage reporting, it now disables the
sys.monitoringsupport due to the lack of support in nedbat/coveragepy#1790
Other changes
Support for Python < 3.8 was removed.
Parsing directly from zlib (or lzma) compressed data is now considered an optional
feature in lxml. It may get removed from libxml2 at some point for security reasons
(compression bombs) and is therefore no longer guaranteed to be available in lxml.
As of this release, zlib support is still normally available in the binary wheels
but may get disabled or removed in later (x.y.0) releases. To test the availability,
use
"zlib" in etree.LIBXML_FEATURES.The
Schematronclass is deprecated and will become non-functional in a future lxml version.The feature will soon be removed from libxml2 and stop being available.
GH#438: Wheels include the
arm7ltarget.GH#465: Windows wheels include the
arm64target.Patch by Finn Womack.
Binary wheels use the library versions libxml2 2.14.4 and libxslt 1.1.43.
Note that this disables direct HTTP and FTP support for parsing from URLs.
Use Python URL request tools instead (which usually also support HTTPS).
To test the availability, use
"http" in etree.LIBXML_FEATURES.Windows binary wheels use the library versions libxml2 2.11.9, libxslt 1.1.39 and libiconv 1.17.
They are now based on VS-2022.
Built using Cython 3.1.2.
The debug methods
MemDebug.dump()andMemDebug.show()were removed completely.libxml2 2.13.0 discarded this feature.
v5.4.0Compare Source
==================
Bugs fixed
(Binary wheels for Windows continue to use a patched libxml2 2.11.9 and libxslt 1.1.39.)
Issue found by Anatoly Katyushin.
v5.3.2Compare Source
==================
This release resolves CVE-2025-24928 as described in
https://gitlab.gnome.org/GNOME/libxml2/-/issues/847
Bugs fixed
Binary wheels use libxml2 2.12.10 and libxslt 1.1.42.
Binary wheels for Windows use a patched libxml2 2.11.9 and libxslt 1.1.39.
v5.3.1Compare Source
==================
Bugs fixed
GH#440: Some tests were adapted for libxml2 2.14.0.
Patch by Nick Wellnhofer.
LP#2097175:
DTD(external_id="…")erroneously required a byte string as ID value.GH#450:
iterparse()internally triggered the `DeprecationWarning`` added in lxml 5.3.0 when parsing HTML.Other changes
-flat_namespace.v5.3.0Compare Source
==================
Bugs fixed
GH#440: Some tests were adapted for libxml2 2.14.0.
Patch by Nick Wellnhofer.
LP#2097175:
DTD(external_id="…")erroneously required a byte string as ID value.GH#450:
iterparse()internally triggered the `DeprecationWarning`` added in lxml 5.3.0 when parsing HTML.Other changes
-flat_namespace.v5.2.2Compare Source
==================
Bugs fixed
GH#417: The
test_feed_parsertest could fail iflxml_html_cleanwas not installed.It is now skipped in that case.
LP#2059910: The minimum CPU architecture for the Linux x86 binary wheels was set back to
"core2", without SSE 4.2.
If libxml2 uses iconv, the compile time version is available as
etree.ICONV_COMPILED_VERSION.v5.2.1Compare Source
==================
Bugs fixed
LP#2059910: The minimum CPU architecture for the Linux x86 binary wheels was set back to
"core2", but with SSE 4.2 enabled.
LP#2059977:
Element.iterfind("//absolute_path")failed with aSyntaxErrorwhere it should have issued a warning.
GH#416: The documentation build was using the non-standard
whichcommand.Patch by Michał Górny.
v5.2.0Compare Source
==================
Other changes
LP#1958539: The
lxml.html.cleanimplementation suffered from several (only if used)security issues in the past and was now extracted into a separate library:
https://github.com/fedora-python/lxml_html_clean
Projects that use lxml without "lxml.html.clean" will not notice any difference,
except that they won't have potentially vulnerable code installed.
The module is available as an "extra" setuptools dependency "lxml[html_clean]",
so that Projects that need "lxml.html.clean" will need to switch their requirements
from "lxml" to "lxml[html_clean]", or install the new library themselves.
The minimum CPU architecture for the Linux x86 binary wheels was upgraded to
"sandybridge" (launched 2011), and glibc 2.28 / gcc 12 (manylinux_2_28) wheels were added.
Built with Cython 3.0.10.
v5.1.1Compare Source
==================
Bugs fixed
LP#2048920:
iterlinks()inlxml.htmlrejectedbytesinput in 5.1.0.High source line numbers from the parser are no longer truncated
(up to a C
long) when using libxml2 2.11 or later.Other changes
GH#407: A compatibility test was adapted to recent expat versions.
Patch by Miro Hrončok.
Binary wheels use the library versions libxml2 2.12.6 and libxslt 1.1.39.
Windows binary wheels use the library versions libxml2 2.11.7 and libxslt 1.1.39.
Built with Cython 3.0.9.
v5.1.0Compare Source
==================
Bugs fixed
LP#2048920:
iterlinks()inlxml.htmlrejectedbytesinput in 5.1.0.High source line numbers from the parser are no longer truncated
(up to a C
long) when using libxml2 2.11 or later.Other changes
GH#407: A compatibility test was adapted to recent expat versions.
Patch by Miro Hrončok.
Binary wheels use the library versions libxml2 2.12.6 and libxslt 1.1.39.
Windows binary wheels use the library versions libxml2 2.11.7 and libxslt 1.1.39.
Built with Cython 3.0.9.
v5.0.2Compare Source
==================
Other changes
GH#407: A compatibility test was adapted to recent expat versions.
Patch by Miro Hrončok.
Binary wheels use the library versions libxml2 2.12.6 and libxslt 1.1.39.
Built with Cython 3.0.9.
v5.0.1Compare Source
==================
Bugs fixed
LP#2046208: Parsing non-BMP Python Unicode strings could fail on macOS.
LP#2044225: When incrementally parsing broken HTML, reporting start events on
missing structural tags failed and could lead to subsequent exceptions.
LP#2045435: Some (not all) issues with stricter C compilers were resolved.
The binary wheels in the 5.0.0 release did not validate cleanly (but installed ok).
.. _latest_release:
v5.0.0Compare Source
==================
Features added
Character escaping in
C14N2serialisation now uses a single pass over the textinstead of searching for each unescaped character separately.
Early support for Python 3.13a2 was added.
Bugs fixed
LP#1976304: The
Element.addnext()method previously inserted the new elementbefore existing tail text. The tail text of both sibling elements now stays on
the respective elements.
LP#1980767, GH#379:
TreeBuilder.close()could fail with aTypeErrorafterparsing incorrect input. Original patch by Enrico Minack.
Element.itertext(with_tail=False)returned the tail text of comments andprocessing instructions, despite the explicit option.
GH#370: A crash with recent libxml2 2.11.x versions was resolved.
Patch by Michael Schlenker.
A compile problem with recent libxml2 2.12.x versions was resolved.
The internal exception handling in C callbacks was improved for Cython 3.0.
The exception declarations of
xmlInputReadCallback,xmlInputCloseCallback,xmlOutputWriteCallbackandxmlOutputCloseCallbackintree.pxdwerecorrected to prevent running Python code or calling into the C-API with a live
exception set.
GH#385: The long deprecated
unittest.m̀akeSuite()function is no longer used.Patch by Miro Hrončok.
LP#1522052: A file-system specific test is now optional and should no longer fail
on systems that don't support it.
GH#392: Some tests were adapted for libxml2 2.13.
Patch by Nick Wellnhofer.
Contains all fixes from lxml 4.9.4.
Other changes
LP#1742885: lxml no longer expands external entities (XXE) by default to prevent
the security risk of loading arbitrary files and URLs. If this feature is needed,
it can be enabled in a backwards compatible way by using a parser with the option
resolve_entities=True. The new default isresolve_entities='internal'.With libxml2 2.10.4 and later (as provided by the lxml 5.0 binary wheels),
parsing HTML tags with "prefixes" no longer builds a namespace dictionary
in
nsmapbut considers theprefix:namestring the actual tag name.With older libxml2 versions, since 2.9.11, the prefix was removed. Before
that, the prefix was parsed as XML prefix.
lxml 5.0 does not try to hide this difference but now changes the ElementPath
implementation to let
element.find("part1:part2")search for the tagpart1:part2in documents parsed as HTML, instead of looking only forpart2.LP#2024343: The validation of the schema file itself is now optional in the
ISO-Schematron implementation. This was done because some lxml distributions
discard the RNG validation schema file due to licensing issues. The validation
can now always be disabled with
Schematron(..., validate_schema=False).It is enabled by default if available and disabled otherwise. The module
constant
lxml.isoschematron.schematron_schema_valid_supportedcan be usedto detect whether schema file validation is available.
Some redundant and long deprecated methods were removed:
parser.setElementClassLookup(),xslt_transform.apply(),xpath.evaluate().Some incorrect declarations were removed from
python.pxd. In general, this fileshould not be used by external Cython code. Use the C-API declarations provided by
Cython itself instead.
Binary wheels use the library versions libxml2 2.12.3 and libxslt 1.1.39.
Built with Cython 3.0.7, updated to follow recent changes in Cython 3.1-dev.
Configuration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.