Make lock behavior controllable, add xxhash.threadsafe submodule, and clean up

CHANGELOG.rst

@@ -5,6 +5,16 @@ NEXT
 ~~~~~~~~~~~~~~~~~
 
 - Drop support for Python 3.8
+- Remove deprecated ``xxhash.VERSION_TUPLE``
+- Default streaming hash objects (``xxh32``, ``xxh64``, ``xxh3_64``,
+  ``xxh3_128``) are no longer thread-safe by default; this removes
+  per-object locking overhead and restores performance as the primary goal
+- Add ``xxhash.threadsafe`` submodule for users who need to share a
+  streaming hash object across threads; it provides the same API with a
+  per-object lock
+- Both the default module and ``xxhash.threadsafe`` are provided on
+  free-threading (no-GIL) Python builds, matching the API on regular GIL
+  builds
 
 v3.7.0 2025-04-25
 ~~~~~~~~~~~~~~~~~

README.rst

@@ -251,6 +251,36 @@ And aliases:
     | xxh128_intdigest = xxh3_128_intdigest
     | xxh128_hexdigest = xxh3_128_hexdigest
 
+Thread safety
+-------------
+
+The default ``xxhash`` module is optimized for speed. Streaming hash objects
+(``xxh32``, ``xxh64``, ``xxh3_64``, ``xxh3_128`` / ``xxh128``) are **not**
+thread-safe: do not call ``update()``, ``digest()``, ``copy()``, ``reset()``,
+or any other mutating method on the same object from multiple threads without
+external synchronization.
+
+One-shot functions (``xxh32_digest``, ``xxh64_hexdigest``, ``xxh3_128_digest``,
+etc.) are stateless and always safe to call concurrently.
+
+Concurrent ``update()`` / ``reset()`` on a shared streaming hash object is
+discouraged even with locking — prefer one-shot functions or per-thread hash
+objects. If you must share a streaming hash across threads, use the
+``xxhash.threadsafe`` submodule. It provides the same API with a per-object
+lock that serializes all access to the internal xxHash state:
+
+.. code-block:: python
+
+    >>> from xxhash import threadsafe
+    >>> h = threadsafe.xxh64()
+    >>> # h can be updated from multiple threads, but concurrent update/reset
+    >>> # still adds overhead and is not recommended
+
+The same two-module split is provided on free-threading (no-GIL) Python
+builds: the default module is unlocked, and ``xxhash.threadsafe`` provides a
+locked variant.
+
+
 Caveats
 -------
 

setup.py

@@ -2,6 +2,7 @@
 from pathlib import Path
 
 from setuptools import Extension, setup
+from setuptools.command.build_ext import build_ext as _build_ext
 
 if os.getenv("XXHASH_LINK_SO"):
     libraries = ["xxhash"]
@@ -12,15 +13,51 @@
     source = ["src/_xxhash.c", "deps/xxhash/xxhash.c"]
     include_dirs = ["deps/xxhash"]
 
+# The default ``xxhash._xxhash`` extension is built without per-object locks
+# for maximum performance.  Users who need to share a streaming hash object
+# across threads can use ``xxhash._xxhash_threadsafe`` (exposed as the public
+# ``xxhash.threadsafe`` submodule), which is compiled from the same source
+# with locking enabled.
+
+_ext_kwargs = {
+    "sources": source,
+    "include_dirs": include_dirs,
+    "libraries": libraries,
+}
+
 ext_modules = [
     Extension(
         "_xxhash",
-        source,
-        include_dirs=include_dirs,
-        libraries=libraries,
-    )
+        **_ext_kwargs,
+    ),
+    Extension(
+        "_xxhash_threadsafe",
+        define_macros=[("XXHASH_WITH_LOCK", "1"), ("XXHASH_MODULE_NAME", "_xxhash_threadsafe")],
+        **_ext_kwargs,
+    ),
 ]
 
+
+class build_ext(_build_ext):
+    """Build each extension in its own temp directory.
+
+    Both extensions are built from the same ``src/_xxhash.c`` source file.
+    Without separate temp directories their object files would overwrite
+    each other, causing one variant to be linked with the wrong macros.
+
+    ``try/finally`` restores ``self.build_temp`` so that incremental builds
+    (where ``build_ext`` may be reused) still work correctly.
+    """
+
+    def build_extension(self, ext):
+        old_build_temp = self.build_temp
+        self.build_temp = os.path.join(old_build_temp, ext.name)
+        try:
+            super().build_extension(ext)
+        finally:
+            self.build_temp = old_build_temp
+
+
 d = Path(__file__).parent
 long_description = d.joinpath("README.rst").read_text() + "\n" + d.joinpath("CHANGELOG.rst").read_text()
 
@@ -58,5 +95,6 @@
     ],
     python_requires=">=3.9",
     ext_modules=ext_modules,
+    cmdclass={"build_ext": build_ext},
     package_data={"xxhash": ["py.typed", "**.pyi"]},
 )