Update semaphore-compat to 2.0.0.0 to fix #9993 by wz1000 · Pull Request #11628 · haskell/cabal

wz1000 · 2026-03-17T11:11:56Z

Patches conform to the coding conventions.
Any changes that could be relevant to users have been recorded in the changelog.
- Is the change significant? If so, remember to add significance: significant in the changelog file.
The documentation has been updated, if necessary.
Manual QA notes have been included.
Tests have been added. (Ask for help if you don’t know how to write them! Ask for an exemption if tests are too complex for too little coverage!)

Update to semaphore-compat 2.0.0 fixing #9993 and https://gitlab.haskell.org/ghc/ghc/-/issues/25087

semaphore-compat now uses a unix sockets based implementation.

Semaphore identifiers are now versioned, and we can really on the versioning
scheme to detect semaphore version mismatch with GHC and fallback gracefully if
possible.

GHC patch: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/15729

After the GHC patch lands, it will include a "Semaphore version" field in its
settings file/--info output that we can use to guide Cabal behaviour.

If this field does not exist (and ghc is 9.8+), then we assume it uses version v1 of the protocol
and this triggers a graceful degradation of behaviour to -jN without semaphore based coordination.

See also

semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8
ghc-proposals change: ghc-proposals/ghc-proposals#673

Mikolaj · 2026-04-09T17:59:37Z

Is this needed for GHC 10 and so for Cabal 3.18? If so, how can we get this merged now?

On Linux and other POSIX platforms, GHC's -jsem jobserver client now speaks v2 of the semaphore-compat protocol, which uses Unix domain sockets in place of POSIX named semaphores. This avoids the libc-ABI issues that affected the old implementation. Windows is unaffected and continues to use the v1 protocol (Win32 named semaphores); its reported protocol version remains v1. When GHC receives a -jsem name whose protocol version it does not support, it emits a -Wsemaphore-version-mismatch warning and falls back to -j<N> rather than crashing. ghc --info exposes the supported version in a new "Semaphore version" entry so cabal-install can detect a mismatch before invoking GHC. Users on a cabal-install that predates the v2 update will continue to build successfully on Linux/POSIX, but will lose the cross-process -jsem coordination and fall back to -j<N> per GHC invocation. Users must upgrade to a cabal-install that supports protocol v2 to recover full parallelism. Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot heldTokens and release them before killing the loop, while the loop's in-flight acquire/release children could still be mutating it. Cleanup now runs inside the loop's own exit handler, after draining the active child via a new activeChild TVar, so the snapshot has no concurrent mutator. See also: - GHC proposal amendment: ghc-proposals/ghc-proposals#673 - cabal-install patch: haskell/cabal#11628 - semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8 Bump semaphore-compat submodule to 2.0.0 Fixes #25087 and #27253

sheaf

Yes, this looks good.

Personally I would like to avoid the "normal parallelism control" phrasing (that predates this PR) as I think we can be more specific about what we are falling back to, but that's just a nitpick.

On Linux and other POSIX platforms, GHC's -jsem jobserver client now speaks v2 of the semaphore-compat protocol, which uses Unix domain sockets in place of POSIX named semaphores. This avoids the libc-ABI issues that affected the old implementation. Windows is unaffected and continues to use the v1 protocol (Win32 named semaphores); its reported protocol version remains v1. When GHC receives a -jsem name whose protocol version it does not support, it emits a -Wsemaphore-version-mismatch warning and falls back to -j<N> rather than crashing. ghc --info exposes the supported version in a new "Semaphore version" entry so cabal-install can detect a mismatch before invoking GHC. Users on a cabal-install that predates the v2 update will continue to build successfully on Linux/POSIX, but will lose the cross-process -jsem coordination and fall back to -j<N> per GHC invocation. Users must upgrade to a cabal-install that supports protocol v2 to recover full parallelism. Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot heldTokens and release them before killing the loop, while the loop's in-flight acquire/release children could still be mutating it. Cleanup now runs inside the loop's own exit handler, after draining the active child via a new activeChild TVar, so the snapshot has no concurrent mutator. See also: - GHC proposal amendment: ghc-proposals/ghc-proposals#673 - cabal-install patch: haskell/cabal#11628 - semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8 Bump semaphore-compat submodule to 2.0.0 Fixes #25087 and #27253

philderbeast · 2026-05-19T15:48:01Z

Could we move forward on this? I recently did a cabal install cabal-install:exe:cabal --overwrite-policy=always from the master branch (in the past week or two) to check behaviour between the master branch and some other branch (can't remember which) and I'm hitting the #9993 issue often. I'm using the following as a build check:

$ cabal build all --enable-tests --enable-benchmarks -j --semaphore --ghc-options="-Werror"

On Linux and other POSIX platforms, GHC's -jsem jobserver client now speaks v2 of the semaphore-compat protocol, which uses Unix domain sockets in place of POSIX named semaphores. This avoids the libc-ABI issues that affected the old implementation. Windows is unaffected and continues to use the v1 protocol (Win32 named semaphores); its reported protocol version remains v1. When GHC receives a -jsem name whose protocol version it does not support, it emits a -Wsemaphore-version-mismatch warning and falls back to -j<N> rather than crashing. ghc --info exposes the supported version in a new "Semaphore version" entry so cabal-install can detect a mismatch before invoking GHC. Users on a cabal-install that predates the v2 update will continue to build successfully on Linux/POSIX, but will lose the cross-process -jsem coordination and fall back to -j<N> per GHC invocation. Users must upgrade to a cabal-install that supports protocol v2 to recover full parallelism. Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot heldTokens and release them before killing the loop, while the loop's in-flight acquire/release children could still be mutating it. Cleanup now runs inside the loop's own exit handler, after draining the active child via a new activeChild TVar, so the snapshot has no concurrent mutator. See also: - GHC proposal amendment: ghc-proposals/ghc-proposals#673 - cabal-install patch: haskell/cabal#11628 - semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8 Bump semaphore-compat submodule to 2.0.0 Fixes #25087 and #27253

wz1000 · 2026-05-21T13:29:42Z

https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8 has been merged, which took longer than expected. I will try to merge the GHC MR tomorrow, and then this should be unblocked for merging.

sheaf · 2026-05-21T13:41:44Z

Could we move forward on this? I recently did a cabal install cabal-install:exe:cabal --overwrite-policy=always from the master branch (in the past week or two) to check behaviour between the master branch and some other branch (can't remember which) and I'm hitting the #9993 issue often.

@philderbeast I don't think that's related. In this PR we are fixing the situation in which GHC and Cabal were linked against different C standard libraries which have incompatible implementations of POSIX semaphores.

If you are running into another issue and are able to provide a reproducer, please open a ticket and I will investigate it.

On Linux and other POSIX platforms, GHC's -jsem jobserver client now speaks v2 of the semaphore-compat protocol, which uses Unix domain sockets in place of POSIX named semaphores. This avoids the libc-ABI issues that affected the old implementation. Windows is unaffected and continues to use the v1 protocol (Win32 named semaphores); its reported protocol version remains v1. When GHC receives a -jsem name whose protocol version it does not support, it emits a -Wsemaphore-version-mismatch warning and falls back to -j<N> rather than crashing. ghc --info exposes the supported version in a new "Semaphore version" entry so cabal-install can detect a mismatch before invoking GHC. Users on a cabal-install that predates the v2 update will continue to build successfully on Linux/POSIX, but will lose the cross-process -jsem coordination and fall back to -j<N> per GHC invocation. Users must upgrade to a cabal-install that supports protocol v2 to recover full parallelism. Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot heldTokens and release them before killing the loop, while the loop's in-flight acquire/release children could still be mutating it. Cleanup now runs inside the loop's own exit handler, after draining the active child via a new activeChild TVar, so the snapshot has no concurrent mutator. See also: - GHC proposal amendment: ghc-proposals/ghc-proposals#673 - cabal-install patch: haskell/cabal#11628 - semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8 Bump semaphore-compat submodule to 2.0.0 Fixes #25087 and #27253

…ab.haskell.org/ghc/ghc/-/issues/25087 Also update index-state and bootstrap plans to pull in new version of semaphore-compat. semaphore-compat now uses a unix sockets based implementation. Semaphore identifiers are now versioned, and we can really on the versioning scheme to detect semaphore version mismatch with GHC and fallback gracefully if possible. GHC patch: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/15729 After the GHC patch lands, it will include a "Semaphore version" field in its settings file/`--info` output that we can use to guide Cabal behaviour. If this field does not exist (and ghc is 9.8+), then we assume it uses version v1 of the protocol and this triggers a graceful degradation of behaviour to `-jN` without semaphore based coordination. See also semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8 ghc-proposals change: ghc-proposals/ghc-proposals#673

source-repository-package has been removed.

wz1000 · 2026-05-25T14:47:23Z

This patch should be ready to merge.

semaphore-compat-2.0.0 is now released and the GHC MR upgrading to sempahore-compat-2.0.0 is in the merge queue. We plan to backport the GHC MR to GHC 9.10.4, 9.12.5, 9.14.2 and 10.0.1. cabal-install releases without these changes will have builds degrading to reduced parallelism with --semaphore on these and later releases, so it would be good to have these changes included in the next release.

On Linux and other POSIX platforms, GHC's -jsem jobserver client now speaks v2 of the semaphore-compat protocol, which uses Unix domain sockets in place of POSIX named semaphores. This avoids the libc-ABI issues that affected the old implementation. Windows is unaffected and continues to use the v1 protocol (Win32 named semaphores); its reported protocol version remains v1. When GHC receives a -jsem name whose protocol version it does not support, it emits a -Wsemaphore-version-mismatch warning and falls back to -j<N> rather than crashing. ghc --info exposes the supported version in a new "Semaphore version" entry so cabal-install can detect a mismatch before invoking GHC. Users on a cabal-install that predates the v2 update will continue to build successfully on Linux/POSIX, but will lose the cross-process -jsem coordination and fall back to -j<N> per GHC invocation. Users must upgrade to a cabal-install that supports protocol v2 to recover full parallelism. Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot heldTokens and release them before killing the loop, while the loop's in-flight acquire/release children could still be mutating it. Cleanup now runs inside the loop's own exit handler, after draining the active child via a new activeChild TVar, so the snapshot has no concurrent mutator. See also: - GHC proposal amendment: ghc-proposals/ghc-proposals#673 - cabal-install patch: haskell/cabal#11628 - semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8 Bump semaphore-compat submodule to 2.0.0 Fixes #25087 and #27253

On Linux and other POSIX platforms, GHC's -jsem jobserver client now speaks v2 of the semaphore-compat protocol, which uses Unix domain sockets in place of POSIX named semaphores. This avoids the libc-ABI issues that affected the old implementation. Windows is unaffected and continues to use the v1 protocol (Win32 named semaphores); its reported protocol version remains v1. When GHC receives a -jsem name whose protocol version it does not support, it emits a -Wsemaphore-version-mismatch warning and falls back to -j<N> rather than crashing. ghc --info exposes the supported version in a new "Semaphore version" entry so cabal-install can detect a mismatch before invoking GHC. Users on a cabal-install that predates the v2 update will continue to build successfully on Linux/POSIX, but will lose the cross-process -jsem coordination and fall back to -j<N> per GHC invocation. Users must upgrade to a cabal-install that supports protocol v2 to recover full parallelism. Also fix a leak in cleanupSem (#27253): cleanupSem used to snapshot heldTokens and release them before killing the loop, while the loop's in-flight acquire/release children could still be mutating it. Cleanup now runs inside the loop's own exit handler, after draining the active child via a new activeChild TVar, so the snapshot has no concurrent mutator. See also: - GHC proposal amendment: ghc-proposals/ghc-proposals#673 - cabal-install patch: haskell/cabal#11628 - semaphore-compat MR: https://gitlab.haskell.org/ghc/semaphore-compat/-/merge_requests/8 Bump semaphore-compat submodule to 2.0.0 Fixes #25087 and #27253 (cherry picked from commit 8db331a)

wz1000 mentioned this pull request Mar 17, 2026

Update jsem proposal to address problems discovered in GHC #25087 (under review) ghc-proposals/ghc-proposals#673

Merged

geekosaur previously requested changes Mar 17, 2026

View reviewed changes

Comment thread cabal.project Outdated

wz1000 force-pushed the wip/semaphore-v2 branch 3 times, most recently from 67769ee to 98685dd Compare March 19, 2026 04:30

Bodigrim approved these changes Mar 19, 2026

View reviewed changes

Comment thread changelog.d/semaphore-version-compat.md Outdated

Comment thread Cabal/src/Distribution/Simple/Compiler.hs Outdated

sheaf reviewed Mar 20, 2026

View reviewed changes

Comment thread cabal-install/src/Distribution/Client/JobControl.hs Outdated

Comment thread cabal-install/src/Distribution/Client/JobControl.hs Outdated

Comment thread changelog.d/semaphore-version-compat.md Outdated

wz1000 force-pushed the wip/semaphore-v2 branch 4 times, most recently from cea0774 to 2758155 Compare May 12, 2026 11:55

wz1000 force-pushed the wip/semaphore-v2 branch from 2758155 to 067e12c Compare May 13, 2026 13:01

wz1000 force-pushed the wip/semaphore-v2 branch 2 times, most recently from d49fb8c to ee96ef5 Compare May 19, 2026 09:05

sheaf approved these changes May 19, 2026

View reviewed changes

wz1000 force-pushed the wip/semaphore-v2 branch 2 times, most recently from d564a18 to 1e9c8ae Compare May 21, 2026 13:27

wz1000 force-pushed the wip/semaphore-v2 branch from 1e9c8ae to b8817fb Compare May 21, 2026 14:00

wz1000 force-pushed the wip/semaphore-v2 branch 2 times, most recently from 0751d73 to 02ae51d Compare May 22, 2026 05:00

wz1000 force-pushed the wip/semaphore-v2 branch 3 times, most recently from eb272bf to 88784cc Compare May 25, 2026 11:16

wz1000 force-pushed the wip/semaphore-v2 branch from 88784cc to 96ccd50 Compare May 25, 2026 11:43

sheaf approved these changes May 25, 2026

View reviewed changes

Mikolaj added the merge me Tell Mergify Bot to merge label May 25, 2026

mergify Bot added the ready and waiting Mergify is waiting out the cooldown period label May 25, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update semaphore-compat to 2.0.0.0 to fix #9993#11628

Update semaphore-compat to 2.0.0.0 to fix #9993#11628
wz1000 wants to merge 1 commit into
haskell:masterfrom
wz1000:wip/semaphore-v2

wz1000 commented Mar 17, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Mikolaj commented Apr 9, 2026

Uh oh!

sheaf left a comment

Uh oh!

philderbeast commented May 19, 2026

Uh oh!

wz1000 commented May 21, 2026

Uh oh!

sheaf commented May 21, 2026

Uh oh!

wz1000 commented May 25, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

wz1000 commented Mar 17, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Mikolaj commented Apr 9, 2026

Uh oh!

sheaf left a comment

Choose a reason for hiding this comment

Uh oh!

philderbeast commented May 19, 2026

Uh oh!

wz1000 commented May 21, 2026

Uh oh!

sheaf commented May 21, 2026

Uh oh!

wz1000 commented May 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

wz1000 commented May 25, 2026 •

edited

Loading