Skip to content

Phase 9: trackpad volume gesture (shared F4)#9

Open
tirthfx wants to merge 1 commit into
phase-8-lockscreen-lyricsfrom
phase-9-volume-gesture
Open

Phase 9: trackpad volume gesture (shared F4)#9
tirthfx wants to merge 1 commit into
phase-8-lockscreen-lyricsfrom
phase-9-volume-gesture

Conversation

@tirthfx

@tirthfx tirthfx commented Jul 10, 2026

Copy link
Copy Markdown
Owner

What changed

Phase 9 — the shared trackpad left-edge volume gesture (PRD §3.3 / F4), the hardest phase. A two-finger vertical slide that starts at the trackpad's physical left edge acts as a system-wide volume fader — regardless of cursor position or which surface (notch / lock screen / neither) is visible.

  • MultitouchSupport.swift — runtime bindings to the private framework via dlopen/dlsym (no public header, can't link directly). MTTouch uses the long-standing community-reverse-engineered 96-byte layout (cited in the source comment); offsets through normalized are the stable part and all we read. The loader asserts the stride (verified = 96, normalized @ offset 32) and disables the gesture gracefully if a future OS changes it, rather than striding through touches incorrectly.
  • VolumeController.swift — reads/sets kAudioHardwareServiceDeviceProperty_VirtualMainVolume on the current default output device via Core Audio, so speakers / headphones / AirPlay all work; re-resolves the device each call (it can change).
  • GestureEngine.swift — arms only when both fingers land in the left-edge strip (x < 0.16), which is what separates a deliberate edge slide from ordinary two-finger scrolling (which starts anywhere); a dead-zone before engaging avoids twitch; vertical delta maps to volume with throttled Core Audio writes. The bare C callback (no context) routes through a shared instance.
  • Opt-in menu toggle "Left-edge Volume Gesture"; enabling prompts for Input Monitoring (IOHIDRequestAccess) the first time. Off by default so nothing claims trackpad space or triggers a permission prompt unexpectedly at launch.

Why

Completes the shared gesture layer from the brief — the notch/lock-screen surfaces handle now-playing; this adds the always-on volume fader. The app is unsandboxed (no App Sandbox entitlement), which is what permits the private-framework dlopen and Core Audio calls (PRD §4 already notes this can't ship on the Mac App Store).

Base

Top of the stack: 5 → 6 → 7 → 8 → 9, based on phase-8-lockscreen-lyrics (open PR #8). Diff is Phase 9 only.

Notes for review — needs your eyes + hands on the trackpad

  • Can't be verified headlessly (needs a real trackpad + the Input Monitoring grant, and the multitouch callback only fires on live hardware). Confirmed: builds clean, launches without crashing, MTTouch stride == 96 checked against the C layout, notch/lock screen unaffected.
  • Please test on the Mac: menu → enable Left-edge Volume Gesture → grant Input Monitoring when prompted → slide two fingers up/down starting at the far-left edge of the trackpad → system volume should move smoothly; normal two-finger scrolling elsewhere should NOT change volume.
  • Tuning knobs if it feels off, all constants at the top of GestureEngine: edgeThreshold (how wide the left strip is), travelForFullRange (sensitivity), deadZone. Slide direction is natural (up = louder) — flip the sign in handleFrame if you'd prefer otherwise.
  • If MultitouchSupport can't start (e.g. permission denied), it logs and no-ops — the toggle just won't stick on.

🤖 Generated with Claude Code

System-wide two-finger left-edge slide → system volume (PRD §3.3 / F4), the
hardest phase. Works regardless of cursor position or which surface is visible.

- MultitouchSupport.swift: runtime bindings to the private framework via
  dlopen/dlsym (no public header, can't link directly). MTTouch uses the
  community-reverse-engineered 96-byte layout (offsets through 'normalized' are
  stable); the loader asserts the stride and bails gracefully otherwise.
- VolumeController.swift: reads/sets kAudioHardwareServiceDeviceProperty_
  VirtualMainVolume on the current default output device via Core Audio, so
  speakers / headphones / AirPlay all work.
- GestureEngine.swift: arms only when BOTH fingers land in the left-edge strip
  (x < 0.16), which separates a deliberate edge slide from ordinary two-finger
  scrolling; dead-zone before engaging; vertical delta → volume with throttled
  Core Audio writes. The bare C callback routes through a shared instance.
- Opt-in menu toggle 'Left-edge Volume Gesture'; enabling prompts for Input
  Monitoring (IOHIDRequestAccess) the first time. Off by default so nothing
  claims trackpad space or prompts unexpectedly at launch.

App is unsandboxed (no App Sandbox entitlement), so the private-framework
dlopen and Core Audio calls are permitted (PRD §4: can't ship on the MAS).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a system-wide trackpad gesture for controlling system volume by reading raw trackpad contacts via the private MultitouchSupport framework and updating the volume using Core Audio. The feedback focuses on ensuring thread safety and improving gesture precision. Key recommendations include introducing a lock to synchronize access to gesture state variables across threads, adjusting the volume throttling logic to guarantee that boundary volumes (0.0 and 1.0) are reachable, and refactoring the volume address property to a file-scope constant to prevent exclusivity violations.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment on lines +33 to +37
private var engaged = false
private var armed = false
private var baselineY: Float = 0
private var startVolume: Float = 0
private var lastAppliedVolume: Float = -1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

To prevent data races on the gesture state variables (engaged, armed, etc.) between the main thread (in disable()) and the background MultitouchSupport callback thread (in handleFrame), we should introduce a lock to synchronize access.

Suggested change
private var engaged = false
private var armed = false
private var baselineY: Float = 0
private var startVolume: Float = 0
private var lastAppliedVolume: Float = -1
private let lock = NSLock()
private var engaged = false
private var armed = false
private var baselineY: Float = 0
private var startVolume: Float = 0
private var lastAppliedVolume: Float = -1

Comment on lines +72 to +73
engaged = false
armed = false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Wrap the state reset in lock.withLock to ensure thread safety when disabling the gesture engine.

        lock.withLock {
            engaged = false
            armed = false
        }

Comment on lines +87 to +127
private func handleFrame(_ touches: UnsafePointer<MTTouch>, _ count: Int) {
// We only care about a clean two-finger contact. Anything else ends the
// gesture, so lifting or adding a finger cleanly disengages.
guard count == 2 else {
engaged = false
armed = false
return
}

let a = touches[0].normalized.position
let b = touches[1].normalized.position
let avgY = (a.y + b.y) / 2

if !engaged {
engaged = true
// Arm only when BOTH fingers land in the left-edge strip — this is
// what separates a deliberate edge slide from ordinary two-finger
// scrolling, which starts anywhere on the surface.
armed = a.x < edgeThreshold && b.x < edgeThreshold
baselineY = avgY
startVolume = volume.currentVolume() ?? 0
lastAppliedVolume = -1
return
}

guard armed else { return }

let delta = avgY - baselineY
guard abs(delta) > deadZone else { return }

// Slide up (y increases) raises volume; one `travelForFullRange` of
// vertical travel sweeps the whole 0...1 range.
let target = startVolume + delta / travelForFullRange
let clamped = min(max(target, 0), 1)

// Throttle Core Audio writes: only when it moves ~1.5% (≈64 steps),
// finer than the 16 hardware increments but without spamming the device.
guard abs(clamped - lastAppliedVolume) >= 0.015 else { return }
lastAppliedVolume = clamped
volume.setVolume(clamped)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

This suggestion addresses two important issues in handleFrame:

  1. Thread Safety / Data Race: The gesture state is mutated on the background MultitouchSupport callback thread, but some state is also reset on the main thread in disable(). Wrapping the body in lock.withLock ensures thread-safe access.
  2. Boundary Volume Reachability: The 0.015 (1.5%) throttle threshold can prevent the volume from reaching exactly 0.0 (muted) or 1.0 (100%) if the final gesture increment is small. Adding || clamped == 0 || clamped == 1 ensures these boundary values are always reachable.
    private func handleFrame(_ touches: UnsafePointer<MTTouch>, _ count: Int) {
        lock.withLock {
            // We only care about a clean two-finger contact. Anything else ends the
            // gesture, so lifting or adding a finger cleanly disengages.
            guard count == 2 else {
                engaged = false
                armed = false
                return
            }

            let a = touches[0].normalized.position
            let b = touches[1].normalized.position
            let avgY = (a.y + b.y) / 2

            if !engaged {
                engaged = true
                // Arm only when BOTH fingers land in the left-edge strip — this is
                // what separates a deliberate edge slide from ordinary two-finger
                // scrolling, which starts anywhere on the surface.
                armed = a.x < edgeThreshold && b.x < edgeThreshold
                baselineY = avgY
                startVolume = volume.currentVolume() ?? 0
                lastAppliedVolume = -1
                return
            }

            guard armed else { return }

            let delta = avgY - baselineY
            guard abs(delta) > deadZone else { return }

            // Slide up (y increases) raises volume; one `travelForFullRange` of
            // vertical travel sweeps the whole 0...1 range.
            let target = startVolume + delta / travelForFullRange
            let clamped = min(max(target, 0), 1)

            // Throttle Core Audio writes: only when it moves ~1.5% (≈64 steps),
            // finer than the 16 hardware increments but without spamming the device.
            guard abs(clamped - lastAppliedVolume) >= 0.015 || clamped == 0 || clamped == 1 else { return }
            lastAppliedVolume = clamped
            volume.setVolume(clamped)
        }
    }

Comment on lines 13 to +18
final class VolumeController {
// TODO: Phase 5 — AudioObjectSetPropertyData on
// kAudioHardwareServiceDeviceProperty_VirtualMainVolume, with smoothing.
private var volumeAddress = AudioObjectPropertyAddress(
mSelector: kAudioHardwareServiceDeviceProperty_VirtualMainVolume,
mScope: kAudioDevicePropertyScopeOutput,
mElement: kAudioObjectPropertyElementMain
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Since VolumeController is called from the background MultitouchSupport callback thread, having volumeAddress as an instance var and passing it by reference (&volumeAddress) to Core Audio functions can lead to thread-safety and exclusivity violations in Swift. Moving volumeAddress to the file scope as a private let constant makes the class completely stateless and thread-safe.

Suggested change
final class VolumeController {
// TODO: Phase 5 — AudioObjectSetPropertyData on
// kAudioHardwareServiceDeviceProperty_VirtualMainVolume, with smoothing.
private var volumeAddress = AudioObjectPropertyAddress(
mSelector: kAudioHardwareServiceDeviceProperty_VirtualMainVolume,
mScope: kAudioDevicePropertyScopeOutput,
mElement: kAudioObjectPropertyElementMain
)
private let volumeAddress = AudioObjectPropertyAddress(
mSelector: kAudioHardwareServiceDeviceProperty_VirtualMainVolume,
mScope: kAudioDevicePropertyScopeOutput,
mElement: kAudioObjectPropertyElementMain
)
final class VolumeController {

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant