Add trampoline preamble with red zone reservation and R11 restart address#812
Open
Add trampoline preamble with red zone reservation and R11 restart address#812
Conversation
…ress The syscall rewriter now emits a 12-byte preamble before each trampoline stub: LEA RSP,[RSP-0x80] to reserve the SysV 128-byte red zone, and LEA R11,[RIP+disp32] to load the call-site restart address into R11 for future SA_RESTART support. Platform callbacks are renamed to syscall_callback_redzone to reflect the new calling convention. The callback recovers the architectural RSP with LEA R11,[RSP+128] and saves the restart address to TLS (saved_restart_addr) before clobbering R11. The rewriter also emits a size=0 header sentinel for binaries with no syscall instructions, allowing the loader to distinguish 'checked, no syscalls' from 'never processed.' The is_already_hooked check treats size=0 as already-hooked. The rtld_audit library is updated to handle both the new calling convention (red zone + R11) and size=0 binaries (via mincore probe before reading trampoline memory).
wdcui
force-pushed
the
wdcui/pr1-trampoline-format
branch
from
75ea7ec to
0bcb6ff
Compare
The pre-built binary had old-format trampolines that jumped to syscall_callback_redzone without reserving the red zone, causing an access violation (0xc0000005) on Windows.
…ore probe) The rtld_audit library uses a real syscall instruction in raw_syscall3() for the mincore probe before syscall_entry is known. Rewriting it would hook that instruction and jump to address 0. Rebuild from source (which already has the new do_syscall preamble) and copy without rewriting.
parse_object is only called for the main binary and interpreter, which always have syscall instructions and mapped trampolines. The mincore probe for the size=0 sentinel case is unnecessary. Removing the raw syscall instruction also means the rewriter can safely process rtld_audit.so (it finds no syscall instructions to hook).
parse_object can be called on binaries without mapped trampolines (e.g. vdso, non-PIE main). The mincore probe safely detects this case. The raw syscall instruction is needed because syscall_entry may not be initialized yet at probe time.
The raw syscall instruction breaks on Windows. This will be fully resolved when runtime patching (PR4) lands and removes rtld_audit.
wdcui
force-pushed
the
wdcui/pr1-trampoline-format
branch
from
a897889 to
fd4fcb7
Compare
|
🤖 SemverChecks 🤖 No breaking API changes detected Note: this does not mean API is unchanged, or even that there are no breaking changes; simply, none of the detections triggered. |
Member
Author
|
This PR is ready for review. Ignore the rtld changes since it will be removed after PR #810 is merged. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the red zone reservation and saves the syscall address for restart in the syscall rewriter and the userland platforms. It also adds an empty trampoline to ELF files that don't have syscalls so the runtime knows it's already patched.