Fix NullReferenceException for null optional non-blittable struct parameters#1740
Merged
jevansaks merged 5 commits intoJul 5, 2026
Merged
Conversation
Optional [In] pointer parameters to non-blittable (managed) structs were emitted with an `in` modifier because a pointer to a managed type is illegal. The friendly overload then passed `ref Unsafe.NullRef<T>()` on the null path, which caused the P/Invoke marshaler to dereference a null reference and throw a NullReferenceException (e.g. MiniDumpWriteDump(..., CallbackParam: null)). Emit such parameters as a `T[]` array instead, as is already done for non-blittable struct fields. A null array marshals to a null pointer and a single-element array marshals to a pointer to the struct. The friendly overload continues to expose the parameter as a nullable `T?`. Fixes #1739 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sergio0694
approved these changes
Jul 3, 2026
The CallbackParameterIsMarshaled runtime test called MiniDumpWriteDump on the current process with a managed callback. Dumping your own process suspends all other threads while the callback runs on the calling thread, which can deadlock when the callback re-enters the CLR. It hung on net9/net10 in CI (blame-hang timeout). Verify the non-null array marshaling deterministically via a generator test instead, and keep the safe null-path runtime regression test. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The non-null marshaling path for optional non-blittable struct parameters (issue #1739) is now verified by actually executing interop: TrySubmitThreadpoolCallback submits a callback with a non-null TP_CALLBACK_ENVIRON_V3 environment (non-blittable, has delegate fields) and asserts the callback runs. This exercises the single-element-array marshaling end to end instead of only inspecting the generated syntax. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Release build treats analyzer warnings as errors; ManualResetEventSlim.Wait must receive the test cancellation token. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MiniDumpWriteDump on the current process suspends all other threads while it walks the process, so it can nondeterministically deadlock when a suspended thread holds a lock the dump needs. This reproduced as ~4/8 net10 test-host hangs locally (and a hang dump in CI) even with no managed callback involved. The #1739 regression (null optional non-blittable struct marshaling) is now covered entirely by ThreadpoolCallbackTests, which exercises both the non-null (single-element array) and null (null array -> null pointer) paths against the native threadpool without self-dumping. Stress-verified 12/12 clean on net10. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Member
Author
|
@Sergio0694 when you have a moment, could you please re-review? All CI is green now (GitHub Actions + the ADO pipeline). Since the earlier reviews, the regression coverage changed based on CI feedback:
Thanks! |
Sergio0694
approved these changes
Jul 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1739.
Calling an extern with a
nulloptional pointer-to-non-blittable-struct parameter threw aNullReferenceExceptioninstead of passing a null pointer. The canonical repro from the issue:Root cause
MINIDUMP_CALLBACK_INFORMATIONis non-blittable in the defaultAllowMarshalingmode because it contains a delegate field (MINIDUMP_CALLBACK_ROUTINE). A pointer to a managed type is illegal, so CsWin32 fell back to emitting the extern parameter with aninmodifier. The friendly overload exposed it asT?and took the null path viaref Unsafe.NullRef<T>(). Because the struct is non-blittable, the P/Invoke marshaler must build a native copy and dereferences the null reference while marshaling it, throwingNullReferenceException.This affects any optional
[In](not[Out]) parameter that points to a non-blittable struct, not justMiniDumpWriteDump.MiniDumpWriteDumpwas actually the motivating case for the original optional-pointer feature (#578), but its test only checked the friendly signature shape, never the runtime null behavior.Fix
Emit such parameters as a
T[]array instead of aninparameter — exactly what CsWin32 already does for non-blittable struct fields:nullarray marshals to a null pointer.The friendly overload continues to expose the parameter as a nullable
T?, and now forwardsparam.HasValue ? new[] { param.Value } : nullto the extern. The public friendly API is unchanged (stillT?); only the internal extern signature changes fromin TtoT[]for these non-blittable cases.Generated code before/after (friendly overload argument for
CallbackParam):Blittable optional struct pointers (e.g.
ExceptionParam,UserStreamParam) are unaffected and keep the existing pointer projection.Tests
GenerationSandbox.Teststhat actually callMiniDumpWriteDump:NullOptionalPointerParametersDoNotThrow— the regression:CallbackParam: nullnow succeeds instead of throwing.CallbackParameterIsMarshaled— passes a real callback and asserts it is invoked, exercising the single-element-array marshaling path.GeneratorTests.MiniDumpWriteDump_AllOptionalPointerParametersAreOptionalcontinues to pass (friendly params are stillT?).Validation
0x1(success) withCallbackParam: null, no NRE.Everything_NoFriendlyOverloadsand theFullMarshaling"Everything" generations pass (all APIs + friendly overloads still compile).