Skip to content

Fix NullReferenceException for null optional non-blittable struct parameters#1740

Merged
jevansaks merged 5 commits into
mainfrom
user/jevansa/fix-1739-null-optional-nonblittable-struct
Jul 5, 2026
Merged

Fix NullReferenceException for null optional non-blittable struct parameters#1740
jevansaks merged 5 commits into
mainfrom
user/jevansa/fix-1739-null-optional-nonblittable-struct

Conversation

@jevansaks

Copy link
Copy Markdown
Member

Summary

Fixes #1739.

Calling an extern with a null optional pointer-to-non-blittable-struct parameter threw a NullReferenceException instead of passing a null pointer. The canonical repro from the issue:

PInvoke.MiniDumpWriteDump(
    process.SafeHandle,
    (uint)Environment.ProcessId,
    dumpStream.SafeFileHandle,
    MINIDUMP_TYPE.MiniDumpWithThreadInfo,
    ExceptionParam: null,
    UserStreamParam: null,
    CallbackParam: null); // -> NullReferenceException

Root cause

MINIDUMP_CALLBACK_INFORMATION is non-blittable in the default AllowMarshaling mode because it contains a delegate field (MINIDUMP_CALLBACK_ROUTINE). A pointer to a managed type is illegal, so CsWin32 fell back to emitting the extern parameter with an in modifier. The friendly overload exposed it as T? and took the null path via ref Unsafe.NullRef<T>(). Because the struct is non-blittable, the P/Invoke marshaler must build a native copy and dereferences the null reference while marshaling it, throwing NullReferenceException.

This affects any optional [In] (not [Out]) parameter that points to a non-blittable struct, not just MiniDumpWriteDump. MiniDumpWriteDump was actually the motivating case for the original optional-pointer feature (#578), but its test only checked the friendly signature shape, never the runtime null behavior.

Fix

Emit such parameters as a T[] array instead of an in parameter — exactly what CsWin32 already does for non-blittable struct fields:

  • A null array marshals to a null pointer.
  • A single-element array marshals to a pointer to the struct.

The friendly overload continues to expose the parameter as a nullable T?, and now forwards param.HasValue ? new[] { param.Value } : null to the extern. The public friendly API is unchanged (still T?); only the internal extern signature changes from in T to T[] for these non-blittable cases.

Generated code before/after (friendly overload argument for CallbackParam):

// before (throws for null):
CallbackParam.HasValue ? &CallbackParamLocal : ref Unsafe.NullRef<...>()
// after:
CallbackParam.HasValue ? new[] { CallbackParam.Value } : null

Blittable optional struct pointers (e.g. ExceptionParam, UserStreamParam) are unaffected and keep the existing pointer projection.

Tests

  • Added runtime tests in GenerationSandbox.Tests that actually call MiniDumpWriteDump:
    • NullOptionalPointerParametersDoNotThrow — the regression: CallbackParam: null now succeeds instead of throwing.
    • CallbackParameterIsMarshaled — passes a real callback and asserts it is invoked, exercising the single-element-array marshaling path.
  • Existing GeneratorTests.MiniDumpWriteDump_AllOptionalPointerParametersAreOptional continues to pass (friendly params are still T?).

Validation

  • Repro project returns 0x1 (success) with CallbackParam: null, no NRE.
  • All 812 non-HighMemory generator tests pass.
  • Everything_NoFriendlyOverloads and the FullMarshaling "Everything" generations pass (all APIs + friendly overloads still compile).

Optional [In] pointer parameters to non-blittable (managed) structs were
emitted with an `in` modifier because a pointer to a managed type is illegal.
The friendly overload then passed `ref Unsafe.NullRef<T>()` on the null path,
which caused the P/Invoke marshaler to dereference a null reference and throw
a NullReferenceException (e.g. MiniDumpWriteDump(..., CallbackParam: null)).

Emit such parameters as a `T[]` array instead, as is already done for
non-blittable struct fields. A null array marshals to a null pointer and a
single-element array marshals to a pointer to the struct. The friendly
overload continues to expose the parameter as a nullable `T?`.

Fixes #1739

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jevansaks jevansaks marked this pull request as ready for review July 3, 2026 16:27
jevansaks and others added 4 commits July 3, 2026 21:29
The CallbackParameterIsMarshaled runtime test called MiniDumpWriteDump on the current process with a managed callback. Dumping your own process suspends all other threads while the callback runs on the calling thread, which can deadlock when the callback re-enters the CLR. It hung on net9/net10 in CI (blame-hang timeout). Verify the non-null array marshaling deterministically via a generator test instead, and keep the safe null-path runtime regression test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The non-null marshaling path for optional non-blittable struct parameters
(issue #1739) is now verified by actually executing interop:
TrySubmitThreadpoolCallback submits a callback with a non-null
TP_CALLBACK_ENVIRON_V3 environment (non-blittable, has delegate fields) and
asserts the callback runs. This exercises the single-element-array marshaling
end to end instead of only inspecting the generated syntax.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Release build treats analyzer warnings as errors; ManualResetEventSlim.Wait
must receive the test cancellation token.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
MiniDumpWriteDump on the current process suspends all other threads while it
walks the process, so it can nondeterministically deadlock when a suspended
thread holds a lock the dump needs. This reproduced as ~4/8 net10 test-host
hangs locally (and a hang dump in CI) even with no managed callback involved.

The #1739 regression (null optional non-blittable struct marshaling) is now
covered entirely by ThreadpoolCallbackTests, which exercises both the non-null
(single-element array) and null (null array -> null pointer) paths against the
native threadpool without self-dumping. Stress-verified 12/12 clean on net10.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jevansaks

Copy link
Copy Markdown
Member Author

@Sergio0694 when you have a moment, could you please re-review? All CI is green now (GitHub Actions + the ADO pipeline).

Since the earlier reviews, the regression coverage changed based on CI feedback:

  • The #1739 fix itself is unchanged.
  • The runtime regression test now lives in ThreadpoolCallbackTests and actually executes interop on TrySubmitThreadpoolCallback, covering both the null path (null → null array → null pointer, the original NRE) and the non-null path (value → single-element array → pointer) by confirming the submitted native threadpool callback runs.
  • Removed the earlier MiniDumpWriteDump runtime test: self-dumping the current process suspends all other threads while walking memory and could nondeterministically deadlock the test host (reproduced as ~4/8 net10 hangs locally and a hang dump in CI), even with no managed callback. The threadpool test exercises the same generated marshaling pattern safely (stress-verified 12/12 clean on net10).

Thanks!

@jevansaks jevansaks merged commit 00bd455 into main Jul 5, 2026
20 checks passed
@jevansaks jevansaks deleted the user/jevansa/fix-1739-null-optional-nonblittable-struct branch July 5, 2026 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

NullReferenceException from passing null as CallbackParam for MiniDumpWriteDump

2 participants