Perftest: use memfd for hugepage host memory#396
Open
Hong-L666 wants to merge 1 commit into
Open
Conversation
--use_hugepages allocates host buffers with SysV shared memory. When --mr_per_qp is used, perftest allocates one buffer per QP, so high QP counts can exhaust kernel.shmmni even when enough huge pages are available. Allocate hugetlb memory through memfd_create and mmap instead. Close the memfd after mmap succeeds and release the mapping with munmap during buffer cleanup. Signed-off-by: Hong-L666 <84577812+Hong-L666@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #395
This changes the
--use_hugepageshost allocation path from SysV shm to a hugetlb memfd mapping.The current code uses
shmget(SHM_HUGETLB)for hugepage-backed host memory. That is fine for the normal path, where perftest allocates one large buffer and shares it between QPs.With
--mr_per_qp, perftest allocates one host buffer per QP. If--use_hugepagesis also set, each buffer allocation creates one SysV shm segment. With a large QP count this can hitkernel.shmmni, even when the hugepage pool still has enough free pages.Using
memfd_create(MFD_HUGETLB)keeps the allocation backed by hugetlb pages, but avoids consuming one SysV shm id per buffer. The memfd is closed aftermmap()succeeds, and the mapping is released withmunmap().Testing:
git diff --check./autogen.sh./configuremakeRuntime test:
Server:
Client:
Result:
The test passed with this patch. The same command failed before this patch
when the SysV shm segment count reached
kernel.shmmni.