--use_hugepages with --mr_per_qp can exhaust kernel.shmmni

[Hong-L666]

Hi,

I found that perftest can fail to create MR when running with
--use_hugepages and --mr_per_qp.

When --use_hugepages is used, perftest allocates hugepage-backed host memory
with SysV shared memory:

shmget(IPC_PRIVATE, buf_size, SHM_HUGETLB | IPC_CREAT | SHM_R | SHM_W)

This is in src/host_memory.c.
Without --mr_per_qp, perftest allocates one large buffer and shares it
between QPs.
With --mr_per_qp, perftest allocates one buffer per QP. When this is combined
with --use_hugepages, it creates one SysV shm segment per QP. For a large QP
count, this exhausts kernel.shmmni even when enough hugepages are available.

Environment:
- kernel: 6.6.0-28.0.0.34.oe2403.x86_64
- RDMA device: hrn5
- kernel.shmmni: 4096
- HugePages_Total: 30000
- HugePages_Free: 30000

Server:

ib_write_bw -d hrn5_0 -s 2 -Q 1 -q 4096 --use_hugepages --run --mr_per_qp

Client:

ib_write_bw -d hrn5_2 -s 2 -Q 1 -q 4096 --use_hugepages --run --mr_per_qp 192.168.100.1

Observed behavior:

Failed to allocate hugepages. Please configure hugepages
Failed to allocate hugepage region.
failed to create mr
Failed to create MR
Couldn't create IB resources

Workaround:

Increasing kernel.shmmni avoids the failure:

sudo sysctl -w kernel.shmmni=<larger_value>

This is only a workaround. The failure happens because the current hugepage
allocation path consumes one SysV shm segment per QP when --use_hugepages
and --mr_per_qp are used together.

Expected behavior:
perftest should not fail because of the SysV shm segment count when enough
hugepages are available.
Possible fix:
Use memfd_create(MFD_HUGETLB) + mmap for --use_hugepages instead of SysV shm.
This still uses hugetlb pages, but does not consume one SysV shm ID per QP.
I have sent a PR for this: #396