Hello
According to the man page, there could be network ports with IBV_QPF_GRH_REQUIRED flag, which require setting is_global and grh header
https://man7.org/linux/man-pages/man3/ibv_modify_qp.3.html
If port flag IBV_QPF_GRH_REQUIRED is set then ah_attr and
alt_ah_attr must be passed with definition of 'struct ibv_ah_attr
{ .is_global = 1; .grh = {...}; }'.
Current logic in perftest_resources.c ctx_modify_qp_to_rtr don't check that IBV_QPF_GRH_REQUIRED flag and just sets
attr->ah_attr.is_global = 0;
On port with GRH requirement current perftest ib_write_bw has critical error
Failed to modify QP 122273 to RTR
Unable to Connect the HCA's through the link
Failed to modify QP 122272 to RTR
Unable to Connect the HCA's through the link
Similar issue was already solved in Mellanox/nccl-rdma-sharp-plugins@e68c599 patch with (portAttr.flags & IBV_QPF_GRH_REQUIRED) and env variable as alternative way to always enable GRH header.
Hello
According to the man page, there could be network ports with IBV_QPF_GRH_REQUIRED flag, which require setting is_global and grh header
https://man7.org/linux/man-pages/man3/ibv_modify_qp.3.html
Current logic in perftest_resources.c ctx_modify_qp_to_rtr don't check that IBV_QPF_GRH_REQUIRED flag and just sets
attr->ah_attr.is_global = 0;
On port with GRH requirement current perftest ib_write_bw has critical error
Failed to modify QP 122273 to RTR
Unable to Connect the HCA's through the link
Failed to modify QP 122272 to RTR
Unable to Connect the HCA's through the link
Similar issue was already solved in Mellanox/nccl-rdma-sharp-plugins@e68c599 patch with (portAttr.flags & IBV_QPF_GRH_REQUIRED) and env variable as alternative way to always enable GRH header.