In pop_request()
if len(group.requests) == 0:
vq.pop_group()
When a group becomes empty, it will be removed from the virtual queue, however, model_slo_group_bimap still keeps the mapping and points to that same (now detached) Group.
In add_request()
if (request.model, request.slo) in self.model_slo_group_bimap:
existing_group = self.model_slo_group_bimap[(request.model, request.slo)]
existing_group.add_request(request)
self.request_to_group[request] = existing_group
Subsequent requests for the same (model, slo) are added to this detached group, which is no longer enqueued in any virtual queue, so they are never dispatched.
In
pop_request()if len(group.requests) == 0:vq.pop_group()When a group becomes empty, it will be removed from the virtual queue, however,
model_slo_group_bimapstill keeps the mapping and points to that same (now detached)Group.In
add_request()if (request.model, request.slo) in self.model_slo_group_bimap:existing_group = self.model_slo_group_bimap[(request.model, request.slo)]existing_group.add_request(request)self.request_to_group[request] = existing_groupSubsequent requests for the same (model, slo) are added to this detached group, which is no longer enqueued in any virtual queue, so they are never dispatched.