Does the reasoning stage require the most space?

[chongchongmao]

Does the inference stage appear to require substantial memory for loading the model? And how much VRAM is needed? The paper mentions that training and testing require 48VRAM – is this the maximum amount utilized during the test/inference phase?

[quqxui]

It depends on the batch size and the number of beams used in beam search. With the settings in our code, it typically requires around 40–48 GB of VRAM.