[2/5] feat: cross-tokenizer collator, Arrow dataset, and eval datasets#2348

avenkateshha wants to merge 3 commits into

NVIDIA-NeMo:mainNVIDIA-NeMo/RL:mainfrom

avenkateshha:avenkateshha/xtoken-off-policy-distillation/02-collator-dataavenkateshha/RL:avenkateshha/xtoken-off-policy-distillation/02-collator-dataCopy head branch name to clipboard