Skip to content

Confusing (wrong? buggy?) behavior of sample ranges with chained trees when using dataframes #22741

Description

@acampove

Check duplicate issues.

  • Checked for duplicates

Description

When using dataframes with a set of files, I see different ways

rdfsampleinfo_.EntryRange();

gets computed. This range is either:

Local: From zero to the number of entries in the file containing the entry at which we are.
Global: From the first entry in the file where we are until the last entry in that file, counted as part of the chain.

I get local ranges when using a dataframe constructor. When using FromSpec to use friend trees AND multithreading, I get global ranges. Without multithreading I again get local ranges.

Reproducer

import ROOT

N_FILES          = 4
ENTRIES_PER_FILE = [1000, 1000, 1000, 1000]  # deliberately uneven sizes
FILES            = [f"repro_file_{i}.root" for i in range(N_FILES)]
TREE             = "Events"

def make_files():
    for path, n in zip(FILES, ENTRIES_PER_FILE):
        df = ROOT.RDataFrame(n).Define("x", "rdfentry_")
        df.Snapshot(TREE, path, ["x"])

def run():
    definition = r'''
    auto range = rdfsampleinfo_.EntryRange();
    auto start = std::get<0>(range);
    auto   end = std::get<1>(range);

    std::cout << start << "  " << end << "\n";

    return 3;
    '''

    df = ROOT.RDataFrame(TREE, FILES)
    df = df.DefinePerSample("val", definition)
    df.Count().GetValue()

if __name__ == "__main__":
    make_files()
    run()
    print('---------------')
    ROOT.EnableImplicitMT(2)
    run()

That produces:

0  1000
0  1000
0  1000
0  1000
---------------
0  1000
0  1000
0  1000
0  1000

i.e. with and without multithreading we get the local range.

However:

import ROOT
from ROOT    import RDF # type: ignore
from pathlib import Path

PATH = Path('files')
PATH.mkdir(exist_ok = True)
# -------------------------------------
def _create_files(index : int) -> None:
    main     = PATH / f'main_{index:03}.root'
    friend   = PATH / f'frnd_{index:03}.root'
    if main.exists() and friend.exists():
        return

    df       = ROOT.RDataFrame(1000)
    df.Define('x', f'{index}').Snapshot('Events', str(main))

    df       = ROOT.RDataFrame(1000)
    df.Define('y', f'{index}').Snapshot('Events', str(friend))
# -------------------------------------
def _load_rdf():
    PATH.mkdir(parents=True, exist_ok=True)
    names = dict()
    names['files/main_001.root'] = 1
    names['files/main_002.root'] = 2
    names['files/main_003.root'] = 3
    names['files/main_004.root'] = 4

    definition = r'''
    auto range = rdfsampleinfo_.EntryRange();
    auto start = std::get<0>(range);
    auto   end = std::get<1>(range);

    std::cout << start << "  " << end << "\n";

    return 3;
    '''

    rdf = RDF.Experimental.FromSpec('spec.json')
    rdf = rdf.DefinePerSample('file_id', definition)
    rdf.Count().GetValue()
# -------------------------------------
for ifile in range(4):
    _create_files(index = ifile)

_load_rdf()
ROOT.EnableImplicitMT(4) # type: ignore
print('-----------------')
_load_rdf()

produces

0  1000
0  1000
0  1000
0  1000
-----------------
0  1000
3000  4000
2000  3000
1000  2000

i.e. multithreading (even with one core) gives the global range. The specification file needed to load the trees is:

spec_maker.py

This makes no sense and must be a bug somewhere.

ROOT version

ROOT Version: 6.38.00
Built for linuxx8664gcc on Feb 19 2026, 03:23:03
From tags/6-38-00@6-38-00

Installation method

mamba

Operating system

almalinux 8

Additional context

Are the global ranges meant to belong to a specific file? i.e. Will the global ranges go across file boundaries? From tests it seems that they are meant to be contained to files. I also see that these boundaries do not agree always with the file but they can be file subsets.

I need this to use:

    rdf = rdf.DefinePerSample('file_id', definition)

Where it is not clear if sample refers to files.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions