ncdu-reader is a parser for binary exports generated by NCDU available since 2.6.
In order to parse a binary export generated by NCDU (i.e., ncdu -O export.bin), you can use the ncdu_reader.ncdu module.
from ncdu_reader import ncdu
export_path = "/tmp/export.bin"
root = ncdu.parse_ncdu(export_path)
For a simple file structure like this:
/tmp/ncdu-playground
└── foo
├── bar
│ ├── qux
│ └── quxx.bin
└── baz.bin
You will get this kind of output:
Parsed output
Directory(reference=AbsoluteReference(block_index=0, offset=126),
name='/tmp/ncdu-playground',
type=<ItemType.Directory: 0>,
prev=None,
asize=4096,
dsize=4096,
uid=None,
gid=None,
mode=None,
mtime=None,
dev=2096,
rderr=None,
cumasize=16384,
cumdsize=16384,
shrasize=0,
shrdsize=0,
items=5,
sub=RelativeReference(offset=-29),
children=[Directory(reference=AbsoluteReference(block_index=0,
offset=97),
name='foo',
type=<ItemType.Directory: 0>,
prev=None,
asize=4096,
dsize=4096,
uid=None,
gid=None,
mode=None,
mtime=None,
dev=0,
rderr=None,
cumasize=12288,
cumdsize=12288,
shrasize=0,
shrdsize=0,
items=4,
sub=RelativeReference(offset=-20),
children=[File(reference=AbsoluteReference(block_index=0,
offset=77),
name='baz.bin',
type=<ItemType.File: 1>,
prev=RelativeReference(offset=-29),
asize=0,
dsize=0,
uid=None,
gid=None,
mode=None,
mtime=None),
Directory(reference=AbsoluteReference(block_index=0,
offset=48),
name='bar',
type=<ItemType.Directory: 0>,
prev=None,
asize=4096,
dsize=4096,
uid=None,
gid=None,
mode=None,
mtime=None,
dev=0,
rderr=None,
cumasize=8192,
cumdsize=8192,
shrasize=0,
shrdsize=0,
items=2,
sub=RelativeReference(offset=-21),
children=[File(reference=AbsoluteReference(block_index=0,
offset=27),
name='quxx.bin',
type=<ItemType.File: 1>,
prev=RelativeReference(offset=-27),
asize=0,
dsize=0,
uid=None,
gid=None,
mode=None,
mtime=None),
Directory(reference=AbsoluteReference(block_index=0,
offset=0),
name='qux',
type=<ItemType.Directory: 0>,
prev=None,
asize=4096,
dsize=4096,
uid=None,
gid=None,
mode=None,
mtime=None,
dev=0,
rderr=None,
cumasize=4096,
cumdsize=4096,
shrasize=0,
shrdsize=0,
items=0,
sub=None,
children=[])])])])
For more information about what the different fields mean, refer to the official documentation.
ncdu-reader does not support lazily reading exports (e.g., parsing the root at first and then parsing children when requested). This means that the entire tree needs to be parsed at once. This can be problematic for very large exports.
It is however possible to only parse down to a certain depth, e.g. ncdu.parse_ncdu(export_path, depth=4). If all you care about is getting information about the top-level item (i.e., the root of your export), pass a depth of 0.
Here is a very unscientific benchmark of parsing exports for varying tree sizes to give an idea of scale, not expected performance. This includes the startup time for the Python interpreter, hence why small exports take around the same time.
| Tree Size | Time (mean ± σ) |
|---|---|
| 5 | 93.0 ms ± 1.6 ms |
| 12,605 | 95.1 ms ± 3.0 ms |
| 1,064,009 | 7.708 s ± 0.114 s |
| 7,905,281 | 87.761 s ± 0.710 s |
Note that RAM availability greatly impacts parsing of larger exports.
ncdu-reader is distributed under the terms of the MIT license.