Storage Layout

HDF5 offers three storage layouts:

• H5D_COMPACT: For scalar or small datasets which can fit in the metadata header. Default on datasets smaller than 32 KB.
• H5D_CONTIGUOUS: For medium size datasets. Default on datasets smaller than 512 KB.
• H5D_CHUNKED: For large datasets. Default on datasets larger than 512 KB. This layout has some additional features:
  • Chunking, portioning of the data to improve IO performance by caching more efficiently. Chunk dimensions are calculated by h5pp if not given by the user.
  • Compression, disabled by default, and only available if HDF5 was built with zlib enabled.
  • Resize datasets. Note that the file size never decreases, for instance after overwriting with a smaller dataset.

h5pp can automatically determine the storage layout for each new dataset. To specify the layout manually, pass it as a third argument when writing a new dataset, for instance:

    file.writeDataset(myData, "science/myChunkedData", H5D_CHUNKED);      // Creates a chunked dataset