[RFC] Use libcramjam for compression codecs

[milesgranger]

(Moved from datafusion-orc repo: datafusion-contrib/datafusion-orc#136)

See if there was any interest in using libcramjam.

There's probably further use/simplification available here, and could anyway help libcramjam's implementation. It's presently used in python's cramjam package that has served quite well.

datafusion-orc could make use of other codecs as well with low overhead that are already part of libcramjam (brotli, bzip2, xz, etc and the ISA-L backed zlib/deflate/gzip algorithms which are quite a bit faster than flate2 w/ zlib-ng [1] (at the expense of more C code and reduced compression level options 0, 1, 3)

I was blissfully unaware there existed a MIT friendly LZO implementation so might add that to libcramjam as well going forward.

[1] https://github.com/milesgranger/isal-rs/tree/main?tab=readme-ov-file#benchmarks

[Jefffrey]

This does seem kinda interesting, though I do need some help understanding the main intention for this. It seems to serve as a thin wrapper over existing crates (please do correct me if I'm wrong 😅 ) that is intended to unify them under a common API, though that doesn't seem to be the case here? Or is that because this is more of a 1-to-1 swap to minimize the changes in this repo?

So we'd still have the original compression crates as transitive dependencies but that would be hidden by libcramjam?