Problem when max error <0.2

[kpannek]

Hi

Thanks for this tool!
I have a problem with I try to set the maximum error to less than 0.2. The decompressed streamlines are "zig-zagging", see screenshot (blue: original streamlines; red: after compression+decompression). This problem doesn't occur when I set the maximum error to 0.2 or larger.
I am using MRtrix3 to create my streamlines.

Kerstin

[jchoude]

Hi Kerstin,

thanks for your interest! First, let me say that this is highly unexpected... I'm routinely using small thresholds (from 0.0001 to 0.1) without any similar errors.

Could you tell me what were the calls you used to compress and decompress your streamlines? That would be a good first step in finding the problem.

[kpannek]

I used the following calls:
$ fibercompression cst.tck cst_maxerr01mm.zfib -e .1 -c
$ fibercompression cst_maxerr01mm.zfib cst_maxerr01mm.tck -d
The exact same calls work fine when I use -e .2. I also tried using the GUI, with the same problems for thresholds <.2

Thanks!

[jchoude]

Ok, everything seems fine with this.

I know this does not remove the potential problem, but, since you only seem to want to "linearize" the streamlines (since you compress and uncompress right after), you can also do that directly, by simply doing

$ fibercompression cst.tck cst_maxerr01mm.tck -e 0.1 -c

This way, you go from .tck to .tck, which is perfectly acceptable and will only run the linearization part of the algorithm, which is the part where the most space gains are made.

Could you try this? This would at least help us find where your problem would be located.

[kpannek]

$ fibercompression cst.tck cst_maxerr01mm.tck -e 0.1 -c
works fine

In practice I wouldn't decompress immediately after compressing, I would only decompress when I need to use those tracks again. I'd also want to compress whole brain tractograms with millions of streamlines, rather than the small example tck file I'm working with now. I'm just compressing and straight away decompressing at the moment while I'm fiddling around trying to work things out.

I noticed that all *.zfib files are the same size when I use thresholds <0.2, and the file size is smaller than it is for threshold=0.2mm.
cst_maxerr03mm.zfib: 239K
cst_maxerr02mm.zfib: 404K
cst_maxerr019mm.zfib: 279K
cst_maxerr01mm.zfib: 279K

[jchoude]

Ok. This seems to confirm that there is an issue only in the intermediate zfib format. I'll try to find time to investigate, but I will be on holiday for three weeks, so it might take some time.

Would you mind sharing your small example tck, to make sure I can exactly reproduce your issue when debugging?

As a side-note, in our day to day use, we always use the syntax that I showed you (from tck to tck). You need to be aware that, in fact, your output tck file is "compressed", even if it doesn't have the .zfib extension. After the publication of the original Fibercompression paper, we continued experimenting, and confirmed that most of the gains in space are obtained through the linearization process (removal of points that are almost collinear). When you run the command as I sent you, you are applying this linearization process, but still saving as a .tck file. If you look at your original file vs your new file, you will see that the size has shrunk.

So, in this use case, you are gaining space, and don't have the need to decompress when you want to use them later on.

However, I just want to make sure that you are aware that, when using compression on your streamlines, you will "lose" information (the same way you lose information when compressing an audio file to the mp3 format). So, in your future analyses, that needs to be taken into account.

[kpannek]

You can download the files from here:
https://www.dropbox.com/sh/r87tk5h7nqey8q4/AADFsN1fpJm3JezVCk2np88La?dl=0
The belong to HCP subject 100307, with diffusion processing as described here:
https://github.com/MRtrix3/mrtrix3/wiki/ISMRM2015-HCP-demonstration

If you do tck to tck, how would you then "uncompress"? I would like to still be able to sample FA/MD/whatever, but for that the streamline points should be (roughly) equally spaced.

[jchoude]

Please note that I'm currently on vacation, so it might take some time for my answers.

However, I can say quickly that, for your "uncompression" question, as soon as you set an error threshold that is different from 0, you will not be able to get directly get back your points after uncompression, since some of them are directly discarded. So, currently, there is no direct way to do this. However, there will soon be an implementation of my ISMRM 2015 abstract (which I'll try to link when I come back from vacation) in Dipy. I presented a method to overcome this kind of problem, and we are able to sample metrics along the streamlines even if the points are not equally spaced...