Error java.nio.file.NoSuchFileException with mcool

[ianakim]

Hi, thanks a lot for the wonderful software!
It works really well on small genomes with hic file. However, I was trying to run it with mcool file using the following command:
java -Xmx10g -jar ${dir}/SIP_HiC_v1.6.1.jar cool ${mcool} ${CHROMSIZE_TEST} ${OUTDIR}/SIP_01 ${cooltoolsPath} ${coolerPath} -res 400 -t 2000 -fdr 0.1 -d 10 -mat 5000 -nbZero 6 -del true -cpu $CPU -g 3 -min 2 -max 2

And got the following error but no output files generated (only parameters.txt). Could you please help me to solve this issue?

cooltools, version 0.5.0

cooler, version 0.8.11

cool mode: 
input: path to mcool
output: path to /SIP_01
cooltools: ~/miniconda3/envs/cooltools_env/bin/cooltools
cooler: ~/.local/bin/cooler
norm: KR
gauss: 3.0
min: 2.0
max: 2.0
matrix size: 5000
diag size: 10
resolution: 400
saturated pixel: 0.01
threshold: 2000
number of zero :6
factor 1
fdr 0.1
del true
cpu 8
-isDroso false

start /home/loops/SIP_01/400kb.expected
Start cooltools compute-expected
java.nio.file.NoSuchFileException: /home/loops/SIP_01/400kb.expected
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
	at java.nio.file.Files.newByteChannel(Files.java:361)
	at java.nio.file.Files.newByteChannel(Files.java:407)
	at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
	at java.nio.file.Files.newInputStream(Files.java:152)
	at java.nio.file.Files.newBufferedReader(Files.java:2784)
	at utils.CoolerExpected.parseExpectedFile(CoolerExpected.java:90)
	at utils.CoolerExpected.dumpExpected(CoolerExpected.java:79)
	at multiProcesing.ProcessCoolerDumpData.go(ProcessCoolerDumpData.java:61)
	at sipMain.Hic_main.main(Hic_main.java:280)
!!!!!!! End /home/loops/SIP_01/400kb.expected
java.nio.file.NoSuchFileException: /home/loops/SIP_01/400kb.expected
	at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
	at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
	at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
	at java.nio.file.Files.newByteChannel(Files.java:361)
	at java.nio.file.Files.newByteChannel(Files.java:407)
	at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:384)
	at java.nio.file.Files.newInputStream(Files.java:152)
	at java.nio.file.Files.newBufferedReader(Files.java:2784)
	at utils.CoolerExpected.parseExpectedFile(CoolerExpected.java:90)
	at multiProcesing.RunnableDumpDataCooler.run(RunnableDumpDataCooler.java:71)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
##### End dump chr_1 400kb
Start loop detction step
/home/loops/SIP_01/400kb/chr_1/
###########End loop detction step
Exception in thread "pool-2-thread-1" java.lang.NullPointerException
	at multiProcesing.RunnableDetectLoops.run(RunnableDetectLoops.java:74)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
End of SIP loops are available in /home/loops/SIP_01

[PouletAxel]

Hi,
Thanks for using SIP,
Can you tell me which cool file are you using in input?
Are you sure that the resolution you want use is present in the mcool file?
Best
Axel

[ianakim]

Hi Axel,

I am using our in-home Micro-C dataset from a non-bilaterian animal (genome size ~200Mb). The file contains the following resolutions:

['/resolutions/200',
 '/resolutions/400',
 '/resolutions/800',
 '/resolutions/1600',
 '/resolutions/3200',
 '/resolutions/6400',
 '/resolutions/12800',
 '/resolutions/25600',
 '/resolutions/51200',
 '/resolutions/102400',
 '/resolutions/204800',
 '/resolutions/409600']

Can the problem be that SIP looks for KR normalization in cool file that was balanced with ICE?

cool mode: 
input: path to mcool
output: path to /SIP_01
cooltools: ~/miniconda3/envs/cooltools_env/bin/cooltools
cooler: ~/.local/bin/cooler
norm: KR

Best,
Iana

[PouletAxel]

Yes it is possible,
I will look at that I need to release a new version soon, so I will correct this problem in the same.
thanks for your patience
Best
Axel

[ianakim]

Thanks Axel!
Looking forward to the new version. Cool Stuff! :)

[xl5525]

I had the same problem yesterday but SIP worked for me a few months ago. And I solved the problem by using cooler=0.8.6, cooltools=0.3.2. I think it is more likely a compiling problem, but not the norm problem.

[ianakim]

thanks, xl5525!
using cooler=0.8.6, cooltools=0.3.2 solved it!!!

[PouletAxel]

I will change that for the next version.Sorry about that

…

On Thu, 28 Apr 2022 at 09:40, ianakim ***@***.***> wrote: thanks, xl5525! using cooler=0.8.6, cooltools=0.3.2 solved it!!! — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABS4UBLDIUONO3HY6VYEPITVHKINPANCNFSM5S4BICOQ> . You are receiving this because you commented.Message ID: ***@***.***>

[MarcoDiS]

using cooler=0.8.6, cooltools=0.3.2 worked for me too! Thanks!

[h3loiz]

Hi Axel,
I would like to try your nice program, but I'm running into the very same problem using my mcool file as input. So I wanted to know if you got a chance to fix it or if you know when this is going to be fixed? Or any other way for me to get around the problem...
Because I have cooltools, version 0.5.1 and cooler, version 0.8.11, and I'd rather not downgrade them because I'm using them for other purposes.
Thanks for your help !

[xl5525]

Hi Axel, I would like to try your nice program, but I'm running into the very same problem using my mcool file as input. So I wanted to know if you got a chance to fix it or if you know when this is going to be fixed? Or any other way for me to get around the problem... Because I have cooltools, version 0.5.1 and cooler, version 0.8.11, and I'd rather not downgrade them because I'm using them for other purposes. Thanks for your help !

Try to setup independent environments for different purposes.

conda create -n SIP cooler=0.8.6 cooltools=0.3.2
conda activate SIP

[h3loiz]

Thanks Axel, this seem to make it work indeed! :-)