Clarification regarding DIANN>2

[josenimo]

For running DIANN>2 in the HPC with quantmsdiann I must create my own singularity-compatible containers, and link them appropiately via .config files right?

To do this I would:

1. Download DIANN releases for Linux
2. Create Docker container for amd64/linux
3. Convert with singularity to .sif
4. Place somewhere I know in HPC env
5. Create personalized .config file that changes the paths of the different versions to the local paths

Any other considerations?
I am rushing this too much? Should I wait for some developments before running the different DIANN versions?
I am very excited for these developments, I am happy to provide feedback from our lab :)

Saludos,
Jose

[enryH]

Indeed. You will need to fork https://github.com/bigbio/quantms-containers and when the action runs it should create private containers on ghcr (github container registry). These you can then use as in the dia_latest test configuration.

There is a plan to document that but I am not sure where this should go at the moment.

[josenimo]

Thank you @enryH for the quick response.
I must say I still don't understand exactly.

I had never heard from ghcr so here is my attempt at making it work.

(1) Fork repo
(2) clone repo locally
(3) Modify .github/workflows/quantms-containers.yml:
(3.1) change log in credentials to:

username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

#Also for these lines, occurs 3x 
export SINGULARITY_DOCKER_USERNAME=${{ github.actor }} 
export SINGULARITY_DOCKER_PASSWORD=${{ secrets.GITHUB_TOKEN }} 

(3.2) replace all instances of bigbio to <GITHUB-USERNAME>, in my case josenimo.
(4) push changes to fork, let CI do its thing

(5) check packages in https://github.com/<GITHUB-USERNAME>?tab=packages; they are separated into diann-sif (singularity) and diann for docker image. These are not private (by default). @enryH should they be private?

[enryH]

So, the fork will be public as with ours, but the images be private. You do not need to do anything since the refactor started by @Skitionek - these variables are all tied to your github user. But I am not sure if you might need an organization account... you will test it:)

See: bigbio/quantms-containers#5

This needs to be merged first...

[enryH]

Our fork is this one, just for you to see it (with a slightly changed main branch).

https://github.com/biosustain/quantms-containers

So if you need it right now, just fork our fork.

[josenimo]

• I understand the fork will be public ✅
• Images (github calls them Packages) were not public by default, I had to actively switch them to be Private ❌
• Ah ok, I forked Commit 4357944 looking forward to those updates :)

Currently I managed to upload the containers to my account, make them private, create a Token that only has read:packages permissions to use in HPC; and this worked

[josenimo]

regarding downsteam usage, I have all my .sif files in my cache folder in the HPC.

I assume that overriding default config, with my own .config file will do the trick, something like this:

singularity {
    enabled = true
    autoMounts = true
    cacheDir = '/path/to/shared/group/data/singularity_cache' 
}
profiles {
    diann_v2_2_0 {
        process.container = '/path/to/shared/group/data/singularity_cache/diann-2.2.0.sif'
    }
     diann_v2_3_2 {
        process.container = '/path/to/shared/group/data/singularity_cache/diann-2.3.2.sif'
    }
}

and then we ensure we use the config file:

nextflow run bigbio/quantmsdiann \
    -c /path/to/shared/group/data/workgroup.config \
    --input 'experiment.sdrf.tsv' \
    --database 'proteins.fasta' \
    --outdir './results' \
    -profile singularity,diann_v2_2_0

Thank you for the sanity checks :D

[enryH]

If there is now only one container, I guess this looks correct.

But in your local config you can do similar things as here:

quantmsdiann/conf/tests/test_latest_dia.config

Lines 35 to 50 in baef81d

	process {
	// thermorawfileparser
	withName: 'BIGBIO_QUANTMSDIANN:QUANTMSDIANN:FILE_PREPARATION:THERMORAWFILEPARSER' {
	publishDir = [path: { "${params.outdir}/${task.process.tokenize(':')[-1].toLowerCase()}" }, pattern: "*.log" ]
	}
	withLabel: diann {
	container = 'ghcr.io/bigbio/diann:2.2.0' // This docker container is private for quantmsdiann
	}
	resourceLimits = [
	cpus: 4,
	memory: '12.GB',
	time: '48.h'
	]

[ypriverol]

Hi @josenimo and @enryH I have added a new page in the documentation https://quantmsdiann.quantms.org/containers/ and in the usage for release 2.0.0 I hope this helps future users.

Thanks a lot for using quantmsdiann.

[enryH]

From where is this website deployed? Is there a separate repo for the docs?

[enryH]

I still cannot see where the docs are for the containers page:

https://github.com/search?q=repo%3Abigbio%2Fquantmsdiann+Building+and+Using+DIA-NN+Containers&type=code

Do you have it on an orphan branch?