add per-sample logic for FASTA

README.md

@@ -21,47 +21,47 @@
 
 ## Introduction
 
-**nf-core/mspepid** is a bioinformatics pipeline that ...
-
-<!-- TODO nf-core:
-   Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the
-   major pipeline sections and the types of output it produces. You're giving an overview to someone new
-   to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction
--->
-
-<!-- TODO nf-core: Include a figure that guides the user through the major workflow steps. Many nf-core
-     workflows use the "tube map" design for that. See https://nf-co.re/docs/community/brand/workflow-schematics#examples for examples.   -->
-<!-- TODO nf-core: Fill in short bullet-pointed list of the default steps in the pipeline -->
+**nf-core/mspepid** is a bioinformatics pipeline for the identification of peptides in mass spectrometry (MS)-based proteomics and peptidomics data. It takes a samplesheet with MS data files (Thermo `.raw`, Bruker TimsTOF `.d`, or open-standard `.mzML`) and a protein sequence database (FASTA) as input, runs one or more configurable database search engines, and performs post-processing with machine learning-based rescoring to produce high-confidence, FDR-filtered peptide-spectrum matches (PSMs).
+
+The pipeline is designed as a modular, identification-focused building block that can in the future feed into downstream quantification or statistical pipelines such as [nf-core/quantms](https://nf-co.re/quantms) and [nf-core/msproteomics](https://nf-co.re/msproteomics).
+
+1. Optional: Create entrapment database for FDR benchmarking ([`FDRBench`](https://github.com/percolator/fdrBench))
+2. Generate decoy sequences ([`OpenMS DecoyDatabase`](https://www.openms.de/)) — skipped with `--skip_decoy_generation` if decoys are already present
+3. Convert vendor spectrum formats to mzML:
+   - [`ThermoRawFileParser`](https://github.com/compomics/ThermoRawFileParser) for Thermo `.raw` files
+   - [`tdf2mzml`](https://github.com/theGreatHerrLebert/tdf2mzml) for Bruker `.d` folders
+4. Peptide-spectrum match (PSM) identification with one or more database search engines (more to come):
+   - [`Comet`](https://uwpr.github.io/Comet/)
+   - [`Sage`](https://github.com/lazear/sage)
+5. Convert search engine results to a common PSM format ([`psm-utils`](https://psm-utils.readthedocs.io/))
+6. PSM rescoring (more to come):
+   - [`Percolator`](https://github.com/percolator/percolator) - semi-supervised machine learning for PSM re-ranking and FDR estimation
+   - [`MS2Rescore`](https://ms2rescore.readthedocs.io/) - MS2PIP spectral prediction-based feature generation followed by Percolator
 
 ## Usage
 
 > [!NOTE]
 > If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/get_started/environment_setup/overview) on how to set-up Nextflow. Make sure to [test your setup](https://nf-co.re/docs/get_started/run-your-first-pipeline) with `-profile test` before running the workflow on actual data.
 
-<!-- TODO nf-core: Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.
-     Explain what rows and columns represent. For instance (please edit as appropriate):
-
 First, prepare a samplesheet with your input data that looks as follows:
 
 `samplesheet.csv`:
 
 ```csv
-sample,fastq_1,fastq_2
-CONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz
+ID,spectrum_file,fasta
+1,/path/to/run1.raw,/path/to/proteins.fasta
+2,/path/to/run2.mzML,/path/to/proteins.fasta
 ```
 
-Each row represents a fastq file (single-end) or a pair of fastq files (paired end).
-
--->
+Each row represents one MS run. The `ID` column must be a unique integer. The `spectrum_file` column accepts Thermo `.raw`, Bruker TimsTOF `.d`, and `.mzML` files (with optional `.gz`, `.zip`, or `.tar.gz` compression). The `fasta` column is optional per row — use `--fasta` to apply a single protein database to all runs instead.
 
 Now, you can run the pipeline using:
 
-<!-- TODO nf-core: update the following command to include all required parameters for a minimal example -->
-
 ```bash
 nextflow run nf-core/mspepid \
    -profile <docker/singularity/.../institute> \
    --input samplesheet.csv \
+   --fasta proteins.fasta \
    --outdir <OUTDIR>
 ```
 
@@ -76,9 +76,12 @@ To see the results of an example test run with a full size dataset refer to the
 For more details about the output files and reports, please refer to the
 [output documentation](https://nf-co.re/mspepid/output).
 
+The main outputs are Percolator-scored PSM files (`.pout`) organised by search engine and rescoring strategy.
+For each enabled search engine and rescoring method, results are written to `<outdir>/<searchengine>/` and respective subfolders `<outdir>/<searchengine>/<rescoring>/`.
+
 ## Credits
 
-nf-core/mspepid was originally written by Julian Uszkoreit.
+nf-core/mspepid was originally written by the team of the Medical Bioinformatics, Ruhr University Bochum, ([@medbioinf](https://github.com/medbioinf)).
 
 We thank the following people for their extensive assistance in the development of this pipeline:
 

assets/samplesheet.csv

@@ -1,3 +1,3 @@
-sample,fastq_1,fastq_2
-SAMPLE_PAIRED_END,/path/to/fastq/files/AEG588A1_S1_L002_R1_001.fastq.gz,/path/to/fastq/files/AEG588A1_S1_L002_R2_001.fastq.gz
-SAMPLE_SINGLE_END,/path/to/fastq/files/AEG588A4_S4_L003_R1_001.fastq.gz,
+ID,spectrum_file,fasta
+1,/path/to/run1.raw,/path/to/proteins.fasta
+2,/path/to/run2.mzML,/path/to/proteins.fasta

assets/schema_input.json

@@ -18,6 +18,13 @@
                 "exists": true,
                 "pattern": "^\\S+\\.(raw|RAW|mzML|mzml|d|)(.tar.(gz|bz2)|.zip|.gz)?$",
                 "errorMessage": "MS file name, cannot contain spaces and must have one of the extensions: raw | RAW | mzML | d"
+            },
+            "fasta": {
+                "type": "string",
+                "format": "path",
+                "exists": true,
+                "pattern": "^\\S+\\.(fasta|fa|fas|faa)(\\.gz)?$",
+                "errorMessage": "FASTA file cannot contain spaces and must have one of the extensions: fasta | fa | fas | faa"
             }
         },
         "required": ["ID", "spectrum_file"]

docs/output.md

@@ -2,17 +2,104 @@
 
 ## Introduction
 
-This document describes the output produced by the pipeline.
+This document describes the output produced by the pipeline. All paths are relative to the top-level results directory specified with `--outdir`.
 
-The directories listed below will be created in the results directory after the pipeline has finished. All paths are relative to the top-level results directory.
+## Pipeline overview
 
-<!-- TODO nf-core: Write this documentation describing your workflow's output -->
+nf-core/mspepid processes mass spectrometry data through the following steps:
 
-## Pipeline overview
+1. **Database preparation** - optional entrapment database creation and decoy sequence generation
+2. **Spectra preparation** - decompression and vendor-format conversion to mzML
+3. **Spectra identification** - database search with Comet and/or Sage
+4. **Rescoring** - FDR-based PSM rescoring with Percolator and/or MS2Rescore
+
+---
+
+## Database preparation
+
+### Decoy generation
+
+Decoy protein sequences are automatically appended to the target database unless `--skip_decoy_generation` is set. Decoys are generated by the [OpenMS DecoyDatabase](https://www.openms.de/) tool using sequence reversal as default strategy.
+
+### Entrapment database
+
+When `--entrapment_fold` is greater than 0, an entrapment database is created by [FDRBench](https://github.com/percolator/fdrBench) prior to decoy generation. Entrapment sequences (shuffled target proteins) are used for independent FDR estimation during benchmarking.
+
+---
+
+## Spectra identification
+
+For each enabled search engine, PSM results are written under a subdirectory named after the search engine (e.g. `comet/` or `sage/`).
+All results are converted to a common format with [psm-utils](https://psm-utils.readthedocs.io/) for downstream processing.
+
+### Comet
+
+<details markdown="1">
+<summary>Output files</summary>
+
+- `comet/`
+  - `*.mzid` - PSMs in mzIdentML standard format (default output of Comet)
+  - `*.comet.params` - the final input parameters of the Comet run.
+  - `*.psmutils.tsv` - PSMs in psm-utils TSV format, used as input to rescoring steps.
+  - `*.pin` - Percolator input file (PIN format) containing search engine features.
+  - Additional outputs can be activated, see module description of Comet
+
+</details>
+
+[Comet](https://uwpr.github.io/Comet/) is a widely used open-source database search engine for tandem mass spectra.
+
+### Sage
+
+<details markdown="1">
+<summary>Output files</summary>
+
+- `sage/`
+  - `*.results.sage.tsv` - PSMs in Sage's default TSV output format.
+  - `*.results.json` - the final input parameters of the Sage run.
+  - `*.psmutils.tsv` - PSMs in psm-utils TSV format, used as input to rescoring steps.
+  - `*.pin` - Percolator input file (PIN format) containing search engine features.
+
+</details>
+
+[Sage](https://github.com/lazear/sage) is a fast, Rust-based database search engine that scales efficiently to large datasets and large protein databases.
+
+---
+
+## Rescoring
+
+PSMs from each search engine are rescored independently. Output directories are nested under the search engine directory.
+
+### Percolator
+
+<details markdown="1">
+<summary>Output files</summary>
+
+- `<searchengine>/percolator/`
+  - `*.target.psms` - Target PSMs scored and filtered by Percolator.
+  - `*.decoy.psms` - Decoy PSMs (used for FDR calibration).
+  - Additional outputs can be activated, see module description of Percolator
+
+</details>
+
+[Percolator](https://github.com/percolator/percolator) applies semi-supervised machine learning (using a SVM) to re-rank PSMs using search engine scores and auxiliary features, then estimates FDR using a target-decoy competition approach.
+
+### MS2Rescore / Tims2Rescore
+
+<details markdown="1">
+<summary>Output files</summary>
+
+- `<searchengine>/ms2rescore/`
+  - `*.target.psms` - Target PSMs rescored after MS2PIP feature augmentation.
+  - `*.decoy.psms` - Decoy PSMs.
+  - `*.pin`- The original PSMs after identification with additional features created my MS2PIP
+
+</details>
+
+[MS2Rescore](https://ms2rescore.readthedocs.io/) generates additional rescoring features by comparing observed fragment ion spectra to spectra predicted by [MS2PIP](https://ms2pip.readthedocs.io/). The augmented feature set is then passed to Percolator for final scoring. This typically improves PSM identifications at a given FDR threshold, particularly for challenging samples such as immunopeptidomes or non-tryptic digests. I TIMS data is used as input, automatically Tims2Rescore is applied (which is the default behaviour of newer MS2Rescore implementations).
 
-The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes data using the following steps:
+The MS2PIP fragmentation model is controlled by `--ms2rescore_model` (default: `HCD`).
 
-- [Pipeline information](#pipeline-information) - Report metrics generated during the workflow execution
+---
 
 ### Pipeline information
 
@@ -21,9 +108,10 @@ The pipeline is built using [Nextflow](https://www.nextflow.io/) and processes d
 
 - `pipeline_info/`
   - Reports generated by Nextflow: `execution_report.html`, `execution_timeline.html`, `execution_trace.txt` and `pipeline_dag.dot`/`pipeline_dag.svg`.
-  - Reports generated by the pipeline: `pipeline_report.html`, `pipeline_report.txt` and `software_versions.yml`. The `pipeline_report*` files will only be present if the `--email` / `--email_on_fail` parameter's are used when running the pipeline.
+  - Reports generated by the pipeline: `pipeline_report.html`, `pipeline_report.txt` and `software_versions.yml`. The `pipeline_report*` files will only be present if the `--email` / `--email_on_fail` parameters are used when running the pipeline.
   - Reformatted samplesheet files used as input to the pipeline: `samplesheet.valid.csv`.
   - Parameters used by the pipeline run: `params.json`.
+  - Software versions used: `nf_core_mspepid_software_versions.yml`.
 
 </details>
 

docs/usage.md

@@ -6,61 +6,70 @@
 
 ## Introduction
 
-<!-- TODO nf-core: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website. -->
+nf-core/mspepid identifies peptides from mass spectrometry data by running one or more database search engines against a protein sequence database and applying machine learning-based rescoring to maximise PSM confidence and identification level.
+
+This page describes how to prepare input data, configure search engines and rescoring tools, and run the pipeline.
 
 ## Samplesheet input
 
-You will need to create a samplesheet with information about the samples you would like to analyse before running the pipeline. Use this parameter to specify its location. It has to be a comma-separated file with 3 columns, and a header row as shown in the examples below.
+You will need to create a samplesheet with information about the MS runs you would like to analyse before running the pipeline. Use this parameter to specify its location. It has to be a comma-separated file with at least two columns and a header row as shown in the examples below.
 
 ```bash
 --input '[path to samplesheet file]'
 ```
 
-### Multiple runs of the same sample
+### Samplesheet columns
+
+| Column          | Required | Description                                                                                                                                                     |
+| --------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `ID`            | Yes      | A unique integer identifier for each MS run.                                                                                                                    |
+| `spectrum_file` | Yes      | Path to the MS data file. Supported formats: Thermo `.raw`, Bruker TimsTOF `.d`, `.mzML`. Files may be compressed with `.gz`, `.zip`, `.tar.gz`, or `.tar.bz2`. |
+| `fasta`         | No       | Path to a per-run protein sequence database (FASTA). Mutually exclusive with the global `--fasta` parameter.                                                    |
+
+### Example samplesheet — global FASTA via `--fasta`
 
-The `sample` identifiers have to be the same when you have re-sequenced the same sample more than once e.g. to increase sequencing depth. The pipeline will concatenate the raw reads before performing any downstream analysis. Below is an example for the same sample sequenced across 3 lanes:
+When all runs share the same protein database, provide it with `--fasta` and omit the `fasta` column:
 
 ```csv title="samplesheet.csv"
-sample,fastq_1,fastq_2
-CONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz
-CONTROL_REP1,AEG588A1_S1_L003_R1_001.fastq.gz,AEG588A1_S1_L003_R2_001.fastq.gz
-CONTROL_REP1,AEG588A1_S1_L004_R1_001.fastq.gz,AEG588A1_S1_L004_R2_001.fastq.gz
+ID,spectrum_file
+1,/path/to/run1.raw
+2,/path/to/run2.mzML
+3,/path/to/run3.d.tar.gz
 ```
 
-### Full samplesheet
+```bash
+nextflow run nf-core/mspepid \
+    --input samplesheet.csv \
+    --fasta /path/to/proteins.fasta \
+    --outdir ./results \
+    -profile docker
+```
 
-The pipeline will auto-detect whether a sample is single- or paired-end using the information provided in the samplesheet. The samplesheet can have as many columns as you desire, however, there is a strict requirement for the first 3 columns to match those defined in the table below.
+### Example samplesheet — per-run FASTA
 
-A final samplesheet file consisting of both single- and paired-end data may look something like the one below. This is for 6 samples, where `TREATMENT_REP3` has been sequenced twice.
+To use a different protein database for each run, add the `fasta` column. This is useful for e.g. proteogenoomics or other sample-specific databases:
 
 ```csv title="samplesheet.csv"
-sample,fastq_1,fastq_2
-CONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz
-CONTROL_REP2,AEG588A2_S2_L002_R1_001.fastq.gz,AEG588A2_S2_L002_R2_001.fastq.gz
-CONTROL_REP3,AEG588A3_S3_L002_R1_001.fastq.gz,AEG588A3_S3_L002_R2_001.fastq.gz
-TREATMENT_REP1,AEG588A4_S4_L003_R1_001.fastq.gz,
-TREATMENT_REP2,AEG588A5_S5_L003_R1_001.fastq.gz,
-TREATMENT_REP3,AEG588A6_S6_L003_R1_001.fastq.gz,
-TREATMENT_REP3,AEG588A6_S6_L004_R1_001.fastq.gz,
+ID,spectrum_file,fasta
+1,/path/to/run1.raw,/path/to/sample1_proteins.fasta
+2,/path/to/run2.mzML,/path/to/sample2_proteins.fasta
 ```
 
-| Column    | Description                                                                                                                                                                            |
-| --------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `sample`  | Custom sample name. This entry will be identical for multiple sequencing libraries/runs from the same sample. Spaces in sample names are automatically converted to underscores (`_`). |
-| `fastq_1` | Full path to FastQ file for Illumina short reads 1. File has to be gzipped and have the extension ".fastq.gz" or ".fq.gz".                                                             |
-| `fastq_2` | Full path to FastQ file for Illumina short reads 2. File has to be gzipped and have the extension ".fastq.gz" or ".fq.gz".                                                             |
-
 An [example samplesheet](../assets/samplesheet.csv) has been provided with the pipeline.
 
 ## Running the pipeline
 
 The typical command for running the pipeline is as follows:
 
 ```bash
-nextflow run nf-core/mspepid --input ./samplesheet.csv --outdir ./results  -profile docker
+nextflow run nf-core/mspepid \
+    --input ./samplesheet.csv \
+    --fasta ./proteins.fasta \
+    --outdir ./results \
+    -profile docker
 ```
 
-This will launch the pipeline with the `docker` configuration profile. See below for more information about profiles.
+This will launch the pipeline with the `docker` configuration profile. All supported search engines and rescoring methods are enabled by default. See parameters for how to customise this.
 
 Note that the pipeline will create the following files in your working directory:
 

main.nf

@@ -30,7 +30,7 @@ include { PIPELINE_COMPLETION     } from './subworkflows/local/utils_nfcore_mspe
 workflow NFCORE_MSPEPID {
 
     take:
-    samplesheet // channel: samplesheet read in from --input
+    samplesheet // channel: [meta, spectrum_file, fasta_file] read in from --input
 
     main:
 
@@ -40,7 +40,6 @@ workflow NFCORE_MSPEPID {
     MSPEPID(
         samplesheet,
         params.outdir,
-        params.fasta,
         params.entrapment_fold,
         params.skip_decoy_generation,
         params.precursor_tol_ppm,

nextflow_schema.json

@@ -47,9 +47,9 @@
                 "fasta": {
                     "type": "string",
                     "fa_icon": "fas fa-file",
-                    "pattern": ".(fasta|faa)$",
-                    "description": "Input FASTA protein database",
-                    "help_text": "Path to the protein database file. Whether it already contains decoys is set by the --skip_decoy_generation parameter."
+                    "pattern": "^\\S+\\.(fasta|fa|fas|faa)(.gz)?$",
+                    "description": "Global input FASTA protein database (mutually exclusive with per-run 'fasta' column in the samplesheet)",
+                    "help_text": "Path to a single protein database file applied to all MS runs. Cannot be used together with the 'fasta' column in the samplesheet. Whether the file already contains decoys is controlled by --skip_decoy_generation."
                 },
                 "entrapment_fold": {
                     "type": "integer",