Skip to content

Feature/studysubmit#36

Merged
ochkalova merged 38 commits intodevfrom
feature/studysubmit
Mar 25, 2026
Merged

Feature/studysubmit#36
ochkalova merged 38 commits intodevfrom
feature/studysubmit

Conversation

@timrozday-mgnify
Copy link
Copy Markdown
Contributor

@timrozday-mgnify timrozday-mgnify commented Mar 12, 2026

Replace registerstudy with new python script that is general to all ENA studies.

For issue: #30

Update from @ochkalova
What was done in this PR:

  • added script submit_study.py and corresponding module registerstudy
  • tests and test profiles for registerstudy module
  • added input parameter --study_metadata, schema updated
  • removed params library and ena_raw_reads_study_accession used by previous version of registerstudy
  • added pipeline level tests for modes mags and metagenomic_assemblies that register study
  • assemblysubmit and genomesubmit updated to use registerstudy
  • updated docs

Minor changes:

TODOs:

  • add checks and validation for required fields in webin receipt (accession should always present)
  • might be worth adding --hold-until to pipeline params

@nf-core-bot
Copy link
Copy Markdown
Member

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.5.1.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 12, 2026

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 55373f8

+| ✅ 210 tests passed       |+
#| ❔   7 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗  22 tests had warnings |!
Details

❗ Test warnings:

  • readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
  • pipeline_todos - TODO string in README.md: Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file.
  • pipeline_todos - TODO string in README.md: Add bibliography of tools and data used in your pipeline
  • pipeline_todos - TODO string in nextflow.config: Optionally, you can add a pipeline-specific nf-core config at https://github.com/nf-core/configs
  • pipeline_todos - TODO string in nextflow.config: Update the field with the details of the contributors to your pipeline. New with Nextflow version 24.10.0
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
  • pipeline_todos - TODO string in test_assembly.config: Specify the paths to your test data on nf-core/test-datasets
  • pipeline_todos - TODO string in test_assembly.config: Give any required params for the test so that command line flags are not needed
  • pipeline_todos - TODO string in test_genome.config: Specify the paths to your test data on nf-core/test-datasets
  • pipeline_todos - TODO string in test_genome.config: Give any required params for the test so that command line flags are not needed
  • pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
  • pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
  • pipeline_todos - TODO string in output.md: Write this documentation describing your workflow's output
  • local_component_structure - rna_detection.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - genome_evaluation.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

  • files_exist - File is ignored: conf/igenomes.config
  • files_exist - File is ignored: conf/igenomes_ignored.config
  • nextflow_config - Config variable ignored: params.input
  • files_unchanged - File ignored due to lint config: .github/PULL_REQUEST_TEMPLATE.md
  • files_unchanged - File ignored due to lint config: assets/nf-core-seqsubmit_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-seqsubmit_logo_light.png
  • files_unchanged - File ignored due to lint config: docs/images/nf-core-seqsubmit_logo_dark.png

❔ Tests fixed:

✅ Tests passed:

Run details

  • nf-core/tools version 3.5.1
  • Run at 2026-03-25 14:17:08

Copy link
Copy Markdown
Collaborator

@vagkaratzas vagkaratzas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left a round of quick-through comments

Comment thread bin/ena_submit_common.py Outdated
Comment thread modules/local/ena_webin_cli/main.nf Outdated
Comment thread modules/local/registerstudy/tests/main.nf.test Outdated
Comment thread tests/default.nf.test Outdated
Comment thread .gitignore Outdated
Comment thread nextflow.config Outdated
Comment thread bin/ena_submit_common.py Outdated
Comment thread bin/ena_submit_common.py Outdated
Comment thread bin/ena_submit_common.py Outdated
Comment thread bin/ena_submit_common.py Outdated
Comment thread bin/ena_submit_common.py Outdated
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py
Comment thread bin/submit_study.py
Comment thread bin/submit_study.py
Comment thread bin/submit_study.py
Comment thread assets/test-fixtures/example_study.csv Outdated
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py Outdated
Comment thread bin/ena_submit_common.py Outdated
Comment thread bin/submit_study.py
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py Outdated
Comment thread modules/local/registerstudy/meta.yml
Comment thread bin/submit_study.py Outdated
Comment thread bin/submit_study.py Outdated
@@ -0,0 +1,774 @@
#!/usr/bin/env python3
"""Submit studies to ENA via the Webin drop-box XML submission service.
Copy link
Copy Markdown
Contributor

@ochkalova ochkalova Mar 24, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry about this 100 line docstring, I needed to show somewhere input file examples, because there is a lot of options and they are not documented anywhere. Even infering from the code what format is required is complicated

@ochkalova ochkalova force-pushed the feature/studysubmit branch from cc29a3e to bbd8305 Compare March 24, 2026 14:35
@ochkalova ochkalova self-requested a review March 24, 2026 15:29
Copy link
Copy Markdown
Collaborator

@mberacochea mberacochea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ochkalova .. I gave it a quick read, and left some tiny comments.

I think we should do another round of cleaning, the python script still does stuff that I'm not sure is relevant for this project.

Comment thread bin/submit_study.py
Comment thread modules/local/registerstudy/main.nf Outdated
Copy link
Copy Markdown
Contributor

@ochkalova ochkalova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to clean everything a bit, still some things left to do, but looks good enough to merge

Copy link
Copy Markdown
Collaborator

@vagkaratzas vagkaratzas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Getting in shape!

Comment thread conf/test_mag_no_study_complete_metadata.config Outdated
Comment thread conf/test_assembly_no_study_complete_metadata.config Outdated
Comment thread modules/local/genome_upload/main.nf
Comment thread tests/assembly_no_study_complete_metadata.nf.test
Copy link
Copy Markdown
Contributor

@KateSakharova KateSakharova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job!

@ochkalova ochkalova dismissed vagkaratzas’s stale review March 25, 2026 15:05

Requested edits cannot be performed in this PR because require merge to more recent branch with bugfixes

@ochkalova ochkalova merged commit 8c3e48d into dev Mar 25, 2026
17 of 24 checks passed
@ochkalova ochkalova deleted the feature/studysubmit branch March 25, 2026 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants