Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 25 additions & 4 deletions .github/workflows/installation-profiles.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
run: |
python -c "import sys, metax, metax.cli.annotate, metax.cli.report; assert not any(name.startswith('PyQt') for name in sys.modules); assert 'metax.report' not in sys.modules"
python -m metax.cli.annotate --help
python -m pytest tests/test_annotation_cli.py -q
python -m pytest tests/test_annotation_cli.py tests/test_annotation_subprocess_cleanup.py tests/test_pep_table_to_otf_dataframe.py tests/test_unit_specific_manifest.py -q

headless-analysis-profiles:
runs-on: ubuntu-latest
Expand All @@ -34,13 +34,15 @@ jobs:
- run: python -m pip install ".[${{ matrix.extra }}]" pytest
- name: Verify Analyzer profile
if: matrix.extra == 'analyzer'
run: python -c "from metax.taxafunc_analyzer.analyzer import TaxaFuncAnalyzer; import sys; assert not any(name.startswith('PyQt') for name in sys.modules)"
run: |
python -c "from metax.taxafunc_analyzer.analyzer import TaxaFuncAnalyzer; import sys; assert not any(name.startswith('PyQt') for name in sys.modules)"
python -m pytest tests/test_analyzer.py tests/test_annotation_cli.py -q
- name: Verify Report profile
if: matrix.extra == 'report'
run: |
python -c "import metax.report; import sys; assert not any(name.startswith('PyQt') for name in sys.modules)"
python -m metax.cli.report --help
- run: python -m pytest tests/test_annotation_cli.py -q
python -m pytest tests/test_auto_report_config.py tests/test_auto_report_workflow.py tests/test_annotation_cli.py -q

gui-and-full-metadata:
runs-on: ubuntu-latest
Expand All @@ -58,4 +60,23 @@ jobs:
env:
QT_QPA_PLATFORM: offscreen
run: python -c "import PyQt5, qtawesome, qt_material; import metax.gui.main_gui"
- run: python -m pytest tests/test_annotation_cli.py -q
- name: Run shared backend and GUI annotation tests
env:
QT_QPA_PLATFORM: offscreen
run: python -m pytest tests/test_annotation_cli.py tests/test_unit_specific_gui_settings_dialog.py -q

windows-annotation:
runs-on: windows-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- run: python -m pip install --upgrade pip
- run: python -m pip install . pytest
- name: Verify lightweight Windows annotation contract
shell: pwsh
run: |
python -c "import sys, metax, metax.cli.annotate; assert not any(name.startswith('PyQt') for name in sys.modules)"
python -m metax.cli.annotate --help
python -m pytest tests/test_annotation_cli.py tests/test_annotation_subprocess_cleanup.py tests/test_pep_table_to_otf_dataframe.py tests/test_unit_specific_manifest.py -q
15 changes: 15 additions & 0 deletions Docs/Annotation_CLI.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ Use `--selection-mode provided` with `--selected-genomes` or
`--genome-list-file` to supply genomes directly. Use `automatic` for MetaX's
internal genome ranking, or `metaumbra-only` to stop after genome scoring.

When `--selection-mode` is omitted, selected genome IDs or a genome-list file
imply `provided`; otherwise digest directories imply `metaumbra`; otherwise the
mode is `automatic`. Explicit modes take precedence, and contradictory genome
sources are rejected instead of ignored.

Exactly one peptide-mapping source is required for OTF generation:
`--peptide-db` or `--digested-genome-folders`.

Expand Down Expand Up @@ -81,6 +86,10 @@ Run it with:
metax-annotate --config annotation.yaml
```

Relative paths loaded from a configuration file are resolved from the
configuration file's directory. Relative paths supplied directly on the command
line remain relative to the current working directory.

## Result JSON

When `--result-json` is supplied, MetaX writes schema `2.0` atomically. The
Expand All @@ -92,6 +101,9 @@ run, inputs, parameters, stages, genome_selection, metrics, outputs, diagnostics

Use `run.status` and `run.exit_code` for the outcome, `outputs` for generated
files, `metrics` for mapping/QC values, and `diagnostics` for warnings or errors.
Scientific outputs are safely renamed when their requested path already exists,
and `outputs` reports the actual paths. The result JSON itself keeps its requested
stable path and is atomically replaced.
See [Annotation Result Schema](Annotation_Result_Schema.md) for the complete
field reference and examples.

Expand All @@ -106,6 +118,9 @@ field reference and examples.
| 5 | Annotation or external scoring failed |
| 130 | Execution was cancelled |

On interruption, MetaX terminates active MetaUmbra or digested-scan process
trees before writing the cancellation result JSON.

## Python subprocess

Invoke MetaX with the same Python environment in which it is installed:
Expand Down
5 changes: 5 additions & 0 deletions Docs/Annotation_Result_Schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,7 @@ annotation workflow API remains `1.0`.
},
"genome_selection": {
"method": "metaumbra",
"source": "metax_temp/OTF_metaumbra_genome_presence.tsv",
"qvalue_cutoff": 0.05,
"genomes_evaluated": 4744,
"genomes_selected": 222,
Expand Down Expand Up @@ -132,6 +133,10 @@ annotation workflow API remains `1.0`.
Fields that do not apply to the selected mode are omitted. Output entries are
only written for artifacts that were actually produced.

`parameters.selection_mode` and `genome_selection.method` contain the same
effective selection mode. `genome_selection.source` identifies explicit genome
IDs, a genome-list file, MetaUmbra output, or automatic genome ranking.

## Unit-specific additions

Unit-specific results use `run.mode: "unit-specific"` and add:
Expand Down
51 changes: 49 additions & 2 deletions metax/cli/annotate.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import argparse
import json
import os
import platform
import sys
import time
Expand Down Expand Up @@ -38,6 +39,16 @@ def error(self, message: str) -> None:


ANNOTATION_RESULT_SCHEMA_VERSION = "2.0"
_CONFIG_PATH_FIELDS = {
"peptide_table",
"unit_specific_manifest",
"taxafunc_db",
"output",
"peptide_db",
"genome_list_file",
"result_json",
}
_CONFIG_MULTI_PATH_FIELDS = {"digested_genome_folders"}


def _decode_separator(value: str) -> str:
Expand Down Expand Up @@ -82,8 +93,38 @@ def _flatten_config(raw_config: Mapping[str, Any]) -> dict[str, Any]:
return flattened


def _resolve_config_path_value(value: Any, config_dir: Path) -> Any:
if value is None or not isinstance(value, (str, Path)):
return value
raw_value = os.path.expanduser(str(value))
if not raw_value.strip():
return raw_value
candidate = Path(raw_value)
return raw_value if candidate.is_absolute() else str(config_dir / candidate)


def _resolve_config_paths(config: Mapping[str, Any], config_dir: Path) -> dict[str, Any]:
resolved = dict(config)
for field_name in _CONFIG_PATH_FIELDS:
if field_name in resolved:
resolved[field_name] = _resolve_config_path_value(
resolved[field_name], config_dir
)
for field_name in _CONFIG_MULTI_PATH_FIELDS:
if field_name not in resolved:
continue
value = resolved[field_name]
if isinstance(value, (str, Path)):
resolved[field_name] = _resolve_config_path_value(value, config_dir)
elif value is not None:
resolved[field_name] = [
_resolve_config_path_value(item, config_dir) for item in value
]
return resolved


def load_config_file(path: str | Path) -> dict[str, Any]:
config_path = Path(path)
config_path = Path(path).expanduser()
if not config_path.is_file():
raise FileNotFoundError(f"Configuration file not found: {config_path}")
suffix = config_path.suffix.lower()
Expand Down Expand Up @@ -112,7 +153,11 @@ def load_config_file(path: str | Path) -> dict[str, Any]:
loaded = {}
if not isinstance(loaded, Mapping):
raise AnnotationConfigurationError("Annotation configuration must be a mapping")
config = _flatten_config(loaded)
config_path = config_path.resolve()
config = _resolve_config_paths(
_flatten_config(loaded),
config_path.parent,
)
requested_api = str(
config.pop("workflow_api_version", config.pop("api_version", ANNOTATION_WORKFLOW_API_VERSION))
)
Expand Down Expand Up @@ -626,6 +671,8 @@ def main(argv: list[str] | None = None) -> int:
stage_started = time.perf_counter()
parser = build_parser(config)
args = parser.parse_args(argv)
if pre_args.config:
args.config = config["config_path"]
mode = args.mode
_validate_scientific_parameters(args)
result_json_path = args.result_json
Expand Down
Loading
Loading