Skip to content

FE-400 - Lungmap modifications #26

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JoshuaFortriede wants to merge 2 commits into
base: main
Choose a base branch
from
+9 −2
Open

FE-400 - Lungmap modifications #26

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
10581fc
dont flag missing sequence files if DRS_URI
JoshuaFortriede May 9, 2025
46bc96e
Finished "TODO: remove unused `metadata_type`"
JoshuaFortriede May 9, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions hca/staging_area_validator.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -258,8 +258,7 @@ def validate_file_description(self, file_description: str) -> None:

def validate_descriptors_file(self, blob: gcs.Blob) -> None:
# Expected syntax: descriptors/{metadata_type}/{metadata_id}_{version}.json
# TODO: remove unused `metadata_type`
metadata_type, descriptor_file = blob.name.split("/")[-2:]
descriptor_file = blob.name.split("/")[-1]
assert descriptor_file.count("_") == 1
assert descriptor_file.endswith(".json")

Expand All @@ -272,6 +271,14 @@ def validate_descriptors_file(self, blob: gcs.Blob) -> None:
if metadata_file := self.metadata_files.get(metadata_id):
metadata_file["crc32c"] = file_json["crc32c"]
metadata_versions = metadata_file["metadata_versions"]

# Sequence file data_files might not be present if they are managed access.
# File Descriptor v2.1.0 allows for the drs_uri to be a string or null.
# In both of these cases, we set found_data_file to True
if metadata_file["entity_type"] == "sequence_file":
Copy link
Copy Markdown
Collaborator

@ncalvanese1 ncalvanese1 May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is necessarily limited to sequencing files in the spec, though that is the use case for LungMap.

Copy link
Copy Markdown
Author

@JoshuaFortriede JoshuaFortriede May 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is correct. This could be change to
if metadata_file["entity_type"].endswith("_file"):

if "drs_uri" in file_json:
metadata_file["found_data_file"] = True

assert (
descriptor_version in metadata_versions
), f"Corresponding metadata version for descriptor version {descriptor_version} not found"
Expand Down