Schema validation does not detect missing properties

[hannes-ucsc]

https://humancellatlas.slack.com/archives/C01360XN04S/p1632505449125000?thread_ts=1631840213.076200&cid=C01360XN04S

It shouldn't even have gotten to the KeyError.

[theathorn]

Spike to validate and estimate.

[dsotirho-ucsc]

The encountered error was:

Error with file: 83f5188e-3bf7-4956-9544-cea4f8997756/staging/links/f4a19d21-7691-5d90-afc9-fcdd462e7529_2021-09-07T15:31:54.000000Z_83f5188e-3bf7-4956-9544-cea4f8997756.json
Traceback (most recent call last):
  File "/Users/npetrill/hca-import-validation/hca/staging_area_validator.py", line 111, in validate_files
    validate_file_fn(blob)
  File "/Users/npetrill/hca-import-validation/hca/staging_area_validator.py", line 138, in validate_links_file
    for entity in link[f'{category}s']:
KeyError: 'protocols'

I verified in hca-import-validation that JSON validation is performed against the JSON file's schema prior to the point this error occurred at.

I also verified through manual testing that a links document would fail JSON validation if a process_link was lacking a required protocols field. (Note: The actual error reported is slightly misleading since a link is defined as being either a process_link or supplementary_file_link, so I believe when the process_link is found to be missing a required field, the validator assumes it is not a valid process_link and therefore must be a supplementary_file_link)

2021-09-28 11:19:54,853 DEBUG   MainThread: Validating JSON of links.json
2021-09-28 11:19:54,853 DEBUG   MainThread: Downloading schema https://schema.staging.data.humancellatlas.org/system/3.0.0/links
Traceback (most recent call last):
  File "scripts/validate_schema.py", line 75, in validate_json
    validate(file_json, schema, format_checker=FormatChecker())
  File "/Users/daniel/repo/azul1/.venv/lib/python3.8/site-packages/jsonschema/validators.py", line 934, in validate
    raise error
jsonschema.exceptions.ValidationError: 'process_link' is not one of ['supplementary_file_link']

Failed validating 'enum' in schema[1]['properties']['link_type']:
    {'description': 'The type of this link.',
     'enum': ['supplementary_file_link'],
     'type': 'string'}

I've reached out to and awaiting reply from Nikelle P. via Slack to see if additional details and/or a copy of the JSON that caused the reported issue would be available to aid in further debugging of this issue.

[dsotirho-ucsc]

After discussion with Nikelle P., the cause of the original issue was identified as a typo in the links JSON document (protocol instead of protocols).

Reproducing the issue, the same error was encountered and revealed a bug in the hca-import-validation tool's exception handling. Currently, exceptions encountered during JSON validation are collected for reporting at the end of the script, however further processing of the JSON file is not then abandoned.

In this case, the method validate_links_file continued to process the links file after the JSON validation failed, which then caused the KeyError exception to be saved for reporting, overwriting the saved JSON validation exception.

Proposed fix for this bug is to not catch & save JSON validation exceptions in validate_file_json, instead letting it travel back up the call stack, halting further processing of the JSON file, to be captured in validate_files.