Skip to content

Bug Report: cell_convert step incorrectly applied to all resources in multi-table validation #1753

Description

@megin1989

Describe the bug
While using the cell_convert step during validation, I encountered an issue where the same error is reported across multiple resources (tables), even when the specified column does not exist in one of them.

To Reproduce
I was validating multiple CSV files (screening_profile_data and screening_observation_data) using a pipeline step like:

 {
                "name": "screening_profile_data",
                "type": "table",
                "valid": false,
                "place": "<memory>",
                "labels": [],
                "stats": {
                    "errors": 1,
                    "warnings": 0,
                    "seconds": 0.197
                },
                "warnings": [],
                "errors": [
                    {
                        "type": "step-error",
                        "title": "Step Error",
                        "description": "Provided step is not valid",
                        "message": "Step is not valid: \"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\"",
                        "tags": [],
                        "note": "\"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\""
                    }
                ]
            },
            {
                "name": "screening_observation_data",
                "type": "table",
                "valid": false,
                "place": "nyher-fhir-ig-example/SDOH_ScreeningObs_CareRidgeSCN_testcase1_20250312040214.csv",
                "labels": [],
                "stats": {
                    "errors": 1,
                    "warnings": 0,
                    "seconds": 0.21,
                    "md5": "38f272992a5cdcf1bf85f896e1192297",
                    "sha256": "60b88259afdcdc3ef94400aca63a9643c4c2f00c38224c0e681753140145d6f7",
                    "bytes": 14676,
                    "fields": 22
                },
                "warnings": [],
                "errors": [
                    {
                        "type": "step-error",
                        "title": "Step Error",
                        "description": "Provided step is not valid",
                        "message": "Step is not valid: \"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\"",
                        "tags": [],
                        "note": "\"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\""
                    }
                ]
            },

Observed Behavior
Both files report the following error even though only one of them actually includes the SCREENING_ENTITY_ID_CODE_SYSTEM column:

"message": "Step is not valid: \"cell_convert\" raises \"selection is not a field or valid field index: 'SCREENING_ENTITY_ID_CODE_SYSTEM'\""
 

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions