Normalise malformed / non-canonical SDRF column headers#32
Conversation
SDRF column names are space- and case-sensitive, so a header that deviates from the canonical name is a *different* (unrecognised) column: parsers and `parse_sdrf validate-sdrf` silently ignore it, so the data in it is dropped downstream while the file still "validates". Three header defects fixed (header row only; no data changed): - comment[modification parameters ] -> comment[modification parameters] (trailing space; PXD017710-silac, 3 cols) - comment[modification parameters].1 -> comment[modification parameters] (pandas de-dup suffix; PXD071075) - comment[ms2 analyzer type] / comment[MS2 analyzer type] -> comment[ms2 mass analyzer] (non-canonical synonym; canonical name per the sdrf-pipelines template; 15 files) 17 files changed, header line only; all re-validate with no new errors.
The file carried both comment[technical replcate] (typo) and the correct comment[technical replicate]; their values are identical (all "1" across the 20 rows), so the typo'd column is a redundant duplicate. Dropped it (31 -> 30 columns); re-validates cleanly.
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
|
Important Review skippedReview was skipped due to path filters ⛔ Files ignored due to path filters (18)
CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
nithujohn
left a comment
There was a problem hiding this comment.
Thank you for fixing it :)
SDRF column names are space- and case-sensitive (per the spec: "The SDRF is sensitive to spaces in column names (
sourcename≠source name)… must NOT have a space before the bracket"). A header that deviates from the canonical name is therefore a different, unrecognised column.parse_sdrf validate-sdrfand downstream parsers silently skip it, so the data in that column is dropped even though the file still "validates" (all affected files here pass validation today, before and after the fix, the defect is invisible to the validator).This PR fixes four header defects across 18 files. The first three are header-row-only renames (no data cells changed); the fourth drops a redundant duplicate column from one file.
Fixes
1. Trailing space inside the bracket in
comment[modification parameters ]datasets/PXD017710/PXD017710-silac.sdrf.tsvhas three columns namedcomment[modification parameters ](note the space before]) alongside one correctcomment[modification parameters]. Fixed to four cleancomment[modification parameters]columns (repeated modification-parameters columns are valid SDRF, one per SILAC mod).2. Pandas de-duplication suffix:
comment[modification parameters].1datasets/PXD071075/PXD071075.sdrf.tsvhascomment[modification parameters].1(the.1is apandasartifact from reading a file with duplicate column names and writing it back). Fixed tocomment[modification parameters](→ two valid repeated columns).3. Non-canonical synonym:
comment[ms2 analyzer type]→comment[ms2 mass analyzer]15 files use
comment[ms2 analyzer type](one ascomment[MS2 analyzer type]) for the concept the spec/sdrf-pipelinestemplate callscomment[ms2 mass analyzer](description: "Mass analyzer used for MS2 acquisition"). The values are identical in kind (NT=Orbitrap;AC=MS:1000484,NT=linear ion trap;AC=MS:1000291), so these rows are simply invisible under a name no tool looks for. Renamed to the canonical column. (No file contained both columns, so there are no collisions.)4. Duplicate typo'd column dropped:
comment[technical replcate]datasets/PXD018241/PXD018241.sdrf.tsvcarried bothcomment[technical replcate](typo) and the correctcomment[technical replicate]. Their values are identical (all1across the 20 rows), so the typo'd column is a redundant duplicate so I dropped it (31 → 30 columns). This is the one file where data rows change (a whole column is removed). It re-validates cleanly.18 files changed
Verification
git diff --numstat→1 / 1per file; no data row touched); PXD018241 (the dropped column) changes every row by design.parse_sdrf validate-sdrf --sdrf_file <f> --use_ols_cache_only: no new errors (same as before the change).