Skip to content

Normalise malformed / non-canonical SDRF column headers#32

Merged
nithujohn merged 2 commits into
bigbio:mainfrom
BioGeek:fix/sdrf-header-normalization
Jul 13, 2026
Merged

Normalise malformed / non-canonical SDRF column headers#32
nithujohn merged 2 commits into
bigbio:mainfrom
BioGeek:fix/sdrf-header-normalization

Conversation

@BioGeek

@BioGeek BioGeek commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

SDRF column names are space- and case-sensitive (per the spec: "The SDRF is sensitive to spaces in column names (sourcenamesource name)… must NOT have a space before the bracket"). A header that deviates from the canonical name is therefore a different, unrecognised column. parse_sdrf validate-sdrf and downstream parsers silently skip it, so the data in that column is dropped even though the file still "validates" (all affected files here pass validation today, before and after the fix, the defect is invisible to the validator).

This PR fixes four header defects across 18 files. The first three are header-row-only renames (no data cells changed); the fourth drops a redundant duplicate column from one file.

Fixes

1. Trailing space inside the bracket in comment[modification parameters ]

datasets/PXD017710/PXD017710-silac.sdrf.tsv has three columns named comment[modification parameters ] (note the space before ]) alongside one correct comment[modification parameters]. Fixed to four clean comment[modification parameters] columns (repeated modification-parameters columns are valid SDRF, one per SILAC mod).

2. Pandas de-duplication suffix: comment[modification parameters].1

datasets/PXD071075/PXD071075.sdrf.tsv has comment[modification parameters].1 (the .1 is a pandas artifact from reading a file with duplicate column names and writing it back). Fixed to comment[modification parameters] (→ two valid repeated columns).

3. Non-canonical synonym: comment[ms2 analyzer type]comment[ms2 mass analyzer]

15 files use comment[ms2 analyzer type] (one as comment[MS2 analyzer type]) for the concept the spec/sdrf-pipelines template calls comment[ms2 mass analyzer] (description: "Mass analyzer used for MS2 acquisition"). The values are identical in kind (NT=Orbitrap;AC=MS:1000484, NT=linear ion trap;AC=MS:1000291), so these rows are simply invisible under a name no tool looks for. Renamed to the canonical column. (No file contained both columns, so there are no collisions.)

4. Duplicate typo'd column dropped: comment[technical replcate]

datasets/PXD018241/PXD018241.sdrf.tsv carried both comment[technical replcate] (typo) and the correct comment[technical replicate]. Their values are identical (all 1 across the 20 rows), so the typo'd column is a redundant duplicate so I dropped it (31 → 30 columns). This is the one file where data rows change (a whole column is removed). It re-validates cleanly.

18 files changed
# comment[modification parameters ] / .1  -> comment[modification parameters]
datasets/PXD017710/PXD017710-silac.sdrf.tsv
datasets/PXD071075/PXD071075.sdrf.tsv

# comment[technical replcate] duplicate column dropped
datasets/PXD018241/PXD018241.sdrf.tsv

# comment[ms2 analyzer type] (or MS2) -> comment[ms2 mass analyzer]
datasets/PXD000534/PXD000534.sdrf.tsv
datasets/PXD000895/PXD000895.sdrf.tsv
datasets/PXD000999/PXD000999.sdrf.tsv
datasets/PXD001061/PXD001061.sdrf.tsv
datasets/PXD002171/PXD002171.sdrf.tsv
datasets/PXD004613/PXD004613.sdrf.tsv
datasets/PXD005507/PXD005507.sdrf.tsv
datasets/PXD011175/PXD011175.sdrf.tsv
datasets/PXD012593/PXD012593-srm.sdrf.tsv
datasets/PXD014565/PXD014565.sdrf.tsv
datasets/PXD015744/PXD015744.sdrf.tsv
datasets/PXD017035/PXD017035.sdrf.tsv
datasets/PXD018970/PXD018970.sdrf.tsv
datasets/PXD019123/PXD019123.sdrf.tsv
datasets/PXD023650/PXD023650.sdrf.tsv

Verification

  • The 17 header renames are confined to the header row (git diff --numstat1 / 1 per file; no data row touched); PXD018241 (the dropped column) changes every row by design.
  • All 18 files re-validate with parse_sdrf validate-sdrf --sdrf_file <f> --use_ols_cache_only: no new errors (same as before the change).

BioGeek added 2 commits July 10, 2026 21:42
SDRF column names are space- and case-sensitive, so a header that deviates from the
canonical name is a *different* (unrecognised) column: parsers and `parse_sdrf
validate-sdrf` silently ignore it, so the data in it is dropped downstream while the
file still "validates". Three header defects fixed (header row only; no data changed):

- comment[modification parameters ]  -> comment[modification parameters]   (trailing space; PXD017710-silac, 3 cols)
- comment[modification parameters].1 -> comment[modification parameters]   (pandas de-dup suffix; PXD071075)
- comment[ms2 analyzer type] / comment[MS2 analyzer type] -> comment[ms2 mass analyzer]
  (non-canonical synonym; canonical name per the sdrf-pipelines template; 15 files)

17 files changed, header line only; all re-validate with no new errors.
The file carried both comment[technical replcate] (typo) and the correct
comment[technical replicate]; their values are identical (all "1" across the 20 rows),
so the typo'd column is a redundant duplicate. Dropped it (31 -> 30 columns);
re-validates cleanly.
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@coderabbitai

coderabbitai Bot commented Jul 10, 2026

Copy link
Copy Markdown

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (18)
  • datasets/PXD000534/PXD000534.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD000895/PXD000895.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD000999/PXD000999.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD001061/PXD001061.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD002171/PXD002171.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD004613/PXD004613.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD005507/PXD005507.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD011175/PXD011175.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD012593/PXD012593-srm.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD014565/PXD014565.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD015744/PXD015744.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD017035/PXD017035.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD017710/PXD017710-silac.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD018241/PXD018241.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD018970/PXD018970.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD019123/PXD019123.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD023650/PXD023650.sdrf.tsv is excluded by !**/*.tsv
  • datasets/PXD071075/PXD071075.sdrf.tsv is excluded by !**/*.tsv

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 45fad6ce-230e-4512-969f-d327e82205c5

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@nithujohn nithujohn left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for fixing it :)

@nithujohn nithujohn merged commit 55057c6 into bigbio:main Jul 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants