Skip to content

Normalize ragged CSV to the widest row, not the header width#647

Open
gaoflow wants to merge 1 commit into
jazzband:masterfrom
gaoflow:fix-csv-ragged-wider-rows
Open

Normalize ragged CSV to the widest row, not the header width#647
gaoflow wants to merge 1 commit into
jazzband:masterfrom
gaoflow:fix-csv-ragged-wider-rows

Conversation

@gaoflow

@gaoflow gaoflow commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Summary

CSV import is documented (issue #226) to normalize ragged rows "to the width of the widest row by filling in empty elements". In practice it only pads rows up to the width of the first/header row, so a data row wider than the header raises InvalidDimensions:

>>> import tablib
>>> tablib.import_set("a,b\nc,d,e\n", format="csv")
tablib.exceptions.InvalidDimensions
>>> tablib.import_set("a,b\nc,d,e\n", format="csv", headers=False)
tablib.exceptions.InvalidDimensions

The analogous shorter case is handled — short rows are padded — so the behaviour is inconsistent, and the existing test_csv_import_set_ragged only exercises the header-is-widest direction.

Fix

Read the header and data rows, compute the width as the maximum across all rows (header included), then pad the header and every data row to that width. Empty data lines are still skipped, and an empty/absent header still yields headers=None.

>>> dataset = tablib.import_set("a,b\nc,d,e\n", format="csv")
>>> dataset.headers
['a', 'b', '']
>>> list(dataset[0])
['c', 'd', 'e']

Verification

  • Reproduced on master (64f0201): a wider-than-header row raises InvalidDimensions; with the fix it normalizes to the widest width (with and without headers).
  • Added test_csv_import_set_ragged_wider_than_header and test_csv_import_set_ragged_wider_no_headers; both fail before the fix and pass after.
  • Full suite: 176 passed (174 baseline + the 2 new tests), no regressions. ruff check clean.
  • Existing behaviour preserved: shorter-row padding, the empty-line-skipping test_csv_import_set_ragged, and empty-stream (headers=None) all unchanged.

This completes the ragged-CSV feature from #226 for the wider-row case.


This pull request was prepared with the assistance of AI, under my direction and review.

CSV import was documented (issue jazzband#226) to normalize ragged rows to the
width of the widest row, but only padded rows up to the width of the
first/header row. A data row wider than the header raised
InvalidDimensions, while the shorter-row case was handled correctly.
Compute the width across all rows and pad the header and every data row
to it.
@codecov

codecov Bot commented Jun 23, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.23%. Comparing base (64f0201) to head (3d3617a).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #647      +/-   ##
==========================================
+ Coverage   93.19%   93.23%   +0.04%     
==========================================
  Files          29       29              
  Lines        3247     3268      +21     
==========================================
+ Hits         3026     3047      +21     
  Misses        221      221              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant