Support ES&S Multi-Contest CVRs

[artoonie]

Closes #981

[artoonie]

This PR found a bug:

While Apple Numbers only shows non-empty rows, the underlying XLSX XML data can store empty rows. That led to some CVRs having hidden empty rows which we erroneously marked as undervotes. We now correctly detect these rows as rows to be completely ignored and thrown out.

[artoonie]

TODO: Audit log should say "read X ballots, of which Y were ignored" or similar, rather than X=only non-empty ballots

[artoonie]

TODO: Audit log should say "read X ballots, of which Y were ignored" or similar, rather than X=only non-empty ballots

Rather than modifying that row, I opted to add a warning so (A) it's more visible, and (B) we don't need to propagate what is likely source-specific values up from the ES&S reader through the generic reader and into the tabulator session.

It says "Ignored %d rows with no votes for any candidates"

[yezr]

Testing I found that setting Treat Blanks As Undeclared Write-In to true has some unexpected behavior now. When it sees a fully blank record it calls handleEmptyCells at the beginning of endCvr(). It writes into each of those blank ranks an undeclared write-in ranking.

However, the hasSeenAnyNonBlankCandidateCells boolean is only updated in the cell data call back, which never gets fired because all cells are blank. The ranking of all Undeclared Write-In ranks is then filtered out.

Maybe we need to set the hasSeenAnyNonBlankCandidateCells to true when we set a UWI?

[artoonie]

It seems that Treat Blanks As Undeclared Write-In is fundamentally incompatible with this approach. Either blanks mean UWI or it means Unrelated Contest. I wonder if we should go back to the drawing board and consider looking at the other columns to disambiguate. Or, better yet -- find out what's actually possible in the ES&S data format so we can support it without guessing.

If we want to keep this approach, I suggest changing the Treat Blanks As Undeclared Write-In from a checkbox to a dropdown: Blanks are... > [Ignored, Treated as Undeclared Write-Ins]

[yezr]

If we want to keep this approach, I suggest changing the Treat Blanks As Undeclared Write-In from a checkbox to a dropdown: Blanks are... > [Ignored, Treated as Undeclared Write-Ins]

I think you are right that it needs to be mutually exclusive. For our options

• default option of Any Blank Invalid - Causes Tabulation to Fail
• for any single blank being treated as a UWI, the current behavior when the Treat Blanks As Undeclared Write-In is checked, it now says Any Blank Valid - Individual Blanks Treated As UWI
• I think for the Ignored option we might change the description to make it explicit that it is only ignored if it is a full ranking of blanks. Maybe Valid Only As An All Blank Ranking Ballot - Not included in tabulation (For Multi-Contest Exports)

We can work on this vocab...

[artoonie]

Makes sense.

Suggested vocab:

1. Invalid (fail tabulation)
2. Undeclared Write-In (valid if only one blank per voter)
3. Irrelevant Contest (skip row if all rankings are blank)

With 3, the CSV field listing the number of voters who didn't vote at all (Undervotes-No Ranking) would always be zero, and that's invalid. Does it make sense to write "unknown" instead of zero for that field?