SXSSFWorkbook fails to set extended information extra field in Zip64 mode

[londonCyclist]

Tested with version 5.5.1 , local file header (LOC) generated while using SXSSFWorkbook with zip64 mode is not set as per the spec (6.3.9) given below;

https://pkwaredownloads.blob.core.windows.net/pkware-general/Documentation/APPNOTE-6.3.9.TXT

4.3.9.2 When compressing files, compressed and uncompressed sizes 
      SHOULD be stored in ZIP64 format (as 8 byte values) when a 
      file's size exceeds 0xFFFFFFFF.   **_However ZIP64 format MAY be 
      used regardless of the size of a file.  When extracting, if 
      the zip64 extended information extra field is present for 
      the file the compressed and uncompressed sizes will be 8
      byte values._**  

I was expecting zip64 extended information extra field to be present in LOC header when compressed and uncompressed sizes were represented as 8 bytes in the data directory record. As a result the application reading data directory record will endup reading it in zip32 mode, this results in both compressed and uncompressed size to be incorrect.

Fig 1: Local file header

extended information extra field is not set (purple) , CRC (green) , compressed size (orange) , uncompressed size(red)

Fig 2: Data directory record

Reading application has no way to determine if the data directory record is represented in Zip32 or Zip64 mode .

[pjfanning]

See https://rzymek.github.io/post/excel-zip64/

SXSSFWorkbook has a method to change the Zip64Mode. You can try a different mode if you are unhappy with the default one.

[londonCyclist]

thanks, I think it is a bug , when Zip64 mode is set then the LOC header should be correctly set. App reading directory record can't figure out if POI is using a 8-byte to represent csize and size or 4-bytes.

[pjfanning]

thanks, I think it is a bug , when Zip64 mode is set then the LOC header should be correctly set. App reading directory record can't figure out if POI is using a 8-byte to represent csize and size or 4-bytes.

What app? And have you tried using a different Zip64 Mode?

[londonCyclist]

My point is when setting Zip64 mode to "Always" POI doesn't set the LOC header information correctly. If I set Zip64 mode to "AsNeeded" , then data directory is correctly represented as 4 bytes. Setting ZipMode to "Always" should be setting extra information flag correctly in LOC header.

[pjfanning]

Have you even read the blog about why the POI code is set this way? Hint: it's because Excel has its own quirky interpretation of the Zip spec.
You can workaround whatever issue you are having by changing the Zip64 Mode.

[londonCyclist]

thanks for the hint! POI sets the data descriptors incorrectly as a workaround for Excel's quirk. However, from POI 5 onwards, it seems to be using the default Zip64mode as "Always"; Considering other libraries do not set data descriptors incorrectly such as Aspose 26.2, LargeXlsx etc. Would it be useful to have the default mode set to "AsNeeded" as it would help with downstream compatibility issues.

[pjfanning]

Not changing the default again. There is a setting that you can use.

[londonCyclist]

Okay, I think it would be helpful to have the odd behaviour of Zip64Mode when set to "Always" documented for SXSSF, and to add guidance recommending "AsNeeded" for better downstream compatibility.