Skip to content

Added UTF-8 encoding to writers for cross-OS compatibility#119

Merged
erickmartins merged 2 commits into
ome:mainfrom
JensWendt:main
Jun 5, 2026
Merged

Added UTF-8 encoding to writers for cross-OS compatibility#119
erickmartins merged 2 commits into
ome:mainfrom
JensWendt:main

Conversation

@JensWendt

@JensWendt JensWendt commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

As I am mainly using Windows I ran into issues with special characters like µ in Key-Value pairs and üäöß (german special characters) in names of images/datasets.
The error was:

INFO:omero_cli_transfer:Unzipping .\image.tar...
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\miniforge\envs\omero-cli-transfer\Scripts\omero.exe\__main__.py", line 5, in <module>
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\omero\main.py", line 125, in main
    rv = omero.cli.argv()
         ^^^^^^^^^^^^^^^^
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\omero\cli.py", line 1771, in argv
    cli.invoke(args[1:])
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\omero\cli.py", line 1208, in invoke
    stop = self.onecmd(line, previous_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\omero\cli.py", line 1285, in onecmd
    self.execute(line, previous_args)
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\omero\cli.py", line 1367, in execute
    args.func(args)
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\omero_cli_transfer.py", line 200, in _wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\omero_cli_transfer.py", line 325, in unpack
    self.__unpack(args)
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\omero_cli_transfer.py", line 618, in __unpack
    hash, ome, folder = self._load_from_pack(args.filepath,
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\omero_cli_transfer.py", line 672, in _load_from_pack
    ome = from_xml(posixpath.join(folder, "transfer.xml"))
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\ome_types\_conversion.py", line 121, in from_xml
    xml_2016 = ensure_2016(
               ^^^^^^^^^^^^
  File "c:\miniforge\envs\omero-cli-transfer\Lib\site-packages\ome_types\_conversion.py", line 479, in ensure_2016
    return ET.parse(normed_source)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "c:\miniforge\envs\omero-cli-transfer\Lib\xml\etree\ElementTree.py", line 1219, in parse
    tree.parse(source, parser)
  File "c:\miniforge\envs\omero-cli-transfer\Lib\xml\etree\ElementTree.py", line 581, in parse
    self._root = parser._parse_whole(source)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^
xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 30, column 33

Setting the encoding to UTF-8 for the writing should alleviate this.
I also set newline='' for the csv.writer as AI was pointing me towards it (https://docs.python.org/3/library/csv.html the footnote at the bottom) as an additional safety net

@erickmartins

Copy link
Copy Markdown
Collaborator

Thanks, Jens! This looks pretty straightforward (and useful) so I'll try to find a minute this week to review it properly.

@erickmartins erickmartins merged commit 84bd563 into ome:main Jun 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants