fix: resolve pylint W1514 (unspecified-encoding) and re-enable it in CI lint#2296
Open
sbnair wants to merge 1 commit into
Open
fix: resolve pylint W1514 (unspecified-encoding) and re-enable it in CI lint#2296sbnair wants to merge 1 commit into
sbnair wants to merge 1 commit into
Conversation
…lint Add explicit encoding="utf-8" to all text-mode open() calls in qlib/, scripts/ and example notebooks, so file reads/writes no longer depend on the platform's locale encoding (e.g. cp936/GBK on Chinese Windows, where reading UTF-8 YAML configs or calendar files could fail or corrupt text). - qlib/utils/file.py: get_io_object() now defaults to UTF-8 only when the file is opened in text mode without an explicit encoding; binary mode and caller-provided encodings are left untouched. - qlib/data/storage/file_storage.py: _write_calendar() keeps its binary mode (np.savetxt already encodes via its own encoding argument) with an explanatory pragma. - Makefile: remove W1514 from the pylint/nbqa disable lists so CI enforces the rule from now on. make pylint now scores 10.00/10 on qlib and scripts, and nbqa reports no W1514 in notebooks. Contributes to microsoft#1007
Author
@microsoft-github-policy-service agree |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contributes to #1007 — completely resolves one of the disabled pylint rules, W1514
unspecified-encoding(13 violations: 12 inqlib/, 1 in an example notebook), and removes it from the disable lists in theMakefileso CI enforces it from now on.Unlike purely cosmetic rules, W1514 guards against a real bug class: without an explicit
encoding, Python'sopen()uses the platform locale encoding. On Chinese-locale Windows (a large share of Qlib's user base) that is cp936/GBK, so reading UTF-8 YAML configs, JSON reports, or calendar files can raiseUnicodeDecodeErroror silently corrupt text.Changes
encoding="utf-8"to every text-modeopen()/Path.open()inqlib/(config loading inqlib/__init__.py,qlib/cli/run.py,qlib/utils/__init__.py,qlib/contrib/rolling/base.py,qlib/rl/contrib/*, TRA'sinfo.jsondump, and calendar/instrument files inqlib/data/storage/file_storage.py) and inexamples/benchmarks/TRA/Reports.ipynb.qlib/utils/file.py::get_io_object(): since it forwards*args, **kwargstoPath.open(), it now defaults to UTF-8 only when the file is opened in text mode without an explicitencoding; binary mode and caller-provided encodings are untouched.qlib/data/storage/file_storage.py::_write_calendar(): kept as-is (itsmodeis always binary"wb"/"ab", andnp.savetxtalready encodes via its ownencoding="utf-8"argument) with an explanatory comment and a targeted pragma, as pylint cannot infer the dynamic mode.Makefile: removedW1514from thepylint(qlib + scripts) andnbqadisable lists.Verification
make pylint(with W1514 now enforced): 10.00/10 on bothqlibandscripts(pylint 4.0.6).nbqa pylint . --enable=W1514: no remaining violations in notebooks.black -l 120 --check: all touched files unchanged.get_io_object(): UTF-8 text read of non-ASCII content, explicitencoding=still respected, binary mode (positional and keyword) unaffected.All changes are behavior-preserving on UTF-8 locales (Linux/macOS defaults) and fix locale-dependent behavior on Windows.