Skip to content

fix: set UTF-8 client charset for SQL Server connections#528

Merged
datlechin merged 2 commits intomainfrom
fix/mssql-unicode-charset
Mar 31, 2026
Merged

fix: set UTF-8 client charset for SQL Server connections#528
datlechin merged 2 commits intomainfrom
fix/mssql-unicode-charset

Conversation

@datlechin
Copy link
Copy Markdown
Collaborator

Summary

  • Set DBSETCHARSET to UTF-8 on the FreeTDS login record so Unicode data (Thai, CJK, Arabic, emoji, etc.) in nvarchar/nchar/ntext columns is delivered as UTF-8 instead of being mangled through the server's default single-byte codepage
  • Updated columnValueAsString to try UTF-8 decoding first for nvarchar types (with UTF-16LE fallback), since FreeTDS converts wire data to the client charset but may retain the original type token

Without this fix, any characters outside the server's default codepage (e.g., cp1252) appear as ? in TablePro. Reported by a user with Thai text in nvarchar columns on Azure SQL Server.

Test plan

  • Connect to a SQL Server instance with an nvarchar column containing Thai text (e.g., N'สวัสดี') — verify it displays correctly
  • Verify Latin/ASCII text in varchar and nvarchar columns still displays correctly
  • Test with other Unicode scripts (CJK, Arabic, emoji) in nvarchar columns
  • Verify ntext columns also render Unicode correctly

@datlechin datlechin merged commit 1be6c3f into main Mar 31, 2026
2 checks passed
@datlechin datlechin deleted the fix/mssql-unicode-charset branch March 31, 2026 06:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant