uchmviewer doesn't display Japanese .chm files correctly

[perdrix52]

Japanese chm files are typically encoded as Shift-JIS. kchmviewer has a View/Encoding menu option that allows me to specify the file encoding so that it displays correctly.

uchmviewer doesn't seem to allow me to do that so my Japanese help which should look like:

is instead displayed as:

which isn't too great.

Can this be addressed please.

[u-235]

I'm not entirely sure what needs to be fixed. I just checked, and the encoding choose menu is still available for CHM files.

[perdrix52]

Silly me - this was macOS so the Menu stuff isn't part of the application but is shown at the top of the screen.

However is there any chance the if the language is set to Japanese then the default encoding for chm files should be set to Shift-JIS

[gonwan]

@perdrix52

However is there any chance the if the language is set to Japanese then the default encoding for chm files should be set to Shift-JIS

This is already the case. uchmviewer does detect and set encodings accordingly. But the encoding here sets both the internal encoding used to decode/display chm menu and index, and the encoding used to display a single html page. If these two encodings do not agree, garbage text is shown.

Maybe it is just fine to let the embeded browser detect the page encoding automatically, as in most modern browsers in 2026.

[u-235]

Maybe it is just fine to let the embeded browser detect the page encoding automatically, as in most modern browsers in 2026.

That's an interesting idea, but it only works if the page includes a valid META tag specifying the character encoding. Unfortunately, some pages lack this information.

[gonwan]

@u-235
That is not quite true. Chrome is able to correctly display following two html pages, even without help of meta tag or http headers.

<!-- save in gbk encoding -->
zhcn_gbk<br/>
123abc<br/>
中文

<!-- save in shiftjis encoding -->
ja_shiftjis<br/>
123abc<br/>
日本語てす

However, a character detector is not a standard part of chrome/firefox/qtwebkit/qtwebengine. They use different aux libraries. See:

• https://superuser.com/questions/1160003/how-do-i-change-the-character-encoding-for-a-webpage-in-chrome
• https://stackoverflow.com/questions/43148464/how-do-browsers-determine-the-encoding-used

[gonwan]

@u-235
I created a chm file for verify. First apply the smallest patch to disable manual encoding settings:

diff --git a/src/qtwebengine/dataprovider.cpp b/src/qtwebengine/dataprovider.cpp
index def2973..4714885 100644
--- a/src/qtwebengine/dataprovider.cpp
+++ b/src/qtwebengine/dataprovider.cpp
@@ -93,8 +93,8 @@ void DataProvider::requestStarted( QWebEngineUrlRequestJob* request )
 	// This is the easiest and most stable way to set the encoding.
 	if ( mimetype == "text/html" )
 	{
-		buf.prepend( QString( "<META http-equiv='Content-Type' content='text/html; charset=%1'>" )
-		             .arg( ::mainWindow->chmFile()->currentEncoding() ).toLatin1() );
+//		buf.prepend( QString( "<META http-equiv='Content-Type' content='text/html; charset=%1'>" )
+//		             .arg( ::mainWindow->chmFile()->currentEncoding() ).toLatin1() );
 	}
 
 	// We will use the buffer because reply() requires the QIODevice.
diff --git a/src/qtwebkit/dataprovider.cpp b/src/qtwebkit/dataprovider.cpp
index ca3d01e..46ac6f1 100644
--- a/src/qtwebkit/dataprovider.cpp
+++ b/src/qtwebkit/dataprovider.cpp
@@ -93,9 +93,7 @@ QByteArray KCHMNetworkReply::loadResource( const QUrl& url )
 
 	if ( mime == "text/html" || mime == "text/xhtml" || mime == "text/xml" )
 	{
-		QString header = QString( "%1; charset=%2" )
-		                 .arg( mime )
-		                 .arg( ::mainWindow->chmFile()->currentEncoding() );
+		QString header = mime;
 		setHeader( QNetworkRequest::ContentTypeHeader, header );
 	}
 

The attched chm file has mixed encodings in separated pages. UTF-8/GBK/Shift-JIS all display correctly. And the QtWebengine version even works without the meta tag. It does make things better. I'll make a PR to address the issue if you agree with me. In addition, The view source function should also be fixed.

testautodetect.chm.zip

[u-235]

I'm not ready to make changes related to page encoding just yet. I've almost finished converting the HTML rendering engines used by the app into plugins. And in the current version, these plugins only accept HTML in Unicode, which means the app is responsible for the encoding conversion.