Migrate to Volar

[AngusMorton]

This is a clean version of #294.

This is a proof of concept that shows what it might look like if the Marko Language Tools migrated to Volar as a base.

This proof-of-concept is roughly complete for our purposes, we've been using it for the past few months.

Motivation and Context

Migrating to Volar resolves the issues we've had with the VSCode plugin on Windows (#240), and it improves the responsiveness of the plugin on Windows.

Why Volar?

• Volar provides the glue code that the Marko Langauge Tools currently implement themselves, which means we could potentially shift the maintenance/implementation burden off the Marko team.
• Vue and Astro use Volar for their Language Tools, so it's fairly well used.
• The patterns in Volar are very similar to what is already here, so the migration is relatively straightforward.
  • MarkoFile -> MarkoVirtualCode
  • Plugin -> LanguageServicePlugin

TODO

This is fairly untested, I mostly hacked it up so it works with our project.

• Test with non-TS projects / more project structures.
  • It doesn't appear to work with Node type stripping - Discord?
• Improve diagnostic mapping?
  • Currently, Volar appears to only map diagnostics if they're wholly contained by a source span, which means we need to do some manual adjustments of diagnostic ranges, which can be less than ideal.

Questions

• How do Processors fit with Volar? Is there an idiomatic way to do that in Volar?
• Could we use Volar for marko-type-check? @volar/kit?

[changeset-bot]

⚠️ No Changeset found

Latest commit: d4a3acf

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[AngusMorton]

This is an awful hack so that I can generate Volar CodeMapping objects from Tokens.

A better solution would be to have the extractor be able to generate CodeMappings itself...

[AngusMorton]

Small QOL improvement, could be split into a separate PR

[AngusMorton]

I'm probably missing something - but I wasn't sure if Marko specific document symbols were still necessary and that couldn't be provided by TS or HTML.

[AngusMorton]

This is necessary to ensure we show diagnostics. Volar doesn't seem to display them when they are larger than the code mappings, so I clamp the diagnostics to the most reasonable code mapping we can find...

There is almost certainly a better way, but this seems good enough for now?

[AngusMorton]

I notice we're using Processors here, but extractors elsewhere... I'm wondering if we should be using Processors in the MarkoVirtualCode?

[kanashimia]

It seems there is a problem here that in the LSP value shorthands don't report errors for some reason:

<never value=size/>
       └─Type '"small" | "large"' is not assignable to type 'never'. 
<never=size/> // no error

Maybe a problem with zero sized spans or something?

[AngusMorton]

Yeah - good catch. There is a zero-sized span for the value shorthand.

Using Volar Labs.

[kanashimia]

Yeah - good catch. There is a zero-sized span for the value shorthand.

Yes indeed, I worked around it like this:

diff --git a/packages/language-server/src/index.ts b/packages/language-server/src/index.ts
index 44fa55a..5fb32ad 100644
--- a/packages/language-server/src/index.ts
+++ b/packages/language-server/src/index.ts
@@ -5,6 +5,7 @@ import {
   loadTsdkByPath,
 } from "@volar/language-server/node";
 import { URI } from "vscode-uri";
+import { defaultMapperFactory } from "@volar/language-core";
 
 import { addMarkoTypes, createMarkoLanguagePlugin } from "./language";
 import { getLanguageServicePlugins } from "./plugins";
@@ -37,13 +38,27 @@ connection.onInitialize((params) => {
             uri.fsPath.replace(/\\/g, "/"),
           ),
         ],
-        setup({ project }) {
+        setup({ project, language }) {
           const { languageServiceHost, configFileName } = project.typescript!;
 
           const rootPath = configFileName
             ? configFileName.split("/").slice(0, -1).join("/")
             : env.workspaceFolders[0]!.fsPath;
 
+          language.mapperFactory = (mappings) => {
+            let def = defaultMapperFactory(mappings);
+
+            for (const map of def.mappings) {
+              for (let i = 0; i < map.lengths.length; i++) {
+                if (map.lengths[i] === 0) {
+                  map.lengths[i] = 1;
+                }
+              }
+            }
+
+            return def;
+          };
+
           addMarkoTypes(rootPath, typescript, languageServiceHost);
         },
       };

Seems to work for me, but I'm not sure if that is an alright solution or not, I know nothing about volar.

Also this mapperFactory api seems to be related to that "Improve diagnostic mapping" point?

[kanashimia]

With regards to "Could we use Volar for marko-type-check?", I actually managed to do that it seems:

Screenshot of it working

This is related to the issue volarjs/volar.js#145
There I found that you can use runTsc for this, from @volar/typescript/lib/quickstart/runTsc.js
Example from mdx thing: https://github.com/karlhorky/poc-mdx-type-checker-cli/blob/main/mdx-tsc/index.ts
Example from vue: https://github.com/vuejs/language-tools/blob/master/packages/tsc/index.ts

Right now it is messy I will try to put a PR into your branch after a cleanup.
I'm not sure it if it should be a separate package from this one, as it is less than 50 lines of code and depends on the code in the language-server. It probably should just be a separate export.
And with this thing I imagine there is a lot of code that isn't necessary anymore.

[kanashimia]

Another thing I found is that there are problems with type inference on dynamic tag imports, even though they work okay for static imports.

<await|Bar|=import("<wl-chip>")>                                                           
                   └─Cannot find module '<wl-chip>' or its corresponding type declarations.
  <Bar />                                                                                  
</await>