Add advanced language features for Csound via custom Tree-sitter–based LSP

[PasqualeMainolfi]

Csound vscode

This PR introduces a new Csound extension for Visual Studio Code built on top of a custom Csound Language Server (LSP) and a Csound Tree-sitter grammar specifically designed for Csound.

The extension aims to provide a modern, semantically aware editing experience for Csound users, fully compatible with Csound7.

Key Features

LSP

• Advanced diagnostics
• Detection of unused variables
• Detection of undefined variables
• Opcode error reporting
• Advanced scoping logic, aware of instruments, UDOs, score blocks, and global contexts
• Precise error localization based on the Tree-sitter syntax tree

Editor

• Format on type
• CodeLens for:
  • Running a script
  • Saving output to an audio file
  • Opening the Csound manual
• Code Actions for quick fixes and navigation

Completion & Hover

• Opcode completion
• Option/flag completion
• Hover support for:
  • Built-in opcodes
  • User-defined opcodes (UDOs)
  • User-defined types

Commands exposed by the Language Server

The server exposes the following commands:

• csound-lsp.run_file — run the current Csound script
• csound-lsp.to_audio_file — render and save output to an audio file
• csound-lsp.open_manual — open the Csound reference manual

Offline Manual (Web Preview)

• The Csound HTML reference manual is bundled with the language server
• A lightweight local HTTP server is started by the server
• The manual is displayed inside a VS Code Webview
• Fully functional offline browsing, including links, searching and assets

Language Injections

• Python and HTML language injections are supported inside Csound files
• Enables proper highlighting and tooling for embedded code

Technical Notes

• Syntax parsing is handled via Tree-sitter
• Semantic analysis is built directly on the syntax tree
• The LSP architecture is modular and designed for future extensions

Future Work

Planned next steps include:

• Support for Csound plugins (not yet tested)
• Deeper support for Cabbage blocks
• Additional semantic checks, completions, and editor actions

This PR lays the groundwork for a complete and modern Csound development environment in VS Code, aligned with current language-server–based tooling.
Feedback and testing are highly appreciated.

[kunstmusik]

Just wanted to note that I saw this PR but it came in while I was traveling for holidays. I'm on another trip now so will look at this when I return home in a couple days. Apologies @PasqualeMainolfi for the delay in review!

[kunstmusik]

@PasqualeMainolfi One thing I did want to ask is if you tested this as a Web Extension (i.e., when installed and running in github.dev). I didn't see code to support that but may have just missed it.

[PasqualeMainolfi]

No worries.
In the meantime, I’m working on fixing a few bugs and improving the parsing of .udo files when they are used as imports. I also need to add support for the additional syntax introduced for raw strings (R{}R).
I haven’t tested the extension as a Web Extension on github.dev.
This extension relies on a language server process and a Tree-sitter grammar, which currently require a Node / native (or WASM) environment and are not compatible with VS Code for the Web out of the box.
For this reason, the extension is intended to run in the desktop VS Code environment only.
If web support becomes a goal in the future, it would likely require a web-based LSP and additional work for Tree-sitter compatibility.

[rorywalsh]

Thank you @PasqualeMainolfi, it's going to be very nice to have this feature. I am going to leave the technical review to @kunstmusik as he is more familiar with these things, but I am concerned that this PR overwrites the project's readme file. This removes all the historical context for the existing project as well as removing valuable information about live-coding and using the UDP server. Would you mind reverting the README to the original master branch version. We can then update accordingly after any merge.

[PasqualeMainolfi]

I’ve restored the original README to preserve the historical context and existing information. Additionally, I’ve added release notes to the PR, without referencing any version numbers.

[rorywalsh]

Thanks @PasqualeMainolfi - I know Steven is extremely busy right now, but hopefully we can get this merged and into the extension in time for the first release of Csound 7 👍

[PasqualeMainolfi]

I was wondering if there is any approximate timeline available for the release of Csound7. Having a rough idea of the release date would help me plan the remaining work and properly schedule the effort needed to deliver a stable version of the LSP.

I am already quite close, and only a few final adjustments are still missing.

Thanks in advance for any information you can share.

[rorywalsh]

Good question. I think most of the tasks have been completed at this stage. So it could be in the next few weeks but I don't think any hard deadline has been set.

[kunstmusik]

This should be in English to match the rest of the plugin. (I haven't done I18N for vscode plugins before; we could investigate that in a separate PR)

[kunstmusik]

@PasqualeMainolfi I ran this in the extension development host. It requested installing the LSP and then reported success in installing the LSP. I checked before and after and CSD and ORC code looks no difference with little syntax highlighting. I did not get any manual entries, autocomplete, etc. I am on MacOS. Unsure if something's gone wrong in installation or what but it's not operating as (I assume) expected.

[PasqualeMainolfi]

It seems very strange. Check the lsp-bin folder and verify the downloaded executable, it’s probably a permissions issue. Also, check /var/folders/hp/.../T/ (temporary files) to see if the csound-lsp-temp-folder exists. This is where it downloads the external resources it uses. I’ve run several tests and everything seems fine. Additionally, I made a modification to the source code to avoid issues with the local manual server, especially on Windows (though I don’t think this is the cause of the problem).

One more thing: there is a check in the extension that disables the LSP in web environments:

  if (vscode.env.uiKind === vscode.UIKind.Web) {
    console.warn("LSP disabled in web");
    return;
  }

[kunstmusik]

I updated to latest and am getting further. Parser seems to have problems with floating point numbers that start with decimal:

I think because of the decimal issue it reports unclosed blocks:

Also multiline comments seem to throw things off:

[PasqualeMainolfi]

I think I have fixed the bugs you showed me.
To give priority to the LSP for handling multi-line comments, I had to disable the comments section in language.
Note that the LSP checks installed plugins on each startup. Could you please verify if everything is working as expected by checking the Developer Tools / Show Dev Logs for the Csound Language Server?
If anything was missed or there are additional things to implement, we can go over the details next.

[kunstmusik]

@PasqualeMainolfi I tried the latest and when I start up the extension host it asks to download a newer version of LSP. I says yet and then it errors:

[PasqualeMainolfi]

Thanks for the report!
The issue was caused by using a for...in loop instead of for...of when iterating over the files returned by fs.readdirSync().
for...in iterates over array indices (e.g. 0, 1, …), which caused the extension to try to unlink non-existent paths like lsp-bin/0.
This has now been fixed by switching to for...of, which correctly iterates over the actual file names.
The installer should work correctly now.

[kunstmusik]

One other note, I asked Opus about using tree-sitter in Javascript so that we can use it in the web extension version of this plugin. Looks like compiling to WASM would be an option, which is promising.

Looking at this PR, your concern is valid. The PR introduces a Csound Language Server built on a custom Tree-sitter grammar, which typically compiles to native binaries. This would indeed be incompatible with running as a VS Code web extension.

The Good News

Yes, Tree-sitter parsers can run in JavaScript/WebAssembly. Tree-sitter supports compiling grammars to WASM, which can then be loaded and run in JavaScript environments (including browsers and VS Code web extensions).

How to Make It Work for Web Extensions

The tree-sitter-csound grammar would need to be compiled to WASM instead of (or in addition to) native binaries. Here's what's typically required:

1. Compile the grammar to WASM:

   tree-sitter build --wasm

2. Use web-tree-sitter - the official JavaScript/WASM binding:

   npm install web-tree-sitter

3. Load the parser in JS:

   const Parser = require('web-tree-sitter');
   await Parser.init();
   const parser = new Parser();
   const Csound = await Parser.Language.load('tree-sitter-csound.wasm');
   parser.setLanguage(Csound);

Recommendations for the PR

1. Ask the PR author if the LSP server (csound-lsp) supports running in a browser/Node.js environment with the WASM-compiled parser, or if it's strictly a native binary.

2. Consider a hybrid approach:

   • Native LSP for desktop (full features, better performance)
   • WASM-based parsing for web extension (potentially reduced features)

3. Check the LSP implementation language - if it's written in a language that can compile to WASM (Rust, TypeScript, etc.), the entire LSP could potentially run in the browser.

[PasqualeMainolfi]

Yes, I can.
I think, the best approach would be to consider a hybrid version: a full-featured implementation for desktop, and a slightly limited version for the web. Certain features, like file watchers and other system-level integrations, simply cannot be ported to a web environment. Rust (and tree-sitter) is not an issue... the main challenge is deciding which parts of the functionality to bring to the web and which to leave out.

[kunstmusik]

I think we're on the same page, offer what we can in each environment. I think this will work out nicely and users will be happy. :)

[PasqualeMainolfi]

@rorywalsh check the LSP version in the lsp-bin folder, it should be 1.4.6. The web version is separate from the desktop one, so they don't touch each other. I did a fresh test from scratch just to be sure, and it all works fine. Additionally, verify that the csound-lsp_temp_folder directory exists within your system's temporary folders. On macOS, you can find it under /var/folders/.../T/, while on Windows it should be in the standard temp directory (usually %TEMP%). This folder is where all the extension's resources are stored. Checking your local repository and perhaps performing a rebase to ensure you are up to date. It really seems like this might be a local environment issue.

[PasqualeMainolfi]

@rorywalsh If you'd like, feel free to send me the entire script. There might be something breaking the syntax that I'm missing just by looking at the photo.

[rorywalsh]

Thank @PasqualeMainolfi I can confirm I'm using the latest build, at this point 1.5.0. I think I also found the source of my woes. When I am in edito mode in Cabbage, i can drag and move UI components. When I do, it dynamically updates the .csd text. It's at this point all my opcodes get squiggly lines under them. I'm pretty sure the problem is on my end. I need to make sure I'm inserting the text in the correct way. 👍

[rorywalsh]

Found the issue, it appears to have been an issue with a stale editor reference. I've updated my extension code and I no longer get any issues when my Cabbage extensions update any JSON code 👍

[PasqualeMainolfi]

@rorywalsh Perfect! At this point, I think we could make sure this no longer causes issues with the syntax.
If I understood correctly, you’re referring to the code that automatically appears after the closing CSD tag (</Cs…>). Is that right?

[rorywalsh]

Yes, or before those tags. My extension inserts json code into a Cabbage block. But the latest syntax errors were a result of my extension breaking the LSP. I've fixed this now on my end, so I think we're in the clear 👍

[rorywalsh]

Hi @PasqualeMainolfi I'm still getting some odd behaviour with this every now and then. For example:

and:

You'll notice how writePointer in the last image is not coloured correctly. This code appear before the first image I sent, maybe it's throwing something off?

[rorywalsh]

Thanks @PasqualeMainolfi, looking good now 👍

[PasqualeMainolfi]

@rorywalsh this one was quite difficult to fix. Csound’s syntax can be very ambiguous when it wants to be!
Anyway, it should now be properly resolved. It would also be good to test assignments to a struct member within the context of an opcode statement to make sure everything behaves correctly.
I have also enabled syntax highlighting for Cabbage widgets. They no longer generate errors even if there is no dedicated resource available to handle their methods.
In the next update, I plan to fix the code that gets inserted outside the CSD block.

[rorywalsh]

FWIW, there is no need to support the old-style Cabbage 2 syntax. Everything will be JSON moving forward. Apologies if you've spent time on this. I thought I had made it clear that it's only Cabbage JSON sections that are relevant here.

[rorywalsh]

@PasqualeMainolfi I have updated the structure of the code block to use a top level object instead of any array, which was the wrong choice to begin with. It now looks like this:

<Cabbage>
{
}
</Cabbage>

This change from [] to {} is causing some errors in the LSP, do you think you could update it?

[PasqualeMainolfi]

@rorywalsh Yes, of course. I also have another fix planned to include. As soon as I finish teaching my students, I’ll add it.

[rorywalsh]

Thanks @PasqualeMainolfi - oh and one other thing, I've add ARA support which is handled by a new JSON block. If you could add that too it would be great :)

[PasqualeMainolfi]

Ok, I can add it, but I need a bit more information. Can the block be placed inside the Cabbage block, or is it a standalone block? Does an ARA block have a specific key, or does it contain something specific? Could you show me a few examples so I can understand how it should look and parse it correctly? Also, please tell me which fields it contains and whether they are fixed by convention or configurable. Sorry, but I’ve never had the chance to work with this block before 😊.

[rorywalsh]

Apologies, I should have been more specific. It's a satandalone block. There shouldn't ever be a Cabbage and CabbageARA block in the same csd, and the contents are a single a top JSON object {} and multiple inner objects and arrays. An example would look like this:

<CabbageARA>
{
  "channels": [
    { "id": "ara_tempo",  "type": "number" },
    { "id": "ara_rms",    "type": "number" },
    { "id": "lufs",       "type": "number" },
    { "id": "ara_source", "type": "string" }
  ],
  "testFile": "/Users/rwalsh/audio/test.wav"
}
</CabbageARA>

[PasqualeMainolfi]

@rorywalsh check now if it’s correct.

[rorywalsh]

Thanks @PasqualeMainolfi this looks super. I won't get a chance to check it properly till later, but I appreciate the prompt turnaround as always.

[kunstmusik]

Checking in, there's a couple concerns I have:

1. It looks like the csound tree-sitter package is not specified in package.json as a dependency. I think it'd be good to get that published on NPM and add that to package.json.

2. It looks like the csound manual opcode entries has been added to this repo (many, many .md files). I think we need a different solution than having that here in this repo.

[PasqualeMainolfi]

@kunstmusik Hi Steven,
Regarding the first point: yes, the csound Tree-sitter package should be published on NPM. This is planned, but I was waiting for you to test the web version first.
For the second point: in the web version we don’t have access to the local filesystem. The current approach is a workaround that allows the web app to read the manual. I could try converting all the manual entries into a single .json file, which can be loaded at runtime. This would require restructuring some sections of the manual, and we might need to omit or simplify certain parts to keep the JSON size manageable.

[kunstmusik]

I tried the latest here and could not run the web version without fixing a typescript error and also updating npm dependency to 11.10.0 due to security constraint. Even then, trying to play the document did not work, it just hangs. I also tried to use npm run compile-web and got an error as the -web commands are expecting webpack.browser.config.js but that doesn't exist here. There's also an "npm" dependency in package.json which I'm not sure we need?

Also to note, once I modified package.json and package-lock.json was rebuilt, seems like I can't build/run the extension here now.

There's also a "Csound: Play Active Document" and a "Csound: Run Active Document". It's awkward as one works in web and other works in desktop, but they both show up for both environments. Would rather we keep to one command that adapts to work in browser or desktop.

For the md files, can't we just load from csound maual github repo? Perhaps cache it? I'm trying to think of a solution that doesn't require having a large resource added to this repo that can go stale.

[PasqualeMainolfi]

@kunstmusik Steven should now be able to compile without errors. I removed the toolbar because the API causes issues in the web version, at least based on what I could see.
Additionally, there was a problem importing CsoundObj in the web context. The type is not exported by @csound/browser in a way that Webpack can resolve, so trying to import it directly caused errors. To work around this, I temporarily replaced CsoundObj with any in the web code. This allows the code to compile, but we lose type checking for the csound object in the web version.
As soon as everything is ready, I’ll publish tree-sitter as an npm package. In the meantime, I’m also thinking about a way to avoid having to add all the .md files manually.