jetbrains-gateway in air-gapped environment can't download the backend

[Heshmatkhah]

Hi
In addition to #22,

When you use JetBrains Client Downloader to clone required IDEs into your air-gapped environment, It will create/update a /products.json file witch contains all required information of your mirror, including

• build number
• version
• download link

But, The module hardcoded the download link of each IDE instead of getting current the releases_base_link/products.json

Also the version info is downloaded from {releases_base_link}/products/releases instead of {releases_base_link}/products.json and that endpoint is only available on official mirror.

Potential fix:

Use {releases_base_link}/products.json to get release information.

When you are generating effective_jetbrains_ide_versions from jetbrains_ide_versions, save the .releases[{INDEX}].downloads[{INDEX_2}].{PLATFORM}.link and use that with the download_base_link!

In most air-gapped environments, the download_base_link and releases_base_link should be the same.

[matifali]

PLease use https://registry.coder.com/modules/coder/jetbrains

[Heshmatkhah]

That module DOES NOT FIT an air-gapped environment:
(please read the issue description one more time)

the JetBrains Client Downloader doesn't store products in the same directory structure as original JetBrains server, But it Fixes the Release JSON (products.json), so you need to use the data returned from the releases_base_link!

The module (both of them) has a download_base_link

BUT, for an air-gapped environment, you SHOULD use the download links that are stored inside the jetbrains_ide_versions.

jetbrains-gateway and jetbrains ignoring this download URLs and constructing their own download URL witch is non existing in an air-gapped environment created using JetBrains Client Downloader!!!

[Heshmatkhah]

Hi @matifali,

Right now, those modules aren’t using the correct products.json that is generated by jetbrains-clients-downloader.

So what’s the expected approach here? Should we manually update the template every time JetBrains publishes a new release?

If that’s the case, it defeats the purpose of using Coder in the first place. The whole point is automation. Otherwise, we might as well skip Coder entirely and run the remote development setup manually end-to-end.

[matifali]

Hi @Heshmatkhah, I am happy to help you resolve this, and if it needs any improvements or fixes in the module, we are happy to accept that. I do not run an air-gapped environment myself, but I am willing to work with you to resolve this.

Do you have any suggestions on how we can fix this?

I recently merged a fix in #836, which should further improve the air-gapped experience by removing unnecessary API calls to resolve IDE versions.

There is another improvement in coder/coder-jetbrains-toolbox#290 that would allow you to configure custom URLs in the plugin settings, too.

[Heshmatkhah]

Hi @matifali, thank you for your response.

Issue #836 is helpful — appreciated.
Also, Please reopen this issue, as the underlying problem still persists.

Mirror Setup

This is my current mirror structure:

Each time I update the mirror, I configure jetbrains-clients-downloader to download only the latest version to minimize disk usage. As a result:

• products.json files are rewritten to include only that latest version.
• The files reflect all IDEs included in the most recent successful run.

Types of products.json in the Mirror

There are two distinct types:

1. Global products.json (Backend Root)

Located at:

mirror/backends/products.json

• Contains all IDE backends.
• Example: mirror/backends/products.json

2. Per-IDE products.json

Located at:

mirror/backends/<PRODUCT>/products.json

• Contains metadata specific to a single IDE backend.
• Example (PY): mirror/backends/PY/products.json

Note

As shown in the screenshot, multiple versions physically exist in the mirror, but only versions that was included in latest run of jetbrains-clients-downloader are listed in products.json.

Key Constraint (Air-Gapped Usage)

The primary purpose of this mirror is use in an air-gapped environment. Unlike the official API:

• All links in products.json is relative to the location of the products.json file.

Expected Resolution Behavior

Given a mirror root:

https://jetbrains.example.com/mirror/

Clients should:

1. Discover available IDEs via either:

   https://jetbrains.example.com/mirror/backends/products.json
   

   or

   https://jetbrains.example.com/mirror/backends/<PRODUCT>/products.json
   

2. Download artifacts using:

   <mirror_root_url> + releases[0].downloads.linux.link
   

Where link is a relative path defined inside products.json.

note that the linux is platform and as --platforms-filter <arg> A list of platforms to download CWM Guests for. Supported values: windows-x64, windows-aarch64, linux-x64, linux-aarch64, osx-x64, osx-aarch64. it may be macM1, mac, windowsARM64, windows, linuxARM64.

[matifali]

We need to rethink this issue in context of JetBrains Toolbox Plugin. I am tagging @Leomedina for further help here.