Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ Ordered by importance:

* `src/s-core-devcontainer/` contains the sources for the Eclipse S-CORE DevContainer.
It uses pre-existing [DevContainer features](https://containers.dev/implementors/features/) to provide some standard tools like Git, LLVM, and others.
In addition, it uses a so-called "local" feature (cf. `src/s-core-devcontainer/.devcontainer/s-core-local`) for the remaining tools and configuration.
In addition, it uses so-called "local" features (cf. `src/s-core-devcontainer/.devcontainer/s-core-local`) for the remaining tools and configuration.
* `scripts/` contains scripts to build and test the container.
* `.devcontainer/` contains the definition of the DevContainer for **this** repository, i.e. the "devcontainer devcontainer".
There should rarely be a need to modify this.
Expand Down
9 changes: 6 additions & 3 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@ One has to take ones own medicin and for that reason the [S-CORE devcontainer](.
│ ┌─────────────────────────────────────────────────────────────────┐ │
│ │ Build S-CORE DevContainer (src/s-core-devcontainer) │ │
│ │ - Dockerfile (Ubuntu base image) │ │
│ │ - Pre-existing features (Git, LLVM/Clang, Rust, …) │ │
│ │ - Pre-existing features (Git, LLVM/Clang, Rust, pre-commit, …) │ │
│ │ - bazel feature (available at /devcontainer/features/…) │ │
│ │ - S-CORE local feature (available at /devcontainer/features/…) │ │
│ └─────────────────────────────────────────────────────────────────┘ │
│ │ │
Expand All @@ -48,21 +49,23 @@ The container images should be able to build all of S-CORE without extra setup.
This requires that the needed tools are preinstalled, but not too much either to keep container image download times in check.

To achieve this, a small base image [based on Ubuntu is chosen](https://github.com/docker-library/buildpack-deps/blob/master/ubuntu/noble/curl/Dockerfile).
To this image, the tools needed to build S-CORE and run its tests are added - either via pre-existing devcontainer features, or our own [S-CORE feature](../src/s-core-devcontainer/.devcontainer/s-core-local/).
To this image, the tools needed to build S-CORE and run its tests are added - either via pre-existing devcontainer features, or our own [S-CORE](../src/s-core-devcontainer/.devcontainer/s-core-local/) and [bazel](../src/s-core-devcontainer/.devcontainer/bazel-feature/) features.
The tools also need to support typical IDE features like enabling code completion.
All of these tools could have been added via a `Dockerfile` as well, but features are the mechanism to achieve composable devcontainer implementations and are preferred instead.

The decision whether to use a pre-existing feature or to add a tool using the S-CORE feature is based on the tradeoff between build time and maintainability.
The chosen features are installed quickly, without us having to maintain them.
Other tools installed via the S-CORE feature either have no corresponding feature, or their feature took so much time to install, that it was quicker done using our own code (example: Python feature).

Bazel and related tools have been moved into their own feature to have its setup better isolated.

### Proxy environments

To support proxy environments, environment variables are set in the [`Dockerfile`](../src/s-core-devcontainer/.devcontainer/Dockerfile) and unset if empty to not interfere with non-proxy environments.

## Tests

After an image build, tests check that each tool expected to be in the image is installed with the specified version for [pre-existing features](../src/s-core-devcontainer/test-project/test.sh) and the [S-CORE feature](../src/s-core-devcontainer/.devcontainer/s-core-local/tests/test_default.sh).
After an image build, tests check that each tool expected to be in the image is installed with the specified version for [pre-existing features](../src/s-core-devcontainer/test-project/test.sh), the [S-CORE feature](../src/s-core-devcontainer/.devcontainer/s-core-local/tests/test_default.sh) and the [bazel feature](../src/s-core-devcontainer/.devcontainer/bazel-feature/tests/test_default.sh).
This may seem overly complex at first, but prevents (1) accidentially wrong versions, (2) completely broken tools that cannot even execute.
Both cases can happen and have happened in the past already, e.g. due to unexpected interactions between devcontainer features.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "Bazel and tools",
"id": "bazel",
"version": "1.0.0",
"description": "Bazel and supplimentary tools for working with Bazel-based projects.",
"dependsOn": {
"./s-core-local": {} // needed for extracting versions (versions.sh)
},
"onCreateCommand": "/devcontainer/features/bazel/on_create_command.sh",
"postCreateCommand": {
// The repos in S-CORE may use different Bazel versions. This ensures that the required version is installed.
// Should be removed once we provide versions of the devcontainer.
"Install matching Bazel version": "bash /devcontainer/features/bazel/install_matching_bazel_version.sh"
},
"mounts": [
{
"source": "eclipse-s-core-bazel-cache",
"target": "/var/cache/bazel",
"type": "volume"
}
]
}
95 changes: 95 additions & 0 deletions src/s-core-devcontainer/.devcontainer/bazel-feature/install.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
#!/usr/bin/env bash
set -euo pipefail

# Copy feature sources and tests to expected location
FEATURES_DIR="/devcontainer/features"
SCRIPT_PATH=$(readlink -f "$0")
SCRIPT_DIR=$(dirname -- "${SCRIPT_PATH}")
mkdir -p "${FEATURES_DIR}"
COPY_TARGET="${FEATURES_DIR}/$(basename "${SCRIPT_DIR%%_*}")"
cp -R "${SCRIPT_DIR}" "${COPY_TARGET}"
rm -f "${COPY_TARGET}/devcontainer-features.env" "${COPY_TARGET}/devcontainer-features-install.sh"

DEBIAN_FRONTEND=noninteractive

# Read tool versions + metadata into environment variables
. /devcontainer/features/s-core-local/versions.sh /devcontainer/features/bazel/versions.yaml

ARCHITECTURE=$(dpkg --print-architecture)

apt-get update

# INSTALL CONTAINER BUILD DEPENDENCIES
# Container build dependencies are not pinned, since they are removed anyway after container creation.
apt-get install apt-transport-https -y

# Bazelisk, directly from GitHub
# Using the existing devcontainer feature is not optimal:
# - it does not check the SHA256 checksum of the downloaded file
# - it cannot pre-install a specific version of Bazel, or prepare bash completion
BAZELISK_VARIANT="amd64"
SHA256SUM="${bazelisk_amd64_sha256}"
if [ "${ARCHITECTURE}" = "arm64" ]; then
BAZELISK_VARIANT="arm64"
SHA256SUM="${bazelisk_arm64_sha256}"
fi
curl -L "https://github.com/bazelbuild/bazelisk/releases/download/v${bazelisk_version}/bazelisk-${BAZELISK_VARIANT}.deb" -o /tmp/bazelisk.deb
echo "${SHA256SUM} /tmp/bazelisk.deb" | sha256sum -c - || exit -1
apt-get install -y --no-install-recommends --fix-broken /tmp/bazelisk.deb
rm /tmp/bazelisk.deb

# Pre-install a fixed Bazel version, setup the bash command completion
export USE_BAZEL_VERSION=${bazel_version}
# bazelisk downloads Bazel into the home directory of the user running the command
# lets assume there is only one user in the devcontainer
su $(ls /home) -c "bazel help completion bash > /tmp/bazel-complete.bash"
mkdir -p /etc/bash_completion.d
mv /tmp/bazel-complete.bash /etc/bash_completion.d/bazel-complete.bash
sh -c "echo 'INSTALLED_BAZEL_VERSION=${bazel_version}' >> /devcontainer/features/bazel/bazel_setup.sh"

# Configure Bazel to use system trust store for SSL/TLS connections
# This is required for corporate environments with custom CA certificates
echo 'startup --host_jvm_args=-Djavax.net.ssl.trustStore=/etc/ssl/certs/java/cacerts --host_jvm_args=-Djavax.net.ssl.trustStorePassword=changeit' >> /etc/bazel.bazelrc

# Buildifier, directly from GitHub (apparently no APT repository available)
# The version is pinned to a specific release, and the SHA256 checksum is provided by the devcontainer-features.json file.
BUILDIFIER_VARIANT="amd64"
SHA256SUM="${buildifier_amd64_sha256}"
if [ "${ARCHITECTURE}" = "arm64" ]; then
BUILDIFIER_VARIANT="arm64"
SHA256SUM="${buildifier_arm64_sha256}"
fi
curl -L "https://github.com/bazelbuild/buildtools/releases/download/v${buildifier_version}/buildifier-linux-${BUILDIFIER_VARIANT}" -o /usr/local/bin/buildifier
echo "${SHA256SUM} /usr/local/bin/buildifier" | sha256sum -c - || exit -1
chmod +x /usr/local/bin/buildifier

# Starlark Language Server, directly from GitHub (apparently no APT repository available)
STARPLS_VARIANT="amd64"
SHA256SUM="${starpls_amd64_sha256}"
if [ "${ARCHITECTURE}" = "arm64" ]; then
STARPLS_VARIANT="aarch64"
SHA256SUM="${starpls_arm64_sha256}"
fi
curl -L "https://github.com/withered-magic/starpls/releases/download/v${starpls_version}/starpls-linux-${STARPLS_VARIANT}" -o /usr/local/bin/starpls
echo "${SHA256SUM} /usr/local/bin/starpls" | sha256sum -c - || exit -1
chmod +x /usr/local/bin/starpls

# Code completion for C++ code of Bazel projects
# (see https://github.com/kiron1/bazel-compile-commands)
source /etc/lsb-release
curl -L "https://github.com/kiron1/bazel-compile-commands/releases/download/v${bazel_compile_commands_version}/bazel-compile-commands_${bazel_compile_commands_version}-${DISTRIB_CODENAME}_${ARCHITECTURE}.deb" -o /tmp/bazel-compile-commands.deb
# Extract correct sha256 for current DISTRIB_CODENAME and check
SHA256SUM="${bazel_compile_commands_amd64_sha256}"
if [ "${ARCHITECTURE}" = "arm64" ]; then
SHA256SUM="${bazel_compile_commands_arm64_sha256}"
fi
echo "${SHA256SUM} /tmp/bazel-compile-commands.deb" | sha256sum -c - || exit -1
apt-get install -y --no-install-recommends --fix-broken /tmp/bazel-compile-commands.deb
rm /tmp/bazel-compile-commands.deb

# Cleanup
# REMOVE CONTAINER BUILD DEPENDENCIES
apt-get remove --purge -y apt-transport-https
apt-get autoremove -y
apt-get clean
rm -rf /var/lib/apt/lists/*
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
#!/usr/bin/env bash
set -eo pipefail

. /devcontainer/features/s-core-local/bazel_setup.sh || true
. /devcontainer/features/bazel/bazel_setup.sh || true

if [ -f .bazelversion ] && [ "$(cat .bazelversion)" != "$INSTALLED_BAZEL_VERSION" ]; then
# Pre-install the matching Bazel version, setup the bash command completion
Expand All @@ -15,5 +15,5 @@ if [ -f .bazelversion ] && [ "$(cat .bazelversion)" != "$INSTALLED_BAZEL_VERSION

bazel help completion ${bash} > /tmp/bazel-complete.bash
sudo mv /tmp/bazel-complete.bash /etc/bash_completion.d/bazel-complete.bash
echo "INSTALLED_BAZEL_VERSION=$USE_BAZEL_VERSION" | sudo tee /devcontainer/features/s-core-local/bazel_setup.sh
echo "INSTALLED_BAZEL_VERSION=$USE_BAZEL_VERSION" | sudo tee /devcontainer/features/bazel/bazel_setup.sh
fi
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/usr/bin/env bash
set -euo pipefail

# Enable persistent Bazel cache
#
# Usually, a volume is mounted to /var/cache/bazel (see
# devcontainer-feature.json). This shall be used as Bazel cache, which is then
# preserved across container re-starts. Since the volume has a fixed
# name ("eclipse-s-core-bazel-cache"), it is even used across all Eclipse
# S-CORE DevContainer instances.
if [ -d /var/cache/bazel ]; then
echo "Bazel Cache: /var/cache/bazel exists. Checking ownership and configuring Bazel to use it as cache..."
current_owner_group=$(stat -c "%U:%G" /var/cache/bazel)
current_user_group="$(id -un):$(id -gn)"
if [ "${current_owner_group}" = "${current_user_group}" ]; then
echo "Bazel Cache: /var/cache/bazel is already owned by ${current_user_group}. "
else
echo "Bazel Cache: /var/cache/bazel is not owned by ${current_owner_group}. Setting ownership (this may take a few seconds) ..."
sudo chown -R "${current_user_group}" /var/cache/bazel
fi
echo "Bazel Cache: Configuring Bazel to use /var/cache/bazel as cache..."
echo "startup --output_user_root=/var/cache/bazel" >> ~/.bazelrc
fi
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/usr/bin/env bash
set -euo pipefail

# Read tool versions + metadata into environment variables
. /devcontainer/features/s-core-local/versions.sh /devcontainer/features/bazel/versions.yaml

# Bazel-related tools
## This is the bazel version preinstalled in the devcontainer.
## A solid test would disable the network interface first to prevent a different version from being downloaded,
## but that requires CAP_NET_ADMIN, which is not yet added.
export USE_BAZEL_VERSION=${bazel_version}
check "validate bazelisk is working and has the correct version" bash -c "bazelisk version | grep '${bazelisk_version}'"
check "validate bazel is working and has the correct version" bash -c "bazel version | grep '${bazel_version}'"
unset USE_BAZEL_VERSION

check "validate buildifier is working and has the correct version" bash -c "buildifier --version | grep '${buildifier_version}'"
check "validate starpls is working and has the correct version" bash -c "starpls version | grep '${starpls_version}'"
check "validate bazel-compile-commands is working and has the correct version" bash -c "bazel-compile-commands --version 2>&1 | grep '${bazel_compile_commands_version}'"
52 changes: 52 additions & 0 deletions src/s-core-devcontainer/.devcontainer/bazel-feature/versions.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
bazel:
# https://github.com/bazelbuild/bazel/releases -- latest version as of 2025-09-24
version: 8.4.1
# no need to define sha256 here, as bazel is installed via bazelisk

buildifier:
version: 8.2.1
amd64:
# The following sha256sum is for the binary buildifier-linux-amd64
# from the GitHub release page of buildtools
# It is generated by running 'sha256sum buildifier-linux-amd64'
sha256: 6ceb7b0ab7cf66fceccc56a027d21d9cc557a7f34af37d2101edb56b92fcfa1a
arm64:
# The following sha256sum is for the binary buildifier-linux-arm64
# from the GitHub release page of buildtools
# It is generated by running 'sha256sum buildifier-linux-arm64'
sha256: 3baa1cf7eb41d51f462fdd1fff3a6a4d81d757275d05b2dd5f48671284e9a1a5

bazelisk:
version: 1.27.0
amd64:
# The following sha256sums are for the deb package bazelisk_<version>_amd64.deb
# It is generated by running 'sha256sum bazelisk_<version>_amd64.deb'
sha256: d8b00ea975c823e15263c80200ac42979e17368547fbff4ab177af035badfa83
arm64:
# The following sha256sums are for the deb package bazelisk_<version>_arm64.deb
# It is generated by running 'sha256sum bazelisk_<version>_arm64.deb'
sha256: 173c5b367b485a30ce58c1d0d560b39d257a2d7a3c859c45d7d05eb61605a2a1

starpls:
version: 0.1.22
amd64:
# The following sha256sum is for the binary starpls-linux-amd64
# from the GitHub release page of starpls
# It is generated by running 'sha256sum starpls-linux-amd64'
sha256: 7c661cdde0d1c026665086d07523d825671e29056276681616bb32d0273c5eab
arm64:
# The following sha256sum is for the binary starpls-linux-arm64
# from the GitHub release page of starpls
# It is generated by running 'sha256sum starpls-linux-arm64'
sha256: 55877ec4c3ff03e1d90d59c76f69a3a144b6c29688747c8ac4d77993e2eef1ad

bazel_compile_commands:
version: 0.18.0
amd64:
# The following sha256sums are for the deb package bazel-compile-commands_<version>-noble_amd64.deb
# It is generated by running 'sha256sum bazel-compile-commands_<version>-noble_amd64.deb'
sha256: 6735ea846241497094719792ad3d4f67b1d3123048693d34fcdf7190f8c2da4e
arm64:
# The following sha256sums are for the deb package bazel-compile-commands_<version>-noble_arm64.deb
# It is generated by running 'sha256sum bazel-compile-commands_<version>-noble_arm64.deb'
sha256: d73998efa01cbd501b82ad6266642464b78ecd9fc6224a60c9cb558182d52d88
8 changes: 5 additions & 3 deletions src/s-core-devcontainer/.devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -37,15 +37,17 @@
"rustfmt"
]
},
"./s-core-local": {}
"./s-core-local": {},
"./bazel-feature": {}
},
"overrideFeatureInstallOrder": [
// this order makes it more convenient to develop the s-core-local feature, since it will be installed last
// this order makes it more convenient to develop the local features, since they will be installed last
// which means changes to it will be applied without needing to rebuild all other features
"ghcr.io/devcontainers/features/common-utils",
"ghcr.io/devcontainers-community/features/llvm",
"ghcr.io/devcontainers/features/rust",
"./s-core-local"
"./s-core-local",
"./bazel-feature"
],
"remoteUser": "vscode",
"customizations": {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,18 +10,10 @@
},
"onCreateCommand": "/devcontainer/features/s-core-local/on_create_command.sh",
"postCreateCommand": {
// The repos in S-CORE may use different Bazel versions. This ensures that the required version is installed.
// Should be removed once we provide versions of the devcontainer.
"Install matching Bazel version": "bash /devcontainer/features/s-core-local/install_matching_bazel_version.sh",
"Setup persistent bash history": "bash /devcontainer/features/s-core-local/setup_command_history.sh",
"Enable pre-commit hooks": "bash /devcontainer/features/s-core-local/enable_pre_commit_hooks.sh"
},
"mounts": [
{
"source": "eclipse-s-core-bazel-cache",
"target": "/var/cache/bazel",
"type": "volume"
},
{
"source": "eclipse-s-core-bash-history-${devcontainerId}",
"target": "/commandhistory",
Expand Down
Loading