- Introduction
- To project collaborators
- Using the final setup
- Building and running
- Prerequisites
- Local setup
- The public Docker based setup case
- Prerequisites for the public Docker based setup case
- Environment variables for the public Docker based setup case
- Building the public Docker based setup case
- Running the public Docker based setup case
- Adding pod contents to public Docker based setup case
- Using the public Docker based setup case
- Viewer URLs and Cypress target
- Bonus: local Viewer (Miravi) for querying public pods
- Stopping the public Docker based setup case
- Other documentation resources
This repository contains the implementation of the Solid4Media IDLab Demo Platform as part of the Solid4Media project.
This repository supports different setup cases. Some are only valid during development. The final and publicly available setup of the Solid4Media IDLab Demo Platform relies on Docker containers and Docker Compose and illustrates the decentralized nature of the Solid-based data sharing platform.
Within the platform, we have set up:
- multiple data providers each publishing their data behind a secure access layer using Solid pods,
- a baseline setup in which a Viewer executes federated queries directly on these Solid pods,
- a second setup, which is the main project goal, in which an aggregator and a UMA authorization server are added to support privacy-preserving aggregation. This setup also includes a UMA-compliant Viewer and LOAMA, an application that supports data owners in managing access to their data.
You may be interested first of all in some quick recipes.
The final setup is available online and needs no further action prior to usage.
Besides the baseline Viewer setup, this repository also contains a second setup.
In this setup, a Solid Community Server (css2) is combined with a UMA authorization server, an aggregator service,
a UMA-compliant Viewer, and a LOAMA frontend.
This setup is the main project goal. It demonstrates the privacy-preserving aggregation scenario shown on the landing page, where a family-level media profile is derived from individual profiles without exposing those individual profiles directly.
To use the included viewers, navigate to https://solid4media.ilabt.imec.be/viewer1/ for the baseline setup or to https://solid4media.ilabt.imec.be/viewer2/ for setup 2.
The list of predefined queries is shown in expandible groups at the left side.
Some queries may require you to login as one of the actors.
That is because read permissions to resources may be restricted to specific actors.
Find actors' email addresses and passwords in the overview of actors' WebIDs, emails and passwords.
To login, click on the icon in the top right.
To execute a query, click on it in the dashboard.
We split up this section for the different setup cases.
Unless specified otherwise, execute all commands below from a bash shell in the repository root.
The different setup cases depend on common files.
Some common files read environment variables, that need to be set upfront by sourcing the environment variables files.
Some files cannot read environment variables. In these cases derived files are created from common templates.
- a bash shell
- Node >= 20 with npm
- yarn classic
For the UMA setup, eye needs to be available
git clone https://github.com/eyereasoner/eye
cd eye
sudo apt-get install software-properties-common
sudo apt-add-repository ppa:swi-prolog/stable
sudo apt-get update
sudo apt-get install swi-prolog
./install.sh --prefix=$HOME/.local
cd ..For the aggregator: install rust + libclang
curl https://sh.rustup.rs -sSf | sh
source $HOME/.cargo/env-
rustup update
sudo apt update
sudo apt install -y clang libclang-dev llvm-dev pkg-configExecute this command in any terminal window, before executing any other command in the remainder of this section:
source env-localhostBefore continuing, make sure the result of a previous build isn't running. See Stopping the localhost setup case.
Next, execute:
# install node dependencies
yarn install
# all further build actions
./scripts/setup/finalize-setup.shRun with new pod contents (mandatory for first run):
./scripts/local-run/start-csss.shRe-run with existing pod contents (requires a previous run and stop):
./scripts/local-run/start-csss.sh -rThe commands above start the pods on the localhost in the background and wait until they are all ready (listening). This takes some time to complete.
The pods log files can be consulted at ./local-run/*.log.
The pods data can be viewed at ./local-run/data/css*/.
Only in the baseline setup is the CSS hosted in Docker.
This assumes the following additional prerequisites to the localhost setup case:
- Docker Engine was installed.
- Docker host networking is available. See also the Docker host networking tutorial.
Execute:
./scripts/stuff-pods/stuff-pods.shUsing the localhost setup case is similar to using the final setup, with the following differences:
- The included Viewers:
-
needs to be started as follows in a separate terminal windows:
- for scenario 1 (baseline):
cd ../applied-in-architecture-miravi-a-linked-data-viewer/main node scripts/select-config.cjs ../../../architecture/actors/viewer1/setup npm run dev -- --port 5174- for setup 2 (project goal):
cd ../applied-in-architecture-miravi-a-linked-data-viewer-uma-compliant/main npm run dev -- --port 5175 -
and can be used at the localhost port reported in the output of above command.
-
./scripts/local-run/stop-csss.sh- a bash shell
- Node >= 20 with npm
- yarn classic
- Docker Engine and Docker Compose
- Depending on your platform, different installation guides are available from the above links.
Execute this command in any terminal window, before executing any other command in the remainder of this section:
source env-docker-publicBefore continuing, make sure the result of a previous build isn't running. See Stopping the public Docker based setup case.
Next, execute:
# install node dependencies
yarn install
# all further build actions
./scripts/setup/finalize-setup.shRun with new pod contents (mandatory for first run):
docker compose --profile backend --profile frontend up --waitRe-run with existing pod contents (requires a previous run and stop):
docker compose -f docker-compose-public-restart.yml --profile backend --profile frontend up --waitThe commands above start all services in a Docker environment and and wait until they are all ready (listening). This takes some time to complete.
Optional: if you're interested in what's happening while these commands execute, you may open a new terminal window and in it, execute:
docker compose --profile backend --profile frontend logs -fThe following command may be executed on the server or on any other computer fulfilling the prerequisites and having access to the public pods, if all above steps where executed before.
Execute:
./scripts/stuff-pods/stuff-pods.shUsage was explained already above in using the final setup. Note that in setup 2, the aggregator, UMA server, and UMA-compliant CSS are not set up in Docker; they are run via PM2 on the server. Also note that in setup 2, granting access to the aggregated family profile is not included in the setup scripts. Access grants still need to be made (for example via LOAMA): each family member (data contributor) must grant access before the aggregated output becomes visible in Viewer2.
The viewers are reachable and intended to coexist at different URLs.
- Localhost setup:
- Viewer1 (Miravi baseline scenario 1): http://localhost:5174/
- Viewer2 (Miravi2 setup 2, with aggregator): http://localhost:5175/
- Public Docker based setup:
- Viewer1 (Miravi baseline scenario 1): https://solid4media.ilabt.imec.be/viewer1/
- Viewer2 (Miravi2 setup 2, with aggregator): https://solid4media.ilabt.imec.be/viewer2/
For Cypress, the default base URL is controlled by TV_CYPRESS_BASE_URL in the sourced environment file.
By default, this points to Viewer (Miravi), so existing yarn run test:viewer usage remains unchanged.
To run Cypress against Viewer2 without changing files, override baseUrl on the command line:
# after sourcing env-localhost or env-docker-public
yarn run test:viewer --config baseUrl=${TV_VIEWER2_BASE_URL}/Interactive mode for Viewer2:
yarn run test:viewer:interactive --config baseUrl=${TV_VIEWER2_BASE_URL}/There is one pleasant side effect of building the public Docker based setup case: the Viewer (Miravi) described in Running the localhost setup case is also available here at the localhost. This time however it addresses the public pods! This is very convenient for further query experiments on the data in the public pods... For more info, see The modified Viewer (Miravi).
docker compose --profile backend --profile frontend down -t 0