diff --git a/docs/deployment-guide-keycloak.md b/docs/deployment-guide-keycloak.md new file mode 100644 index 0000000..2cf55e8 --- /dev/null +++ b/docs/deployment-guide-keycloak.md @@ -0,0 +1,1406 @@ +# Governance Platform Deployment Guide (Keycloak) + +End-to-end guide for deploying the EQTY Lab Governance Platform on Kubernetes with Keycloak as the identity provider. + +## Table of Contents + +1. [Overview](#1-overview) +2. [Prerequisites](#2-prerequisites) +3. [Infrastructure Setup](#3-infrastructure-setup) +4. [Domain & TLS Configuration](#4-domain--tls-configuration) +5. [Deploying Keycloak](#5-deploying-keycloak) +6. [Running Keycloak Bootstrap](#6-running-keycloak-bootstrap) +7. [Creating Kubernetes Secrets](#7-creating-kubernetes-secrets) +8. [Configuring values.yaml](#8-configuring-valuesyaml) +9. [Deploying the Governance Platform](#9-deploying-the-governance-platform) +10. [Post-Install Setup & Verification](#10-post-install-setup--verification) + +--- + +## 1. Overview + +### What You're Deploying + +The Governance Platform consists of five microservices deployed via a single Helm umbrella chart (`governance-platform`), backed by a PostgreSQL database, and integrated with an external Keycloak instance for identity and access management. + +### Architecture + +```mermaid +flowchart TD + + A[πŸ‘₯ Users] --> PE[🌍 Public Endpoint] + PE --> K8s + + subgraph K8s[☸️ Kubernetes Cluster] + I[🚦 Ingress - NGINX + TLS] --> GS[πŸ–₯️ Governance Studio] + + GS --> GSV[βš™οΈ Governance Service] + GS --> AUTH[πŸ” Auth Service] + GS --> INT[πŸ›‘οΈ Integrity Service] + + GSV --> DB[πŸ—„οΈ PostgreSQL] + AUTH --> DB + INT --> DB + end + + subgraph EXT[🧩 External Dependencies] + KC[πŸ”‘ Keycloak - IdP] + OS[πŸ“¦ Object Storage] + KV[πŸ—οΈ Azure Key Vault] + end + + K8s --> EXT +``` + +### Platform Services + +| Service | Language | Description | Ingress Path | +| ---------------------- | -------- | --------------------------------------------- | --------------------- | +| **governance-studio** | React | Web UI for governance workflows | `/` | +| **governance-service** | Go | Backend API, workflow engine, worker | `/governanceService/` | +| **auth-service** | Go | Authentication, authorization, token exchange | `/authService/` | +| **integrity-service** | Rust | Verifiable credentials and lineage tracking | `/integrityService/` | +| **PostgreSQL** | β€” | Shared database (Bitnami Helm chart) | Internal only | + +All four application services are exposed through a single domain via NGINX Ingress with path-based routing. PostgreSQL is internal to the cluster. + +### External Dependencies + +These components live **outside** the `governance-platform` Helm chart and must be provisioned separately before deploying. + +| Dependency | Purpose | Required? | +| -------------------- | ------------------------------------------------------------------ | --------- | +| **Keycloak** | Identity provider β€” manages users, realms, OAuth clients | Yes | +| **Object Storage** | Artifact and document storage (Azure Blob, GCS, or AWS S3) | Yes | +| **Azure Key Vault** | DID signing key management for verifiable credentials (VC signing) | Yes | +| **DNS** | A-record or CNAME pointing your domain to the cluster ingress | Yes | +| **TLS Certificates** | cert-manager with a ClusterIssuer/Issuer, or pre-provisioned certs | Yes | + +### Helm Chart Structure + +The deployment uses an **umbrella chart pattern**. You deploy a single chart (`governance-platform`) which pulls in all subcharts as dependencies: + +``` +charts/ +β”œβ”€β”€ governance-platform/ # Umbrella chart β€” deploy this +β”‚ β”œβ”€β”€ Chart.yaml # Declares subchart dependencies +β”‚ β”œβ”€β”€ values.yaml # Default values for all services +β”‚ β”œβ”€β”€ templates/ # Shared resources (secrets, config, hooks) +β”‚ └── examples/ # Ready-to-use values files +β”‚ β”œβ”€β”€ values-keycloak.yaml # Keycloak deployment example +β”‚ β”œβ”€β”€ values-auth0.yaml # Auth0 deployment example +β”‚ └── secrets-sample.yaml # Secrets template +β”œβ”€β”€ governance-studio/ # Frontend subchart +β”œβ”€β”€ governance-service/ # Backend API subchart +β”œβ”€β”€ integrity-service/ # Credentials/lineage subchart +β”œβ”€β”€ auth-service/ # Authentication subchart +└── keycloak-bootstrap/ # Keycloak realm/client configuration (standalone) +``` + +The `keycloak-bootstrap` chart is deployed **separately** β€” it runs a one-time Kubernetes Job that configures the Keycloak realm, OAuth clients, scopes, and an initial admin user. + +### OAuth Clients + +The Keycloak bootstrap creates three OAuth clients in the `governance` realm: + +| Client ID | Type | Purpose | +| ------------------------------ | ----------------------------------- | -------------------------------------------------------------------------------------- | +| `governance-platform-frontend` | Public (SPA) | Browser-based authentication for governance-studio | +| `governance-platform-backend` | Confidential | Service-to-service auth, has service account with `query-users` and `view-users` roles | +| `governance-worker` | Confidential (service account only) | Automated governance workflow execution | + +### Deployment Flow + +The end-to-end deployment follows this order: + +``` +1. Provision infrastructure (storage, key vault, DNS, TLS) + β”‚ +2. Deploy Keycloak (if self-hosted) + β”‚ +3. Run keycloak-bootstrap (creates realm, clients, admin user in Keycloak) + β”‚ +4. Create Kubernetes secrets (uses Keycloak-generated client secrets) + β”‚ +5. Configure values.yaml + β”‚ +6. Deploy governance-platform (Helm umbrella chart) + β”‚ + β”œβ”€β”€ PostgreSQL starts, initializes databases + β”œβ”€β”€ governance-service starts, runs migrations + β”œβ”€β”€ Post-install hook creates organization + admin user in DB + β”œβ”€β”€ auth-service, integrity-service, governance-studio start + β”‚ +7. Post-install verification +``` + +> **Key ordering note:** The `keycloak-bootstrap` chart must be run **before** deploying the governance-platform, because the platform services need valid OAuth client credentials at startup. The governance-platform chart includes a post-install hook that automatically creates the organization and platform-admin user in the database after migrations complete. + +--- + +## 2. Prerequisites + +### Tools + +| Tool | Minimum Version | Purpose | +| ----------- | --------------- | ---------------------------------------- | +| **kubectl** | 1.21+ | Kubernetes cluster management | +| **Helm** | 3.8+ | Chart deployment | +| **jq** | 1.6+ | JSON processing (used by helper scripts) | +| **curl** | β€” | API calls (used by helper scripts) | +| **openssl** | β€” | Generating random secrets | + +### Kubernetes Cluster + +- Kubernetes **1.21+** with RBAC enabled +- **NGINX Ingress Controller** installed and configured as the default ingress class (see [`scripts/nginx.sh`](../scripts/nginx.sh)) +- **cert-manager** installed with a ClusterIssuer or Issuer configured for TLS (see [`scripts/cert-issuer.sh`](../scripts/cert-issuer.sh)) +- Sufficient resources for the platform (recommended minimums): + +| Component | CPU Request | Memory Request | Storage | +| ------------------ | ----------- | -------------- | -------- | +| governance-service | 250m | 256Mi | β€” | +| auth-service | 250m | 256Mi | β€” | +| integrity-service | 250m | 256Mi | β€” | +| governance-studio | 100m | 128Mi | β€” | +| PostgreSQL | 500m | 1Gi | 10Gi PVC | + +### Keycloak Instance + +A running Keycloak server accessible from within the Kubernetes cluster. This can be: + +- **Self-hosted in the same cluster** β€” deployed via the [Bitnami Keycloak Helm chart](https://github.com/bitnami/charts/tree/main/bitnami/keycloak) or the official [Keycloak Operator](https://www.keycloak.org/operator/installation) +- **Self-hosted on a separate cluster or VM** +- **Managed Keycloak service** (e.g., Red Hat SSO) + +Requirements: + +- Keycloak admin credentials available (username + password for the `master` realm) +- Network connectivity from the governance namespace pods to Keycloak's HTTP port +- If using an external Keycloak, a publicly accessible URL (e.g., `https://keycloak.your-domain.com`) +- If using an in-cluster Keycloak, internal service DNS is sufficient (e.g., `http://keycloak:8080/keycloak`) + +### Container Registry Access + +Platform images are hosted on GitHub Container Registry (GHCR). You need: + +- A **GitHub Personal Access Token (PAT)** with `read:packages` scope +- Or access to a mirror registry containing the platform images + +### Cloud Provider Resources + +Depending on your cloud provider, provision the following **before** deployment: + +**Object Storage** (one of): + +- **Azure Blob Storage** β€” storage account + container(s) for governance artifacts and integrity store +- **Google Cloud Storage** β€” bucket(s) + service account with storage admin permissions +- **AWS S3** β€” bucket(s) + IAM user/role with read/write access + +**Key Vault** (for verifiable credential signing): + +- **Azure Key Vault** β€” vault instance + service principal with key sign/verify permissions + +### DNS + +A domain name (or subdomain) that you control, with the ability to create A-records or CNAMEs pointing to your cluster's ingress controller external IP. + +The platform uses a **single domain** with path-based routing: + +| URL Path | Service | +| ------------------------------------------------------- | ------------------------ | +| `https://governance.your-domain.com/` | governance-studio (UI) | +| `https://governance.your-domain.com/governanceService/` | governance-service (API) | +| `https://governance.your-domain.com/authService/` | auth-service | +| `https://governance.your-domain.com/integrityService/` | integrity-service | + +Keycloak typically runs on a **separate domain** (e.g., `https://keycloak.your-domain.com`) or on the **same domain** under a subpath (e.g., `https://governance.your-domain.com/keycloak`). + +### Checklist + +Before proceeding, confirm: + +- [ ] Kubernetes cluster is running and `kubectl` is configured +- [ ] NGINX Ingress Controller is installed +- [ ] cert-manager is installed with a working Issuer/ClusterIssuer +- [ ] Keycloak is deployed and accessible +- [ ] Keycloak admin credentials are known +- [ ] Object storage is provisioned (Azure Blob, GCS, or S3) +- [ ] Azure Key Vault is provisioned (if using VC signing) +- [ ] DNS domain is available and you can create records +- [ ] GitHub PAT with `read:packages` scope is available +- [ ] Helm 3.8+ and kubectl 1.21+ are installed locally + +--- + +## 3. Infrastructure Setup + +Provision the following cloud resources before deploying. Both **governance-service** and **integrity-service** require object storage; **auth-service** requires a key vault for DID signing. + +### Object Storage + +Choose one storage provider. Both governance-service and integrity-service must use the same provider. + +#### Option A: Azure Blob Storage + +Create a storage account and two containers: + +```bash +# Create storage account +az storage account create \ + --name yourstorageaccount \ + --resource-group your-resource-group \ + --location eastus \ + --sku Standard_LRS + +# Create containers +az storage container create --name governance-artifacts --account-name yourstorageaccount +az storage container create --name integrity-store --account-name yourstorageaccount + +# Get the account key (needed for secrets later) +az storage account keys list --account-name yourstorageaccount --query '[0].value' -o tsv +``` + +You'll need these values for your `values.yaml`: + +| Value | governance-service field | integrity-service field | +| -------------------- | --------------------------- | -------------------------------- | +| Storage account name | `azureStorageAccountName` | `integrityAppBlobStoreAccount` | +| Artifacts container | `azureStorageContainerName` | β€” | +| Integrity container | β€” | `integrityAppBlobStoreContainer` | + +#### Option B: Google Cloud Storage + +Create two buckets and a service account: + +```bash +# Create buckets +gcloud storage buckets create gs://your-governance-artifacts --location=us-central1 +gcloud storage buckets create gs://your-integrity-store --location=us-central1 + +# Create service account +gcloud iam service-accounts create governance-storage \ + --display-name="Governance Platform Storage" + +# Grant access +for BUCKET in your-governance-artifacts your-integrity-store; do + gcloud storage buckets add-iam-policy-binding gs://$BUCKET \ + --member="serviceAccount:governance-storage@your-project.iam.gserviceaccount.com" \ + --role="roles/storage.objectAdmin" +done + +# Create key (needed for secrets later) +gcloud iam service-accounts keys create service-account.json \ + --iam-account=governance-storage@your-project.iam.gserviceaccount.com +``` + +You'll need these values for your `values.yaml`: + +| Value | governance-service field | integrity-service field | +| --------------------------- | ------------------------ | -------------------------------- | +| Artifacts bucket | `gcsBucketName` | β€” | +| Integrity bucket | β€” | `integrityAppBlobStoreGcsBucket` | +| Integrity folder (optional) | β€” | `integrityAppBlobStoreGcsFolder` | + +#### Option C: AWS S3 + +Create two buckets and an IAM user: + +```bash +# Create buckets +aws s3 mb s3://your-governance-artifacts --region us-east-1 +aws s3 mb s3://your-integrity-store --region us-east-1 + +# Create IAM user with programmatic access +aws iam create-user --user-name governance-storage +aws iam attach-user-policy --user-name governance-storage \ + --policy-arn arn:aws:iam::policy/AmazonS3FullAccess # Or a scoped policy + +# Create access key (needed for secrets later) +aws iam create-access-key --user-name governance-storage +``` + +You'll need these values for your `values.yaml`: + +| Value | governance-service field | integrity-service field | +| --------------------------- | ------------------------ | -------------------------------- | +| Region | `awsS3Region` | `integrityAppBlobStoreAwsRegion` | +| Artifacts bucket | `awsS3BucketName` | β€” | +| Integrity bucket | β€” | `integrityAppBlobStoreAwsBucket` | +| Integrity folder (optional) | β€” | `integrityAppBlobStoreAwsFolder` | + +### Azure Key Vault + +The auth-service uses Azure Key Vault for DID signing key management. Create a vault and a service principal with appropriate permissions: + +```bash +# Create Key Vault +az keyvault create \ + --name your-keyvault \ + --resource-group your-resource-group \ + --location eastus + +# Create service principal +az ad sp create-for-rbac --name governance-keyvault-sp + +# Grant key permissions to the service principal +az keyvault set-policy \ + --name your-keyvault \ + --spn \ + --key-permissions get list sign verify create +``` + +You'll need these values for your `values.yaml`: + +| Value | Field | +| ------------------------------- | ---------------------------------------------------- | +| Vault URL | `auth-service.config.keyVault.azure.vaultUrl` | +| Tenant ID | `auth-service.config.keyVault.azure.tenantId` | +| Service principal client ID | Secret: `platform-azure-key-vault` β†’ `client-id` | +| Service principal client secret | Secret: `platform-azure-key-vault` β†’ `client-secret` | + +### Summary of Provisioned Resources + +After completing this section, you should have: + +| Resource | What You Need for Later | +| --------------------------- | --------------------------------------------------- | +| Object storage (1 provider) | Account name/keys, 2 container/bucket names | +| Azure Key Vault | Vault URL, tenant ID, service principal credentials | + +These values will be used in [Section 7 (Creating Secrets)](#7-creating-kubernetes-secrets) and [Section 8 (Configuring values.yaml)](#8-configuring-valuesyaml). + +--- + +## 4. Domain & TLS Configuration + +### DNS Setup + +The platform requires one domain for the governance services. Keycloak can run on a separate domain or on the same domain under `/keycloak`. + +Create DNS records pointing to your NGINX Ingress Controller's external IP: + +```bash +# Get the ingress controller external IP +kubectl get svc -n ingress-nginx ingress-nginx-controller \ + -o jsonpath='{.status.loadBalancer.ingress[0].ip}' +``` + +Then create A-records (or CNAMEs if using a load balancer hostname): + +| Record | Type | Value | +| ----------------------------------------------- | ---- | ----------------------- | +| `governance.your-domain.com` | A | `` | +| `keycloak.your-domain.com` (if separate domain) | A | `` | + +### NGINX Ingress Controller + +If not already installed, use the provided helper script: + +```bash +./scripts/nginx.sh +``` + +This installs the `ingress-nginx` Helm chart into the `ingress-nginx` namespace. + +### TLS with cert-manager + +The platform uses cert-manager to automatically provision TLS certificates from Let's Encrypt. + +#### Install cert-manager + +If not already installed, use the provided helper script: + +```bash +./scripts/cert-issuer.sh +``` + +This installs cert-manager into the `ingress-nginx` namespace. To install into a different namespace: + +```bash +./scripts/cert-issuer.sh --namespace cert-manager +``` + +#### Create a Let's Encrypt Issuer + +After cert-manager is running, create an Issuer in your governance namespace: + +```bash +kubectl apply -f - < **Note:** The Issuer name (`letsencrypt-prod`) must match the `cert-manager.io/issuer` annotation in your ingress configuration. The example values files use `letsencrypt-prod`. + +### How TLS Works in the Platform + +Each service's ingress is configured with: + +1. A `cert-manager.io/issuer` annotation that references the Issuer +2. A `tls` block specifying the TLS secret name and hostname + +For example, from [`values-keycloak.yaml`](../charts/governance-platform/examples/values-keycloak.yaml): + +```yaml +ingress: + enabled: true + className: "nginx" + annotations: + cert-manager.io/issuer: "letsencrypt-prod" + hosts: + - host: governance.your-domain.com + paths: + - path: "/authService(/|$)(.*)" + pathType: ImplementationSpecific + tls: + - secretName: prod-tls-secret + hosts: + - governance.your-domain.com +``` + +cert-manager watches for ingress resources with the `cert-manager.io/issuer` annotation and automatically requests and renews certificates. The certificate is stored in the Kubernetes secret specified by `secretName` (e.g., `prod-tls-secret`). + +All four services share the **same TLS secret name and hostname** since they run on the same domain with different paths. + +### Verify DNS and TLS + +After DNS propagation: + +```bash +# Verify DNS resolution +dig governance.your-domain.com + +# After deploying (Section 9), verify TLS certificate +kubectl get certificate -n governance +kubectl describe certificate -n governance +``` + +--- + +## 5. Deploying Keycloak + +The Governance Platform requires a running Keycloak instance. This section covers deploying Keycloak into the same Kubernetes cluster. If you already have a Keycloak instance running, skip to [creating the required secrets](#pre-bootstrap-secrets) and then proceed to [Section 6](#6-running-keycloak-bootstrap). + +### Create Namespace + +If not already created: + +```bash +kubectl create namespace governance +``` + +### Deploy Keycloak with Bitnami Helm Chart + +The recommended approach for in-cluster Keycloak is the Bitnami Helm chart: + +```bash +helm repo add bitnami https://charts.bitnami.com/bitnami +helm repo update +``` + +Create a values file for your Keycloak deployment (e.g., `keycloak-values.yaml`): + +```yaml +# Keycloak server configuration +auth: + adminUser: admin + adminPassword: "" # Will be set via existing secret + existingSecret: "keycloak-admin" + passwordSecretKey: "password" + +# Run Keycloak under /keycloak subpath +httpRelativePath: "/keycloak/" + +# Production mode with TLS termination at ingress +production: true +proxy: edge + +# PostgreSQL - use a dedicated database or the platform's shared database +postgresql: + enabled: true + auth: + postgresPassword: "" # Set via secret or generate + database: keycloak + +# Ingress configuration +ingress: + enabled: true + ingressClassName: "nginx" + hostname: governance.your-domain.com # Or keycloak.your-domain.com + path: /keycloak + annotations: + cert-manager.io/issuer: "letsencrypt-prod" + tls: true + +# Resource limits +resources: + requests: + cpu: 500m + memory: 512Mi + limits: + cpu: 1000m + memory: 1Gi +``` + +### Pre-Bootstrap Secrets + +Before deploying Keycloak, create the secrets that both Keycloak and the bootstrap job will need: + +```bash +# Keycloak admin password (master realm) +kubectl create secret generic keycloak-admin \ + --from-literal=password="$(openssl rand -base64 24)" \ + --namespace governance + +# Platform admin password (governance realm user β€” created by bootstrap) +kubectl create secret generic platform-admin \ + --from-literal=password="$(openssl rand -base64 16)" \ + --namespace governance +``` + +### Install Keycloak + +```bash +helm upgrade --install keycloak bitnami/keycloak \ + --namespace governance \ + --values keycloak-values.yaml \ + --wait \ + --timeout 10m +``` + +### Verify Keycloak is Running + +```bash +# Check pod status +kubectl get pods -l app.kubernetes.io/name=keycloak -n governance + +# Check readiness +kubectl get pod -l app.kubernetes.io/name=keycloak -n governance \ + -o jsonpath='{.items[0].status.conditions[?(@.type=="Ready")].status}' + +# Test internal connectivity (should return HTML or redirect) +kubectl run curl-test --rm -it --image=curlimages/curl --restart=Never -n governance -- \ + curl -s -o /dev/null -w "%{http_code}" http://keycloak:8080/keycloak/health/ready +``` + +You should see `Ready: True` and an HTTP 200 from the health endpoint. + +### Using an External Keycloak + +If Keycloak is running outside the cluster, you need to ensure: + +1. **Network reachability** β€” pods in the governance namespace can reach the Keycloak URL +2. **Internal URL** β€” the bootstrap chart defaults to `http://keycloak:8080/keycloak`. Override this in the bootstrap values if your Keycloak uses a different internal URL: + +```yaml +keycloak: + url: "https://keycloak.your-domain.com" +``` + +3. **Admin credentials** β€” the `keycloak-admin` secret must still be created in the governance namespace with the external Keycloak's admin password + +### What's Next + +With Keycloak running, proceed to [Section 6](#6-running-keycloak-bootstrap) to configure the governance realm, OAuth clients, and initial admin user. + +--- + +## 6. Running Keycloak Bootstrap + +The `keycloak-bootstrap` chart runs a Kubernetes Job that configures Keycloak via its Admin REST API. It creates the governance realm, OAuth clients, custom scopes, service account roles, and an initial platform-admin user. + +### Prepare the Bootstrap Values + +Start from the example values file and customize it for your environment: + +```bash +cp charts/keycloak-bootstrap/examples/values.yaml bootstrap-values.yaml +``` + +Edit `bootstrap-values.yaml` and replace all `CHANGE_ME_DOMAIN_HERE` placeholders with your actual domain: + +```yaml +# Client redirect URIs and web origins +clients: + frontend: + redirectUris: + - "https://governance.your-domain.com/*" + - "http://localhost:5173/*" + webOrigins: + - "https://governance.your-domain.com" + - "http://localhost:5173" + + backend: + redirectUris: + - "https://governance.your-domain.com/authService/*" + webOrigins: + - "https://governance.your-domain.com" + +# Admin user email +users: + admin: + email: "admin@governance.your-domain.com" +``` + +If your Keycloak is not reachable at the default `http://keycloak:8080/keycloak`, update the connection settings: + +```yaml +keycloak: + url: "https://keycloak.your-domain.com" # External URL + # or + url: "http://keycloak.other-namespace.svc:8080/keycloak" # Cross-namespace +``` + +### Run the Bootstrap + +#### Option A: Using the Helper Script (Recommended) + +```bash +./scripts/keycloak/bootstrap-keycloak.sh -f /path/to/bootstrap-values.yaml -n governance +``` + +The script validates prerequisites (Keycloak running, secrets exist), runs the Helm chart, monitors the job, and displays the results. + +#### Option B: Using Helm Directly + +```bash +helm upgrade --install keycloak-bootstrap ./charts/keycloak-bootstrap \ + --namespace governance \ + --values /path/to/bootstrap-values.yaml \ + --wait \ + --timeout 10m +``` + +Monitor the job: + +```bash +# Watch job status +kubectl get jobs -l app.kubernetes.io/instance=keycloak-bootstrap -n governance -w + +# View logs +kubectl logs job/keycloak-bootstrap -n governance -f +``` + +### What the Bootstrap Creates + +| Resource | Details | +| ----------------------- | ------------------------------------------------------------------------------------------------------- | +| **Realm** | `governance` with brute force protection, SSO sessions, token lifespans | +| **Frontend client** | `governance-platform-frontend` β€” public SPA client | +| **Backend client** | `governance-platform-backend` β€” confidential, service account with `query-users` and `view-users` roles | +| **Worker client** | `governance-worker` β€” confidential, service account only | +| **Custom scopes** | 8 authorization scopes (governance, integrity, organizations, projects, evaluations) | +| **Platform admin user** | `platform-admin` in the governance realm | + +### Retrieve Auto-Generated Client Secrets + +The backend and worker client secrets are **auto-generated by Keycloak** during bootstrap. You must retrieve them to create the platform's Kubernetes secrets in the next step. + +```bash +# Get Keycloak admin token +KEYCLOAK_URL="http://keycloak:8080/keycloak" # Adjust if different +ADMIN_PASS=$(kubectl get secret keycloak-admin -n governance -o jsonpath='{.data.password}' | base64 -d) + +TOKEN=$(kubectl run get-token --rm -it --restart=Never -n governance \ + --image=curlimages/curl -- \ + curl -s -X POST "$KEYCLOAK_URL/realms/master/protocol/openid-connect/token" \ + -d "username=admin" \ + -d "password=$ADMIN_PASS" \ + -d "grant_type=password" \ + -d "client_id=admin-cli" | jq -r '.access_token') + +# Get backend client secret +kubectl run get-backend-secret --rm -it --restart=Never -n governance \ + --image=curlimages/curl -- \ + curl -s -H "Authorization: Bearer $TOKEN" \ + "$KEYCLOAK_URL/admin/realms/governance/clients?clientId=governance-platform-backend" \ + | jq -r '.[0].secret' + +# Get worker client secret +kubectl run get-worker-secret --rm -it --restart=Never -n governance \ + --image=curlimages/curl -- \ + curl -s -H "Authorization: Bearer $TOKEN" \ + "$KEYCLOAK_URL/admin/realms/governance/clients?clientId=governance-worker" \ + | jq -r '.[0].secret' +``` + +Alternatively, retrieve them from the **Keycloak Admin Console**: + +1. Navigate to `https://governance.your-domain.com/keycloak/admin` +2. Select the **governance** realm +3. Go to **Clients** > **governance-platform-backend** > **Credentials** tab +4. Copy the **Client secret** +5. Repeat for **governance-worker** + +> **Save these secrets** β€” you'll need them in [Section 7](#7-creating-kubernetes-secrets) to create the `platform-keycloak` and `platform-governance-worker` Kubernetes secrets. + +### Retrieve the Platform Admin Keycloak ID + +The bootstrap also creates the `platform-admin` user in Keycloak. Retrieve their Keycloak user ID now β€” you'll need it when configuring `keycloak.platformAdminKeycloakId` in [Section 8](#keycloak-organization-hook). + +```bash +# Using the same admin token from above +kubectl run get-admin-id --rm -it --restart=Never -n governance \ + --image=curlimages/curl -- \ + curl -s -H "Authorization: Bearer $TOKEN" \ + "$KEYCLOAK_URL/admin/realms/governance/users?username=platform-admin&exact=true" \ + | jq -r '.[0].id' +``` + +Or from the **Keycloak Admin Console**: go to **governance** realm > **Users** > **platform-admin** and copy the user ID from the URL. + +### Verify the Bootstrap + +```bash +# Test realm discovery endpoint +curl -s https://governance.your-domain.com/keycloak/realms/governance/.well-known/openid-configuration | jq '.issuer' + +# Expected output: "https://governance.your-domain.com/keycloak/realms/governance" +``` + +### Troubleshooting + +| Issue | Solution | +| ------------------------------------------ | ---------------------------------------------------------------------------------------- | +| Job fails with "Failed to get admin token" | Verify `keycloak-admin` secret password matches the actual Keycloak admin password | +| Job fails with connection refused | Check `keycloak.url` in values β€” ensure Keycloak is reachable from within the cluster | +| Realm already exists | The bootstrap is idempotent β€” it updates existing resources rather than failing | +| Job times out | Check Keycloak pod logs: `kubectl logs -l app.kubernetes.io/name=keycloak -n governance` | + +--- + +## 7. Creating Kubernetes Secrets + +The governance-platform chart expects secrets to be pre-created in the namespace. There are two approaches: + +- **Option A (Recommended for production):** Create secrets manually with `kubectl` (documented below) +- **Option B:** Use the [`secrets-sample.yaml`](../charts/governance-platform/examples/secrets-sample.yaml) template with `global.secrets.create: true` to have Helm create them + +> **Note:** The `keycloak-admin` and `platform-admin` secrets were already created in [Section 5](#pre-bootstrap-secrets). The commands below cover all remaining secrets. + +### Secret Reference + +| Secret Name | Used By | Keys | +| -------------------------------- | --------------------------------------------------- | ------------------------------------------------------------ | +| `keycloak-admin` | Keycloak, bootstrap | `password` | +| `platform-admin` | Bootstrap | `password` | +| `platform-database` | governance-service, auth-service, integrity-service | `username`, `password` | +| `platform-keycloak` | auth-service, governance-service | `service-account-client-id`, `service-account-client-secret` | +| `platform-auth-service` | auth-service | `api-secret`, `jwt-secret` | +| `platform-encryption-key` | governance-service, auth-service | `encryption-key` | +| `platform-governance-worker` | governance-service worker | `encryption-key`, `client-id`, `client-secret` | +| `platform-azure-blob` | governance-service, integrity-service (Azure) | `account-key`, `connection-string` | +| `platform-aws-s3` | governance-service, integrity-service (AWS) | `access-key-id`, `secret-access-key` | +| `platform-gcs` | governance-service, integrity-service (GCS) | `service-account-json` | +| `platform-azure-key-vault` | auth-service | `client-id`, `client-secret`, `tenant-id`, `vault-url` | +| `platform-governance-service-ai` | governance-service (optional) | `api-key` | +| `platform-image-pull-secret` | All services | Docker registry credentials | + +### Create Secrets + +Run these commands in order, replacing placeholder values with your actual credentials. + +#### Database + +```bash +kubectl create secret generic platform-database \ + --from-literal=username=postgres \ + --from-literal=password="$(openssl rand -base64 24)" \ + --namespace governance +``` + +#### Keycloak (Service Account Credentials) + +Use the backend client secret retrieved from Keycloak in [Section 6](#retrieve-auto-generated-client-secrets): + +```bash +kubectl create secret generic platform-keycloak \ + --from-literal=service-account-client-id=governance-platform-backend \ + --from-literal=service-account-client-secret=YOUR_BACKEND_CLIENT_SECRET \ + --namespace governance +``` + +#### Auth Service + +```bash +kubectl create secret generic platform-auth-service \ + --from-literal=api-secret="$(openssl rand -base64 32)" \ + --from-literal=jwt-secret="$(openssl rand -base64 32)" \ + --namespace governance +``` + +#### Encryption Key + +```bash +kubectl create secret generic platform-encryption-key \ + --from-literal=encryption-key="$(openssl rand -base64 32)" \ + --namespace governance +``` + +#### Governance Worker + +Use the worker client secret retrieved from Keycloak in [Section 6](#retrieve-auto-generated-client-secrets): + +```bash +kubectl create secret generic platform-governance-worker \ + --from-literal=encryption-key="$(openssl rand -base64 32)" \ + --from-literal=client-id=governance-worker \ + --from-literal=client-secret=YOUR_WORKER_CLIENT_SECRET \ + --namespace governance +``` + +#### Storage Credentials (choose one) + +**Azure Blob:** + +```bash +kubectl create secret generic platform-azure-blob \ + --from-literal=account-key=YOUR_AZURE_STORAGE_ACCOUNT_KEY \ + --from-literal=connection-string="DefaultEndpointsProtocol=https;AccountName=yourstorageaccount;AccountKey=YOUR_KEY;EndpointSuffix=core.windows.net" \ + --namespace governance +``` + +**AWS S3:** + +```bash +kubectl create secret generic platform-aws-s3 \ + --from-literal=access-key-id=YOUR_AWS_ACCESS_KEY_ID \ + --from-literal=secret-access-key=YOUR_AWS_SECRET_ACCESS_KEY \ + --namespace governance +``` + +**GCS:** + +```bash +kubectl create secret generic platform-gcs \ + --from-file=service-account-json=service-account.json \ + --namespace governance +``` + +#### Azure Key Vault + +```bash +kubectl create secret generic platform-azure-key-vault \ + --from-literal=client-id=YOUR_AZURE_CLIENT_ID \ + --from-literal=client-secret=YOUR_AZURE_CLIENT_SECRET \ + --from-literal=tenant-id=YOUR_AZURE_TENANT_ID \ + --from-literal=vault-url=https://your-vault.vault.azure.net/ \ + --namespace governance +``` + +#### AI Credentials (Optional) + +Only required if `governance-service.config.ai.enabled` is set to `true` in your values file. AI features are **disabled by default**. + +```bash +kubectl create secret generic platform-governance-service-ai \ + --from-literal=api-key=YOUR_ANTHROPIC_API_KEY \ + --namespace governance +``` + +#### Image Pull Secret + +```bash +kubectl create secret docker-registry platform-image-pull-secret \ + --docker-server=ghcr.io \ + --docker-username=YOUR_GITHUB_USERNAME \ + --docker-password=YOUR_GITHUB_PAT \ + --docker-email=YOUR_EMAIL \ + --namespace governance +``` + +### Verify Secrets + +```bash +# List all platform secrets +kubectl get secrets -n governance | grep platform + +# Verify a specific secret has the expected keys +kubectl get secret platform-keycloak -n governance -o jsonpath='{.data}' | jq 'keys' +``` + +--- + +## 8. Configuring values.yaml + +The governance-platform Helm chart is configured through a single values file. Start from the Keycloak example and customize it for your environment. + +### Start from the Example + +```bash +cp charts/governance-platform/examples/values-keycloak.yaml my-values.yaml +``` + +The example file ([`values-keycloak.yaml`](../charts/governance-platform/examples/values-keycloak.yaml)) has all four services pre-configured for Keycloak with placeholder values you need to replace. + +### Global Configuration + +Set the domain and auth provider at the top of your values file: + +```yaml +global: + domain: "governance.your-domain.com" + environmentType: "production" # Options: development, staging, production +``` + +The `global.secrets.create` setting defaults to `false`, which means secrets must be pre-created (as done in [Section 7](#7-creating-kubernetes-secrets)). If you prefer Helm-managed secrets, set `create: true` and provide values via [`secrets-sample.yaml`](../charts/governance-platform/examples/secrets-sample.yaml). + +### Auth Service + +The auth-service handles authentication, authorization, and token exchange. Key configuration areas: + +```yaml +auth-service: + config: + # Identity Provider β€” must match your Keycloak setup + idp: + provider: "keycloak" + issuer: "https://governance.your-domain.com/keycloak/realms/governance" + keycloak: + realm: "governance" + adminUrl: "https://governance.your-domain.com/keycloak" + clientId: "governance-platform-frontend" + enableUserManagement: true + + # Token Exchange β€” enables service-to-service token exchange + tokenExchange: + enabled: true + keyId: "auth-service-prod-001" # Unique key identifier + + # Key Vault β€” for DID signing keys + keyVault: + provider: "azure_key_vault" + azure: + vaultUrl: "https://your-keyvault.vault.azure.net/" + tenantId: "your-azure-tenant-id" +``` + +| Field | Description | Where to Get It | +| ------------------------- | -------------------------------------------- | ----------------------------------------------------- | +| `idp.issuer` | Keycloak realm issuer URL | `https:///keycloak/realms/governance` | +| `idp.keycloak.adminUrl` | Keycloak base URL (used for Admin API calls) | Your Keycloak URL without `/realms/...` | +| `idp.keycloak.clientId` | Frontend client ID | Set during [bootstrap](#6-running-keycloak-bootstrap) | +| `keyVault.azure.vaultUrl` | Azure Key Vault URL | From [Section 3](#azure-key-vault) | +| `keyVault.azure.tenantId` | Azure AD tenant ID | From your Azure subscription | + +### Governance Service + +The governance-service is the main backend API. Configure storage and Keycloak: + +```yaml +governance-service: + config: + # Storage β€” choose one provider + storageProvider: "azure_blob" # Options: azure_blob, gcs, aws_s3 + azureStorageAccountName: "your-storage-account" + azureStorageContainerName: "your-governance-artifacts" + + # GCS alternative: + # storageProvider: "gcs" + # gcsBucketName: "your-governance-artifacts-bucket" + + # AWS S3 alternative: + # storageProvider: "aws_s3" + # awsS3Region: "us-east-1" + # awsS3BucketName: "your-governance-artifacts-bucket" + + # Keycloak β€” must match auth-service config + keycloakUrl: "https://governance.your-domain.com/keycloak" + keycloakRealm: "governance" +``` + +### Governance Studio + +The frontend application. Configure Keycloak connection and feature flags: + +```yaml +governance-studio: + config: + keycloakUrl: "https://governance.your-domain.com/keycloak" + keycloakRealm: "governance" + keycloakClientId: "governance-platform-frontend" + + # Feature flags + features: + governance: true # Governance workflows + lineage: true # Lineage tracking + guardianEnabled: false # Guardian features + agentManagement: false # Agent management +``` + +> **Important:** The `keycloakClientId` must match the frontend client ID created during [bootstrap](#6-running-keycloak-bootstrap) (`governance-platform-frontend`). + +### Integrity Service + +The integrity-service handles verifiable credentials. Configure its storage (must use the same provider as governance-service): + +```yaml +integrity-service: + config: + integrityAppBlobStoreType: "azure_blob" # Must match governance-service storageProvider + integrityAppBlobStoreAccount: "your-storage-account" + integrityAppBlobStoreContainer: "your-integrity-store" + + # AWS S3 alternative: + # integrityAppBlobStoreType: "aws_s3" + # integrityAppBlobStoreAwsRegion: "us-east-1" + # integrityAppBlobStoreAwsBucket: "your-integrity-store-bucket" + + # GCS alternative: + # integrityAppBlobStoreType: "gcs" + # integrityAppBlobStoreGcsBucket: "your-integrity-store-bucket" +``` + +### Ingress Configuration + +Each service needs an ingress block. All four services share the same domain with different paths. The pattern is identical across services β€” here's the template: + +```yaml +ingress: + enabled: true + className: "nginx" + annotations: + cert-manager.io/issuer: "letsencrypt-prod" + nginx.ingress.kubernetes.io/use-regex: "true" + nginx.ingress.kubernetes.io/rewrite-target: "/$2" + nginx.ingress.kubernetes.io/enable-cors: "true" + nginx.ingress.kubernetes.io/proxy-body-size: "64m" + hosts: + - host: governance.your-domain.com + paths: + - path: "/(/|$)(.*)" + pathType: ImplementationSpecific + tls: + - secretName: prod-tls-secret + hosts: + - governance.your-domain.com +``` + +| Service | Path Pattern | Notes | +| ------------------ | ------------------------------ | ---------------------------------- | +| governance-studio | `/` (Prefix) | No regex rewrite needed | +| governance-service | `/governanceService(/\|$)(.*)` | Regex rewrite to `/$2` | +| auth-service | `/authService(/\|$)(.*)` | Extra buffer size annotations | +| integrity-service | `/integrityService(/\|$)(.*)` | `proxy-body-size: "0"` (unlimited) | + +> **Note:** The `secretName: prod-tls-secret` must be the same across all four services. cert-manager creates this secret automatically when it provisions the TLS certificate. + +### Keycloak Organization Hook + +The umbrella chart includes a post-install hook that creates the organization and platform-admin user in the database. Enable it in your values: + +```yaml +keycloak: + createOrganization: true + realmName: "governance" # Must match auth-service idp.keycloak.realm + displayName: "Governance Platform" + createPlatformAdmin: true + platformAdminKeycloakId: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" # From Section 6 +``` + +> **Note:** The `platformAdminKeycloakId` is the Keycloak user ID for the `platform-admin` user, retrieved in [Section 6](#retrieve-the-platform-admin-keycloak-id). This links the database record to the Keycloak user so login works correctly. + +### PostgreSQL + +The Bitnami PostgreSQL chart is included as a dependency. Configure storage and resources: + +```yaml +postgresql: + enabled: true + primary: + persistence: + enabled: true + size: 10Gi + storageClass: "standard" # Adjust for your cluster (e.g., "managed-premium" on AKS) + resources: + requests: + cpu: 500m + memory: 1Gi + limits: + cpu: 2000m + memory: 2Gi +``` + +The database password is pulled from the `platform-database` secret created in [Section 7](#database). + +### Configuration Checklist + +Before deploying, verify your values file has: + +- [ ] `global.domain` set to your actual domain +- [ ] `auth-service.config.idp.issuer` pointing to your Keycloak realm +- [ ] `auth-service.config.idp.keycloak.adminUrl` pointing to your Keycloak +- [ ] `auth-service.config.keyVault.azure.vaultUrl` and `tenantId` set +- [ ] `governance-service.config.storageProvider` and storage fields set +- [ ] `governance-studio.config.keycloakUrl` and `keycloakRealm` set +- [ ] `integrity-service.config.integrityAppBlobStoreType` matching governance-service +- [ ] All ingress `host` fields set to your domain +- [ ] All ingress `tls` blocks using the same `secretName` +- [ ] `keycloak.createOrganization` set to `true` + +--- + +## 9. Deploying the Governance Platform + +### Update Chart Dependencies + +Before installing, pull the subchart dependencies: + +```bash +helm dependency update ./charts/governance-platform +``` + +This downloads the Bitnami PostgreSQL chart and links the local subcharts (auth-service, governance-service, governance-studio, integrity-service). + +### Install + +```bash +helm upgrade --install governance-platform ./charts/governance-platform \ + --namespace governance \ + --create-namespace \ + --values /path/to/my-values.yaml \ + --wait \ + --timeout 15m +``` + +> If using Helm-managed secrets instead of pre-created secrets, include the secrets file: +> +> ```bash +> --values /path/to/secrets.yaml --values /path/to/my-values.yaml +> ``` + +### What Happens During Install + +The Helm install proceeds in this order: + +1. **PostgreSQL** starts and initializes the `governance` database +2. **governance-service** starts, runs database migrations on startup +3. **auth-service** and **integrity-service** start (depend on database being ready) +4. **governance-studio** starts (static frontend, no database dependency) +5. **Post-install hook** runs (weight 20) β€” waits for migrations to complete, then creates the organization and platform-admin user in the database + +The `--wait` flag ensures Helm waits for all pods to reach `Ready` state before returning. + +### Monitor the Deployment + +```bash +# Watch all pods come up +kubectl get pods -n governance -w + +# Check deployment status +kubectl get deployments -n governance + +# Check the post-install hook job +kubectl get jobs -n governance +``` + +Expected pod status once healthy: + +``` +NAME READY STATUS AGE +governance-platform-auth-service-xxxxx-xxxxx 1/1 Running 2m +governance-platform-governance-service-xxxxx-xxxxx 1/1 Running 2m +governance-platform-governance-studio-xxxxx-xxxxx 1/1 Running 2m +governance-platform-integrity-service-xxxxx-xxxxx 1/1 Running 2m +governance-platform-postgresql-0 1/1 Running 3m +governance-platform-create-keycloak-org-xxxxx 0/1 Completed 1m +``` + +### Troubleshooting Deployment Issues + +**Pod stuck in CrashLoopBackOff:** + +```bash +# Check pod logs +kubectl logs -l app.kubernetes.io/instance=governance-platform -n governance --all-containers + +# Check specific service +kubectl logs deployment/governance-platform-auth-service -n governance +``` + +**Pod stuck in ImagePullBackOff:** + +```bash +# Verify image pull secret exists and is correct +kubectl get secret platform-image-pull-secret -n governance -o jsonpath='{.data.\.dockerconfigjson}' | base64 -d | jq . +``` + +**Database connection errors:** + +```bash +# Check PostgreSQL is running +kubectl get pod governance-platform-postgresql-0 -n governance + +# Verify database secret +kubectl get secret platform-database -n governance -o jsonpath='{.data.password}' | base64 -d +``` + +**Post-install hook failed:** + +```bash +# Check the hook job logs +kubectl logs job/governance-platform-create-keycloak-org -n governance + +# Common cause: migrations haven't completed yet β€” the init container retries for up to 5 minutes +``` + +**Ingress not working:** + +```bash +# Check ingress resources were created +kubectl get ingress -n governance + +# Check cert-manager certificate status +kubectl get certificate -n governance +kubectl describe certificate -n governance +``` + +--- + +## 10. Post-Install Setup & Verification + +### Post-Install Script (Optional) + +If the post-install Helm hook didn't run correctly, or if you need to re-run the database setup, use the helper script: + +```bash +./scripts/keycloak/post-install-keycloak-setup.sh \ + --namespace governance \ + --realm governance \ + --display-name "Governance Platform" +``` + +The script will: + +1. Wait for the database and governance-service to be ready +2. Create (or update) the organization in the database +3. Look up the `platform-admin` user ID from Keycloak +4. Create (or update) the platform-admin user in the database with the correct Keycloak ID +5. Set up the organization membership with `organization_owner` role +6. Run verification checks + +This script is idempotent β€” it's safe to run multiple times. + +### Verify Service Health + +```bash +# All services should return healthy responses +DOMAIN="governance.your-domain.com" + +# Governance Studio (should return 200) +curl -s -o /dev/null -w "%{http_code}" https://$DOMAIN/ + +# Governance Service health +curl -s https://$DOMAIN/governanceService/health | jq . + +# Auth Service health +curl -s https://$DOMAIN/authService/health | jq . + +# Integrity Service health +curl -s https://$DOMAIN/integrityService/health | jq . +``` + +### Verify Keycloak Integration + +```bash +# OpenID Connect discovery endpoint (should return JSON with issuer) +curl -s https://$DOMAIN/keycloak/realms/governance/.well-known/openid-configuration | jq '.issuer' + +# Test token exchange β€” get a token using the backend service account +BACKEND_SECRET=$(kubectl get secret platform-keycloak -n governance -o jsonpath='{.data.service-account-client-secret}' | base64 -d) + +curl -s -X POST "https://$DOMAIN/keycloak/realms/governance/protocol/openid-connect/token" \ + -d "grant_type=client_credentials" \ + -d "client_id=governance-platform-backend" \ + -d "client_secret=$BACKEND_SECRET" \ + | jq '.access_token | split(".") | .[1] | @base64d | fromjson | {sub, azp, realm_access}' +``` + +### Verify Database Records + +```bash +# Check organization was created +kubectl exec -n governance governance-platform-postgresql-0 -- \ + env PGPASSWORD=$(kubectl get secret platform-database -n governance -o jsonpath='{.data.password}' | base64 -d) \ + psql -U postgres -d governance -c \ + "SELECT id, name, display_name, idp_provider FROM organization;" + +# Check platform-admin user exists +kubectl exec -n governance governance-platform-postgresql-0 -- \ + env PGPASSWORD=$(kubectl get secret platform-database -n governance -o jsonpath='{.data.password}' | base64 -d) \ + psql -U postgres -d governance -c \ + "SELECT u.email, u.display_name, u.idp_provider, uom.roles + FROM users u + JOIN user_organization_memberships uom ON u.id = uom.user_id + WHERE u.email LIKE 'admin@%';" +``` + +### Test Login + +1. Navigate to `https://governance.your-domain.com` in your browser +2. You should be redirected to the Keycloak login page for the `governance` realm +3. Log in with the platform-admin credentials: + - **Username:** `platform-admin` + - **Password:** retrieve from the secret: + ```bash + kubectl get secret platform-admin -n governance -o jsonpath='{.data.password}' | base64 -d + ``` +4. After login, you should be redirected back to Governance Studio with full access + +### Deployment Complete + +Your Governance Platform is now running with: + +- Keycloak managing identity and access for the `governance` realm +- Three OAuth clients (frontend, backend, worker) +- Platform-admin user with `organization_owner` role +- All four services accessible via path-based routing on a single domain +- TLS certificates managed by cert-manager +- PostgreSQL with all required schemas + +### Next Steps + +#### Adding Users + +Users must be created in **Keycloak first** before they can be added to Governance Studio: + +1. **Create the user in Keycloak:** + - Go to the Keycloak Admin Console > **governance** realm > **Users** > **Add user** + - Set username, email, first/last name, and enable the account + - Under the **Credentials** tab, set a password (or configure email verification) + +2. **Add the user in Governance Studio:** + - Log in as `platform-admin` + - Navigate to **Organization** > **Members** (`https://governance.your-domain.com/organization/members`) + - Add the user by email and assign a role + +The user can then log in to Governance Studio with their Keycloak credentials. + +### Quick Reference + +| Resource | URL | +| ----------------------- | ------------------------------------------------------------------------------------------------ | +| Governance Studio | `https://governance.your-domain.com/` | +| Governance Service API | `https://governance.your-domain.com/governanceService/` | +| Auth Service API | `https://governance.your-domain.com/authService/` | +| Integrity Service API | `https://governance.your-domain.com/integrityService/` | +| Keycloak Admin Console | `https://governance.your-domain.com/keycloak/admin` | +| Keycloak Realm Settings | `https://governance.your-domain.com/keycloak/admin/governance/console` | +| OIDC Discovery | `https://governance.your-domain.com/keycloak/realms/governance/.well-known/openid-configuration` | diff --git a/scripts/cert-issuer.sh b/scripts/cert-issuer.sh index 9d8a252..cc91fb5 100755 --- a/scripts/cert-issuer.sh +++ b/scripts/cert-issuer.sh @@ -1,40 +1,76 @@ #!/usr/bin/env bash - -## This script installs the cert-manager set -e -echo "Installing cert-manager" - -# Label the ingress-basic namespace to disable resource validation -kubectl label namespace ingress-basic cert-manager.io/disable-validation=true - -# Add the Jetstack Helm repository -helm repo add jetstack https://charts.jetstack.io - -# Update your local Helm chart repository cache -helm repo update - -# Install the cert-manager Helm chart -helm install \ - cert-manager jetstack/cert-manager \ - --namespace cert-manager \ - --create-namespace \ - # --set installCRDs=true - -kubectl apply -f - < Namespace for cert-manager (default: $NAMESPACE) + -h, --help Show this help message + +Examples: + $0 + $0 --namespace cert-manager +" +} + +# Install cert-manager +install() { + print_info "Installing cert-manager" + + # Add the Jetstack Helm repository + helm repo add jetstack https://charts.jetstack.io + + # Update your local Helm chart repository cache + helm repo update + + # Install the cert-manager Helm chart + helm install \ + cert-manager jetstack/cert-manager \ + --namespace "$NAMESPACE" \ + --create-namespace \ + --set crds.enabled=true + + print_info "cert-manager installed" +} + +# Default values +NAMESPACE="ingress-nginx" + +# Parse arguments +while [[ $# -gt 0 ]]; do + case $1 in + -n | --namespace) + NAMESPACE="$2" + shift 2 + ;; + -h | --help) + usage + exit 0 + ;; + *) + print_error "Unknown option: $1" + usage + exit 1 + ;; + esac +done + +# Validate prerequisites +assert_is_installed "helm" +assert_is_installed "kubectl" + +# Install cert-manager +install diff --git a/scripts/helpers/array.sh b/scripts/helpers/array.sh new file mode 100755 index 0000000..c7f45e4 --- /dev/null +++ b/scripts/helpers/array.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash + +# Returns 0 if the given item (needle) is in the given array (haystack); returns 1 otherwise. +function array_contains { + local -r needle="$1" + shift + local -ra haystack=("$@") + + local item + for item in "${haystack[@]}"; do + if [[ "$item" == "$needle" ]]; then + return 0 + fi + done + + return 1 +} + +# Joins the elements of the given array into a string with the given separator between each element. +# +# Examples: +# +# array_join "," ("A" "B" "C") +# Returns: "A,B,C" +# +function array_join { + local -r separator="$1" + shift + local -ra values=("$@") + + local out="" + for (( i=0; i<"${#values[@]}"; i++ )); do + if [[ "$i" -gt 0 ]]; then + out="${out}${separator}" + fi + out="${out}${values[i]}" + done + + echo -n "$out" +} diff --git a/scripts/helpers/assert.sh b/scripts/helpers/assert.sh new file mode 100755 index 0000000..bb70c19 --- /dev/null +++ b/scripts/helpers/assert.sh @@ -0,0 +1,117 @@ +#!/usr/bin/env bash +# A collection of useful assertions. Each one checks a condition and if the +# condition is not satisfied, exits the program. This is useful for defensive +# programming. + +# shellcheck source=./log.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/log.sh" +# shellcheck source=./array.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/array.sh" +# shellcheck source=./string.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/string.sh" +# shellcheck source=./os.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/os.sh" + +# Check that the given binary is available on the PATH. If it's not, exit with +# an error. +function assert_is_installed { + local -r name="$1" + + if ! os_command_is_installed "$name"; then + log_error "The command '$name' is required by this script but is not installed or in the system's PATH." + exit 1 + fi +} + +# Check that the value of the given arg is not empty. If it is, exit with an +# error. +function assert_not_empty { + local -r arg_name="$1" + local -r arg_value="$2" + local -r reason="$3" + + if [[ -z "$arg_value" ]]; then + log_error "The value for '$arg_name' cannot be empty. $reason" + exit 1 + fi +} + +# Check that the value of the given arg is empty. If it isn't, exit with an +# error. +function assert_empty { + local -r arg_name="$1" + local -r arg_value="$2" + local -r reason="$3" + + if [[ ! -z "$arg_value" ]]; then + log_error "The value for '$arg_name' must be empty. $reason" + exit 1 + fi +} + +# Check that the given response from AWS is not empty or null (the null often +# comes from trying to parse AWS responses with jq). If it is, exit with an +# error. +function assert_not_empty_or_null { + local -r response="$1" + local -r description="$2" + + if string_is_empty_or_null "$response"; then + log_error "Got empty response for $description" + exit 1 + fi +} + +# Check that the given value is one of the values from the given list. If not, exit with an error. +function assert_value_in_list { + local -r arg_name="$1" + local -r arg_value="$2" + shift 2 + local -ar list=("$@") + + if ! array_contains "$arg_value" "${list[@]}"; then + log_error "'$arg_value' is not a valid value for $arg_name. Must be one of: [${list[@]}]." + exit 1 + fi +} + +# Check that this script is running as root or sudo and exit with an error if it's not +function assert_uid_is_root_or_sudo { + if ! os_user_is_root_or_sudo; then + log_error "This script should be run using sudo or as the root user" + exit 1 + fi +} + +# Check that the path provided exsists +function assert_path_exists { + local -r arg_name="$1" + local -r arg_value="$2" + + if [ ! -d "$arg_value" ]; then + log_error "The $arg_name provided does not exists." + exit 1 + fi +} + +# Check that the git repository url is valid +function assert_git_validity { + local -r arg_name="$1" + local -r arg_value="$2" + + if [ "$(git ls-remote "$arg_value" > /dev/null 2>&1 && echo $?)" != 0 ]; then + log_error "The $arg_name Git repository URL '$arg_value' is invalid." + exit 1 + fi +} + +# Check that the date is properly formatted +function assert_date_formatting { + local -r arg_value="$1" + + if [ "$(date +"%Y%m%d" -d "$arg_value" 2>/dev/null)" != "$arg_value" ]; then + EXAMPLE_DATE=$(date -d "1 day ago" +'%Y%m%d') + log_error "Incorrect date formatting: Ex. $EXAMPLE_DATE" + exit 1 + fi +} diff --git a/scripts/helpers/log.sh b/scripts/helpers/log.sh new file mode 100755 index 0000000..cbc7789 --- /dev/null +++ b/scripts/helpers/log.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash + +# Log the given message at the given level. All logs are written to stderr with a timestamp. +function log { + local -r level="$1" + local -r message="$2" + local -r timestamp=$(date +"%Y-%m-%d %H:%M:%S") + local -r script_name="$(basename "$0")" + >&2 echo -e "${timestamp} [${level}] [$script_name] ${message}" +} + +# Log the given message at INFO level. All logs are written to stderr with a timestamp. +function log_info { + local -r message="$1" + log "INFO" "$message" +} + +# Log the given message at WARN level. All logs are written to stderr with a timestamp. +function log_warn { + local -r message="$1" + log "WARN" "$message" +} + +# Log the given message at ERROR level. All logs are written to stderr with a timestamp. +function log_error { + local -r message="$1" + log "ERROR" "$message" +} diff --git a/scripts/helpers/os.sh b/scripts/helpers/os.sh new file mode 100755 index 0000000..2a846da --- /dev/null +++ b/scripts/helpers/os.sh @@ -0,0 +1,86 @@ +#!/usr/bin/env bash + +# shellcheck source=./modules/bash-commons/src/log.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/log.sh" + +# Return the available memory on the current OS in MB +function os_get_available_memory_mb { + free -m | awk 'NR==2{print $2}' +} + +# Returns true (0) if this is an Amazon Linux server at the given version or false (1) otherwise. The version number +# can use regex. If you don't care about the version, leave it unspecified. +function os_is_amazon_linux { + local -r version="$1" + grep -q "Amazon Linux * $version" /etc/*release +} + +# Returns true (0) if this is an Ubuntu server at the given version or false (1) otherwise. The version number +# can use regex. If you don't care about the version, leave it unspecified. +function os_is_ubuntu { + local -r version="$1" + grep -q "Ubuntu $version" /etc/*release +} + +# Returns true (0) if this is a CentOS server at the given version or false (1) otherwise. The version number +# can use regex. If you don't care about the version, leave it unspecified. +function os_is_centos { + local -r version="$1" + grep -q "CentOS Linux release $version" /etc/*release +} + +# Returns true (0) if this is a RedHat server at the given version or false (1) otherwise. The version number +# can use regex. If you don't care about the version, leave it unspecified. +function os_is_redhat { + local -r version="$1" + grep -q "Red Hat Enterprise Linux Server release $version" /etc/*release +} + + +# Returns true (0) if this is an OS X server or false (1) otherwise. +function os_is_darwin { + [[ $(uname -s) == "Darwin" ]] +} + +# Validate that the given file has the given checksum of the given checksum type, where type is one of "md5" or +# "sha256". +function os_validate_checksum { + local -r filepath="$1" + local -r checksum="$2" + local -r checksum_type="$3" + + case "$checksum_type" in + sha256) + log_info "Validating sha256 checksum of $filepath is $checksum" + echo "$checksum $filepath" | sha256sum -c + ;; + md5) + log_info "Validating md5 checksum of $filepath is $checksum" + echo "$checksum $filepath" | md5sum -c + ;; + *) + log_error "Unsupported checksum type: $checksum_type." + exit 1 + esac +} + +# Returns true (0) if this the given command/app is installed and on the PATH or false (1) otherwise. +function os_command_is_installed { + local -r name="$1" + command -v "$name" > /dev/null +} + +# Get the username of the current OS user +function os_get_current_users_name { + id -u -n +} + +# Get the name of the primary group for the current OS user +function os_get_current_users_group { + id -g -n +} + +# Returns true (0) if the current user is root or sudo and false (1) otherwise. +function os_user_is_root_or_sudo { + [[ "$EUID" == 0 ]] +} diff --git a/scripts/helpers/output.sh b/scripts/helpers/output.sh new file mode 100755 index 0000000..25ebba7 --- /dev/null +++ b/scripts/helpers/output.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +# output.sh contains helper functions for formatted script output and colors + +# info is a function that will take an argument and echo it to output. The +# output gets emitted in green. +print_info() { + echo -e "${Green}[I] -> $1${Color_Off}" +} + +# warn is a function that will take an argument and echo it to output. The +# output gets emitted in yellow. +print_warn() { + echo -e "${Yellow}[W] -> $1${Color_Off}" +} + +# error is a function that will take an argument and echo it to output. The +# output gets emitted in red. +print_error() { + echo -e "${Red}[E] -> $1${Color_Off}" +} + +# Reset +Color_Off='\033[0m' # Text Reset + +# Regular Colors +Black='\033[0;30m' # Black +Red='\033[0;31m' # Red +Green='\033[0;32m' # Green +Yellow='\033[0;33m' # Yellow +Blue='\033[0;34m' # Blue +Purple='\033[0;35m' # Purple +Cyan='\033[0;36m' # Cyan +White='\033[0;37m' # White + +# Bold +BBlack='\033[1;30m' # Black +BRed='\033[1;31m' # Red +BGreen='\033[1;32m' # Green +BYellow='\033[1;33m' # Yellow +BBlue='\033[1;34m' # Blue +BPurple='\033[1;35m' # Purple +BCyan='\033[1;36m' # Cyan +BWhite='\033[1;37m' # White + +# Underline +UBlack='\033[4;30m' # Black +URed='\033[4;31m' # Red +UGreen='\033[4;32m' # Green +UYellow='\033[4;33m' # Yellow +UBlue='\033[4;34m' # Blue +UPurple='\033[4;35m' # Purple +UCyan='\033[4;36m' # Cyan +UWhite='\033[4;37m' # White + +# Background +On_Black='\033[40m' # Black +On_Red='\033[41m' # Red +On_Green='\033[42m' # Green +On_Yellow='\033[43m' # Yellow +On_Blue='\033[44m' # Blue +On_Purple='\033[45m' # Purple +On_Cyan='\033[46m' # Cyan +On_White='\033[47m' # White + +# High Intensity +IBlack='\033[0;90m' # Black +IRed='\033[0;91m' # Red +IGreen='\033[0;92m' # Green +IYellow='\033[0;93m' # Yellow +IBlue='\033[0;94m' # Blue +IPurple='\033[0;95m' # Purple +ICyan='\033[0;96m' # Cyan +IWhite='\033[0;97m' # White + +# Bold High Intensity +BIBlack='\033[1;90m' # Black +BIRed='\033[1;91m' # Red +BIGreen='\033[1;92m' # Green +BIYellow='\033[1;93m' # Yellow +BIBlue='\033[1;94m' # Blue +BIPurple='\033[1;95m' # Purple +BICyan='\033[1;96m' # Cyan +BIWhite='\033[1;97m' # White + +# High Intensity backgrounds +On_IBlack='\033[0;100m' # Black +On_IRed='\033[0;101m' # Red +On_IGreen='\033[0;102m' # Green +On_IYellow='\033[0;103m' # Yellow +On_IBlue='\033[0;104m' # Blue +On_IPurple='\033[0;105m' # Purple +On_ICyan='\033[0;106m' # Cyan +On_IWhite='\033[0;107m' # White diff --git a/scripts/helpers/string.sh b/scripts/helpers/string.sh new file mode 100755 index 0000000..a7eca79 --- /dev/null +++ b/scripts/helpers/string.sh @@ -0,0 +1,58 @@ +#!/usr/bin/env bash + +# Return true (0) if the first string (haystack) contains the second string (needle), and false (1) otherwise. +function string_contains { + local -r haystack="$1" + local -r needle="$2" + + [[ "$haystack" == *"$needle"* ]] +} + +# Returns true (0) if the first string (haystack), which is assumed to contain multiple lines, contains the second +# string (needle), and false (1) otherwise. The needle can contain regular expressions. +function string_multiline_contains { + local -r haystack="$1" + local -r needle="$2" + + echo "$haystack" | grep -q "$needle" +} + +# Convert the given string to uppercase +function string_to_uppercase { + local -r str="$1" + echo "$str" | awk '{print toupper($0)}' +} + +# Strip the prefix from the given string. Supports wildcards. +# +# Example: +# +# string_strip_prefix "foo=bar" "foo=" ===> "bar" +# string_strip_prefix "foo=bar" "*=" ===> "bar" +# +# http://stackoverflow.com/a/16623897/483528 +function string_strip_prefix { + local -r str="$1" + local -r prefix="$2" + echo "${str#$prefix}" +} + +# Strip the suffix from the given string. Supports wildcards. +# +# Example: +# +# string_strip_suffix "foo=bar" "=bar" ===> "foo" +# string_strip_suffix "foo=bar" "=*" ===> "foo" +# +# http://stackoverflow.com/a/16623897/483528 +function string_strip_suffix { + local -r str="$1" + local -r suffix="$2" + echo "${str%$suffix}" +} + +# Return true if the given response is empty or "null" (the latter is from jq parsing). +function string_is_empty_or_null { + local -r response="$1" + [[ -z "$response" || "$response" == "null" ]] +} diff --git a/scripts/keycloak/bootstrap-keycloak.sh b/scripts/keycloak/bootstrap-keycloak.sh index 334a654..42f24a4 100755 --- a/scripts/keycloak/bootstrap-keycloak.sh +++ b/scripts/keycloak/bootstrap-keycloak.sh @@ -1,172 +1,250 @@ -#!/bin/bash - -# Simplified bootstrap script for Keycloak without SPI mappers or groups -# Groups/projects are now managed in the governance service - +#!/usr/bin/env bash set -e -# Color codes for output -RED='\033[0;31m' -GREEN='\033[0;32m' -YELLOW='\033[1;33m' -BLUE='\033[0;34m' -NC='\033[0m' # No Color - -# Configuration -NAMESPACE="governance" -BOOTSTRAP_RELEASE="keycloak-bootstrap" - -# Get script directory -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -CHART_DIR="$(cd "$SCRIPT_DIR/../../charts/keycloak-bootstrap" && pwd)" - -echo -e "${BLUE}Simplified Keycloak Bootstrap for Governance Platform${NC}" -echo -e "${BLUE}=================================================${NC}" -echo "" -echo "This will create:" -echo "- Governance realm with token exchange enabled" -echo "- Platform admin user (no groups)" -echo "- Three OAuth clients:" -echo " - Frontend (public client)" -echo " - Backend (confidential with service account)" -echo " - Worker (service account only)" -echo "- Custom authorization scopes" -echo "" - -# Check prerequisites -echo -e "${YELLOW}Checking prerequisites...${NC}" - -# Check if Keycloak pod is running -KEYCLOAK_READY=$(kubectl get pod -l app=keycloak -n "$NAMESPACE" -o jsonpath='{.items[0].status.conditions[?(@.type=="Ready")].status}' 2>/dev/null) -if [ "$KEYCLOAK_READY" != "True" ]; then - echo -e "${RED}Error: Keycloak pod is not ready${NC}" - kubectl get pod -l app=keycloak -n "$NAMESPACE" - exit 1 -fi -echo -e "${GREEN}βœ“ Keycloak is running${NC}" +SOURCE="${BASH_SOURCE[0]}" +while [ -h "$SOURCE" ]; do SOURCE="$(readlink "$SOURCE")"; done +ROOTDIR="$(cd -P "$(dirname "$SOURCE")/../.." && pwd)" + +# shellcheck source=../helpers/output.sh +source "$ROOTDIR/scripts/helpers/output.sh" +# shellcheck source=../helpers/assert.sh +source "$ROOTDIR/scripts/helpers/assert.sh" + +# Function to display usage +usage() { + echo -e "\ +Bootstrap Keycloak realm, clients, scopes, and users for the Governance Platform + +Usage: $0 -f [options] + -f, --values Helm values file for keycloak-bootstrap chart (required) + -n, --namespace Kubernetes namespace (default: $NAMESPACE) + -r, --release Helm release name (default: $BOOTSTRAP_RELEASE) + -c, --chart-dir Chart directory (default: $CHART_DIR) + -h, --help Show this help message + +Examples: + $0 -f $CHART_DIR/examples/values.yaml + $0 -f my-values.yaml --namespace governance-stag +" +} + +# Verify Keycloak is running and required secrets exist +check_prerequisites() { + print_warn "Checking prerequisites..." + + # Check if Keycloak pod is running + local keycloak_ready + keycloak_ready=$(kubectl get pod -l app=keycloak -n "$NAMESPACE" -o jsonpath='{.items[0].status.conditions[?(@.type=="Ready")].status}' 2>/dev/null) + if [ "$keycloak_ready" != "True" ]; then + print_error "Keycloak pod is not ready" + kubectl get pod -l app=keycloak -n "$NAMESPACE" + exit 1 + fi + print_info "Keycloak is running" + + # Check required secrets + # Only keycloak-admin and platform-admin are mounted by the bootstrap job + # Backend/worker client secrets are auto-generated by Keycloak + local missing_secrets=() + for secret in keycloak-admin platform-admin; do + if ! kubectl get secret "$secret" -n "$NAMESPACE" &>/dev/null; then + missing_secrets+=("$secret") + fi + done + + if [ ${#missing_secrets[@]} -gt 0 ]; then + print_error "Missing required secrets:" + printf '%s\n' "${missing_secrets[@]}" + echo "" + echo "Create them with:" + echo " kubectl create secret generic keycloak-admin --from-literal=password= -n $NAMESPACE" + echo " kubectl create secret generic platform-admin --from-literal=password= -n $NAMESPACE" + exit 1 + fi + print_info "All required secrets exist" +} + +# Deploy the keycloak-bootstrap Helm chart and monitor the job to completion +run_bootstrap() { + # Clean up any existing bootstrap jobs + print_warn "Cleaning up any existing bootstrap jobs..." + kubectl delete job -l app.kubernetes.io/instance="$BOOTSTRAP_RELEASE" -n "$NAMESPACE" --ignore-not-found + + # Run the bootstrap + print_warn "Running Keycloak bootstrap..." + + helm upgrade --install "$BOOTSTRAP_RELEASE" "$CHART_DIR" \ + -n "$NAMESPACE" \ + -f "$VALUES_FILE" \ + --wait \ + --timeout 10m + + # Get the job name + local job_name + job_name=$(kubectl get job -l app.kubernetes.io/instance="$BOOTSTRAP_RELEASE" -n "$NAMESPACE" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) + + if [ -z "$job_name" ]; then + print_error "Bootstrap job not found" + exit 1 + fi -# Check required secrets -MISSING_SECRETS=() -for secret in keycloak-admin keycloak-backend-client keycloak-worker-client keycloak-admin-user; do - if ! kubectl get secret "$secret" -n "$NAMESPACE" &>/dev/null; then - MISSING_SECRETS+=("$secret") + echo "Bootstrap job: $job_name" + + # Monitor job completion + print_warn "Monitoring bootstrap job..." + + local timeout=300 + local elapsed=0 + while [ $elapsed -lt $timeout ]; do + local job_status + local job_failed + job_status=$(kubectl get job "$job_name" -n "$NAMESPACE" -o jsonpath='{.status.conditions[?(@.type=="Complete")].status}' 2>/dev/null) + job_failed=$(kubectl get job "$job_name" -n "$NAMESPACE" -o jsonpath='{.status.conditions[?(@.type=="Failed")].status}' 2>/dev/null) + + if [ "$job_status" = "True" ]; then + print_info "Bootstrap completed successfully" + break + elif [ "$job_failed" = "True" ]; then + print_error "Bootstrap job failed" + echo "Job logs:" + kubectl logs job/"$job_name" -n "$NAMESPACE" --tail=50 + exit 1 + fi + + echo -n "." + sleep 5 + elapsed=$((elapsed + 5)) + done + + if [ $elapsed -ge $timeout ]; then + print_error "Bootstrap job timed out" + kubectl logs job/"$job_name" -n "$NAMESPACE" --tail=50 + exit 1 fi -done -if [ ${#MISSING_SECRETS[@]} -gt 0 ]; then - echo -e "${RED}Error: Missing required secrets:${NC}" - printf '%s\n' "${MISSING_SECRETS[@]}" + # Show logs echo "" - echo "Create them with:" - echo " kubectl create secret generic keycloak-admin --from-literal=password= -n $NAMESPACE" - echo " kubectl create secret generic keycloak-backend-client --from-literal=client-secret= -n $NAMESPACE" - echo " kubectl create secret generic keycloak-worker-client --from-literal=client-secret= -n $NAMESPACE" - echo " kubectl create secret generic keycloak-admin-user --from-literal=password= -n $NAMESPACE" - exit 1 -fi -echo -e "${GREEN}βœ“ All required secrets exist${NC}" + print_warn "Bootstrap logs (last 20 lines):" + kubectl logs job/"$job_name" -n "$NAMESPACE" --tail=20 -# Clean up any existing bootstrap jobs -echo "" -echo -e "${YELLOW}Cleaning up any existing bootstrap jobs...${NC}" -kubectl delete job -l app.kubernetes.io/instance="$BOOTSTRAP_RELEASE" -n "$NAMESPACE" --ignore-not-found + # Cleanup completed job + echo "" + print_warn "Cleaning up completed job..." + kubectl delete job "$job_name" -n "$NAMESPACE" --ignore-not-found +} + +# Display credentials, OAuth clients, and Keycloak URLs +show_summary() { + echo "" + print_info "Bootstrap Complete!" + echo "" + echo "Keycloak URLs:" + echo "- Admin Console: https://DOMAIN/keycloak/admin" + echo "- Governance Realm: https://DOMAIN/keycloak/admin/governance/console" + echo "" -# Run the bootstrap -echo "" -echo -e "${YELLOW}Running Keycloak bootstrap...${NC}" + # Show credentials + if kubectl get secret keycloak-admin -n "$NAMESPACE" &>/dev/null; then + local admin_password + admin_password=$(kubectl get secret --namespace "$NAMESPACE" keycloak-admin -o jsonpath="{.data.password}" | base64 -d) + echo "Keycloak Admin (master realm):" + echo " Username: admin" + echo " Password: $admin_password" + echo "" + fi -# Install the bootstrap chart with values -helm upgrade --install "$BOOTSTRAP_RELEASE" "$CHART_DIR" \ - -n "$NAMESPACE" \ - -f "$SCRIPT_DIR/values-bootstrap-keycloak.yaml" \ - --wait \ - --timeout 10m + if kubectl get secret platform-admin -n "$NAMESPACE" &>/dev/null; then + local platform_admin_password + platform_admin_password=$(kubectl get secret --namespace "$NAMESPACE" platform-admin -o jsonpath="{.data.password}" | base64 -d) + echo "Platform Admin (governance realm):" + echo " Username: platform-admin" + echo " Password: $platform_admin_password" + echo "" + fi -# Get the job name -JOB_NAME=$(kubectl get job -l app.kubernetes.io/instance="$BOOTSTRAP_RELEASE" -n "$NAMESPACE" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) + echo "OAuth Clients:" + echo "- Frontend: governance-platform-frontend (public client)" + echo "- Backend: governance-platform-backend (confidential with service account)" + echo "- Worker: governance-worker (service account only)" + echo "" + echo "Backend service account roles: query-users, view-users (realm-management)" -if [ -z "$JOB_NAME" ]; then - echo -e "${RED}Error: Bootstrap job not found${NC}" - exit 1 -fi + echo "" + print_info "Bootstrap process completed!" +} -echo "Bootstrap job: $JOB_NAME" - -# Monitor job completion -echo "" -echo -e "${YELLOW}Monitoring bootstrap job...${NC}" - -# Wait for job to complete -TIMEOUT=300 -ELAPSED=0 -while [ $ELAPSED -lt $TIMEOUT ]; do - JOB_STATUS=$(kubectl get job "$JOB_NAME" -n "$NAMESPACE" -o jsonpath='{.status.conditions[?(@.type=="Complete")].status}' 2>/dev/null) - JOB_FAILED=$(kubectl get job "$JOB_NAME" -n "$NAMESPACE" -o jsonpath='{.status.conditions[?(@.type=="Failed")].status}' 2>/dev/null) - - if [ "$JOB_STATUS" = "True" ]; then - echo -e "${GREEN}βœ“ Bootstrap completed successfully${NC}" - break - elif [ "$JOB_FAILED" = "True" ]; then - echo -e "${RED}Bootstrap job failed${NC}" - echo "Job logs:" - kubectl logs job/"$JOB_NAME" -n "$NAMESPACE" --tail=50 +# Default values +NAMESPACE="governance" +BOOTSTRAP_RELEASE="keycloak-bootstrap" +CHART_DIR="$ROOTDIR/charts/keycloak-bootstrap" +VALUES_FILE="" + +# Parse arguments +while [[ $# -gt 0 ]]; do + case $1 in + -f | --values) + VALUES_FILE="$2" + shift 2 + ;; + -n | --namespace) + NAMESPACE="$2" + shift 2 + ;; + -r | --release) + BOOTSTRAP_RELEASE="$2" + shift 2 + ;; + -c | --chart-dir) + CHART_DIR="$2" + shift 2 + ;; + -h | --help) + usage + exit 0 + ;; + *) + print_error "Unknown option: $1" + usage exit 1 - fi - - echo -n "." - sleep 5 - ELAPSED=$((ELAPSED + 5)) + ;; + esac done -if [ $ELAPSED -ge $TIMEOUT ]; then - echo -e "${RED}Bootstrap job timed out${NC}" - kubectl logs job/"$JOB_NAME" -n "$NAMESPACE" --tail=50 - exit 1 -fi - -# Show logs -echo "" -echo -e "${YELLOW}Bootstrap logs (last 20 lines):${NC}" -kubectl logs job/"$JOB_NAME" -n "$NAMESPACE" --tail=20 - -# Display results -echo "" -echo -e "${BLUE}Bootstrap Complete!${NC}" -echo -e "${BLUE}=================${NC}" -echo "" -echo "Keycloak URLs:" -echo "- Admin Console: https://DOMAIN/keycloak/admin" -echo "- Governance Realm: https://DOMAIN/keycloak/admin/governance/console" -echo "" - -# Show credentials -if kubectl get secret keycloak-admin -n "$NAMESPACE" &>/dev/null; then - ADMIN_PASSWORD=$(kubectl get secret --namespace "$NAMESPACE" keycloak-admin -o jsonpath="{.data.password}" | base64 -d) - echo "Keycloak Admin (master realm):" - echo " Username: admin" - echo " Password: $ADMIN_PASSWORD" - echo "" -fi +# Validate prerequisites +assert_is_installed "helm" +assert_is_installed "kubectl" -if kubectl get secret keycloak-admin-user -n "$NAMESPACE" &>/dev/null; then - PLATFORM_ADMIN_PASSWORD=$(kubectl get secret --namespace "$NAMESPACE" keycloak-admin-user -o jsonpath="{.data.password}" | base64 -d) - echo "Platform Admin (governance realm):" - echo " Username: platform-admin" - echo " Password: $PLATFORM_ADMIN_PASSWORD" - echo "" -fi +# Validate required arguments +assert_not_empty "values-file" "$VALUES_FILE" "Use -f or --values to provide a Helm values file." -echo "OAuth Clients:" -echo "- Frontend: governance-platform-frontend (public client)" -echo "- Backend: governance-platform-backend (confidential with service account)" -echo "- Worker: governance-worker (service account only)" -echo "" -echo "Token Exchange: Enabled for Auth Service integration" +# Validate path to file exists +assert_path_exists "chart-dir" "$CHART_DIR" -# Cleanup -echo "" -echo -e "${YELLOW}Cleaning up completed job...${NC}" -kubectl delete job "$JOB_NAME" -n "$NAMESPACE" --ignore-not-found +if [ ! -f "$VALUES_FILE" ]; then + print_error "Values file not found: $VALUES_FILE" + exit 1 +fi -echo "" -echo -e "${GREEN}Bootstrap process completed!${NC}" +echo -e "\ +Namespace: $NAMESPACE +Release: $BOOTSTRAP_RELEASE +Chart: $CHART_DIR +Values: $VALUES_FILE + +This will create: +- Governance realm +- Platform admin user +- Three OAuth clients: + - Frontend (public client) + - Backend (confidential with service account) + - Worker (service account only) +- Custom authorization scopes +- Service account roles for backend client +" + +# Verify Keycloak +check_prerequisites +# Deploy the keycloak-bootstrap +run_bootstrap +# Display outputs +show_summary diff --git a/scripts/keycloak/create-keycloak-organization.sh b/scripts/keycloak/create-keycloak-organization.sh deleted file mode 100755 index d11c19b..0000000 --- a/scripts/keycloak/create-keycloak-organization.sh +++ /dev/null @@ -1,339 +0,0 @@ -#!/bin/bash - -# Script to create organization in governance service database for Keycloak realm -# This should be run AFTER the governance platform is deployed - -set -e - -# Color codes for output -RED='\033[0;31m' -GREEN='\033[0;32m' -YELLOW='\033[1;33m' -BLUE='\033[0;34m' -NC='\033[0m' # No Color - -# Default values -DEFAULT_NAMESPACE="governance" -DEFAULT_REALM="governance" -DEFAULT_DB_NAME="governance" - -# Function to print usage -usage() { - echo "Usage: $0 [OPTIONS]" - echo "" - echo "Options:" - echo " -n, --namespace Kubernetes namespace (default: $DEFAULT_NAMESPACE)" - echo " -r, --realm Keycloak realm name (default: $DEFAULT_REALM)" - echo " -d, --database Database name (default: $DEFAULT_DB_NAME)" - echo " -p, --pod Specific pod name (optional, will auto-detect)" - echo " -h, --help Show this help message" - echo "" - echo "Example:" - echo " $0 --namespace governance-stag --realm governance" -} - -# Parse arguments -NAMESPACE=$DEFAULT_NAMESPACE -REALM_NAME=$DEFAULT_REALM -DB_NAME=$DEFAULT_DB_NAME -POD_NAME="" - -while [[ $# -gt 0 ]]; do - case $1 in - -n | --namespace) - NAMESPACE="$2" - shift 2 - ;; - -r | --realm) - REALM_NAME="$2" - shift 2 - ;; - -d | --database) - DB_NAME="$2" - shift 2 - ;; - -p | --pod) - POD_NAME="$2" - shift 2 - ;; - -h | --help) - usage - exit 0 - ;; - *) - echo "Unknown option: $1" - usage - exit 1 - ;; - esac -done - -echo -e "${BLUE}Creating Keycloak Organization in Governance Database${NC}" -echo -e "${BLUE}==================================================${NC}" -echo "" -echo "Namespace: $NAMESPACE" -echo "Realm/Organization: $REALM_NAME" -echo "Database: $DB_NAME" -echo "" - -# Function to get PostgreSQL password -get_postgres_password() { - # Try common secret names - local password="" - - # Try platform-database first (current standard) - password=$(kubectl get secret -n "$NAMESPACE" platform-database -o jsonpath="{.data.password}" 2>/dev/null | base64 -d) - - if [ -z "$password" ]; then - # List available secrets for debugging - echo -e "${RED}Error: Could not find PostgreSQL password${NC}" - echo "Available secrets that might contain database credentials:" - kubectl get secrets -n "$NAMESPACE" | grep -E "(postgres|database|platform)" - return 1 - fi - - echo "$password" -} - -# Function to find governance database pod -find_db_pod() { - echo -e "${YELLOW}Finding governance database pod...${NC}" - - if [ -n "$POD_NAME" ]; then - echo "Using specified pod: $POD_NAME" - return - fi - - # Try to find the governance PostgreSQL pod - POD_NAME=$(kubectl get pods -n "$NAMESPACE" -l "app.kubernetes.io/name=postgresql,app.kubernetes.io/instance=governance-platform" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) - - if [ -z "$POD_NAME" ]; then - # Try alternative label - POD_NAME=$(kubectl get pods -n "$NAMESPACE" -l "app=postgresql" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) - fi - - if [ -z "$POD_NAME" ]; then - echo -e "${RED}Error: Could not find PostgreSQL pod${NC}" - echo "Please specify pod name with -p option" - kubectl get pods -n "$NAMESPACE" - exit 1 - fi - - echo "Found pod: $POD_NAME" -} - -# Function to check if organization exists -check_org_exists() { - echo -e "${YELLOW}Checking if organization already exists...${NC}" - - RESULT=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -tA -c "SELECT COUNT(*) FROM organization WHERE name = '$REALM_NAME';" 2>/dev/null) - - if [ "$RESULT" = "1" ]; then - echo -e "${GREEN}Organization '$REALM_NAME' already exists${NC}" - - # Show existing organization - echo "" - echo "Existing organization details:" - kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -c "SELECT id, name, display_name, idp_provider, created_at FROM organization WHERE name = '$REALM_NAME';" - return 0 - else - return 1 - fi -} - -# Function to create organization -create_organization() { - echo -e "${YELLOW}Creating organization '$REALM_NAME'...${NC}" - - # Prepare SQL statement - SQL="INSERT INTO organization (name, description, display_name, idp_provider, settings, created_at, updated_at) VALUES ('$REALM_NAME', '$REALM_NAME', '$REALM_NAME', 'keycloak', '{}', NOW(), NOW()) RETURNING id, name, display_name, idp_provider;" - - # Execute SQL - RESULT=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -c "$SQL" 2>&1) - - if [ $? -eq 0 ]; then - echo -e "${GREEN}Organization created successfully!${NC}" - echo "" - echo "$RESULT" - else - echo -e "${RED}Failed to create organization${NC}" - echo "$RESULT" - exit 1 - fi -} - -# Function to verify tables exist -verify_tables() { - echo -e "${YELLOW}Verifying database tables exist...${NC}" - - # Check for all required tables - local required_tables=("organization" "users" "user_organization_memberships") - local all_exist=true - local missing_tables=() - - for table in "${required_tables[@]}"; do - local count=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -- \ - env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -tA -c \ - "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'public' AND table_name = '$table';" 2>/dev/null) - - if [ "$count" != "1" ]; then - all_exist=false - missing_tables+=("$table") - fi - done - - if [ "$all_exist" = true ]; then - echo -e "${GREEN}βœ“ All required tables exist${NC}" - return 0 - else - echo -e "${RED}Error: Required tables are missing:${NC}" - printf ' - %s\n' "${missing_tables[@]}" - echo "" - echo "This usually means:" - echo "1. The governance platform is not deployed yet, OR" - echo "2. The application hasn't started and run migrations yet" - echo "" - echo "Please ensure:" - echo "1. Governance platform is deployed" - echo "2. Wait for the governance-service to be running" - echo "3. Try again in a minute" - - # Show existing tables for debugging - echo "" - echo "Existing tables in database:" - kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -c "\dt" 2>/dev/null || echo "Could not list tables" - - exit 1 - fi -} - -# Main execution -main() { - # Find database pod - find_db_pod - - # Get PostgreSQL password - echo -e "${YELLOW}Getting database credentials...${NC}" - PG_PASSWORD=$(get_postgres_password) - if [ -z "$PG_PASSWORD" ]; then - exit 1 - fi - export PG_PASSWORD # Make it available to all functions - - # Verify tables exist - verify_tables - - # Check if organization exists - if check_org_exists; then - read -p "Organization already exists. Do you want to update it? (y/n) [n]: " UPDATE_ORG - UPDATE_ORG=${UPDATE_ORG:-n} - - if [[ "$UPDATE_ORG" =~ ^[Yy]$ ]]; then - echo -e "${YELLOW}Updating organization...${NC}" - UPDATE_SQL="UPDATE organization SET idp_provider = 'keycloak', updated_at = NOW() WHERE name = '$REALM_NAME' RETURNING id, name, display_name, idp_provider;" - kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -c "$UPDATE_SQL" - echo -e "${GREEN}Organization updated${NC}" - fi - else - # Create organization - create_organization - fi - - # Create platform-admin user if requested - echo "" - read -p "Create platform-admin user in auth service? (y/n) [y]: " CREATE_USER - CREATE_USER=${CREATE_USER:-y} - - if [[ "$CREATE_USER" =~ ^[Yy]$ ]]; then - create_platform_admin_user - fi - - echo "" - echo -e "${GREEN}Done!${NC}" - echo "" - echo "Next steps:" - echo "1. Run Keycloak bootstrap to create the '$REALM_NAME' realm" - echo "2. Users authenticated through Keycloak will be associated with this organization" -} - -# Function to create platform-admin user -create_platform_admin_user() { - echo -e "${YELLOW}Creating platform-admin user in auth service...${NC}" - - # If existing KEYCLOAK_USER_ID, use it - if [ -n "$KEYCLOAK_USER_ID" ]; then - echo -e "${YELLOW}Using existing Keycloak user ID: $KEYCLOAK_USER_ID${NC}" - else - # Get Keycloak admin user ID from bootstrap (we'll use a placeholder for now) - local KEYCLOAK_USER_ID="${KEYCLOAK_USER_ID:-$(uuidgen | tr '[:upper:]' '[:lower:]')}" - fi - - # # Get Keycloak admin user ID from bootstrap (we'll use a placeholder for now) - # local KEYCLOAK_USER_ID="${KEYCLOAK_USER_ID:-$(uuidgen | tr '[:upper:]' '[:lower:]')}" - local USER_ID=$(uuidgen | tr '[:upper:]' '[:lower:]') - - # Check if user already exists - local USER_EXISTS=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -tA -c "SELECT COUNT(*) FROM users WHERE email = 'admin@$REALM_NAME.local';" 2>/dev/null) - - if [ "$USER_EXISTS" = "1" ]; then - echo -e "${YELLOW}Platform admin user already exists${NC}" - # Get existing user ID - USER_ID=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -tA -c "SELECT id FROM users WHERE email = 'admin@$REALM_NAME.local';" 2>/dev/null | tr -d ' ') - else - # Create user - USER_SQL="INSERT INTO users (id, idp_provider, idp_user_id, email, email_verified, display_name, given_name, family_name, active, app_metadata, created_at, updated_at, is_service_account, service_config) VALUES ('$USER_ID', 'keycloak', '$KEYCLOAK_USER_ID', 'admin@$REALM_NAME.local', true, 'Platform Admin', 'Platform', 'Admin', true, '{}', NOW(), NOW(), false, '{}') RETURNING id;" - - RESULT=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -c "$USER_SQL" 2>&1) - - if [ $? -eq 0 ]; then - echo -e "${GREEN}Platform admin user created successfully${NC}" - else - echo -e "${RED}Failed to create platform admin user${NC}" - echo "$RESULT" - return 1 - fi - fi - - # Get organization ID - local ORG_ID=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -tA -c "SELECT id FROM organization WHERE name = '$REALM_NAME';" 2>/dev/null | tr -d ' ') - - if [ -z "$ORG_ID" ]; then - echo -e "${RED}Error: Organization not found${NC}" - return 1 - fi - - # Check if membership already exists - local MEMBERSHIP_EXISTS=$(kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -tA -c "SELECT COUNT(*) FROM user_organization_memberships WHERE user_id = '$USER_ID' AND organization_id = $ORG_ID;" 2>/dev/null) - - if [ "$MEMBERSHIP_EXISTS" = "1" ]; then - echo -e "${YELLOW}Organization membership already exists${NC}" - # Update to ensure owner role - UPDATE_SQL="UPDATE user_organization_memberships SET roles = '{organization_owner}', status = 'active' WHERE user_id = '$USER_ID' AND organization_id = $ORG_ID;" - kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -c "$UPDATE_SQL" - else - # Create membership - MEMBERSHIP_ID=$(uuidgen | tr '[:upper:]' '[:lower:]') - MEMBERSHIP_SQL="INSERT INTO user_organization_memberships (id, user_id, organization_id, roles, invited_at, joined_at, status) VALUES ('$MEMBERSHIP_ID', '$USER_ID', $ORG_ID, '{organization_owner}', NOW(), NOW(), 'active');" - - kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -c "$MEMBERSHIP_SQL" - - if [ $? -eq 0 ]; then - echo -e "${GREEN}Organization membership created successfully${NC}" - else - echo -e "${RED}Failed to create organization membership${NC}" - return 1 - fi - fi - - # Show the created user and membership - echo "" - echo "Platform admin user:" - kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -c "SELECT id, email, display_name, idp_provider FROM users WHERE email = 'admin@$REALM_NAME.local';" - - echo "" - echo "Organization membership:" - kubectl exec -n "$NAMESPACE" "$POD_NAME" -- env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" -c "SELECT m.*, o.name as org_name FROM user_organization_memberships m JOIN organization o ON m.organization_id = o.id WHERE m.user_id = '$USER_ID';" -} - -# Run main function -main diff --git a/scripts/keycloak/post-install-keycloak-setup.sh b/scripts/keycloak/post-install-keycloak-setup.sh index 89b1d16..4c6af94 100755 --- a/scripts/keycloak/post-install-keycloak-setup.sh +++ b/scripts/keycloak/post-install-keycloak-setup.sh @@ -1,104 +1,54 @@ -#!/bin/bash - -# Post-install script for Keycloak database setup -# Run this AFTER: -# 1. Keycloak is deployed -# 2. Keycloak bootstrap is complete (realm, clients, users created) -# 3. Governance platform is deployed -# -# NOTE: Database migrations run automatically when the governance-service starts, -# so this script waits for the service to be running and verifies the schema exists. -# -# This script will: -# - Wait for database schema to be ready (migrations complete) -# - Create organization in governance database -# - Create platform-admin user in auth service tables -# - Set up organization membership with owner role -# - Verify the integration - +#!/usr/bin/env bash set -e -# Color codes for output -RED='\033[0;31m' -GREEN='\033[0;32m' -YELLOW='\033[1;33m' -BLUE='\033[0;34m' -NC='\033[0m' # No Color - -# Get script directory -SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" -ROOT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)" +SOURCE="${BASH_SOURCE[0]}" +while [ -h "$SOURCE" ]; do SOURCE="$(readlink "$SOURCE")"; done +ROOTDIR="$(cd -P "$(dirname "$SOURCE")/../.." && pwd)" -# Default values -NAMESPACE="governance" -REALM_NAME="governance" -ENVIRONMENT="dev" -OVERRIDE_KEYCLOAK_URL="" +# shellcheck source=../helpers/output.sh +source "$ROOTDIR/scripts/helpers/output.sh" +# shellcheck source=../helpers/assert.sh +source "$ROOTDIR/scripts/helpers/assert.sh" -# Function to print usage +# Function to display usage usage() { - echo "Usage: $0 [OPTIONS]" - echo "" - echo "Options:" - echo " -n, --namespace Kubernetes namespace (default: $NAMESPACE)" - echo " -r, --realm Keycloak realm name (default: $REALM_NAME)" - echo " -e, --env Environment: dev|stag|prod (default: $ENVIRONMENT)" - echo " -k, --keycloak-url Keycloak URL" - echo " -h, --help Show this help message" - echo "" - echo "Example:" - echo " $0 --namespace governance-stag --realm governance --env stag" + echo -e "\ +Post-install database setup for Keycloak integration + +Usage: $0 -k [options] + -k, --keycloak-url Keycloak URL (required, e.g. https://keycloak.example.com) + -n, --namespace Kubernetes namespace (default: $NAMESPACE) + -r, --realm Keycloak realm name (default: $REALM_NAME) + -D, --display-name Organization display name (default: $DISPLAY_NAME) + -d, --database Database name (default: $DB_NAME) + -h, --help Show this help message + +Examples: + $0 -k https://governance.example.com/keycloak + $0 -k https://governance.example.com/keycloak --realm governance --display-name 'Governance Platform' +" } -# Parse arguments -while [[ $# -gt 0 ]]; do - case $1 in - -n | --namespace) - NAMESPACE="$2" - shift 2 - ;; - -r | --realm) - REALM_NAME="$2" - shift 2 - ;; - -e | --env) - ENVIRONMENT="$2" - shift 2 - ;; - -k | --keycloak-url) - OVERRIDE_KEYCLOAK_URL="$2" - shift 2 - ;; - -h | --help) - usage - exit 0 - ;; - *) - echo "Unknown option: $1" - usage - exit 1 - ;; - esac -done +# Execute psql via kubectl exec on the discovered database pod +run_psql() { + kubectl exec -n "$NAMESPACE" "$DB_POD" -- \ + env PGPASSWORD="$PG_PASSWORD" psql -U postgres -d "$DB_NAME" "$@" +} -echo -e "${BLUE}Post-Install Keycloak Database Setup${NC}" -echo -e "${BLUE}===================================${NC}" -echo "" -echo "Namespace: $NAMESPACE" -echo "Environment: $ENVIRONMENT" -echo "Realm/Organization: $REALM_NAME" -echo "" +# Execute psql and return a trimmed scalar result +run_psql_scalar() { + run_psql -tA -c "$1" 2>/dev/null | tr -d ' ' +} -# Function to wait for governance platform +# Wait for governance-service deployment and database pod to be ready wait_for_platform() { - echo -e "${YELLOW}Waiting for governance platform components...${NC}" + print_warn "Waiting for governance platform components..." # Check for governance service deployment echo "Checking for governance service deployment..." if ! kubectl get deployment -l app.kubernetes.io/name=governance-service -n "$NAMESPACE" &>/dev/null; then - # Try alternative names if ! kubectl get deployment governance-platform-governance-service -n "$NAMESPACE" &>/dev/null; then - echo -e "${RED}Error: Governance service deployment not found${NC}" + print_error "Governance service deployment not found" echo "Available deployments:" kubectl get deployments -n "$NAMESPACE" return 1 @@ -118,184 +68,180 @@ wait_for_platform() { done if [ "$db_ready" = true ]; then - echo -e "${GREEN}βœ“ Database pod is running${NC}" + print_info "Database pod is running" else - echo -e "${RED}Database pod not ready after 150 seconds${NC}" + print_error "Database pod not ready after 150 seconds" return 1 fi - # Check if database is accepting connections - echo "Checking database connectivity..." - local db_pod=$(kubectl get pods -n "$NAMESPACE" -l "app.kubernetes.io/name=postgresql" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) - - if [ -n "$db_pod" ]; then - for i in {1..10}; do - if kubectl exec -n "$NAMESPACE" "$db_pod" -- pg_isready -h localhost -U postgres &>/dev/null; then - echo -e "${GREEN}βœ“ Database is accepting connections${NC}" - return 0 - fi - echo -n "." - sleep 3 - done - fi + # Discover and store DB pod name for reuse + DB_POD=$(kubectl get pods -n "$NAMESPACE" -l "app.kubernetes.io/name=postgresql" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) - echo -e "${YELLOW}Platform components are starting up${NC}" -} - -# Function to get PostgreSQL password -get_postgres_password() { - # Try common secret names - local password="" - - # Try platform-database first (current standard) - password=$(kubectl get secret -n "$NAMESPACE" platform-database -o jsonpath="{.data.password}" 2>/dev/null | base64 -d) - - if [ -z "$password" ]; then - # List available secrets for debugging - echo -e "${RED}Error: Could not find PostgreSQL password${NC}" - echo "Available secrets that might contain database credentials:" - kubectl get secrets -n "$NAMESPACE" | grep -E "(postgres|database|platform)" + if [ -z "$DB_POD" ]; then + print_error "Could not find PostgreSQL pod" return 1 fi - echo "$password" -} - -# Function to verify database schema exists -verify_database_schema() { - local db_pod=$1 - local pg_password=$2 - - # Check for required tables - local required_tables=("organization" "users" "user_organization_memberships") - local all_exist=true - - for table in "${required_tables[@]}"; do - # Use COUNT for more reliable results with password - local count=$(kubectl exec -n "$NAMESPACE" "$db_pod" -- \ - env PGPASSWORD="$pg_password" psql -U postgres -d governance -tA -c \ - "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'public' AND table_name = '$table';" 2>/dev/null) - - # Check if count is 1 (table exists) or 0 (doesn't exist) - if [ "$count" = "1" ]; then - echo -e "${GREEN} βœ“ Table '$table' exists${NC}" - else - echo -e "${RED} Table '$table' does not exist yet (count: ${count:-unknown})${NC}" - all_exist=false + # Check if database is accepting connections + echo "Checking database connectivity..." + for i in {1..10}; do + if kubectl exec -n "$NAMESPACE" "$DB_POD" -- pg_isready -h localhost -U postgres &>/dev/null; then + print_info "Database is accepting connections" + return 0 fi + echo -n "." + sleep 3 done - if [ "$all_exist" = true ]; then - return 0 - else - return 1 - fi + print_warn "Platform components are starting up" } -# Function to ensure database is ready (migrations run on app startup) +# Wait for migrations and verify required database tables exist ensure_database_ready() { - echo -e "${YELLOW}Ensuring database is ready...${NC}" + print_warn "Ensuring database is ready..." - # 1. Wait for governance service to be running + # Wait for governance service to be running (migrations run on startup) echo "Waiting for governance service to be available..." kubectl wait --for=condition=available --timeout=300s \ deployment/governance-platform-governance-service \ -n "$NAMESPACE" 2>/dev/null || true - # 2. Give the app time to run migrations on startup echo "Waiting for application to initialize and run migrations..." sleep 5 - # 3. Get database pod - local db_pod=$(kubectl get pods -n "$NAMESPACE" -l "app.kubernetes.io/name=postgresql" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) - - if [ -z "$db_pod" ]; then - echo -e "${RED}Error: Database pod not found${NC}" - return 1 - fi - - # 4. Get PostgreSQL password + # Get PostgreSQL password echo "Getting database credentials..." - local pg_password=$(get_postgres_password) - if [ -z "$pg_password" ]; then - echo -e "${RED}Error: Could not get PostgreSQL password${NC}" + PG_PASSWORD=$(kubectl get secret -n "$NAMESPACE" platform-database -o jsonpath="{.data.password}" 2>/dev/null | base64 -d) + + if [ -z "$PG_PASSWORD" ]; then + print_error "Could not find PostgreSQL password" + echo "Available secrets that might contain database credentials:" + kubectl get secrets -n "$NAMESPACE" | grep -E "(postgres|database|platform)" return 1 fi - # 5. Verify schema with retries + # Verify schema with retries (matching chart's init container: 30 attempts, 10s sleep) echo "Verifying database schema..." - local retries=5 - for i in $(seq 1 $retries); do - if verify_database_schema "$db_pod" "$pg_password"; then - echo -e "${GREEN}Database schema is ready${NC}" + local required_tables=("organization" "users" "user_organization_memberships") + local retries=30 + + for attempt in $(seq 1 $retries); do + local all_exist=true + for table in "${required_tables[@]}"; do + local exists=$(run_psql_scalar "SELECT EXISTS (SELECT FROM information_schema.tables WHERE table_name = '$table');") + if [ "$exists" != "t" ]; then + all_exist=false + break + fi + done + + if [ "$all_exist" = true ]; then + for table in "${required_tables[@]}"; do + print_info " Table '$table' exists" + done + print_info "Database schema is ready" return 0 fi - if [ $i -lt $retries ]; then - echo "Retry $i/$retries - waiting 30 seconds for migrations to complete..." - sleep 30 + if [ $attempt -lt $retries ]; then + echo "Attempt $attempt/$retries - Waiting 10 seconds for migrations..." + sleep 10 fi done - # 6. Final check - if critical tables exist, proceed with warning - local org_count=$(kubectl exec -n "$NAMESPACE" "$db_pod" -- \ - env PGPASSWORD="$pg_password" psql -U postgres -d governance -tA -c \ - "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = 'public' AND table_name = 'organization';" 2>/dev/null) + print_error "Database may not be fully initialized after $retries attempts" + return 1 +} - if [ "$org_count" = "1" ]; then - echo -e "${YELLOW}Proceeding - organization table exists${NC}" - return 0 +# Create or update the organization record in the governance database +create_organization() { + print_warn "Creating organization '$REALM_NAME'..." + + # Check if organization already exists + local exists=$(run_psql_scalar "SELECT COUNT(*) FROM organization WHERE name = '$REALM_NAME';") + + if [ "$exists" = "1" ]; then + print_info "Organization '$REALM_NAME' already exists" + # Update to ensure idp_provider is set correctly + run_psql -c "UPDATE organization SET idp_provider = 'keycloak', updated_at = NOW() WHERE name = '$REALM_NAME';" + echo "Updated organization to use Keycloak IDP" else - echo -e "${RED}Warning: Database may not be fully initialized${NC}" - return 1 + # Create new organization + run_psql -c "INSERT INTO organization (name, description, display_name, idp_provider, settings, created_at, updated_at) \ + VALUES ('$REALM_NAME', '$REALM_NAME', '$DISPLAY_NAME', 'keycloak', '{}', NOW(), NOW());" + print_info "Created organization '$REALM_NAME'" fi + + # Show the organization + run_psql -c "SELECT id, name, display_name, idp_provider FROM organization WHERE name = '$REALM_NAME';" } -# Function to create organization -create_organization() { - echo -e "${YELLOW}Creating organization in database...${NC}" +# Create or update the platform-admin user and organization membership +create_platform_admin_user() { + local keycloak_user_id=$1 - # Run the organization creation script - "$SCRIPT_DIR/create-keycloak-organization.sh" \ - --namespace "$NAMESPACE" \ - --realm "$REALM_NAME" \ - --database "governance" -} + print_warn "Creating platform-admin user..." -# Function to get platform-admin Keycloak ID -get_platform_admin_keycloak_id() { - echo -e "${YELLOW}Getting platform-admin user ID from Keycloak...${NC}" >&2 + # Generate UUID using PostgreSQL (matching chart approach) + local user_id=$(run_psql_scalar "SELECT gen_random_uuid();") - # Get external Keycloak URL - local keycloak_url="$OVERRIDE_KEYCLOAK_URL" - local ingress_host=$(kubectl get ingress -n "$NAMESPACE" -o jsonpath='{.items[0].spec.rules[0].host}' 2>/dev/null) + # Check if user exists (by email or by IDP composite key) + local user_exists=$(run_psql_scalar "SELECT COUNT(*) FROM users WHERE email = 'admin@$REALM_NAME.local' OR (idp_provider = 'keycloak' AND idp_user_id = '$keycloak_user_id');") - if [ -z "$keycloak_url" ]; then - if [ -n "$ingress_host" ]; then - keycloak_url="https://$ingress_host/keycloak" - else - echo -e "${YELLOW}No ingress found, trying port-forward method${NC}" >&2 - # Start port-forward in background - kubectl port-forward -n "$NAMESPACE" svc/keycloak 8080:80 &>/dev/null & - local pf_pid=$! - sleep 3 - keycloak_url="http://localhost:8080/keycloak" - fi + if [ "$user_exists" != "0" ]; then + print_warn "Platform admin user already exists" + user_id=$(run_psql_scalar "SELECT id FROM users WHERE email = 'admin@$REALM_NAME.local' OR (idp_provider = 'keycloak' AND idp_user_id = '$keycloak_user_id') LIMIT 1;") + else + # Create user + run_psql -c "INSERT INTO users (id, idp_provider, idp_user_id, email, email_verified, display_name, given_name, family_name, active, app_metadata, created_at, updated_at, is_service_account, service_config) \ + VALUES ('$user_id', 'keycloak', '$keycloak_user_id', 'admin@$REALM_NAME.local', true, 'Platform Admin', 'Platform', 'Admin', true, '{}', NOW(), NOW(), false, '{}');" + print_info "Created platform admin user" + fi + + # Get organization ID + local org_id=$(run_psql_scalar "SELECT id FROM organization WHERE name = '$REALM_NAME';") + + if [ -z "$org_id" ]; then + print_error "Organization not found" + return 1 + fi + + # Check if membership exists + local membership_exists=$(run_psql_scalar "SELECT COUNT(*) FROM user_organization_memberships WHERE user_id = '$user_id' AND organization_id = '$org_id';") + + if [ "$membership_exists" = "1" ]; then + print_warn "Membership already exists, updating to ensure owner role" + run_psql -c "UPDATE user_organization_memberships SET roles = '{organization_owner}', status = 'active' WHERE user_id = '$user_id' AND organization_id = '$org_id';" + else + # Create membership + local membership_id=$(run_psql_scalar "SELECT gen_random_uuid();") + run_psql -c "INSERT INTO user_organization_memberships (id, user_id, organization_id, roles, invited_at, joined_at, status) \ + VALUES ('$membership_id', '$user_id', '$org_id', '{organization_owner}', NOW(), NOW(), 'active');" + print_info "Created organization membership" fi + # Show created user + echo "" + echo "Platform admin user:" + run_psql -c "SELECT id, email, display_name, idp_provider FROM users WHERE email = 'admin@$REALM_NAME.local';" +} + +# Look up the platform-admin user ID from Keycloak Admin API +get_platform_admin_keycloak_id() { + print_warn "Getting platform-admin user ID from Keycloak..." >&2 + # Get admin password - local admin_pass=$(kubectl get secret keycloak-admin -n "$NAMESPACE" -o jsonpath='{.data.password}' | base64 -d) + local admin_pass=$(kubectl get secret keycloak-admin -n "$NAMESPACE" -o jsonpath='{.data.password}' 2>/dev/null | base64 -d) if [ -z "$admin_pass" ]; then - echo -e "${YELLOW}Could not get admin password, using placeholder ID${NC}" >&2 - [ -n "$pf_pid" ] && kill $pf_pid 2>/dev/null + print_warn "Could not get admin password, using placeholder ID" >&2 echo "00000000-0000-0000-0000-000000000000" return fi # Get admin token from master realm echo "Getting admin token..." >&2 - local token_response=$(curl -sk -X POST "$keycloak_url/realms/master/protocol/openid-connect/token" \ + local token_response=$(curl -sk -X POST "$KEYCLOAK_URL/realms/master/protocol/openid-connect/token" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "username=admin" \ -d "password=$admin_pass" \ @@ -305,8 +251,7 @@ get_platform_admin_keycloak_id() { local token=$(echo "$token_response" | jq -r '.access_token // empty') if [ -z "$token" ]; then - echo -e "${YELLOW}Could not get Keycloak token, using placeholder ID${NC}" >&2 - [ -n "$pf_pid" ] && kill $pf_pid 2>/dev/null + print_warn "Could not get Keycloak token, using placeholder ID" >&2 echo "00000000-0000-0000-0000-000000000000" return fi @@ -314,50 +259,30 @@ get_platform_admin_keycloak_id() { # Get platform-admin user from realm echo "Looking up platform-admin user in $REALM_NAME realm..." >&2 local user_data=$(curl -sk -H "Authorization: Bearer $token" \ - "$keycloak_url/admin/realms/$REALM_NAME/users?username=platform-admin&exact=true" 2>/dev/null) + "$KEYCLOAK_URL/admin/realms/$REALM_NAME/users?username=platform-admin&exact=true" 2>/dev/null) local user_id=$(echo "$user_data" | jq -r '.[0].id // empty') - # Clean up port-forward if we started it - [ -n "$pf_pid" ] && kill $pf_pid 2>/dev/null - if [ -n "$user_id" ]; then - echo -e "${GREEN}Found platform-admin user ID: $user_id${NC}" >&2 + print_info "Found platform-admin user ID: $user_id" >&2 echo "$user_id" else - echo -e "${YELLOW}Platform-admin user not found in Keycloak, using placeholder${NC}" >&2 + print_warn "Platform-admin user not found in Keycloak, using placeholder" >&2 echo "00000000-0000-0000-0000-000000000000" fi } -# Function to verify integration +# Verify Keycloak realm, database records, and organization membership verify_integration() { - echo -e "${YELLOW}Verifying Keycloak integration...${NC}" + print_warn "Verifying Keycloak integration..." - # Check Keycloak via external URL + # --- Verify Keycloak realm and user --- echo "Checking Keycloak realm..." - # Get external Keycloak URL - local keycloak_url="" - local ingress_host=$(kubectl get ingress -n "$NAMESPACE" -o jsonpath='{.items[0].spec.rules[0].host}' 2>/dev/null) - - if [ -n "$ingress_host" ]; then - keycloak_url="https://$ingress_host/keycloak" - else - echo -e "${YELLOW}No ingress found, trying port-forward method${NC}" - # Start port-forward in background - kubectl port-forward -n "$NAMESPACE" svc/keycloak 8080:80 &>/dev/null & - local pf_pid=$! - sleep 3 - keycloak_url="http://localhost:8080/keycloak" - fi - - # Get admin password - local admin_pass=$(kubectl get secret keycloak-admin -n "$NAMESPACE" -o jsonpath='{.data.password}' | base64 -d) + local admin_pass=$(kubectl get secret keycloak-admin -n "$NAMESPACE" -o jsonpath='{.data.password}' 2>/dev/null | base64 -d) if [ -n "$admin_pass" ]; then - # Get admin token - local token_response=$(curl -sk -X POST "$keycloak_url/realms/master/protocol/openid-connect/token" \ + local token_response=$(curl -sk -X POST "$KEYCLOAK_URL/realms/master/protocol/openid-connect/token" \ -H "Content-Type: application/x-www-form-urlencoded" \ -d "username=admin" \ -d "password=$admin_pass" \ @@ -367,174 +292,169 @@ verify_integration() { local token=$(echo "$token_response" | jq -r '.access_token // empty') if [ -n "$token" ]; then - # Check if realm exists local realm_check=$(curl -sk -o /dev/null -w "%{http_code}" \ -H "Authorization: Bearer $token" \ - "$keycloak_url/admin/realms/$REALM_NAME") + "$KEYCLOAK_URL/admin/realms/$REALM_NAME") if [ "$realm_check" = "200" ]; then - echo -e "${GREEN}βœ“ Keycloak realm '$REALM_NAME' exists${NC}" + print_info "Keycloak realm '$REALM_NAME' exists" - # Check platform-admin user local user_data=$(curl -sk -H "Authorization: Bearer $token" \ - "$keycloak_url/admin/realms/$REALM_NAME/users?username=platform-admin&exact=true" 2>/dev/null) - + "$KEYCLOAK_URL/admin/realms/$REALM_NAME/users?username=platform-admin&exact=true" 2>/dev/null) local user_count=$(echo "$user_data" | jq '. | length // 0') if [ "$user_count" -gt 0 ]; then - echo -e "${GREEN}βœ“ Platform-admin user exists in Keycloak${NC}" + print_info "Platform-admin user exists in Keycloak" else - echo -e "${RED}βœ— Platform-admin user not found in Keycloak${NC}" + print_error "Platform-admin user not found in Keycloak" fi else - echo -e "${RED}βœ— Keycloak realm '$REALM_NAME' not found${NC}" + print_error "Keycloak realm '$REALM_NAME' not found" fi else - echo -e "${YELLOW}Could not verify Keycloak - unable to get token${NC}" + print_warn "Could not verify Keycloak - unable to get token" fi else - echo -e "${YELLOW}Could not verify Keycloak - no admin password found${NC}" + print_warn "Could not verify Keycloak - no admin password found" fi - # Clean up port-forward if we started it - [ -n "$pf_pid" ] && kill $pf_pid 2>/dev/null - - # Check organization in database + # --- Verify database records --- echo "Checking organization in database..." - local db_pod=$(kubectl get pods -n "$NAMESPACE" -l "app.kubernetes.io/name=postgresql" -o jsonpath='{.items[0].metadata.name}' 2>/dev/null) - - if [ -n "$db_pod" ]; then - # Get PostgreSQL password - local pg_password=$(get_postgres_password) - - if [ -n "$pg_password" ]; then - local org_count=$(kubectl exec -n "$NAMESPACE" "$db_pod" -- \ - env PGPASSWORD="$pg_password" psql -U postgres -d governance -tA -c "SELECT COUNT(*) FROM organization WHERE name = '$REALM_NAME' AND idp_provider = 'keycloak';" 2>/dev/null) - - if [ "$org_count" = "1" ]; then - echo -e "${GREEN}βœ“ Organization '$REALM_NAME' exists with idp_provider=keycloak${NC}" - else - echo -e "${RED}βœ— Organization not found or incorrect idp_provider (count: ${org_count:-unknown})${NC}" - fi - - # Check platform-admin user in auth service - local user_count=$(kubectl exec -n "$NAMESPACE" "$db_pod" -- \ - env PGPASSWORD="$pg_password" psql -U postgres -d governance -tA -c "SELECT COUNT(*) FROM users WHERE email = 'admin@$REALM_NAME.local' AND idp_provider = 'keycloak';" 2>/dev/null) - if [ "$user_count" = "1" ]; then - echo -e "${GREEN}βœ“ Platform-admin user exists in auth service${NC}" + local org_count=$(run_psql_scalar "SELECT COUNT(*) FROM organization WHERE name = '$REALM_NAME' AND idp_provider = 'keycloak';") + if [ "$org_count" = "1" ]; then + print_info "Organization '$REALM_NAME' exists with idp_provider=keycloak" + else + print_error "Organization not found or incorrect idp_provider (count: ${org_count:-unknown})" + fi - # Check membership - local membership_count=$(kubectl exec -n "$NAMESPACE" "$db_pod" -- \ - env PGPASSWORD="$pg_password" psql -U postgres -d governance -tA -c "SELECT COUNT(*) FROM user_organization_memberships uom JOIN users u ON uom.user_id = u.id WHERE u.email = 'admin@$REALM_NAME.local' AND 'organization_owner' = ANY(uom.roles);" 2>/dev/null) + local user_count=$(run_psql_scalar "SELECT COUNT(*) FROM users WHERE email = 'admin@$REALM_NAME.local' AND idp_provider = 'keycloak';") + if [ "$user_count" = "1" ]; then + print_info "Platform-admin user exists in auth service" - if [ "$membership_count" = "1" ]; then - echo -e "${GREEN}βœ“ Platform-admin has organization_owner role${NC}" - else - echo -e "${RED}βœ— Platform-admin missing organization_owner role (count: ${membership_count:-unknown})${NC}" - fi - else - echo -e "${RED}βœ— Platform-admin user not found in auth service (count: ${user_count:-unknown})${NC}" - fi + local membership_count=$(run_psql_scalar "SELECT COUNT(*) FROM user_organization_memberships uom JOIN users u ON uom.user_id = u.id WHERE u.email = 'admin@$REALM_NAME.local' AND 'organization_owner' = ANY(uom.roles);") + if [ "$membership_count" = "1" ]; then + print_info "Platform-admin has organization_owner role" else - echo -e "${YELLOW}Could not verify database - unable to get PostgreSQL password${NC}" + print_error "Platform-admin missing organization_owner role (count: ${membership_count:-unknown})" fi else - echo -e "${YELLOW}Could not find database pod${NC}" + print_error "Platform-admin user not found in auth service (count: ${user_count:-unknown})" fi } -# Function to show summary +# Display setup summary with URLs and next steps show_summary() { echo "" - echo -e "${BLUE}Setup Summary${NC}" - echo -e "${BLUE}=============${NC}" + print_info "Setup Summary" echo "" echo "Database Setup Completed:" echo " - Organization: $REALM_NAME (idp_provider=keycloak)" + echo " - Display Name: $DISPLAY_NAME" echo " - Platform Admin: admin@$REALM_NAME.local" echo " - Role: organization_owner" - # Get Keycloak URL - local ingress_host=$(kubectl get ingress -n "$NAMESPACE" -o jsonpath='{.items[0].spec.rules[0].host}' 2>/dev/null) - echo "" echo "Keycloak Information:" - if [ -n "$ingress_host" ]; then - echo " - Admin Console: https://$ingress_host/keycloak/admin" - echo " - Realm: https://$ingress_host/keycloak/admin/$REALM_NAME/console" - else - echo " - Use port-forward: kubectl port-forward -n $NAMESPACE svc/keycloak 8080:80" - echo " - Then access: http://localhost:8080/keycloak/admin" - fi + echo " - Admin Console: $KEYCLOAK_URL/admin" + echo " - Realm: $KEYCLOAK_URL/admin/$REALM_NAME/console" echo "" echo "Next Steps:" echo " 1. Test login with platform-admin user" echo " 2. Verify token exchange with auth service" echo " 3. Check that users can access the governance platform" - - echo "" - echo "To sync Keycloak user IDs (if needed):" - echo " $SCRIPT_DIR/sync-keycloak-user-ids.sh --namespace $NAMESPACE --realm $REALM_NAME" } -# Main execution +# Orchestrate the full post-install setup workflow main() { echo "This script sets up database entries after Keycloak bootstrap" echo "" - echo -e "${YELLOW}Prerequisites:${NC}" + print_warn "Prerequisites:" echo " - Keycloak must be deployed and running" echo " - Keycloak bootstrap must be complete (realm, clients, users created)" echo " - Governance platform must be deployed" echo "" - # Step 1: Wait for platform + # Step 1: Wait for platform and discover DB pod wait_for_platform # Step 2: Ensure database is ready (migrations run on startup) ensure_database_ready - # Step 3: Create organization + # Step 3: Create organization and platform-admin user echo "" - read -p "Create organization in database? (y/n) [y]: " CREATE_ORG - CREATE_ORG=${CREATE_ORG:-y} - - if [[ "$CREATE_ORG" =~ ^[Yy]$ ]]; then - create_organization - fi + create_organization - # Step 4: Create platform-admin user in database echo "" - read -p "Create platform-admin user in auth service? (y/n) [y]: " CREATE_ADMIN - CREATE_ADMIN=${CREATE_ADMIN:-y} - - if [[ "$CREATE_ADMIN" =~ ^[Yy]$ ]]; then - # Get Keycloak user ID from existing platform-admin in Keycloak - - local user_id=$(get_platform_admin_keycloak_id) - echo "USER ID LINE 521 -- $user_id" - export KEYCLOAK_USER_ID=$user_id - - # Run the organization script with user creation - "$SCRIPT_DIR/create-keycloak-organization.sh" \ - --namespace "$NAMESPACE" \ - --realm "$REALM_NAME" \ - --database "governance" <<<"n -y" # Answer n to create org (already done), y to create user - fi + local keycloak_user_id=$(get_platform_admin_keycloak_id) + create_platform_admin_user "$keycloak_user_id" - # Step 5: Verify integration + # Step 4: Verify integration echo "" verify_integration - # Step 6: Show summary + # Step 5: Show summary show_summary echo "" - echo -e "${GREEN}Post-install setup complete!${NC}" + print_info "Post-install setup complete!" } -# Run main function +# Default values +NAMESPACE="governance" +REALM_NAME="governance" +DISPLAY_NAME="Governance Platform" +DB_NAME="governance" +KEYCLOAK_URL="" + +# Shared state (populated during setup) +DB_POD="" +PG_PASSWORD="" + +# Parse arguments +while [[ $# -gt 0 ]]; do + case $1 in + -n | --namespace) + NAMESPACE="$2" + shift 2 + ;; + -r | --realm) + REALM_NAME="$2" + shift 2 + ;; + -D | --display-name) + DISPLAY_NAME="$2" + shift 2 + ;; + -d | --database) + DB_NAME="$2" + shift 2 + ;; + -k | --keycloak-url) + KEYCLOAK_URL="$2" + shift 2 + ;; + -h | --help) + usage + exit 0 + ;; + *) + print_error "Unknown option: $1" + usage + exit 1 + ;; + esac +done + +# Validate prerequisites +assert_is_installed "kubectl" +assert_is_installed "curl" +assert_is_installed "jq" + +# Validate required arguments +assert_not_empty "keycloak-url" "$KEYCLOAK_URL" "Use -k or --keycloak-url to provide the Keycloak URL." + +# Run post-install setup main diff --git a/scripts/keycloak/values-bootstrap-keycloak.yaml b/scripts/keycloak/values-bootstrap-keycloak.yaml deleted file mode 100644 index 997a285..0000000 --- a/scripts/keycloak/values-bootstrap-keycloak.yaml +++ /dev/null @@ -1,161 +0,0 @@ -# Simplified Keycloak Bootstrap values for staging environment -# This configures the bootstrap job without SPI mappers or group management -# Groups/projects are now managed in the governance service - -# Bootstrap job configuration -bootstrap: - enabled: true - - # Use an image that already has curl and jq - image: - repository: dwdraju/alpine-curl-jq - tag: "latest" - pullPolicy: IfNotPresent - - # Job settings - backoffLimit: 3 - ttlSecondsAfterFinished: 300 - activeDeadlineSeconds: 600 - - # Disable the built-in wait since it checks the wrong port - wait: - enabled: false - - # Resources - resources: - limits: - cpu: 500m - memory: 256Mi - requests: - cpu: 100m - memory: 128Mi - -# Keycloak connection -keycloak: - # Internal service URL - url: "http://keycloak:8080/keycloak" - adminUsername: "admin" - adminPasswordSecret: - name: "keycloak-admin" - key: "password" - - # Realm configuration - realm: - name: "governance" - displayName: "Governance Platform" - loginWithEmailAllowed: true - registrationAllowed: false - resetPasswordAllowed: true - rememberMe: true - verifyEmail: false - sslRequired: "external" - bruteForceProtected: true - - # Token configuration - tokens: - accessTokenLifespan: 300 # 5 minutes - ssoSessionIdleTimeout: 1800 # 30 minutes - ssoSessionMaxLifespan: 36000 # 10 hours - -# Client configuration -clients: - # Frontend client (public) - frontend: - clientId: "governance-platform-frontend" - name: "Governance Platform Frontend" - description: "Frontend application for Governance Platform" - publicClient: true - redirectUris: - - "https://CHANGE_ME_DOMAIN_HERE/*" - - "http://localhost:5173/*" - webOrigins: - - "https://CHANGE_ME_DOMAIN_HERE" - - "http://localhost:5173" - defaultScopes: - - "openid" - - "profile" - - "email" - - "roles" - optionalScopes: - - "offline_access" - - # Backend client (confidential with service account) - backend: - clientId: "governance-platform-backend" - name: "Governance Platform Backend" - description: "Backend service for Governance Platform" - publicClient: false - serviceAccountsEnabled: true - secretName: "keycloak-backend-client" - secretKey: "client-secret" - redirectUris: - - "https://CHANGE_ME_DOMAIN_HERE/auth/*" - webOrigins: - - "https://CHANGE_ME_DOMAIN_HERE" - defaultScopes: - - "openid" - - "profile" - - "email" - - "roles" - - # Worker client (service account only) - worker: - clientId: "governance-worker" - name: "Governance Worker" - description: "Worker service for Governance Platform" - publicClient: false - serviceAccountsEnabled: true - secretName: "keycloak-worker-client" - secretKey: "client-secret" - defaultScopes: - - "openid" - - "profile" - - "email" - - "roles" - -# Custom scopes for authorization -scopes: - - name: "governance:declarations:create" - description: "Create governance declarations" - - name: "integrity:statements:create" - description: "Create integrity statements" - - name: "read:organizations" - description: "Read access to organizations" - - name: "write:organizations" - description: "Write access to organizations" - - name: "read:projects" - description: "Read access to projects" - - name: "write:projects" - description: "Write access to projects" - - name: "read:evaluations" - description: "Read access to evaluations" - - name: "write:evaluations" - description: "Write access to evaluations" - -# Initial users -users: - # Platform admin user - admin: - enabled: true - username: "platform-admin" - email: "admin@governance.eqtylab.io" - firstName: "Platform" - lastName: "Admin" - emailVerified: true - temporaryPassword: false - secretName: "keycloak-admin-user" - secretKey: "password" - - # Test users - disabled by default - testUsers: - enabled: false - users: [] - -# Output configuration -output: - generateEnvFile: true - createSecrets: true - secrets: - frontend: "keycloak-frontend-config" - backend: "keycloak-backend-config" - worker: "keycloak-worker-config" diff --git a/scripts/nginx.sh b/scripts/nginx.sh index 587bc18..784d817 100755 --- a/scripts/nginx.sh +++ b/scripts/nginx.sh @@ -1,16 +1,74 @@ #!/usr/bin/env bash - -# This script installs an nginx ingress controller set -e -NAMESPACE=ingress-nginx +SOURCE="${BASH_SOURCE[0]}" +while [ -h "$SOURCE" ]; do SOURCE="$(readlink "$SOURCE")"; done +ROOTDIR="$(cd -P "$(dirname "$SOURCE")/.." && pwd)" + +# shellcheck source=./helpers/output.sh +source "$ROOTDIR/scripts/helpers/output.sh" +# shellcheck source=./helpers/assert.sh +source "$ROOTDIR/scripts/helpers/assert.sh" + +# Function to display usage +usage() { + echo -e "\ +Install an NGINX ingress controller via Helm + +Usage: $0 [options] + -n, --namespace Namespace for ingress-nginx (default: $NAMESPACE) + -h, --help Show this help message + +Examples: + $0 + $0 --namespace somewhere-else +" +} + +# Install NGINX ingress controller into the specified namespace +install() { + print_info "Installing ingress-nginx to $NAMESPACE namespace" + + # Add the ingress-nginx Helm repository + helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx + + # Update your local Helm chart repository cache + helm repo update + + # Install the ingress-nginx Helm chart + helm install ingress-nginx ingress-nginx/ingress-nginx \ + --create-namespace \ + --namespace "$NAMESPACE" \ + --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz + + print_info "ingress-nginx installed" +} + +# Default values +NAMESPACE="ingress-nginx" -echo "Installing ingress-nginx to $NAMESPACE namespace" +# Parse arguments +while [[ $# -gt 0 ]]; do + case $1 in + -n | --namespace) + NAMESPACE="$2" + shift 2 + ;; + -h | --help) + usage + exit 0 + ;; + *) + print_error "Unknown option: $1" + usage + exit 1 + ;; + esac +done -helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx -helm repo update +# Validate prerequisites +assert_is_installed "helm" +assert_is_installed "kubectl" -helm install ingress-nginx ingress-nginx/ingress-nginx \ - --create-namespace \ - --namespace $NAMESPACE \ - --set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-health-probe-request-path"=/healthz +# Install ingress-nginx +install