KubeActivityEngine

A Kubernetes operator for managing Activity resources with distributed worker execution tracking.

Overview

KubeActivityEngine is a Kubernetes operator that manages custom Activity and ActivityWorker resources. It provides a framework for orchestrating activities with configurable inputs and automatically creating workers to execute those activities. When an Activity enters the Initialization state, the controller creates ActivityWorker resources that define job specifications. The operator provides a declarative way to manage distributed activity execution in Kubernetes clusters with full lifecycle tracking.

Features

• Dual Custom Resource Definitions (CRDs): Define activities and workers as Kubernetes resources
  • Activity CRD: Orchestrates activity lifecycle with configurable inputs
  • ActivityWorker CRD: Defines job specifications with ActivityJob structure
• Automatic Worker Creation: Controller automatically creates ActivityWorker resources during Activity initialization
• Flexible Job Configuration: ActivityJob spec supports custom images, commands, args, environment variables, and pull policies
• Worker Status Tracking: Monitor the state of multiple workers processing activities
• Kubernetes-Native: Built using the Kubebuilder framework with full Kubernetes API integration
• Status Conditions: Track activity lifecycle with standard Kubernetes condition types
• RBAC Support: Fine-grained access control with admin, editor, and viewer roles

Architecture

The operator manages two custom resource types that work together:

Activity Resource

The Activity CRD represents a unit of work to be executed. Each Activity contains:

• Spec: Desired state including activity name and input parameters (key-value pairs)
• Status: Observed state with conditions tracking the activity lifecycle

When an Activity is created and reaches the "Initialization" condition state, the controller automatically creates ActivityWorker resources.

ActivityWorker Resource

The ActivityWorker CRD defines the execution specification for workers. Each ActivityWorker contains:

• Spec: Job specification using the ActivityJob structure
  • Name: Unique identifier for the job
  • Image: Container image to run (e.g., bash:5.0)
  • Command: Command array to execute (e.g., ["bash", "-c"])
  • Args: Arguments array for the command (e.g., ["echo done!"])
  • Env: Environment variables for the container
  • ImagePullPolicy: Pull policy for the container image
• Status: Observed state with conditions tracking the worker lifecycle
• Labels: Automatically labeled with kube-activity-engine.algatux.dev/activity to link to parent Activity

Activity States

Workers can be in one of four states:

• READY: Worker is prepared to execute
• RUNNING: Worker is currently executing
• COMPLETED: Worker successfully completed execution
• FAILED: Worker encountered an error

Prerequisites

• Go version v1.24.0+
• Docker version 17.03+
• kubectl version v1.11.3+
• Access to a Kubernetes v1.11.3+ cluster

Installation

Quick Start

1. Install the CRDs into your cluster:

make install

2. Build and push the operator image:

make docker-build docker-push IMG=<your-registry>/kubeactivityengine:tag

3. Deploy the operator:

make deploy IMG=<your-registry>/kubeactivityengine:tag

Note: If you encounter RBAC errors, ensure you have cluster-admin privileges or are logged in as admin.

Deploy Using Pre-built Manifests

You can also deploy using the bundled YAML installer:

kubectl apply -f https://raw.githubusercontent.com/<org>/kubeactivityengine/<tag or branch>/dist/install.yaml

Usage

Creating an Activity

Create an Activity resource with the following manifest:

apiVersion: kube-activity-engine.algatux.dev/v1beta1
kind: Activity
metadata:
  name: my-activity
spec:
  name: "data-processing-task"
  inputs:
    dataset: "production-data"
    batch-size: "1000"
    region: "us-west-2"

Apply the resource:

kubectl apply -f activity.yaml

Viewing Activities

List all activities:

kubectl get activities

View detailed information:

kubectl describe activity my-activity

Quick Demo

After applying an Activity, the controller automatically creates an ActivityWorker resource that defines a job specification. The ActivityWorker uses the new ActivityJob CRD structure:

Default ActivityJob Configuration:

• Image: bash:5.0
• Command: ["bash", "-c"]
• Args: ["echo done!"]
• Environment: TEST=FAKE (example environment variable)
• ImagePullPolicy: IfNotPresent

Verify the created resources:

List ActivityWorkers for your activity:

kubectl get activityworkers -l kube-activity-engine.algatux.dev/activity=my-activity

View ActivityWorker details:

kubectl describe activityworker <worker-name>

Check the ActivityWorker spec:

kubectl get activityworker <worker-name> -o yaml

Expected Output: The ActivityWorker will show the job configuration including the bash command that echoes "done!". The worker's status conditions will track its lifecycle state.

Troubleshooting the demo:

• If no ActivityWorkers are created, check the Activity status conditions:

  kubectl get activity my-activity -o jsonpath='{.status.conditions}'

• Check the controller logs for any errors:

  kubectl logs -n kubeactivityengine-system deployment/kubeactivityengine-controller-manager

• Verify that the Activity has the "Initialization" condition set to "True"

Checking Worker Status

Monitor the status of workers processing your activity:

kubectl get activity my-activity -o jsonpath='{.status.workersStatus}'

Sample Activities

Apply the provided sample activities:

kubectl apply -k config/samples/

Development

Running Tests

Run the controller tests:

make test

Running Locally

Run the controller locally against your configured Kubernetes cluster:

make run

Generating Code

After modifying the API types, regenerate the code:

make generate
make manifests

Uninstallation

1. Delete Activity instances:

kubectl delete -k config/samples/

2. Remove the operator:

make undeploy

3. Uninstall CRDs:

make uninstall

Distribution Options

YAML Bundle

Generate a complete installation bundle:

make build-installer IMG=<your-registry>/kubeactivityengine:tag

This creates dist/install.yaml with all necessary resources.

Helm Chart

Generate a Helm chart for the operator:

kubebuilder edit --plugins=helm/v1-alpha

The chart will be available in dist/chart/. Use --force flag when updating if webhooks are added.

Project Structure

├── api/v1beta1/                      # API definitions
│   ├── activity_types.go             # Activity CRD types
│   ├── activityworker_types.go       # ActivityWorker CRD types (includes ActivityJob spec)
│   └── states/                       # State type definitions
├── internal/
│   ├── controller/                   # Controller implementations
│   │   ├── activity_controller.go    # Activity reconciler
│   │   └── activityworker_controller.go # ActivityWorker reconciler
│   └── state_machine/                # State machine logic
│       ├── activity_state_machine.go # Handler retrieval logic
│       └── transitions/              # State transition handlers
│           ├── initialization.go     # Initialization state handler (creates ActivityWorkers)
│           └── empty_conditions.go   # Initial state handler
├── config/                           # Kubernetes manifests
│   ├── crd/                         # CRD definitions
│   ├── rbac/                        # RBAC configurations
│   ├── samples/                     # Example resources
│   └── manager/                     # Controller deployment
└── cmd/                              # Application entrypoint

RBAC Roles

The operator provides predefined roles for both Activity and ActivityWorker resources:

Activity Roles:

• activity-admin-role: Full access to Activity resources
• activity-editor-role: Create, update, and delete activities
• activity-viewer-role: Read-only access to activities

ActivityWorker Roles:

• activityworker-admin-role: Full access to ActivityWorker resources
• activityworker-editor-role: Create, update, and delete activity workers
• activityworker-viewer-role: Read-only access to activity workers

Contributing

We welcome contributions! To contribute:

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Make your changes and add tests
4. Run tests and ensure they pass (make test)
5. Commit your changes (git commit -m 'Add amazing feature')
6. Push to the branch (git push origin feature/amazing-feature)
7. Open a Pull Request

Please ensure your code follows the project's coding standards and includes appropriate tests.

Development Guidelines

• Follow Go best practices and conventions
• Add unit tests for new functionality
• Update documentation for user-facing changes
• Run make help to see all available make targets

Monitoring

The operator exposes Prometheus metrics at /metrics endpoint. Deploy the ServiceMonitor to scrape metrics:

kubectl apply -k config/prometheus/

Troubleshooting

Common Issues

RBAC Permission Errors

• Ensure you have cluster-admin privileges
• Verify service account has proper RBAC bindings

CRD Installation Fails

• Check that kubectl has access to the cluster
• Verify API server is reachable

Operator Pod Not Starting

• Check image pull secrets are configured
• Verify image registry is accessible
• Review pod logs: kubectl logs -n kubeactivityengine-system deployment/kubeactivityengine-controller-manager

Resources

• Kubebuilder Documentation
• Kubernetes API Conventions
• Controller Runtime

License

Copyright 2025.

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.