RocketGraph Multi-Cloud Infrastructure

Deploy RocketGraph graph analytics platform on AWS, Azure, or GCP using a unified Python deployment script.

Overview

This repository provides a unified deployment experience across multiple cloud providers with:

• Single Python script for AWS, Azure, and GCP deployments
• Automated Docker and Docker Compose installation
• RocketGraph graph analytics platform setup using official installer
• Data pipeline scripts for loading graph datasets
• Terraform Infrastructure as Code with automatic configuration

Directory Structure

.
├── deploy.py               # 🚀 Main deployment script
├── setup.sh               # One-time setup for dependencies
├── requirements.txt       # Python dependencies
├── config.example.json    # Example configuration file
├── terraform/             # Infrastructure as Code
│   ├── aws/              # AWS-specific configuration
│   ├── azure/            # Azure-specific configuration
│   ├── gcp/              # GCP-specific configuration
│   ├── modules/          # Reusable Terraform modules
│   └── scripts/          # User data initialization scripts
├── etl/                  # Data pipeline code
│   ├── extractors/       # Data extraction modules
│   ├── loaders/          # Graph data loading modules
│   └── run_pipeline.py   # Main pipeline orchestrator
└── scripts/              # Additional setup scripts
    └── setup/            # RocketGraph installation scripts

Prerequisites

The setup script will install most dependencies automatically, but you'll need:

• Python 3.7+
• SSH key pair for instance access
• Cloud provider account and credentials

Quick Start

1. Setup (One-time)

Run the setup script to install dependencies:

git clone <this-repo>
cd rocketworx
./setup.sh

This installs:

• Terraform
• Cloud provider CLIs (instructions provided)
• Python dependencies

2. Configure Cloud Credentials

Choose your provider and configure credentials:

# AWS
aws configure

# Azure
az login

# GCP
gcloud auth application-default login

3. Deploy RocketGraph

Use the unified deployment script for any cloud provider:

# Deploy on AWS
python deploy.py --provider aws --ssh-key-name my-key

# Deploy on Azure (uses SSH public key from ~/.ssh/id_rsa.pub)
python deploy.py --provider azure --region "West US"

# Deploy on GCP
python deploy.py --provider gcp --gcp-project-id my-project-123

# See all options
python deploy.py --help

4. Access Your Instance

After deployment, you'll see:

🌐 Public IP: 54.123.45.67
🔗 RocketGraph URL: http://54.123.45.67:8080
💻 SSH Command: ssh -i ~/.ssh/my-key.pem ubuntu@54.123.45.67

Advanced Usage

Using Configuration Files

Create a config file for repeated deployments:

cp config.example.json my-config.json
# Edit my-config.json with your settings
python deploy.py --config-file my-config.json --provider aws

Planning Changes

Preview changes before applying:

python deploy.py --provider aws --plan --ssh-key-name my-key

Destroying Infrastructure

Clean up resources:

python deploy.py --provider aws --destroy --auto-approve

Command Line Options

python deploy.py --help

Key options:

• --provider: aws, azure, or gcp
• --region: Cloud provider region
• --instance-type: Instance size (provider-specific)
• --ssh-key-name: AWS key pair name
• --ssh-public-key-path: Path to public key (Azure/GCP)
• --gcp-project-id: Required for GCP
• --auto-approve: Skip confirmation prompts
• --plan: Preview changes only
• --destroy: Remove infrastructure

Instance Types by Provider

Provider	Small	Medium	Large
AWS	t4g.small	t4g.medium	t4g.large
Azure	Standard_B1ms	Standard_B2ms	Standard_B4ms
GCP	e2-small	e2-medium	e2-standard-2

Example:

python deploy.py --provider aws --instance-type t4g.large

Manual Terraform Deployment (Advanced Users Only)

AWS Deployment

1. Configure AWS credentials:

aws configure

2. Navigate to AWS directory:

cd terraform/aws

3. Create terraform.tfvars:

cp terraform.tfvars.example terraform.tfvars
# Edit terraform.tfvars with your values

4. Deploy:

terraform init
terraform plan
terraform apply

5. Access RocketGraph:

# Get the instance IP
terraform output public_ip

# SSH into instance
terraform output ssh_command

# Access RocketGraph UI
http://<public_ip>:8080

Azure Deployment

1. Login to Azure:

az login

2. Navigate to Azure directory:

cd terraform/azure

3. Create terraform.tfvars:

cat > terraform.tfvars <<EOF
project_name = "rocketgraph"
location = "East US"
ssh_public_key_path = "~/.ssh/id_rsa.pub"
EOF

4. Deploy:

terraform init
terraform apply

GCP Deployment

1. Authenticate with GCP:

gcloud auth application-default login

2. Navigate to GCP directory:

cd terraform/gcp

3. Create terraform.tfvars:

cat > terraform.tfvars <<EOF
project_id = "your-gcp-project-id"
ssh_public_key_path = "~/.ssh/id_rsa.pub"
EOF

4. Deploy:

terraform init
terraform apply

What Gets Installed

Each deployed instance automatically includes:

1. Ubuntu 22.04 LTS - Latest stable Ubuntu server
2. Docker & Docker Compose - Latest container runtime
3. RocketGraph - Graph analytics platform on port 8080
4. XGT REST API - rocketgraph/xgt-rest image for programmatic graph access
5. Python Data Tools - pandas, requests, sqlalchemy for data pipelines
6. Monitoring - Basic logging to /var/log/user-data.log

Deployment Flow

When you run python deploy.py, here's what happens:

1. Validation - Checks credentials, tools, and parameters
2. Terraform Generation - Creates provider-specific configuration
3. Infrastructure Provisioning - Creates cloud resources (VPC, instance, security groups)
4. Automated Installation - Runs user-data script on instance:
   • Updates system packages
   • Installs Docker and Docker Compose
   • Downloads and runs RocketGraph installer
   • Configures services to start automatically
5. Output - Displays connection info and next steps

Total deployment time: 3-5 minutes depending on provider.

Data Pipeline

The data pipeline supports:

• API data extraction
• CSV/JSON file processing
• Database extraction
• Graph data formatting and loading
• Batch processing for large datasets

Running ETL Pipeline

SSH into your instance and run:

# Run pipeline once
cd /opt/rocketgraph
python3 /path/to/etl/run_pipeline.py --run-once

# Schedule pipeline
python3 /path/to/etl/run_pipeline.py --schedule

ETL Configuration

Create a config file:

{
  "sources": [
    {
      "name": "api_source",
      "type": "api",
      "url": "https://api.example.com/data",
      "target_table": "api_data",
      "if_exists": "append"
    },
    {
      "name": "csv_source",
      "type": "csv",
      "path": "/data/input.csv",
      "target_table": "csv_data",
      "if_exists": "replace"
    }
  ]
}

Security Considerations

1. SSH Access: Update ssh_allowed_ips to restrict SSH access
2. API Keys: Store sensitive credentials securely
3. Database Password: Use strong passwords for any databases
4. Network Security: Review and restrict security group rules
5. HTTPS: Configure SSL certificates for production

Customization

Using Custom Docker Compose

To use your own docker-compose.yml:

# In terraform.tfvars
compose_url = "https://raw.githubusercontent.com/your-org/config/main/docker-compose.yml"
env_url = "https://raw.githubusercontent.com/your-org/config/main/.env"

Instance Sizing

Adjust instance types in terraform.tfvars:

• AWS: instance_type = "t4g.large"
• Azure: vm_size = "Standard_B4ms"
• GCP: machine_type = "e2-standard-2"

Troubleshooting

Check Installation Logs

ssh ubuntu@<instance_ip>
sudo tail -f /var/log/user-data.log

Check RocketGraph Status

cd /opt/rocketgraph
docker compose ps
docker compose logs

Restart Services

cd /opt/rocketgraph
docker compose down
docker compose up -d

Clean Up

To destroy infrastructure:

terraform destroy

Contributing

1. Fork the repository
2. Create a feature branch
3. Commit your changes
4. Push to the branch
5. Create a Pull Request

License

MIT License - See LICENSE file for details

Support

• RocketGraph Documentation: https://docs.rocketgraph.io
• Terraform Documentation: https://www.terraform.io/docs
• Issues: Create an issue in this repository