Home

QE Tech Talks CI/CD Workshop

Pre-pre requisites:

For this workshop you will need:

• Github account
• Rackspace account
• Machine with administrative access (or a VM)

Installation

Quick note: you may not have administrator access to install these tools. If you aren't able to, you will have to spin up a cloud virtual machine (can be a Windows vm) and install everything there:

1. Install VirtualEnv (https://virtualenv.pypa.io/en/latest/installation.html)

Ansible

We will use Ansible to both spin up our CI/CD environment as well as spin up our target infrastructure. To install ansible:

1. Set up your virtual environment

virtualenv .
. bin/activate

2. Install ansible

git clone https://github.com/dimtruck/workshop_prereq workshop_prereq
cd workshop_prereq
pip install -r ansible-requirements.txt

3. Create .raxpub file in the ansible directory with your credentials in the following format:

[rackspace_cloud]
username = my_username
api_key = 01234567890abcdef 

4. Update ansible/inventory/cloudserver/group_vars/all file creds_path key to .raxpub file's path.

Set up ssh key

1. Create ssh key pair for this workshop (please don't forget to remove it after done or secure your workshop environment!)

cd ansible && ssh-keygen -t rsa -N "" -f workshop.key && cd ..

Set up Gerrit with your public key

1. Navigate to http://23.253.229.56/ and create a new user
2. Click on Register
3. In OpenID input field, type in https://login.launchpad.net (alternatively, you can also use your Yahoo login).
4. Go through the steps to tie your launchpad id (or yahoo! id) with gerrit.
5. Register your user name and copy your public ssh key you generated earlier.

Repositories

Fork the following repositories:

• https://github.com/Rax-io-CI-CD/workshop_demo
• https://github.com/dimtruck/workshop_deployment
• https://github.com/dimtruck/workshop_build
• https://github.com/dimtruck/workshop_autocreator

Install Jenkins environment

1. Spin up the environment. Note that you'll need to edit creds_path in the following command. Also, you will need to put in your github user for github_user variable, put in your gerrit user name for gerrit_user variable, and put in your gerrit email address for gerrit_email variable.

export RAX_CREDS_FILE=/path/to/your/ansible/.raxpub
export GITHUB_USERNAME=<YOUR GITHUB USER>
export GERRIT_USERNAME=<YOUR GERRIT USER>
export GERRIT_EMAIL=<YOUR GERRIT EMAIL>


export ANSIBLE_FORCE_COLOR=true
export ANSIBLE_HOST_KEY_CHECKING=false
export ANSIBLE_SSH_ARGS='\
-o UserKnownHostsFile=/dev/null \
-o ControlMaster=auto \
-o ControlPersist=60s \
-o PubkeyAuthentication=yes'


ansible-playbook ansible/cleanup_cloud.yml \
--inventory ansible/inventory/cloudserver \
--extra-vars "creds_path=/path/to/your/ansible/.raxpub" \
 && ansible-playbook ansible/cloudserver.yml \
--inventory ansible/inventory/cloudserver \
--extra-vars "creds_path=$RAX_CREDS_FILE github_user=$GITHUB_USERNAME gerrit_user=$GERRIT_USERNAME gerrit_email=$GERRIT_EMAIL"

Your output should be something like this:

    "msg": "ssh root@23.253.226.217 with password abcdefg"
}

TASK: [jenkins_output | show Jenkins] ***************************************** 
ok: [127.0.0.1] => {
    "var": {
        "jenkins_load_balancer.balancer.virtual_ips[0].address": "104.239.245.95"
    }
}

TASK: [jenkins_output | show Sonar] ******************************************* 
ok: [127.0.0.1] => {
    "var": {
        "sonar_load_balancer.balancer.virtual_ips[0].address": "104.130.254.66"
    }
}

TASK: [jenkins_output | show Gerrit] ****************************************** 
ok: [127.0.0.1] => {
    "var": {
        "gerrit_load_balancer.balancer.virtual_ips[0].address": "104.130.255.32"
    }
}

PLAY RECAP ******************************************************************** 
127.0.0.1                  : ok=15   changed=7    unreachable=0    failed=0   
jenkins1                   : ok=40   changed=36   unreachable=0    failed=0   

Troubleshooting

If you got the following error, you'll need to install sshpass on your computer:

fatal: [jenkins1] => to use the 'ssh' connection type with passwords, you must install the sshpass program

cd ~/Downloads
curl -O -L http://downloads.sourceforge.net/project/sshpass/sshpass/1.05/sshpass-1.05.tar.gz
tar xvzf sshpass-1.05.tar.gz
cd sshpass-1.05
./configure
make
sudo make install

If you get the following error, it means that docker registry cannot be reached. This happens frequently.

Error pulling image (latest) from larrycai/gerrit, Get https://registry-1.docker.io/v1/images/0baf5c828bac0fb6284fd43bcb3be1626eb399491aa7801bb4618739cc3923ce/ancestry: dial tcp: lookup registry-1.docker.io on 72.3.128.241:53: no such host

Just rerun the ansible-playbook command again.

Gerrit and Jenkins Setup

Install git-review

1. Navigate to http://docs.openstack.org/infra/manual/developers.html#installing-git-review

Update "OpenStack" deploy key

1. Add your ssh public key to your workshop-demo forked repository. This is under Settings -> Deploy Keys on the right side of your repository. Make sure you check the checkbox to allow the write access
2. Add your JENKINS IP/ghprbhook/ to Webhooks with set individual events to Pull Requests and Issue Comments

Update Jenkins github pull request builder in Jenkins

Navigate to Manage Jenkins -> Configure System

Under "GitHub Pull Request Builder" click on "Advanced" button and enter your jenkins IP address in "Published Jenkins URL" text input.

Jenkins gerrit plugin

Sometimes Gerrit Trigger just doesn't get set up properly. You can check if everything's ok by navigating to Manage Jenkins -> Gerrit Trigger and validating that you see a green ball next to Openstack. If you see a red ball, you'll need to restart Jenkins. To do that, simply log into the Jenkins server and as root run service jenkins restart. Once Jenkins is up, navigate back to Gerrit Trigger configuration again and validate that the ball is now green.

Deployment

To successfully deploy a new release, you'll need to export your GITHUB_TOKEN in deployment job (staging_release_deployer). Note that this is going to be in clear text! You should secure your environment by removing anonymous access.

export GITHUB_TOKEN=my_token

Clean Up

When you want to delete the created servers and load balancers after the workshop, you can remove them manually through the Cloud Control Panel or by running the following command:

export RAX_CREDS_FILE=/path/to/your/ansible/.raxpub

export ANSIBLE_FORCE_COLOR=true
export ANSIBLE_HOST_KEY_CHECKING=false
export ANSIBLE_SSH_ARGS='\
-o UserKnownHostsFile=/dev/null \
-o ControlMaster=auto \
-o ControlPersist=60s \
-o PubkeyAuthentication=yes'

ansible-playbook ansible/cleanup_cloud.yml --inventory ansible/inventory/cloudserver --extra-vars "creds_path=/path/to/your/ansible/.raxpub"