Deploying Kolla-Ansible release Train with Ext Ceph

TAGS: linux ceph ansible openstack kolla kolla-ansible

Intro

Mission: Deploy Kolla-ansible release Train with 1 controller, 2 computes and external Ceph (business purpose - test bug with live-migration in Train);

Prerequisites

  • Working Ceph (3 nodes) within same network and root access;
  • 3 VM for Openstack env (for test, 8 cpu/12ram/100GB local + 3 network interfaces(mgmt/storage/external));
  • Deployment VM Ubuntu 18 (1 cpu/1ram/30GB);

Deployment

Prepare deployment host

SSH to deployment VM and begin:
cd /opt git clone https://github.com/openstack/kolla-ansible.git cd kolla-ansible git checkout stable/train apt update; apt -y install virtualenv virtualenv /opt/venv source /opt/venv/bin/activate pip install -r requirements.txt pip install ./ #install kolla-ansible package mkdir -p /etc/kolla chown $USER:$USER /etc/kolla cp -r etc/kolla/* /etc/kolla cp ansible/inventory/* .

Prepare target hosts

  1. Configure all network interfaces (for data plane and control)
  2. Install python-minimal
  3. Configure passwordless access from deploymen VM

Configuration of interfaces

cat /etc/netplan/01-netcfg.yaml
network: version: 2 renderer: networkd ethernets: ens160: addresses: [ 10.220.104.53/24 ] gateway4: 10.220.104.254 nameservers: addresses: - "8.8.8.8" ens192: addresses: [ 10.220.103.53/24 ] ens256: addresses: [ 10.220.102.53/24 ]
netplan apply

Install python-minimal

apt update; apt -y install python-minimal

Configure passwordless access

deployhost:
ssh-keygen; cat /root/.ssh/id_rsa.pub (copy in buffer)
target1-3:
echo $(key) >> /root/.ssh/authorized_keys

Don’t forget to enable hardware virtualization

On compute nodes verify:
grep -r vmx /proc/cpuinfo

Also, check that nova user exists in Ceph, or create it

ceph auth list; ceph auth add client.nova ceph auth caps client.nova mon 'profile rbd' osd 'profile rbd pool=images, profile rbd pool=vms, profile rbd pool=volumes, profile rbd pool=backups'

Create files for ceph integration in /etc/kolla/config (without cinder-backup)

mkdir -p /etc/kolla/config/{nova,cinder,glance}
tree /etc/kolla/config/
. ├── cinder │   ├── ceph.conf │   ├── cinder-volume │   │   └── ceph.client.cinder.keyring │   └── cinder-volume.conf ├── glance │   ├── ceph.client.glance.keyring │   ├── ceph.conf │   └── glance-api.conf └── nova ├── ceph.client.cinder.keyring ├── ceph.client.nova.keyring ├── ceph.conf └── nova-compute.conf
paste there keyrings, ceph.conf from ceph and other stuff, more info here - https://docs.openstack.org/kolla-ansible/queens/reference/external-ceph-guide.html

Generate passwords

kolla-genpwd
Optionally, change keystone_admin_password there to something shorter for convenience;

Configure variables

vim /etc/kolla/globals.yml
/etc/kolla/globals.yml
vim /opt/kolla-ansible/multinode /opt/kolla-ansible/multinode

Other checks

In inventory file (multinode, for example), your nodes also should be placed under [baremetal:children] section (bootstrap).

Bootstrap

kolla-ansible -i multinode bootstrap-servers

Deploy

kolla-ansible deploy -i multinode

Post-Deploy

kolla-ansible post-deploy

Check basic stuff

If you installed kolla in venv, u also need openstack clients.
apt install -y build-essential python-dev source /opt/venv/bin/activate pip install openstackclient
Now, check basic statuses:
cinder service-list; nova service-list; neutron agent-list

Other stuff

Customize roles

If you are using venv as I do then path is:
/opt/venv/share/kolla-ansible/ansible/roles

Mirror images in registry

for img in $(docker images | awk '{print $1":"$2}' | grep -v REPOSITORY | tr '\n' ' '); do docker tag $img harbor.cognoz/$img; docker push harbor.cognoz/$img; done

Destroing Environment

cd /opt/kolla-ansible source /etc/kolla/admin-openrc.sh source /opt/venv/bin/activate nova list --all-tenants #first, you need to delete all instances kolla-ansible -i multinode destroy --yes-i-really-really-mean-it

https://docs.openstack.org/kolla-ansible/latest/user/operating-kolla.html

Written on December 9, 2019

Archive

linux openstack kubernetes gluster ceph ansible docker systemctl python openshift log centos sed registry kolla kibana keepalived elasticsearch deploy cloud-init auth HA zabbix vsphere vmware venv tools swift ssl ssh scm ruby rsyslog rhel rbac rabbitmq prometheus postgres policy pgpool2 patrole pacemaker ntp nfs net mq monitoring mongo mbr lvm logstash kubespray kubeadm kolla-ansible jenkins jekyII java ironic helm heketi harbor grub github ganesha fluentd fluent-bit fio drbd diskimage-builder devstack debug deb db cisco cicd ceph-ansible calico build bitbucket benchmarking apparmor api NUMA Linux Golang Go ELK 3par