We'll install a HA Origin cluster with Container Native Storage.
- Digital ocean account
- if you dont have one register here https://m.do.co/c/bc801d22c7b9
ssh-keygenon your machine and copy~/.ssh/id_rsa.pub
- When provisioning masters-lb make sure 80,443 are forwarded
- for certificate on https port, use
passthroughmechanism. - for Healthcheck using TCP 443.
- We will add master vm's later in the process.
- When provisioning infra-lb make sure 80,443 are forwarded
- for certificate on https port, use
passthroughmechanism. - We will add node vm's later in the process.
- Make sure to add your public key that is generated in step #2 as authentication mechanism to VM's
- Select private networking checkbox on
- Take 8GB Mem (lastest centos) size for all three VM's
- for each master add additional docker storage volume of size 30GB
- Add these 3 vm's to masters-lb
- Make sure to add your public key that is generated in step #2 as authentication mechanism to VM's
- Take 8GB Mem (lastest centos) size for all VM's
- Select private networking checkbox on
- for each infra vm add additional docker storage volume of size 30GB
- Add these two vm's to infra LB.
- Make sure to add your public key that is generated in step #2 as authentication mechanism to VM's
- Take 8GB Mem (lastest centos) size for all VM's
- for each compute vm add additional docker storage volume of size 50GB
- From now on we'll do all the acticity on bastion host. example master1
- First copy id_rsa from your laptop to master1
- It require to run everytime reboot master1
scp -r ~/.ssh/id_rsa root@<master-ip>:/tmp/id_rsa
ssh root@<master-ip>
run all the next steps on master1 (bastion)
ssh-agent $SHELL
ssh-add /tmp/id_rsa
export ANSIBLE_HOST_KEY_CHECKING=False
rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
yum update -y
yum install ansible -y
easy_install pip
pip install -U pyopenssl
- Use the inventory https://github.com/debianmaster/okd-hackathon.git as reference for your inventory file.
git clone https://github.com/debianmaster/okd-hackathon.git && cd okd-hackathon- remove /etc/ansible/hosts
- make a hard link of hosts file in this repo to /etc/ansible/hosts using
sudo ln hosts /etc/ansible/hosts - Update inventory file with your own ip values. Since we dont have a working dns we'll use nip.io appended to ip address.
All the steps from here on are on bastion host i.e master1 in our case
export ANSIBLE_HOST_KEY_CHECKING=False
ansible all -m ping
ansible all -a "rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm"
ansible all -a "yum install ansible -y"
ansible all -a "cat /etc/hosts"
ansible all -m lineinfile -a "path=/etc/hosts regexp={{inventory_hostname}} state=absent"
ansible all -m lineinfile -a "path=/etc/hosts regexp=master state=absent"
ansible all -m lineinfile -a "path=/etc/hosts regexp=infra state=absent"
ansible all -m lineinfile -a "path=/etc/hosts regexp=node state=absent"
This will make sure controller,api pods are able to reach out to etc without dns resolution issues. Otherwise controllers will try to make an attempt to connect to etcd on 127.0.0.1:2379 which is not accurate.
ansible all -a "yum install NetworkManager -y"
ansible all -a "systemctl restart NetworkManager"
ansible all -a "yum install dnsmasq -y"
ansible all -a "systemctl restart dnsmasq"
ansible all -a "getenforce"
ansible all -a "setenforce 1"
ansible all -a "date"
ansible all -m shell -a "timedatectl set-timezone UTC"
ansible all -a "sysctl -w vm.max_map_count=262144"
ansible all -a "ping -c 1 google.com"
ansible all -a "cp -f /etc/resolv.conf /etc/resolv.conf.upstream"Tip : DONOT run this unless required. to restore /etc/resolv.conf use following command
#ansible all -a "cp -f /etc/resolv.conf.upstream /etc/resolv.conf"
ansible all -a "systemctl status firewalld"
ansible all -a "systemctl stop firewalld"
ansible all -a "systemctl disable firewalld"
- did not required for this lab
ansible all -a "hostname"
ansible all -a "hostname -f "
For your digital ocean VM this might be different. since we dont have a working dns, and we need a full FQDN
We'll rest the hosntames with their nip.io equivalent dns names. so its consistent everywhere.
ansible all -a "hostnamectl set-hostname {{inventory_hostname}}"
you can skip 9.10.1 if your VM's hostname and hostname -f are both pointing to same FQDN.
9.11 Make sure VM's can reachout to themselves and responding with correct similar interface ip on all vm's
ansible all -a "ping -c 1 {{inventory_hostname}}"
9.12 Your VM's may have two network interfaces. openshift will use an interface which satisfies this condition
ansible all -a "ip -4 route get 8.8.8.8"
if you need to change this interface you need to use this command
#ansible all -a "route add -net 8.8.8.8 netmask 255.255.255.255 <interfacename>"
- I did changed to eth1 using above command.
change the interface name if necessary
ansible all -m lineinfile -a "path=/etc/sysconfig/network-scripts/ifcfg-eth1 line=NM_CONTROLLED='yes'"
ansible all -m lineinfile -a "path=/etc/sysconfig/network-scripts/ifcfg-eth1 line=PEERDNS='yes'"
ansible all -m sysctl -a "name=net.ipv4.ip_forward value=1 sysctl_set=yes state=present reload=yes"
ansible all -m sysctl -a "name=net.ipv6.conf.all.disable_ipv6 value=1 sysctl_set=yes state=present reload=yes"
ansible all -m sysctl -a "name=net.ipv6.conf.default.disable_ipv6 value=1 sysctl_set=yes state=present reload=yes"
ansible all -m shell -a 'yum install epel-release wget git net-tools bind-utils yum-utils iptables-services bridge-utils bash-completion kexec-tools sos psacct -y'
ansible masters -m shell -a "yum install httpd-tools -y"
ansible masters -m shell -a "yum install centos-release-openshift-origin311.noarch -y"Need this for most of the storage clients
ansible all -m shell -a "yum install samba-client samba-common cifs-utils iscsi-initiator-utils -y"
ansible all -a "systemctl restart iscsid"
ansible all -a "systemctl status iscsid"
ansible all -m shell -a "yum install docker -y"
ansible all -m shell -a "/usr/bin/needs-restarting -r"
#ansible all -a "systemctl reboot"
create file named origin.repo
[origin-repo]
name=Origin RPMs
baseurl=http://mirror.centos.org/centos/7/paas/x86_64/openshift-origin/
enabled=1
gpgcheck=0
ansible all -m copy -a "src=origin.repo dest=/etc/yum.repos.d/origin.repo"
git clone https://github.com/openshift/openshift-ansible.git ~/openshift-ansible
cd openshift-ansible && git checkout release-3.11
ansible-playbook ~/openshift-ansible/playbooks/byo/openshift_facts.yml
ansible-playbook ~/openshift-ansible/playbooks/prerequisites.yml
ansible-playbook ~/openshift-ansible/playbooks/deploy_cluster.yml