Skip to content
This repository has been archived by the owner on Jul 27, 2023. It is now read-only.

Commit

Permalink
Merge branch 'master' of https://github.com/redhat-sap/sap-netweaver-…
Browse files Browse the repository at this point in the history 
…ha-pacemaker into test
  • Loading branch information
@rickgcv
rickgcv committed Oct 29, 2021
2 parents 952051b + 0b9c8d8 commit 7cf5c31
Show file tree
Hide file tree
Showing 9 changed files with 393 additions and 25 deletions.
1 change: 1 addition & 0 deletions .gitignore
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.cache/
96 changes: 94 additions & 2 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,96 @@
# This is a **WIP** role

# sap-netweaver-ha-pacemaker
HA configuration of pacemaker for SAP Netweaver software

### Role variables

| variable | info | required? |
|:----------------------------------------------------------:|:-----------------------------------------------------------------------------------:|:-----------------------:|
| sap_netweaver_ha_pacemaker_hacluster_manage_shared_storage | Attempt to manage shared (eg. nfs) storage? | no, default is yes |
| sap_netweaver_ha_pacemaker_hacluster_node1_ip | IP address of first node in cluster | yes |
| sap_netweaver_ha_pacemaker_hacluster_node1_fqdn | FWDN of first node in cluster | yes |
| sap_netweaver_ha_pacemaker_hacluster_node2_ip | IP address of second node in cluster | yes |
| sap_netweaver_ha_pacemaker_hacluster_node2_fqdn | FWDN of second node in cluster | yes |
| sap_netweaver_ha_pacemaker_ensa_ver | ENSA/ERS version | no, default 2 |
| sap_netweaver_ha_pacemaker_hacluster_manage_azure_lb | Default: no. Deal with Azure load balancer? | no |
| sap_netweaver_ha_pacemaker_sid | SID of this instance | yes |
| sap_netweaver_ha_pacemaker_profile_path | Full path of directory holding profiles | no, is generated sanely |
| sap_netweaver_ha_pacemaker_alias_ascs | ASCS host alias | yes |
| sap_netweaver_ha_pacemaker_instance_number_ascs | ASCS instance number | yes |
| sap_netweaver_ha_pacemaker_instance_name_ascs | ASCS Instance Name (profile file name) | no, is generated sanely |
| sap_netweaver_ha_pacemaker_profile_file_ascs | Full path and name of ASCS profile | no, is generated sanely |
| sap_netweaver_ha_pacemaker_profile_path_ascs | Full path to directory holding ASCS profile | no, is generated sanely |
| sap_netweaver_ha_pacemaker_vip_ascs | Virtual (cluster managed) IP address for ASCS | yes |
| sap_netweaver_ha_pacemaker_alias_ers | ERS host alias | yes |
| sap_netweaver_ha_pacemaker_instance_number_ers | ERS instance number | yes |
| sap_netweaver_ha_pacemaker_instance_name_ers | ASCS Instance Name (profile file name) | no, is generated sanely |
| sap_netweaver_ha_pacemaker_profile_file_ers | Full path and file name of ASCS profile | no, is generated sanely |
| sap_netweaver_ha_pacemaker_profile_path_ers | Full path to directory holding ASCS profile | no, is generated sanely |
| sap_netweaver_ha_pacemaker_vip_ers | Virtual (cluster managed) IP address for ASCS | yes |
| sap_netweaver_ha_pacemaker_e4s_repos_ppc64le | List of required extended support repos for ppc64le arch | no |
| sap_netweaver_ha_pacemaker_e4s_repos_x86_64 | List of required extended support repos for x86_64 arch | no |
| sap_netweaver_ha_pacemaker_standard_repos_ppc64le | List of required standard support repos for ppc64le arch | no |
| sap_netweaver_ha_pacemaker_standard_repos_x86_64 | List of required standard support repos for x86_64 arch | no |
| sap_netweaver_ha_pacemaker_fsdevice_ascs | ascs profile filesystem resource | yes |
| sap_netweaver_ha_pacemaker_fsdevice_ers | ers profile filesystem resource | yes |
| sap_netweaver_ha_pacemaker_fsdevice_sapmnt | sapmnt filesystem resource | yes |
| sap_netweaver_ha_pacemaker_fsdevice_sys | sys filesystem resource | yes |
| sap_netweaver_ha_pacemaker_fsdevice_trans | trans filesystem resource | yes |
| sap_netweaver_ha_pacemaker_mntdir_ascs | ASCS filesystem mount point directory | yes |
| sap_netweaver_ha_pacemaker_mntdir_ers | ERS filesystem mount point directory | yes |
| sap_netweaver_ha_pacemaker_mntdir_sapmnt | sapmnt filesystem mount point directory | yes |
| sap_netweaver_ha_pacemaker_mntdir_sys | sys filesystem mount point directory | yes |
| sap_netweaver_ha_pacemaker_mntdir_trans | trans filesystem mount point directory | yes |
| sap_netweaver_ha_pacemaker_fstype | Type of filesystem resource of managed filesystems see: pcs fstype=xxx | yes |
| sap_netweaver_ha_pacemaker_fstype_<thing> | Type of filesystem resource for <thing> (inherited value from above) | no |
| sap_netweaver_ha_pacemaker_fsr_name_<thing> | PCS resource name for managed filesystem of <thing> | no |
| sap_netweaver_ha_pacemaker_auto_recover_ascs | Sets SAPInstance AUTOMATIC_RECOVER=true if ansible truthy (otherwise skips setting) | no |
| sap_netweaver_ha_pacemaker_auto_recover_ers | Sets SAPInstance AUTOMATIC_RECOVER=true if ansible truthy (otherwise skips setting) | no |
| sap_netweaver_ha_pacemaker_ocf_check | Default for OCF_CHECK_LEVEL= value for mount points | no |
| sap_netweaver_ha_pacemaker_ocf_check_<thing> | OCF_CHECK_LEVEL= value for <thing> mount points (inherited value from above) | no |
| sap_netweaver_ha_pacemaker_startup_delay | Delay in integer seconds for systemd PCS startup | no |
| sap_netweaver_ha_pacemaker_save_backups | Save backups of remote files that we change | no|
#### Paths, Files, Filesystems
`sap_netweaver_ha_pacemaker_profile_path` is the root directory holding the "profiles" of ASCS and ERS. It is generated based on convention and the provided SID. `sap_netweaver_ha_pacemaker_profile_path_ascs` and `sap_netweaver_ha_pacemaker_profile_path_ers` are set to it (or overridden), and the respective `profile_file` values are generated based on the particular `_alias_` names and IDs (or overridden).

For each of thing={'ascs', 'ers', 'sapmnt', 'sys', 'trans'}, we create a PCS filesystem resource named `sap_netweaver_ha_pacemaker_fsr_name_<thing>`, which is set sanely, but can be overridden. It mounts `sap_netweaver_ha_pacemaker_fsdevice_<thing>` at `sap_netweaver_ha_pacemaker_mntdir_<type>` as a filesystem type of `sap_netweaver_ha_pacemaker_fstype_<thing>`. `_mntdir` is set by convention (but can be overridden), the `device` need to be specified. `fstype` can be specified generally, or per `fstype_<thing>`.

### STONITH fencing
The following variables are common to all fencing devices (except those on hyperscalers)

| variable | info | required? |
|:--------------------------------------------------------:|:----------------------------------------------------------------------------------------------:|:---------:|
| sap_netweaver_ha_pacemaker_fencing_device.name | Name of the fencing device | yes |
| sap_netweaver_ha_pacemaker_fencing_device.type | Fencing device type | yes * |
| sap_netweaver_ha_pacemaker_fencing_device.ip | IP address of the fencing device | yes |
| sap_netweaver_ha_pacemaker_fencing_device.user | Username to connect to the fencing device | yes |
| sap_netweaver_ha_pacemaker_fencing_device.pwd | Password to connect to the fencing device | yes |
| sap_netweaver_ha_pacemaker_fencing_device.pcmk_host_list | List of nodes controlled by the fencing device | yes * |
| sap_netweaver_ha_pacemaker_fencing_device.password_file | Full path/filename to 'password script' to use for fencing device configuration | no |
| sap_netweaver_ha_pacemaker_fencing_device.pcmk_host_map | Mapping of the hostnames/ip of the nodes | yes* |
| sap_netweaver_ha_pacemaker_fencing_device.custom_options | Additional options to pass to fence device creation, e.g. "ssl=1 ssl_insecure=1 power_wait=30" | no |

\* Must provide either a pcmk_host_list or a pcmk_host_map, per https://access.redhat.com/solutions/2619961 for a discussion on host_list vs host_map

This is a list of the most common fencing devices:
- fence_vmware_soap - VMWare
- fence_vmware_rest - VMWare
- fence_ipmilan - IPMI
- fence_ilo4_ssh - HP ILO
- fence_rhevm - RHV
- fence_azure_arm - Azure


## TODO and Notes
* Fencing should perhaps be extracted to own role to share with redhat_sap.sap-hana-ha-pacemaker
* Verify drivers other than fence_vmware_rest support passwd_script option
* Support >2 nodes
* Support >2 links
* This remains mostly untested against a range of systems.
* Var docs
* FS paths should match samples from docs
* Is this the right place for Azure LB? (commented out as wasn't going to work, anyway)
* If using clustering, the sap_s4netweaver_deployment is likely going to need /etc/hosts setup, so this is a dupe
* /etc/hosts perhaps should have ansible tags in it, either way, so the several roles can not clobber each other
* Should instance aliases be in /etc/hosts?
* Rename 'ASCS' to 'APP'(?) everywhere if this "works" with more app stacks than ASCS, like the java thing
*
26 changes: 16 additions & 10 deletions defaults/main.yml
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,22 @@
sap_hana_ha_pacemaker_secondary_read: false
sap_hana_ha_pacemaker_configure_firewall: false
sap_hana_ha_pacemaker_stickiness: false
sap_hana_ha_pacemaker_threshold: false
sap_netweaver_ha_pacemaker_secondary_read: false
sap_netweaver_ha_pacemaker_configure_firewall: false
sap_netweaver_ha_pacemaker_stickiness: false
sap_netweaver_ha_pacemaker_threshold: false

sap_hana_ha_pacemaker_use_e4s: true
sap_netweaver_ha_pacemaker_use_e4s: true

sap_hana_ha_pacemaker_standard_repos_x86_64: rhel-8-for-x86_64-highavailability-rpms
sap_hana_ha_pacemaker_e4s_repos_x86_64: rhel-8-for-x86_64-highavailability-e4s-rpms
sap_hana_ha_pacemaker_standard_repos_ppc64le: rhel-8-for-ppc64le-highavailability-rpms
sap_hana_ha_pacemaker_e4s_repos_ppc64le: rhel-8-for-ppc64le-highavailability-e4s-rpm
sap_netweaver_ha_pacemaker_e4s_repos_x86_64: rhel-8-for-x86_64-highavailability-e4s-rpms
sap_netweaver_ha_pacemaker_standard_repos_x86_64: rhel-8-for-x86_64-highavailability-rpms
sap_netweaver_ha_pacemaker_e4s_repos_ppc64le: rhel-8-for-ppc64le-highavailability-e4s-rpm
sap_netweaver_ha_pacemaker_standard_repos_ppc64le: rhel-8-for-ppc64le-highavailability-rpms

sap_hana_ha_pacemaker_cluster_name: "hana_cluster"
sap_netweaver_ha_pacemaker_hacluster_cluster_name: "ascs_cluster"

sap_netweaver_ha_pacemaker_ensa_ver: 2
sap_netweaver_ha_pacemaker_ascs_key: 'ASCS' #sometimes ASCS isn't actually ASCS

sap_netweaver_ha_pacemaker_hacluster_manage_shared_storage: true
sap_netweaver_ha_pacemaker_hacluster_manage_azure_lb: false
sap_netweaver_ha_pacemaker_startup_delay: 0 #integer seconds, passed to `sleep`
sap_netweaver_ha_pacemaker_save_backups: yes

52 changes: 45 additions & 7 deletions tasks/cluster-config.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,25 +23,29 @@
path: /etc/hosts
line: "{{ item }}"
loop:
- "{{ sap_netweaver_ha_pacemaker_node1_ip }} {{ sap_netweaver_ha_pacemaker_node1_fqdn }}"
- "{{ sap_netweaver_ha_pacemaker_node2_ip }} {{ sap_netweaver_ha_pacemaker_node2_fqdn }}"
- "{{ sap_netweaver_ha_pacemaker_hacluster_node1_ip }} {{ sap_netweaver_ha_pacemaker_hacluster_node1_fqdn }}"
- "{{ sap_netweaver_ha_pacemaker_hacluster_node2_ip }} {{ sap_netweaver_ha_pacemaker_hacluster_node2_fqdn }}"

- name: Authenticate the cluster Nodes
command: |
pcs host auth \
pcs {{ _subject }} auth {{ _auth }} \
{{ sap_netweaver_ha_pacemaker_hacluster_node1_fqdn }} {{ sap_netweaver_ha_pacemaker_hacluster_node2_fqdn }} \
-u hacluster -p {{ sap_netweaver_ha_pacemaker_hacluster_password }}
register: auth_cluster
changed_when: "'Authorized' in auth_cluster.stdout"
vars:
_subject: "{{ 'cluster' if ansible_facts['distribution_major_version'] | int == 7 else 'host' }}"
_auth: "{{ '--auth' if ansible_facts['distribution_major_version'] | int == 7 else '' }}"

- name: Create RHEL HA Cluster
command: |
pcs cluster setup \
{{ sap_netweaver_ha_pacemaker_hacluster_cluster_name }} \
pcs cluster setup {{ _name }} {{ sap_netweaver_ha_pacemaker_hacluster_cluster_name }} \
{{ sap_netweaver_ha_pacemaker_hacluster_node1_fqdn }} \
{{ sap_netweaver_ha_pacemaker_hacluster_node2_fqdn }} \
{{ sap_netweaver_ha_pacemaker_hacluster_node2_fqdn }} --force --wait=6000 --start
register: create_cluster
changed_when: "'Cluster has been successfully set up' in create_cluster.stdout"
vars:
_name: "{{ '--name' if ansible_facts['distribution_major_version'] | int == 7 else '' }}"
run_once: true

- name: Enable the RHEL HA Cluster
Expand All @@ -51,7 +55,41 @@
run_once: true

- name: Start the RHEL HA Cluster
command: pcs cluster start --all
command: pcs cluster start --all --wait=600 --request-timeout=600
register: start_cluster
changed_when: "'Starting Cluster' in start_cluster.stdout"
run_once: true

- name: Set quorum votes
command: pcs quorum expected-votes 2
run_once: true

- name: Configure second link
command: pcs cluster link add {{ sap_netweaver_ha_pacemaker_hacluster_node1_fqdn }}={{ sap_netweaver_ha_pacemaker_node1_link2_ip }} {{ sap_netweaver_ha_pacemaker_hacluster_node2_fqdn }}={{ sap_netweaver_ha_pacemaker_node2_link2_ip }} options linknumber=2
run_once: true
when: sap_netweaver_ha_pacemaker_node1_link2_ip is defined and sap_netweaver_ha_pacemaker_node2_link2_ip is defined


# For fenced nodes, they can come up too quickly, and like a zombie, get double tapped by pacemaker too fast.
# Note: We enable this *after* the initial cluster setup/start, so we don't need to to have ansible know how to
# wait on the initial, one time, startup.
- name: Ensure /etc/systemd/system/corosync.service.d directory exists
file:
path: /etc/systemd/system/corosync.service.d
owner: root
group: root
state: directory
mode: '0755'
when: sap_netweaver_ha_pacemaker_startup_delay > 0

- name: Setup /etc/systemd/system/corosync.service.d/override.conf
blockinfile:
owner: root
group: root
mode: '0644'
create: yes
path: "/etc/systemd/system/corosync.service.d/override.conf"
block: |
[Service]
ExecStartPre=/bin/sleep {{ sap_netweaver_ha_pacemaker_startup_delay }}
when: sap_netweaver_ha_pacemaker_startup_delay > 0
Loading

0 comments on commit 7cf5c31

Please sign in to comment.