​This article will help you to setup High Availability NFS server using Pacemaker on Redhat Enterprise Linux 7. From the scratch ,we will build the pacemaker blocks which includes package installation , configuring the HA resources, fencing etc. NFS shares are used for setting up the home directories and sharing the same content across multiple servers. NFS HA will suit for customers who can’t afford NAS storage. You might have followed Pacemaker articles on UnixArena where we have setup Failover KVM VM and GFS earlier. If not , please go through it to understand the various component of pacemaker and how it works. This article is not going to cover in-depth.

 NFS HA - Pacemaker UnixArena

NFS HA – Pacemaker 

Assumptions:

  • Two servers installed with RHEL 7.x (Hosts- UA-HA1 / UA-HA2)
  • Access to Redhat Repository or Local Repository to install packages.
  • SELINUX & Firewalld can be turned off.

 

1. Login to the each node as root user and install the package .

# yum install pcs fence-agents-all

 

2. Disable SELINUX on both the nodes.

# setenforce 0
setenforce: SELinux is disabled
# cat /etc/selinux/config |grep SELINUX |grep -v "#"
SELINUX=disabled
SELINUXTYPE=targeted

 

3. Disable firewalld on both the hosts

UA-HA# systemctl stop firewalld.service
UA-HA# systemctl disable firewalld.service
UA-HA# iptables --flush
UA-HA#

 

4. Enable and Start the Services on both the Nodes.

# systemctl start pcsd.service
# systemctl enable pcsd.service
# systemctl status pcsd.service

 

5. On each nodes, set the password for hauser.

# passwd hacluster
Changing password for user hacluster.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.

 

6.Login to any of the cluster node and authenticate “hacluster” user.

# pcs cluster auth UA-HA1 UA-HA2

 

7. Create a new cluster using pcs command. The cluster name is “UACLS”.

# pcs cluster setup --name UACLS UA-HA1 UA-HA2

 

8. Start the cluster using pcs command. “–all” will start the cluster on all the configured nodes.

# pcs cluster start –all

 

9. Check the corosync communication status.This command output should show that which IP has been used for heartbeat. Refer configure-redundant-corosync
# corosync-cfgtool –s

 

10. Disable STONITH to avoid issues while configuring the resources. Once we complete the cluster setup, we will enable the fencing back.

#pcs property set stonith-enabled=false
#pcs property show stonith-enabled

 

10. Configure the fencing (STONITH) using ipmilan.

# pcs stonith create UA-HA1_fen fence_ipmilan pcmk_host_list="UA-HA1" ipaddr=192.168.10.24 login=root  passwd=test123 lanplus=1 cipher=1 op monitor interval=60s
#pcs stonith create UA-HA2_fen fence_ipmilan pcmk_host_list="UA-HA2" ipaddr=192.168.10.25 login=root  passwd=test123 lanplus=1 cipher=1 op monitor interval=60s

These IP’s are IDRAC console IP used for fencing.

 

11. Verify the cluster configuration.

# crm_verify -L –V

 

12. Configure the volume group and Logical volume.

# vgcreate UAVG1 /dev/disk_name1
# vgcreate UAVG2 /dev/disk_#name2

# lvcreate -L sizeM -n /dev/UAVG1/UAVOL1
# lvcreate -L sizeM -n /dev/UAVG2/UAVOL2

 

12. Create a filesystem. (Let’s go with XFS)

#mkfs.xfs /dev/UAVG1/UAVOL1
#mkfs.xfs /dev/UAVG2/UAVOL2

 

13. Modify LVM configuration similar to below. Assuming that all the volume groups are used in cluster. If you have root vg , you need to specify in lvm.conf for automatic import.

# grep use_lvmetad /etc/lvm/lvm.conf |grep -v "#"
use_lvmetad = 0

 

14.Configure symmetric cluster property and check the status.

# pcs property set symmetric-cluster=true
[[email protected] tmp]# pcs status
Cluster name: UACLS
Stack: corosync
Current DC: UA-HA2 (2) - partition with quorum
2 Nodes configured
2 Resources configured

Online: [ UA-HA1 UA-HA2 ]

Full list of resources:

 UA-HA1_fen   (stonith:fence_ipmilan):        Started UA-HA2
 UA-HA2_fen   (stonith:fence_ipmilan):        Started UA-HA1

PCSD Status:
  UA-HA1: Online
  UA-HA2: Online

Daemon Status:
  corosync: active/enabled
  pacemaker: active/enabled
  pcsd: active/enabled
[[email protected] tmp]# pcs stonith show
 UA-HA1_fen   (stonith:fence_ipmilan):        Started
 UA-HA2_fen   (stonith:fence_ipmilan):        Started
[[email protected] tmp]#

 

15.Configure VG & Mount resources .

#pcs resource create UAVG1_res LVM volgrpname="UAVG1" exclusive=true  --group UANFSHA
#pcs resource create UAVOL1_res Filesystem  device="/dev/UAVG1/UAVOL1" directory="/cm/shared" fstype="xfs" --group UANFSHA


#pcs resource create UAVG2_res LVM volgrpname="UAVG2" exclusive=true --group  UANFSHA
#pcs resource create UAVOL2_res Filesystem  device="/dev/UAVG2/UAVOL2" directory="/global/home" fstype="xfs" --group UANFSHA

 

16.Configure VIP for NFS share. This IP will be used on NFS client to mount the shares.

# pcs resource create ClusterIP ocf:heartbeat:IPaddr2 ip=192.168.2.90  nic="eth0" cidr_netmask=24 op monitor interval=30s --group UANFSHA

 

17. Configure NFS server resources.

[[email protected] ~]# pcs resource create NFS-D nfsserver nfs_shared_infodir=/global/nfsinfo nfs_ip=192.168.2.90  --group UANFSHA

 

18.Check the cluster status.

[[email protected] ~]# pcs status
Cluster name: UACLS
Last updated: Tue Aug 16 12:39:22 2016
Last change: Tue Aug 16 12:39:19 2016 via cibadmin on UA-HA1
Stack: corosync
Current DC: UA-HA1 (1) - partition with quorum
Version: 1.1.10-29.el7-368c726
2 Nodes configured
8 Resources configured

Online: [ UA-HA1 UA-HA2 ]

Full list of resources:

 UA-HA1_fen   (stonith:fence_ipmilan):        Started UA-HA1
 UA-HA2_fen   (stonith:fence_ipmilan):        Started UA-HA1
 Resource Group: UANFSHA
     UAVG1_res  (ocf::heartbeat:LVM):   Started UA-HA1
     UAVG2_res  (ocf::heartbeat:LVM):   Started UA-HA1
     UAVOL1_res  (ocf::heartbeat:Filesystem):    Started UA-HA1
     UAVOL2_res  (ocf::heartbeat:Filesystem):    Started UA-HA1
     ClusterIP  (ocf::heartbeat:IPaddr2):       Started UA-HA1
     NFS-D      (ocf::heartbeat:nfsserver):     Started UA-HA1

 

19. Configure the HA NFS shares.

[[email protected] ~]# pcs resource create nfs-cm-shared exportfs clientspec=192.168.2.0/255.255.255.0 options=rw,sync,no_root_squash directory=/SAP_SOFT fsid=0 --group UANFSHA
[[email protected] ~]# pcs resource create nfs-global-home  exportfs clientspec=10.248.102.0/255.255.255.0 options=rw,sync,no_root_squash directory=/users1/home fsid=1 --group UANFSHA

 

20.Final cluster status will looks like similar to the following.

[[email protected] ~]# pcs status
Cluster name: UACLS
Last updated: Tue Aug 16 12:52:43 2016
Last change: Tue Aug 16 12:51:56 2016 via cibadmin on UA-HA1
Stack: corosync
Current DC: UA-HA1 (1) - partition with quorum
Version: 1.1.10-29.el7-368c726
2 Nodes configured
10 Resources configured


Online: [ UA-HA1 UA-HA2 ]

Full list of resources:

 UA-HA1_fen   (stonith:fence_ipmilan):        Started UA-HA1
 UA-HA2_fen   (stonith:fence_ipmilan):        Started UA-HA1
 Resource Group: UANFSHA
     UAVG1_res  (ocf::heartbeat:LVM):   Started UA-HA1
     UAVG2_res  (ocf::heartbeat:LVM):   Started UA-HA1
     UAVOL1_res  (ocf::heartbeat:Filesystem):    Started UA-HA1
     UAVOL2_res  (ocf::heartbeat:Filesystem):    Started UA-HA1
     ClusterIP  (ocf::heartbeat:IPaddr2):       Started UA-HA1
     NFS-D      (ocf::heartbeat:nfsserver):     Started UA-HA1
     nfs-cm-shared      (ocf::heartbeat:exportfs):      Started UA-HA1
     nfs-global-home    (ocf::heartbeat:exportfs):      Started UA-HA1

 

21. Configure resource dependencies.

[[email protected] ~]# pcs constraint order start UAVG1_res then UAVOL1_res
[[email protected] ~]# pcs constraint order start UAVG2_res then UAVOL2_res
[[email protected] ~]# pcs constraint order start UAVOL1_res then ClusterIP
[[email protected] ~]# pcs constraint order start UAVOL2_res then ClusterIP 
[[email protected] ~]# pcs constraint order start ClusterIP then NFS-D
[[email protected] ~]# pcs  constraint order start NFS-D then nfs-cm-shared
[[email protected] ~]# pcs  constraint order start NFS-D then nfs-global-home
[[email protected] ~]# pcs constraint
Location Constraints:
  Resource: UANFSHA
    Enabled on: UA-HA1 (role: Started)
Ordering Constraints:
  start UAVG1_res then start UAVOL1_res
  start UAVG2_res then start UAVOL2_res
  start UAVOL1_res then start ClusterIP
  start UAVOL2_res then start ClusterIP
  start ClusterIP then start NFS-D
  start NFS-D then start nfs-cm-shared
  start NFS-D then start nfs-global-home
Colocation Constraints:
[[email protected] ~]#

 

22. You can also verify the NFS shares using the following command. (Have to execute where the resources are running currently)

[[email protected] ~]# showmount -e 192.168.2.90
Export list for 192.168.2.90:
/SAP_SOFT   192.168.2.0/255.255.255.0
/users1/home 192.168.2.0/255.255.255.0
[[email protected] ~]#

[[email protected] ~]# ifconfig eth0
ib0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 2044
        inet 192.168.2.90  netmask 255.255.255.0  broadcast 0.0.0.0
        inet6 fe80::7efe:9003:a7:851  prefixlen 64  scopeid 0x20

 

23. Enable the STONITH.

#pcs property set stonith-enabled=true
#pcs property show stonith-enabled

 

24. Login to NFS clients and mount the shares .

# mkdir /users1/home 
# mkdir /SAP_SOFT
# mount -t nfs -o vers=4 192.168.2.90:/SAP_SOFT  /SAP_SOFT
# mount -t nfs -o vers=4 192.168.2.90:/users1/home  /users1/home

We have successfully setup high availability NFS server v4 using pacemaker cluster suite on Redhat Enterprise Linux 7.x.

If you have any trouble with resources , use the following command to clear the state. Resource might be automatically banned if its faulted more than one twice.

 [[email protected] init.d]# pcs resource clear UANFSHA