Summary
When I tried to add REDFERN2 to the REDFERN RAC, I got the following errors as described in “ REDFERN2 is unable to share disks with REDFERN1 ”.
File /OVS/shareDisk/REDFERN/FRA_01 is loopback-mounted through /dev/loop3, which is mounted in a guest domain, and so cannot be mounted now.
I will now use the allocated LUNs on the NARRABRI NAS for the shared disks.
References
Oracle Manuals
Procedure
The iSCSI target is the NARRABRI NAS.
This procedure follows that of “ 20.7.2 Configuring an iSCSI Initiator ”.
Install iSCSI Initiator Software
Ran the following command as root on REDFERN1 to install the software for the iSCSI client (initiator):
yum install iscsi-initiator-utils
The output is:
Loaded plugins: ulninfo ol7_UEKR4 | 1.2 kB 00:00 ol7_latest | 1.4 kB 00:00 (1/5): ol7_UEKR4/x86_64/updateinfo | 191 kB 00:01 (2/5): ol7_latest/x86_64/group | 659 kB 00:04 (3/5): ol7_latest/x86_64/updateinfo | 1.8 MB 00:04 (4/5): ol7_latest/x86_64/primary | 17 MB 00:48 (5/5): ol7_UEKR4/x86_64/primary | 36 MB 01:20 ol7_UEKR4 669/669 ol7_latest 26617/26617 Resolving Dependencies --> Running transaction check ---> Package iscsi-initiator-utils.x86_64 0:6.2.0.874-7.0.1.el7 will be installed --> Processing Dependency: iscsi-initiator-utils-iscsiuio >= 6.2.0.874-7.0.1.el7 for package: iscsi-initiator-utils-6.2.0.874-7.0.1.el7.x86_64 --> Running transaction check ---> Package iscsi-initiator-utils-iscsiuio.x86_64 0:6.2.0.874-7.0.1.el7 will be installed --> Finished Dependency Resolution Dependencies Resolved ================================================================================ Package Arch Version Repository Size ================================================================================ Installing: iscsi-initiator-utils x86_64 6.2.0.874-7.0.1.el7 ol7_latest 421 k Installing for dependencies: iscsi-initiator-utils-iscsiuio x86_64 6.2.0.874-7.0.1.el7 ol7_latest 90 k Transaction Summary ================================================================================ Install 1 Package (+1 Dependent package) Total download size: 511 k Installed size: 2.6 M Is this ok [y/d/N]: y Downloading packages: warning: /var/cache/yum/x86_64/7Server/ol7_latest/packages/iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY Public key for iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7.x86_64.rpm is not installed (1/2): iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7. | 90 kB 00:00 (2/2): iscsi-initiator-utils-6.2.0.874-7.0.1.el7.x86_64.rp | 421 kB 00:00 -------------------------------------------------------------------------------- Total 700 kB/s | 511 kB 00:00 Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle Importing GPG key 0xEC551F03: Userid : "Oracle OSS group (Open Source Software group) <build@oss.oracle.com>" Fingerprint: 4214 4123 fecf c55b 9086 313d 72f9 7b74 ec55 1f03 Package : 7:oraclelinux-release-7.4-1.0.4.el7.x86_64 (@anaconda/7.4) From : /etc/pki/rpm-gpg/RPM-GPG-KEY-oracle Is this ok [y/N]: y Running transaction check Running transaction test Transaction test succeeded Running transaction Installing : iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7.x86_64 1/2 Installing : iscsi-initiator-utils-6.2.0.874-7.0.1.el7.x86_64 2/2 Verifying : iscsi-initiator-utils-6.2.0.874-7.0.1.el7.x86_64 1/2 Verifying : iscsi-initiator-utils-iscsiuio-6.2.0.874-7.0.1.el7.x86_64 2/2 Installed: iscsi-initiator-utils.x86_64 0:6.2.0.874-7.0.1.el7 Dependency Installed: iscsi-initiator-utils-iscsiuio.x86_64 0:6.2.0.874-7.0.1.el7 Complete!
Discover Targets on Narrabri
Ran the following command as root on REDFERN1 to discover what iSCSI targets are available on the NARRABRI NAS:
iscsiadm -m discovery -t sendtargets -p narrabri
The output is:
192.168.1.253:3260,1 iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern
This is what I set up for the REDFERN cluster. See the following screen-shot from the NAS console:
Display Target Information on Narrabri
Ran the following command as root on REDFERN1 to populate the local iSCSI discovery database with information about the iSCSI targets presented by the NARRABRI NAS:
iscsiadm -m discoverydb -t sendtargets -p narrabri
The output is:
# BEGIN RECORD 6.2.0.874-7 discovery.startup = manual discovery.type = sendtargets discovery.sendtargets.address = narrabri discovery.sendtargets.port = 3260 discovery.sendtargets.auth.authmethod = None discovery.sendtargets.auth.username = <empty> discovery.sendtargets.auth.password = <empty> discovery.sendtargets.auth.username_in = <empty> discovery.sendtargets.auth.password_in = <empty> discovery.sendtargets.timeo.login_timeout = 15 discovery.sendtargets.use_discoveryd = No discovery.sendtargets.discoveryd_poll_inval = 30 discovery.sendtargets.reopen_max = 5 discovery.sendtargets.timeo.auth_timeout = 45 discovery.sendtargets.timeo.active_timeout = 30 discovery.sendtargets.iscsi.MaxRecvDataSegmentLength = 32768
Establish Session with REDFERN Target on Narrabri
Ran the following command as root on REDFERN1 to start an iSCSI initiator session with the NARRABRI NAS:
iscsiadm -m node -T iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern -p narrabri:3260 -l
The output is:
Login to [iface: default, target: iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern, portal: 192.168.1.253,3260] successful.
Establish Session with REDFERN Target on Narrabri
Ran the following command as root on REDFERN1 to verify that an iSCSI initiator session has been established with the NARRABRI NAS:
iscsiadm -m session -P 3
The output is:
iSCSI Transport Class version 2.0-870
version 6.2.0.874-7
Target: iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern (non-flash)
Current Portal: 192.168.1.253:3260,1
Persistent Portal: 192.168.1.253:3260,1
**********
Interface:
**********
Iface Name: default
Iface Transport: tcp
Iface Initiatorname: iqn.1988-12.com.oracle:d3436e6a9eca
Iface IPaddress: 192.168.1.141
Iface HWaddress: <empty>
Iface Netdev: <empty>
SID: 1
iSCSI Connection State: LOGGED IN
iSCSI Session State: LOGGED_IN
Internal iscsid Session State: NO CHANGE
*********
Timeouts:
*********
Recovery Timeout: 120
Target Reset Timeout: 30
LUN Reset Timeout: 30
Abort Timeout: 15
*****
CHAP:
*****
username: <empty>
password: ********
username_in: <empty>
password_in: ********
************************
Negotiated iSCSI params:
************************
HeaderDigest: None
DataDigest: None
MaxRecvDataSegmentLength: 262144
MaxXmitDataSegmentLength: 262144
FirstBurstLength: 65536
MaxBurstLength: 262144
ImmediateData: Yes
InitialR2T: Yes
MaxOutstandingR2T: 1
************************
Attached SCSI devices:
************************
Host Number: 2 State: running
scsi2 Channel 00 Id 0 Lun: 0
Attached scsi disk sda State: running
scsi2 Channel 00 Id 0 Lun: 1
Attached scsi disk sdc State: running
scsi2 Channel 00 Id 0 Lun: 2
Attached scsi disk sdd State: running
scsi2 Channel 00 Id 0 Lun: 3
Attached scsi disk sde State: running
scsi2 Channel 00 Id 0 Lun: 4
Attached scsi disk sdb State: running
A less verbose output is generated by:
iscsiadm -m session
The output is:
tcp: [1] 192.168.1.253:3260,1 iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern (non-flash)
However, the first version gives a list of disks that have been mounted through iSCSI:
************************
Attached SCSI devices:
************************
Host Number: 2 State: running
scsi2 Channel 00 Id 0 Lun: 0
Attached scsi disk sda State: running
scsi2 Channel 00 Id 0 Lun: 1
Attached scsi disk sdc State: running
scsi2 Channel 00 Id 0 Lun: 2
Attached scsi disk sdd State: running
scsi2 Channel 00 Id 0 Lun: 3
Attached scsi disk sde State: running
scsi2 Channel 00 Id 0 Lun: 4
Attached scsi disk sdb State: running
See What Disks Have Been Mounted
Ran the following command on REDFERN1 to see what iSCSI LUNs have been mounted from the NARRABRI NAS:
ls -l /dev/disk/by-path/
The output is:
total 0 lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-0 -> ../../sda lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-1 -> ../../sdc lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-2 -> ../../sdd lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-3 -> ../../sde lrwxrwxrwx. 1 root root 9 Jun 2 21:27 ip-192.168.1.253:3260-iscsi-iqn.1994-11.com.netgear:narrabri:4eca4c51:redfern-lun-4 -> ../../sdb lrwxrwxrwx. 1 root root 9 Jun 2 20:19 pci-0000:00:01.1-ata-2.0 -> ../../sr0