5.1 Adding a SPARCstorage Array

Additional SPARCstorage Arrays can be added to a Solstice HA configuration at any time.

You must review the metadevice distribution in your Solstice HA configuration before adding a SPARCstorage Array. The discussions in the SPARCcluster High Availability Software Planning and Installation Guide, and Chapter 8, "Metadevice and Diskset Administration," in this manual will help determine the impact of the SPARCstorage Array on the distribution of metadevices.

· How to Add a SPARCstorage Array

1. Shut down one of the Solstice HA servers (host1). Use the procedure in Section 9.3, "Shutting Down Solstice HA Servers," on page 9-5 to shut down the server.

2. Install the Fibre Channel SBus card (FC/S) in the server.

   Use the instructions in the SPARCcluster High Availability Server Service Manual to install the FC/S card.

---

Note - Install the FC/S card in the first available empty SBus slot, following all other cards in the server. This will ensure the controller numbering will be preserved if the Solaris operating environment is reinstalled. Refer to Section 2.4, "Instance Numbering," on page 2-4 for more information.

---

3. Connect the cables to the SPARCstorage Array and FC/S card.

   Use the instructions in the SPARCcluster High Availability Server Service Manual.

4. Perform a reconfiguration reboot of the server.

ok boot -r

5. When the server reboots, switch ownership of the Solstice HA services to the other host.

   Use the haswitch(1M) command.

host1# haswitch host1 logicalhost1 logicalhost2

6. Repeat Step 1 through Step 4 on the sibling Solstice HA server.

---

Note - The hardware must be installed identically on each of the servers. This means the new SBus card must be installed on the same System Board and SBus slot on each server.

---

7. Switch ownership of the logical hosts back to the appropriate default master.

   For example:

host1# haswitch host2 logicalhost2

8. Add the disks in the SPARCstorage Arrays to the selected diskset.

   Use the instructions in Section 8.3.1, "Adding a Disk to a Diskset," on page 8-4 to add the disks to disksets.

5.2 Adding a Disk to a SPARCstorage Array

Adding a disk to a SPARCstorage Array involves taking all the metadevices in the tray offline. It is likely that the tray will contain disks from each of the disksets in a symmetric configuration.

· How to Add a Disk to a SPARCstorage Array

1. Switch ownership of both logical hosts to one of the Solstice HA servers.

   In order to run the ssaadm(1M) command you must switch over both logical hosts to a single server. If you have an asymmetric configuration no switchover is required.

host1# haswitch host1 logicalhost1 logicalhost2

2. Determine the controller number of the SPARCstorage Array where the disk will be added.

   The World Wide Name displayed on the front of the SPARCstorage Array also appears as part of the /devices entry to which the /dev entry containing the controller number points. For example:

host1# ls -l /dev/rdsk | grep -i world_wide_number | tail -1

If the World Wide Name displayed on the front of the SPARCstorage Array is 36cc, the following output would be displayed and you would find the controller number to be c2:

host1# ls -l /dev/rdsk | grep -i 36cc | tail -1 lrwxrwxrwx 1 root root 94 Jun 25 22:39 c2t5d2s7 -> ../../devices/io- unit@f,e1200000/sbi@0,0/SUNW,soc@3,0/SUNW,pln@a0000800,201836cc/ssd@5,2:h,raw host1#

3. Locate an appropriate empty disk tray slot in the SPARCstorage Array for the disk that is being added.

   Use the ssaadm(1M) command with the display option to view the empty slots in the SPARCstorage Array. The empty slots are shown with a NO SELECT status.

host1# ssaadm display c2 DEVICE STATUS TRAY 1 TRAY 2 TRAY 3 slot 1 Drive: 0,0 Drive: 2,0 Drive: 4,0 2 Drive: 0,1 Drive: 2,1 Drive: 4,1 3 NO SELECT NO SELECT NO SELECT 4 NO SELECT NO SELECT NO SELECT 5 NO SELECT NO SELECT NO SELECT 6 Drive: 1,0 Drive: 3,0 Drive: 5,0 7 Drive: 1,1 NO SELECT NO SELECT 8 NO SELECT NO SELECT NO SELECT 9 NO SELECT NO SELECT NO SELECT 10 NO SELECT NO SELECT NO SELECT host1#

4. Determine the tray to which you will add the new disk.

   In the remainder of the procedure, tray 2 will be used as an example.

   The slot selected for the new disk is tray 2 slot 7. The new disk will be known as c2t3d1.

5. Locate all hot spares in the tray of the SPARCstorage Array.

   To discover the status and location of all hot spares, run the metahs(1M) command with the -i option on each of the logical hosts.

host1# metahs -s logicalhost1 -i ... host1# metahs -s logicalhost2 -i ...

---

Note - Save a list of the hot spares. The list will be used later in this maintenance procedure. Be sure to note the hot spare devices and the hot spare pools they are in.

---

6. Delete all hot spares in the tray.

   Use the metahs command with the -d option to delete the hot spares.

host1# metahs -s logicalhost1 -d hot_spare_pool host1# metahs -s logicalhost2 -d hot_spare_pool

7. Locate all metadevice state database replicas that are on disks in the tray of the SPARCstorage Array.

   Run the metadb(1M) command on each of the logical hosts to locate all metadevice state databases. In an asymmetric configuration there will be only one logical host. Direct the output into temporary files.

host1# metadb -s logicalhost1 > /tmp/mddb1 host1# metadb -s logicalhost2 > /tmp/mddb2

---

Note - Save the list of the metadevice state database replicas. The list will be used later in this maintenance procedure.

---

Use the metadb command to determine on which disks in this tray metadevice state database replicas reside. Save this information for the step in which you restore the replicas.

8. Delete the metadevice state database replicas that are on disks in the tray where the disk will be added.

   Keep a record of the number and local of the replicas you delete. These must be restored in a later step.

host1# metadb -s logicalhost1 -d replicas host1# metadb -s logicalhost2 -d replicas

9. Run the metastat command to determine all the metadevice components in the tray.

   The output from metastat should be directed to a temporary file so the information can be used later when deleting and re-adding the metadevices.

host1# metastat -s logicalhost1 > /tmp/log1 host1# metastat -s logicalhost2 > /tmp/log2

10. Take all submirrors on the tray offline.

    Use the temporary files to create a script to take all submirrors on the tray offline. If there are only a few submirrors on the tray you can run the metaoffline(1M) command to take each offline. The following is an example script.

#!/bin/sh # metaoffline -s <setname> <metamirror> <submirror> metaoffline -s logicalhost1 d15 d35 metaoffline -s logicalhost2 d15 d35 ...

11. Spin down the disks in the SPARCstorage Array tray.

    Use the ssaadm(1M) command to perform this step.

host1# ssaadm stop -t 2 c2

12. Remove the tray, insert the disk, and replace the tray.

    Use the instructions in the SPARCstorage Array Model 100 Series Service Manual to perform the hardware procedure of adding the disk.

13. Make sure all disks in the SPARCstorage Array tray spin up.

    The disks in the SPARCstorage Array tray should automatically spin up following the hardware procedure. If the tray fails to spin up automatically within two minutes, force the action by using the following command:

host1# ssaadm start -t 2 c2

14. Bring the submirrors in the tray back online.

    Modify the script you created in Step 10 to bring the submirrors back online.

#!/bin/sh # metaonline -s <setname> <metamirror> <submirror> metaonline -s logicalhost1 d15 d35 metaonline -s logicalhost2 d15 d35 ...

15. Add back the hot spares that were deleted in Step 6.

host1# metahs -s logicalhost1 -a hot_spare_devices host1# metahs -s logicalhost2 -a hot_spare_devices

16. Be sure to restore the original count of metadevice state database replicas to the devices. The replicas were removed in Step 8.

host1# metadb -s logicalhost1 -a replicas host1# metadb -s logicalhost2 -a replicas

17. Run the drvconfig(1M) and disks(1M) commands to create the new entries in /devices, /dev/dsk, and /dev/rdsk for the drive that was added.

host1# drvconfig host1# disks

18. Switch ownership of both logical hosts to host2.

host1# haswitch host2 logicalhost1 logicalhost2

19. Run the drvconfig and disks commands on host2.

host2# drvconfig host2# disks

20. Add the disk to a diskset.

    In this example, the disk is being added to the diskset that is mastered by host2.

host2# metaset -s logicalhost2 -a drivename

21. Perform usual administration actions.

    You can now perform the usual administration steps that are done when a new drive is brought into service. These include partitioning the disk, adding it to the configuration as a hot spare, or configuring it as a metadevice.

22. Switch each logical host back to its default master.

host2# haswitch host1 logicalhost1

5.3 Adding a Public Network

Adding a public network connection in a Solstice HA configuration involves both software and hardware procedures.

· How to Add a Public Network Connection

1. Run haswitch to move the data services that are running to a single host. In this example, host2 will be the first to receive the new public network connection.

host1# haswitch host1 logicalhost1 logicalhost2

2. Stop the membership monitor.

host2# /etc/init.d/SUNWhadf stop

3. Create the entries in /etc/inet/hosts, /etc/inet/netmasks, and network name services.

   Add the new hostnames to the local /etc/inet/hosts file on both Solstice HA servers and the network name service. If this is a new network number make the appropriate entries in the /etc/inet/netmasks file.Assign additional logical hostnames on this new network for each logical host. Make the entries to the /etc/inet/hosts files and network name service at any time, before the new network is added.

4. Create the /etc/hostname.xxn file for the new interface. The /etc/hostname.xxn file must contain the hostname associated with the new interface. (Replace the xxn suffix with the type and number of the interface (for example qe3 or le4). If this is performed before the server is halted, ifconfig(1M) will automatically assign an address to the network interface when the server is rebooted.

   If the name of the new interface (for instance, qe3) cannot be determined before the reboot, run the prtconf(1M) command after the next reboot and use that output to learn the new address.

   In that case either another reboot or several manual invocations of ifconfig are needed to bring the interface into service.

5. Halt the server.

host2# halt

6. Install the network hardware using the instructions in the appropriate hardware manual.

7. Perform a reconfiguration reboot on the server.

   The reboot will automatically start the membership monitor, however the server will not take back ownership of the diskset or data services.

ok boot -r

8. Repeat the entire procedure on the sibling server.

9. Edit the hadfconfig(4) file on both servers. After the new interfaces are appropriately configured and entries have been added to the name service, /etc/hosts file, and the /etc/hostname.xxn file, the interface will be added to the

   /etc/opt/SUNWhadf/hadf/hadfconfig file. Entries to the hadfconfig file are made in the following format:.

HOSTNAME host1-nnn logicalhost1-nnn host2-nnn logicalhost2-nnn

Following the naming convention, in the above example the nnn represents the third octet of the associated network number.

---

Note - If this is an asymmetric configuration with a single logical host (diskset), the string logicalhost2-nnn is replaced with a hyphen (-). These changes must be made manually on both machines for correct operation.

---

Services will be offered via the logical hosts on the next membership reconfiguration. Refer to "Forcing a Membership Reconfiguration" on page 9-2.

10. Run the hacheck(1M) command on both hosts.

host2# hacheck

5.4 Adding Board-Level Modules

Adding or replacing board-level modules such as SIMM and CPUs involves both software and hardware procedures.

· How to Add Board-Level Modules

1. Run haswitch to move the data services that are running to a single host. In this example, host2 will be the first to receive the board-level module.

host1# haswitch host1 logicalhost1 logicalhost2

2. Stop the membership monitor.

host2# /etc/init.d/SUNWhadf stop

3. Halt the server.

host2# halt

4. Power off the server.

5. Perform a reconfiguration reboot.

ok boot -r

6. Install the board-level module using the instructions in the appropriate hardware manual.

7. Power on the server.

   The server will rejoin the configuration.

8. Run haswitch to move the data services to the host that has just received the additional board-level modules.

9. Repeat Step 2 through Step 6 on the sibling server.

   In order to maintain a symmetric hardware configuration, both servers must have exactly the same hardware installed.

10. Switch each logical host back to its default master.