8.1 Overview of Metadevice and Diskset Administration

Metadevices and disksets are created and administered using either Solstice DiskSuite 4.0 command-line utilities or the DiskSuite Tool (metatool(1M)) graphical user interface.

Your primary source of information about administration of Solstice DiskSuite devices will be the Solstice DiskSuite 4.0 Administration Guide and Solstice DiskSuite Tool 4.0 User's Guide.

Read the information in this chapter before using the DiskSuite documentation to administer disksets and metadevices in a Solstice HA configuration.

Disksets are groups of disks. The primary administration task you will perform on disksets involves adding and removing disks.

Before using a physical device that you have placed in a diskset, you must set up a metadevice using the component. A metadevice consist of concatenations, stripes, mirrors, UFS logs, or hot spares.

---

Note - Metadevice names begin with "d" and are followed by a number. By default there are 128 unique metadevices in the range 0 to 127. Each UFS logging device you create will use at least seven metadevice names. Thus, in a large Solstice HA configuration, you may need more than the 128 default metadevice names. Refer to Appendix A of the Solstice DiskSuite 4.0 Administration Guide for instructions on changing the number.

The Solstice DiskSuite 4.0 Administration Guide tells you the only modifiable field in the /kernel/drv/md.conf file is the nmd field. You can also modify the md_nsets field from 4 to 3.

---

8.2 Mirroring Guidelines

Corresponding submirror components must be on different SPARCstorage Array chassis. This ensures all mirrored data will survive a SPARCstorage chassis failure.

The need for online service of Solstice HA servers dictates a somewhat stronger requirement, that no two components in different submirrors of a single mirror may be in the same SPARCstorage Array chassis or tray. This allows submirrors to be taken offline and either the chassis to be powered off or the tray to be spun down and removed.

Additional information about mirroring guidelines can be found in the SPARCcluster High Availability Software Planning and Installation Guide.

8.3 Diskset Administration

Diskset administration consists of adding and removing disks from the diskset. The steps for these procedures are included in the following subsections.

---

Note - If the logical hosts are up and running you should never perform diskset administration using either the -t (take ownership) or -r (release ownership) options of the metaset(1M) command. These options are used internally by the Solstice HA software and must be coordinated between the two servers.

If the logical host is in maintenance mode, as reported by hastat(1M), you can safely use the metaset -t command to take ownership of the diskset. However, before returning the logical host to service you must release the diskset ownership using the metaset -r command.

To place a logical host in maintenance mode, run the haswitch -m command.

---

If a SPARCstorage Array tray must be spun down to perform the administration tasks, you must first have ownership of any diskset on the tray. Ownership may be changed using the haswitch(1M) command.

You cannot use the ssaadm(1M) command to perform maintenance on a SPARCstorage Array if the sibling host has ownership of the logical host. If the logical host is either in maintenance mode or is locally owned you can use the ssaadm command.

Before using the ssaadm command, you must make sure that no drives in the tray are owned by the sibling server. If you see the following message when running the ssaadm command with the display argument, there are drives in the tray owned by the sibling server.

host1# ssaadm display c1 ssaadm: Close: I/O error host1#

The procedures in Chapter 4, "Hardware Replacement and Repair," or Chapter 5, "Adding Hardware," provide additional details about performing maintenance procedures.

8.3.1 Adding a Disk to a Diskset

If the disk that is being added to a diskset will be used as a submirror, you must have two disks available on two different SPARCstorage Arrays to allow for mirroring. However, if the disk will be used as a hot spare, you can add only one.

· How to Add a Disk to a Diskset

1. Ensure there is no data on the disk because the partition table will be rewritten and space for a metadevice state database replica will be allocated on the disk.

2. Insert the disk device into the SPARCstorage Array using the instructions in the SPARCstorage Array Model 100 Series Service Manual.

3. Invoke the following command from the Solstice HA server that owns the diskset:

# metaset -s logicalhost -a diskname

8.3.2 Removing a Disk From a Diskset

You can remove a disk from a diskset at any time, provided that none of the slices on the disk are currently in use in metadevices or hot spare pools. To find out if any of the slices are in use, you would use the metastat(1M) command.

The steps to follow when removing a disk from a diskset are in the following section.

· How to Remove a Disk From a Diskset

1. Ensure that none of the slices are in use as metadevices or as hot spares.

2. Invoke the following command from the Solstice HA server that owns the diskset:

# metaset -s logicalhost -d diskname

The metaset command automatically discovers if a metadevice state database replica existed on the disk and if so it finds a suitable location for a replacement replica on another disk.

8.4 Multi-host Metadevice Administration

The following subsections contain information about the differences in administering metadevices in the multi-host Solstice HA environment versus a single host environment.

Unless noted in the following subsections, the instructions in the Solstice DiskSuite 4.0 Administration Guide and the Solstice DiskSuite Tool 4.0 User's Guide can be used.

---

Note - Before using the instructions in either of the Solstice DiskSuite manuals, check in the SPARCcluster documentation set. The instructions in the Solstice DiskSuite manuals deal only with single host configurations.

---

The following subsections tell you the Solstice DiskSuite command-line programs you should use when performing a task. Optionally, you can use the metatool(1M) graphical user interface for all the tasks unless directed otherwise. You must remember to use the -s option when running metatool. The -s option allows you to specify the diskset name.

8.4.1 Managing Metadevices

For ongoing management of metadevices, you must constantly monitor the metadevices for errors in operation, as discussed in Chapter 3, "Monitoring the Solstice HA Servers."

Use the hastat(1M) command to monitor the status of the disksets. When hastat reports a problem with a diskset, you can use the metastat(1M) command to locate the errored metadevice.

You must use the -s option when running either metastat or metatool. The -s option allows you to specify the diskset name.

8.4.2 Adding a Mirror to a Diskset

Mirrored metadevices may be used directly by HA-DBMS applications. They may also be used as part of a logging UFS file system for either HA-NFS or HA-ORACLE applications.

Idle slices on disks within a diskset can be configured into metadevices by using the metainit(1M) command.

8.4.3 Removing a Mirror from a Diskset

HA-ORACLE applications may use raw mirrored metadevices for database storage. While these are not mentioned in the dfstab.logicalhost or vfstab.logicalhost files, they appear in the related HA-ORACLE configuration files. The mirror must be removed from these files and the HA-ORACLE system made to stop using the mirror. At that point the mirror may be cleared by using the metaclear(1M) command.

8.4.4 Taking Submirrors Offline

Before replacing or adding a disk drive in a SPARCstorage Array tray, all the metadevices on that tray must be taken offline.

In symmetric configurations, taking the submirrors offline for maintenance is complex because there may be disks from each of the two disksets on the same tray in the SPARCstorage Array. That means you must take the metadevices from each diskset offline before removing the tray.

You will use the metaoffline(1M) command to take all submirrors on every disk in the tray off line.

8.4.5 Creating New Metadevices

After a disk is added to a diskset, you can create new metadevices using metainit(1M) or metatool. If the new devices are going to be hot spares, you will use the metahs(1M) command to place the hot spares in a hot spare pool.

8.4.6 Replacing Errored Components

When replacing an errored metadevice component, you will use the metareplace(1M) command.

You can return drives to service that have sustained transient errors (for example, a chassis power failure) by using the metareplace -e command.

A replacement slice (or disk) must be available as a replacement. This device could be an existing device that is not in use or a new device you have added to the diskset.

8.4.7 Deleting a Metadevice

Before deleting a metadevice, you must ensure that none of the components in the metadevice is in use, either by a database or by HA-NFS. You will use the metaclear(1M) command to delete the metadevice.

8.4.8 Growing a Metadevice

To grow a metadevice you must have another two slices (disks) in different SPARCstorage Arrays available. Each of the two new slices will be added to a different submirror with the metainit command. You then use the growfs(1M) command to grow the file system.

---

Caution - When running the growfs command, clients can experience an interruption of service.

---

If a takeover occurs while the file system is growing, the file system will not be grown. You must reissue the growfs command (from the command line) after the takeover completes.

---

Note - The file system that contains /logicalhost/statmon cannot be grown. This is because the statd(1M) program modifies this directory it would be blocked for extended periods while the file system is growing. This would have unpredictable effects on the network lock protocol. This is only a problem for configurations using HA-NFS.

---

The following example shows the growfs command used to enlarge the d30 metadevice on host1 by 512 Mbytes. The warning printed by growfs reports the number of inode blocks that will not be allocated. The errors indicate that while the file system was being grown it was locked against modifications. These errors are part of the expected behavior.

# growfs -s 1000000 -M /host1/ufsd30 /dev/md/host1/rdsk/d30 Warning: inode blocks/cyl group (133) >= data blocks (4) in last cylinder group. This implies 64 sector(s) cannot be allocated. /dev/md/host1/rdsk/d30: 999936 sectors in 992 cylinders of 14 tracks, 72 sectors 488.2MB in 62 cyl groups (16 c/g, 7.88MB/g, 3776 i/g) super-block backups (for fsck -F ufs -o b=#) at: 32, 16240, 32448, 48656, 64864, 81072, 97280, 113488, 129696, 145904, 162112, 178320, 194528, 210736, 226944, 243152, 258080, 274288, 290496, 306704, 322912, 339120, 355328, 371536, 387744, 403952, 420160, 436368, 452576, 468784, 484992, 501200, 516128, 532336, 548544, 564752, 580960, 597168, 613376, 629584, 645792, 662000, 678208, 694416, 710624, 726832, 743040, 759248, 774176, 790384, 806592, 822800, 839008, 855216, 871424, 887632, 903840, 920048, 936256, 952464, 968672, 984880, NFS2 setattr failed for server ha-drag: RPC: Timed out Jun 30 14:40:55 ha-drag hadf: ERROR: Error: nfs_mon: ftruncate of '/var/opt/SUNWhadf/hadf/nfs_probe_mountpoints/_host1_ufsd30.locking/.probe_nfs/probefile' failed, errno=145 'Connection timed out' Jun 30 14:40:55 host1 hadf: ERROR: Error: nfs_mon: This problem is with this host. #

8.4.9 Managing Hot Spare Pools

Hot spare devices can be added and deleted from hot spare pools at any time, providing they are not in use. In addition, you can create new hot spare pools and associate them with submirrors by using the metahs(1M) command.

8.4.10 Managing UFS Logs

All UFS logs on multi-host disks are mirrored. When a submirror fails, it is reported as an errored component. You repair the failure by using either metareplace or metatool.

If the entire mirror that contains the UFS log fails, you must unmount the file system, back up any accessible data, repair the error, repair the file system (using fsck(1M)), and remount the file system.

8.4.10.1 Adding UFS Logging to a Logical Host

All UFS file systems within a logical host must be logging UFS file systems to ensure the failover or haswitch timeout criteria can be met. A logging UFS file system may be used by HA-NFS or by an HA-DBMS data service.

The logging UFS file system is set up by first creating a trans device with a mirrored log and a mirrored UFS master file system. Both the log and UFS master device must be mirrored following the mirroring guidelines explained in Section 8.2, "Mirroring Guidelines," on page 8-2.

During Solstice HA configuration, the hasetup(1M) command optionally reserves space on slice 6 of each drive in a diskset for use as a UFS log. The size of these slices was determined during configuration. The slices can be used for UFS log submirrors. If the slices are smaller than the desired log size, several can be concatenated. Typically one Mbyte per 100 Mbytes is adequate for UFS logs. Ideally, log slices would be drive-disjoint from the UFS master device.

---

Note - If you must repartition the disk to gain space for UFS logs, you must preserve the existing slice 7 which starts on cylinder 0 and contains at least two Mbytes. This space is required and reserved for metadevice state database replicas. The Tag and Flag fields (as reported by the format(1M) command) must be preserved for slice 7. The metaset(1M) command sets the Tag and Flag fields correctly when the initial configuration is built.

---

After the trans device has been configured, the UFS file system must be created by using newfs(1M) on the trans device. This typically takes about two minutes per Gbyte of UFS master device size.

After the newfs process is completed, the UFS file system can be added to the vfstab.logicalhost file with the hafstab(1M) command. If this file system is for use by an HA-DBMS application, the HA-DBMS configuration files should be updated.

If the file system will be shared by HA-NFS, follow the procedure in Section 6.3, "Adding an HA-NFS File System to a Logical Host," on page 6-2.

The new file system will be automatically mounted at the next membership monitor reconfiguration. It may be manually mounted if membership monitor reconfiguration is not possible or desirable.

8.5 Local Metadevice Administration

Local disks are not mirrored, however some local file systems may have UFS logs. If an error occurs, you will perform the same actions as when the entire mirror fails, as specified in "Managing UFS Logs" on page 8-8.

If the entire root disk fails, use the instructions in "How to Replace a Failed Local Boot Disk" on page 4-13.

8.6 Destructive Metadevice Actions

The metadevice actions that are not supported in Solstice HA configurations include:

• Creation of a diskset with fewer than three disks
• Creation of a diskset that is attached to fewer than three controllers
• Creation of a one-way mirror in a diskset
• Creation of a configuration with too few metadevice state database replicas on the local disks
• Creation of metadevice state database replicas on multi-host disks without the use of the metaset(1M) command, unless directed to do so in explicit instructions in this or another SPARCcluster manual.