Overview of Hot Spares

A hot spare is a component that is running (but not being used) which can be substituted for a broken component in a submirror of a two- or three-way metamirror or RAID device. Failed components in a one-way metamirror cannot be replaced by a hot spare, since no other copy of the data is available.

A hot spare is to a metamirror or RAID device what a spare tire is to a car. A spare is meant to quickly replace a flat tire, thus increasing the availability of your car. A spare tire is a temporary fix that is made with the intention that the flat tire will be fixed or replaced with a new one and the spare returned to the trunk.

This is exactly how hot spares should be treated. They are not intended to be used as permanent ''fixes'' when a component has failed. They are temporary fixes that can be used until a failed component is either fixed or replaced.

Components designated as hot spares cannot be used in submirrors or another metadevice in the md.tab file. They must remain ready for immediate use in the event of a component failure.

Hot spares are always in one of three states:

• ''Available'' hot spares are running and ready to accept data, but are not currently being written to or read from.
• ''In-use'' hot spares are currently being written to and read from.
• ''Broken'' hot spares are out of service. A hot spare is placed in the broken state when an I/O error occurs.

You display the states by running the metahs command with the -i option.

Hot spares can be defined as part of a hot spare pool in the md.tab file. This is performed by editing the md.tab file to create hot spares and hot spare pools and then running the metainit(1M) command.

Hot spare definitions, replacements, additions, and deletions can also be performed by using the metahs(1M) utility. The number of hot spare pools is limited to 1000.

Defining Hot Spares

After a hot spare pool is defined, it can be associated with one or more submirrors. Hot spare pools are named hspnnn where nnn is a number in the range 000-999. A metadevice cannot be configured as a hot spare.

A hot spare can be used in one or more hot spare pools, thus allowing for the maximum amount of security from the minimum number of components. DiskSuite looks for the first available hot spare from the designated pool when errors are reported.

With DiskSuite, you can also define empty hot spare pools so hot spares can be added when they become available.

The procedure for defining hot spare pools is provided below. Note that the hot spares used as examples in these steps are shared in the three defined hot spare pools.

1. Determine the names of the devices that you are designating as hot spares.

   In the following steps, the names /dev/dsk/c0t0d0s2, /dev/dsk/c1t0d0s2, and /dev/dsk/c2t0d0s2 are used as examples.

2. Edit the md.tab file and set up the hot spare pool(s) using the devices that have been chosen as hot spares.

   In this example, the hot spare pools are named: hsp001, hsp002, and hsp003. Note that the same hot spares are used in each hot spare pool, but the order is changed.

hsp001 /dev/dsk/c0t0d0s2 /dev/dsk/c1t0d0s2 /dev/dsk/c2t0d0s2 hsp002 /dev/dsk/c1t0d0s2 /dev/dsk/c2t0d0s2 /dev/dsk/c0t0d0s2 hsp003 /dev/dsk/c2t0d0s2 /dev/dsk/c0t0d0s2 /dev/dsk/c1t0d0s2

3. Once the hot spare pools are defined, you associate the pools with submirrors using the metaparam command with the -h option.

# metaparam -h hsp002 d9 # metaparam -h hsp003 d10 # metaparam -h hsp001 d11

Once the hot spare pools are defined and associated with a submirror, the hot spares are ''available'' for use. If a component failure occurs, DiskSuite searches through the list of hot spares in the assigned pool and selects the first ''available'' component that is equal or greater in disk capacity to the failed component.

If a hot spare of adequate size is found, the hot spare's state changes to ''in-use'' and a resync operation is automatically performed. The resync operation brings the hot spare into sync with the other submirrors.

If a component of adequate size is not found in the list of hot spares, the submirror that failed is considered erred and that portion of the submirror no longer replicates the data.

Hot Spare Conditions to Avoid

As stated earlier, hot spares are intended to be a temporary remedy for a failed submirror or RAID component. They are not meant to be used as a permanent replacement. Additionally, there are three conditions that should be avoided when using hot spares:

• Associating hot spares of the wrong size with submirrors. This condition occurs when hot spare pools are defined and associated with a submirror and none of the hot spares in the hot spare pool are equal to or greater than the smallest component in the submirror. This would occur, for example, when 661-Mbyte disk drives make up the hot spare pool that is associated with a submirror that is made up of a 1-Gbyte drive.
• Having all hot spares within the hot spare pool in-use. When the administrator notices that all hot spares are in-use, immediate action is required. There are two possible solutions when this occurs. The first is to add additional hot spares. The second is to repair some of the components that have been hot spare replaced. If all hot spares are in use and a submirror fails due to errors, that portion of the mirror will no longer be replicated.
• Assigning a hot spare pool to a submirror in a one-way metamirror.

Manipulating Hot Spare Pools

DiskSuite offers facilities to dynamically add, delete, replace, and enable hot spares within existing hot spare pools. The command used to perform these functions is metahs.

metahs is the primary command used to manipulate hot spares and hot spare pools. This command provides the following utilities:

• Adding hot spares to hot spare pools
• Deleting hot spares from hot spare pools
• Replacing hot spares in hot spare pools
• Enabling hot spares
• Checking the status of hot spares

The metaparam command is used to add or change the hot spare pool associated with a metamirror or RAID device.

Adding Hot Spares

A hot spare can be added to one or more hot spare pools, using the metahs command with the -a option. When a hot spare is added, the existing order of the hot spares already in the pool is preserved. The new hot spare is added at the end of list of hot spares in the hot spare pool that is specified.

The following example illustrates adding a hot spare (/dev/dsk/c0t0d0s7) to a hot spare pool (hsp003).

# /usr/opt/SUNWmd/sbin/metahs -a hsp003 /dev/dsk/c0t0d0s7

If the hot spare pool, hsp003, was defined with two hot spares (for example, /dev/dsk/c0t1d0s7 and /dev/dsk/c0t2d0s7), the hot spare added (/dev/dsk/c0t0d0s7) would follow the two that were already associated with that hot spare pool. If the hot spare pool, hsp003, didn't already exist, it would be automatically created.

To add a hot spare to the hot spare pools that are currently defined, enter the following:

# /usr/opt/SUNWmd/sbin/metahs -a all /dev/dsk/c0t0d0s7

The keyword all in the above example specifies adding the hot spare, /dev/dsk/c0t0d0s7, to all the hot spare pools.

Deleting Hot Spares

Hot spares can be deleted from any or all the hot spare pools to which they have been associated, using the metahs command with the -d option. DiskSuite will not allow a hot spare to be deleted if it is in the ''in-use'' state.

When a hot spare is deleted from a hot spare pool, the position of the remaining hot spares changes to reflect the new position. For example, if the second of three hot spares in a hot spare pool is deleted, the third hot spare moves to the second position.

The following example shows the deletion of the hot spare, /dev/dsk/c0t0d0s7, from the hot spare pool, hsp003.

# /usr/opt/SUNWmd/sbin/metahs -d hsp003 /dev/dsk/c0t0d0s7

If the hot spare, /dev/dsk/c0t0d0s7, is associated with other hot spare pools, it would be removed from all of them if the all option is used. For example:

# /usr/opt/SUNWmd/sbin/metahs -d all /dev/dsk/c0t0d0s7

The -d option can also be used to delete a hot spare pool. Before deleting a hot spare pool, all hot spares associated with the hot spare pool must first be deleted. A hot spare pool can not be deleted if it is associated with a submirror.

In the following example, three hot spares (/dev/dsk/c0t0d0s1, /dev/dsk/c1t0d0s1, and /dev/dsk/c2t0d0s1) are associated with hot spare pool hsp001. The hot spare pool is currently associated with the metadevice, d16. In this example, the metadevice is disassociated with the hot spare pool with the metaparam command. Each of the three hot spares are then deleted from the hot spare pool, using metahs. Finally, the hot spare pool is deleted.

# /usr/opt/SUNWmd/sbin/metaparam -h none d16 # /usr/opt/SUNWmd/sbin/metahs -d hsp001 /dev/dsk/c0t0d0s1 # /usr/opt/SUNWmd/sbin/metahs -d hsp001 /dev/dsk/c1t0d0s1 # /usr/opt/SUNWmd/sbin/metahs -d hsp001 /dev/dsk/c2t0d0s1 # /usr/opt/SUNWmd/sbin/metahs -d hsp001

Replacing Hot Spares

Hot spares can be replaced in any or all the hot spare pools to which they have been associated, using the metahs command with the -r option. However, hot spares that are in the in-use state cannot be replaced by other hot spares.

The order of hot spares in the hot spare pools is not changed when a replacement occurs.

The following example shows the replacement of hot spare /dev/dsk/c0t0d0s1 with the component /dev/dsk/c0t1d0s1 in the hsp003 hot spare pool.

# /usr/opt/SUNWmd/sbin/metahs -r hsp003 /dev/dsk/c0t0d0s1 \ /dev/dsk/c0t1d0s1

The metahs command with the -r command can also be used to replace a hot spare in all the hot spare pools where it is associated. The following example shows the replacement of hot spare /dev/dsk/c0t0d0s2 with

/dev/dsk/c0t1d0s2.

# /usr/opt/SUNWmd/sbin/metahs -r all /dev/dsk/c0t0d0s2 \ /dev/dsk/c0t1d0s2

The keyword all in the above example specifies replacing the hot spare, /dev/dsk/c0t0d0s2, with /dev/dsk/c0t1d0s2 in all the hot spare pools.

Enabling Hot Spares

When a hot spare that has been placed in the ''broken'' state is repaired, it is brought back to the ''available'' state by using the metahs command with the -e option. Hot spares are placed in the ''broken'' state after an I/O error occurs. After the hardware is repaired, the component can be brought back to the available state.

The following example illustrates placing the hot spare /dev/dsk/c0t0d0s2 in the available state after it has been repaired.

# /usr/opt/SUNWmd/sbin/metahs -e /dev/dsk/c0t0d0s2

Changing the Associated Hot Spare Pool

Each submirror or RAID component can optionally be associated with a hot spare pool. The hot spare pool association can be changed with the metaparam command while the system is running, providing none of the hot spares in the current hot spare pool are currently being used by the submirror or RAID component.

In the following example, the hot spare pool, hsp005, is currently associated with d80. This command changes the hot spare pool association to hsp001.

# /usr/opt/SUNWmd/sbin/metaparam -h hsp005 d80

Checking the Status of Hot Spares

The metahs command can be used to display the status of hot spare pools. When the -i option is used, metahs displays the status for all hot spare pools if one is not specified. For example:

# /usr/opt/SUNWmd/sbin/metahs -i hsp001: 1 hot spare /dev/dsk/c0t0d0s0 (Available) blocks 263220 hsp002: 1 hot spare /dev/dsk/c0t1d0s0 (In-use) blocks 263220 hsp003: 1 hot spare /dev/dsk/c0t2d0s0 (Broken) blocks 263220

The above example shows three hot spare pools (hsp001, hsp002, and hsp003) and gives the status of the components. If one of the three hot spare pools had been specified at the command line, only that hot spare pool would have been displayed.

Examples

A series of examples are provided in this section that show, in a step-by-step fashion, how to define hot spares and set up hot spare pools. These examples first define a hot spare pool in the /etc/opt/SUNWmd/md.tab file, then add, delete, and replace hot spares in that pool. The examples are:

• Setting up hot spare pools
• Adding hot spares to hot spare pools
• Deleting hot spares from hot spare pools
• Replacing hot spares within hot spare pools

Setting up Hot Spare Pools

Hot spare pools can be defined with or without hot spares. The following steps show how to define the hot spare pool, attach two hot spares, and associate the hot spares with submirrors.

1. Empty hot spare pools can be defined in the /etc/opt/SUNWmd/md.tab file, as follows:

hsp001 hsp002

If you are defining an empty hot spare pool, skip attaching hot spares to the hot spare pool and go to Step 3.

2. You can attach hot spares to the hot spare pool by entering the following:

hsp001 /dev/dsk/c0t1d0s0 hsp002 /dev/dsk/c0t2d0s0

3. Associate the hot spare pool with a metadevice that will use the hot spares.

   In this example, the previously defined hot spare pool is associated with both sides of a submirror.

d8 -m d9 d9 1 1 /dev/dsk/c0t2d0s7 -h hsp001 d10 1 1 /dev/dsk/c0t3d0s7 -h hsp002 hsp001 /dev/dsk/c0t1d0s0 hsp002 /dev/dsk/c0t2d0s0

4. Use the metainit command to initialize the two hot spare pools. The submirrors d9 and d10 must also be initialized for the hot spares to take affect.

# /usr/opt/SUNWmd/sbin/metainit hsp001 # /usr/opt/SUNWmd/sbin/metainit hsp002 # /usr/opt/SUNWmd/sbin/metainit d9 # /usr/opt/SUNWmd/sbin/metainit d10 # /usr/opt/SUNWmd/sbin/metainit d8 # metattach d8 d10

Adding Hot Spares to Hot Spare Pools

This section shows how to add one hot spare to the previously defined hot spare pools. In this example, the hot spare /dev/dsk/c0t3d0s0 is added to both hsp001 and hsp002.

* Use the following command to add the hot spare to both hot spare pools:

# /usr/opt/SUNWmd/sbin/metahs -a all /dev/dsk/c0t3d0s0

Using metahs -i, the two hot spare pools, hsp001 and hsp002, would appear as follows:

# /usr/opt/SUNWmd/sbin/metahs -i hsp001: 2 hot spares /dev/dsk/c0t1d0s0 (Available) blocks 263220 /dev/dsk/c0t3d0s0 (Available) blocks 263220 hsp002: 2 hot spares /dev/dsk/c0t2d0s0 (Available) blocks 263220 /dev/dsk/c0t3d0s0 (Available) blocks 263220

Deleting Hot Spares From Hot Spare Pools

This section shows how to delete a hot spare that was previously added to the two hot spare pools. In this example, the hot spare /dev/dsk/c0t3d0s0 is being deleted from both hsp001 and hsp002.

* Use the following command to delete the hot spare from both hot spare pools:

# /usr/opt/SUNWmd/sbin/metahs -d all /dev/dsk/c0t3d0s0

Using metahs -i, the two hot spare pools, hsp001 and hsp002, would appear as follows:

# /usr/opt/SUNWmd/sbin/metahs -i hsp001: 1 hot spare /dev/dsk/c0t1d0s0 (Available) blocks 263220 hsp002: 1 hot spare /dev/dsk/c0t2d0s0 (Available) blocks 263220

Replacing Hot Spares Within Hot Spare Pools

This section shows how to replace a hot spare that was previously associated with a hot spare pool in the /etc/opt/SUNWmd/md.tab file. In this example, the hot spare /dev/dsk/c0t1d0s0 is replaced with /dev/dsk/c0t3d0s0.

* Use the following command to replace a hot spare in a hot spare pool:

# /usr/opt/SUNWmd/sbin/metahs -r hsp001 /dev/dsk/c0t1d0s0 \ /dev/dsk/c0t3d0s0

Using metahs -i, the hot spare pool hsp001 would appear as follows:

# /usr/opt/SUNWmd/sbin/metahs -i hsp001 hsp001: 1 hot spare /dev/dsk/c0t3d0s0 (Available) blocks 263220