CHAPTER 5

Hotplugging in the Sun Enterprise Network Array

---

---

Hotplugging Sun Enterprise Network Arrays

The procedure for hotplugging whole Sun Enterprise Network Array enclosures is

very similar to the procedure for removing and replacing individual disk drives.

Instead of specifying an enclosure,dev you only need to specify the enclosure.

For hotplugging Sun Enterprise Network Arrays, use the procedures in "Adding a

Disk Drive" on page 38, "Replacing a Disk Drive" on page 41, and "Removing a

Disk Drive" on page 52 except do not specify a device (dev). You only need to specify

the enclosure.

---

Hotplugging Disks

---

Caution - As with other high-RAS products, drives should not be pulled out

randomly. The Sun Enterprise Network Array design provides support to replace

failed drives in redundant (mirrored or RAID) configurations. If the drive is active,

you must stop that activity before removing it. This can be done without bringing

down the operating system or powering down the unit. The Sun Enterprise Network

Array hardware fully supports hot-plugging, but there are software considerations that must be taken into account. Follow the procedures in this section when removing, replacing, and adding drives.

---

While there is no Solaris^(TM) system software that provides hot-plug functionality to shield the operating system from the physical removal and replacement of a disk drive, there are several scenarios where a disk drive may be safely removed or added to the enclosure as long as the software framework managing the disk drives in the enclosure is taken into account.

Overview

Hot-plug reconfiguration or hot-plug operations cannot be performed on an active disk drive. All disk access activity must be stopped prior to a disk drive being removed or replaced.

In general, hot-plug reconfiguration operations involve three stages:

1. Preparing for hot-plug reconfiguration

2. Adding, replacing, or removing a disk drive

3. Reconfiguring the operating environment.

Three specific cases exist where the hot-plug feature is useful.

• Adding a disk drive to a system to increase storage capacity
• Replacing a faulty disk drive while the system is running:
• Removing a drive from a system that no longer needs it

Adding a Disk Drive

This section contains information on how to configure your system when you add a disk drive while the power is on and the operating system is running.

The way you add a disk drive depends on the application you are using. Each application requires that you decide where to install the new disk drive, add the drive, and then reconfigure the operating environment. Each application is different.

In all cases, you must select a slot, install the disk drive, and configure the Solaris environment to recognize the drive. Then you must configure your application to accept the new disk drive.

---

Caution - These procedures should be performed only by a qualified system

administrator. Performing hot-plug operations on an active disk drive may result in

data loss and/or data corruption.

---

Selecting a Slot for the New Disk Drive

Select any available slot in the Sun Enterprise Network Array for the new disk

drive. For reference when you configure the software environment, make a note of

which enclosure and slot you chose.

Configuring the Solaris Environment

A new device entry needs to be created for the drive in the /devices and /dev/

dsk and /dev/rdsk hierarchy. The new drive is assigned a name associated with

the slot into which the drive was installed.

1. Use the luxadm insert_device command to add the new device.

   This command is interactive. You will be guided through the procedure for inserting

   a new device or chain of devices.

# luxadm insert_device Please hit <enter> when you have finished adding the device(s):

2. You must physically insert the disk drive at this time. After the drive is installed,

   hit Return. The following is displayed:

Waiting for Loop Initialization to complete... New Logical Nodes under /dev/es: New Logical Nodes under /dev/dsk and /dev/rdsk : c1t3d0s0 c1t3d0s1 c1t3d0s2 c1t3d0s3 c1t3d0s4 c1t3d0s5 c1t3d0s6 c1t3d0s7 #

The new disk drive is now available for use as a block or character device. Refer to

the sd(7) man pages for further details.

Configuring the New Disk Drive for a Unix File System (UFS)

Use the following procedure to configure a slice (single physical partition) on a disk to be used with a UFS file system. For instructions about adding a file system to a Volume Manager logical disk, refer to the documentation that came with your application.

1. Verify that the device label meets your requirements.

   You can use the prtvtoc command to inspect the label for your disk. To modify the label, use the format command. Refer to the prtvtoc(1M) and format(1M) man pages for more information.

2. Once you have selected a disk slice for your UFS file system, create a file system on the slice:

# newfs /dev/rdsk/cwtxdysz

Refer to the newfs(1M) man page for more information.

3. If necessary, create a mountpoint for the new file system:

# mkdir mount_point

where: mount_point is a fully qualified pathname. Refer to the mount(1M) man page for more information.

4. After the file system and mountpoint have been created, modify the /etc/vfstab file to reflect the new file system.

   See the vfstab(4) man page for more details.

5. Mount the new file system using the mount command:

# mount mount_point

where: mount_point is the directory you created.

The file system is ready to be used.

Adding the New Device to a Sun Enterprise Volume Manager Disk Group

Use the following procedure to configure the new device to be used with a new or existing Volume Manager disk group.

1. Configure the Volume Manager to recognize the disk drive.

# vxdctl enable

2. Add the new disk to a new or existing Volume Manager disk group:

# vxdiskadd cwtxdysz

where: cwtxdysz is the new disk.

Refer to the vxdiskadd(1M) man page for further details.

The disk is now ready for use with Volume Manager: as part of a new volume, added to an existing volume as a plex, or to increase an existing volume. Refer to your Sun Enterprise Volume Manager User's Guide for more information.

Replacing a Disk Drive

This chapter contains information on configuring your system to replace a disk drive while the power is on and the operating system is running.

The way you replace a faulty disk drive depends on the application you are using. Each application is different, but requires that you

1. Determine which disk drive is failing or has failed

2. Remove the disk

3. Add the replacement drive

4. Reconfigure the operating environment.

In all cases you must stop any activity on the disk; physically remove the old drive and install the new one; and configure the Solaris environment to recognize the drive. Then you must configure your application to accept the new disk drive.

---

Caution - These procedures should be performed only by a qualified system

administrator. Performing hot-plug operations on an active disk drive may result in

data loss and/or data corruption.

---

Identifying the Faulty Disk Drive

Different applications provide various levels of error logging. In general, you can

find messages about failing or failed disks in your system console window. The

information is also logged in the /usr/adm/messages file(s). See the

documentation that came with your application for more information.

UNIX File System (UFS)

The following procedure describes how to unconfigure a disk being used by one or

more UFS file systems.

---

Caution - These procedures should be performed only by a qualified system

administrator. Performing hot-plug operations on an active disk drive can result in

data loss.

---

Preparing to Replace the Disk Drive

1. Stop any application processes on the file systems to be unconfigured.

2. Back up your system.

3. Determine what file system(s) are on the disk:

# mount | grep cwtxdysz

For example, if the device to be removed is c1t3d0, enter the following:

# mount | grep c1t3d0 /export/home (/dev/dsk/c1t3d0s7 ): 98892 blocks 142713 files /export/home2 (/dev/dsk/c1t3d0s5 ): 153424 blocks 112107 files

4. Determine and save the partition table for the disk.

If the replacement disk is the same type as the faulty disk, you can use the format command to save the partition table of the disk. Use the save command in format to save a copy of the partition table to the /etc/format.dat file. This will allow you to configure the replacement disk so that its layout matches the current disk.

Refer to the format(1M) man page for more information.

5. Unmount any file systems on the disk.

   Note - If the file system(s) are on a disk that is failing or has failed, the umount operation may not complete. A large number of error messages may be displayed in the system console and in the /var directory during the umount operation. If the umount operation does not complete, you may have to reboot the system.

   For each file system from Step 3 returned, type:

# umount filesystem

where filesystem is the first field for each line returned in Step 3.

For example:

# umount /export/home # umount /export/home2

6. Using the df command, verify that the file system has been unmounted.

Removing the Disk Drive

1. Use the luxadm remove_device command to remove the disk.

   The luxadm remove_device command is interactive. You will be guided through the procedure for removing a device or chain of devices.

# luxadm remove_device /dev/rdsk/c1t3d0 WARNING!!! Please ensure that no filesystems are mounted on these device(s). All data on these devices should have been backed up. The list of devices which will be removed is: 1: Box Name "macs3" rear slot 1 Please enter 'q' to Quit OR <Return> to Continue: offlining: Drive in "macs3" rear slot 1 Hit any key after inserting/removing drives:

2. You must physically remove the disk drive at this time. After the drive is removed, hit any key. The following is displayed:

---

Note - The yellow LED on the designated disk drive(s) should be flashing.

---

Drive in Box Name "macs3" rear slot 1 Removing Logical Nodes: Removing /dev/dsk/c1t3d0s0 Removing /dev/rdsk/c1t3d0s0 Removing /dev/dsk/c1t3d0s1 Removing /dev/rdsk/c1t3d0s1 Removing /dev/dsk/c1t3d0s2 Removing /dev/rdsk/c1t3d0s2 Removing /dev/dsk/c1t3d0s3 Removing /dev/rdsk/c1t3d0s3 Removing /dev/dsk/c1t3d0s4 Removing /dev/rdsk/c1t3d0s4 Removing /dev/dsk/c1t3d0s5 Removing /dev/rdsk/c1t3d0s5 Removing /dev/dsk/c1t3d0s6 Removing /dev/rdsk/c1t3d0s6 Removing /dev/dsk/c1t3d0s7 Removing /dev/rdsk/c1t3d0s7 #

Installing the New Disk Drive

1. Use the luxadm insert_device command to add the new device.

   This command is interactive. You will be guided through the procedure for inserting a new device or chain of devices.

# luxadm insert_device Please hit <enter> when you have finished adding the device(s):

2. You must physically insert the disk drive at this time. After the drive is installed, hit Return. The following is displayed:

Waiting for Loop Initialization to complete... New Logical Nodes under /dev/es: New Logical Nodes under /dev/dsk and /dev/rdsk : c1t3d0s0 c1t3d0s1 c1t3d0s2 c1t3d0s3 c1t3d0s4 c1t3d0s5 c1t3d0s6 c1t3d0s7 #

The new disk drive is now available for use as a block or character device. Refer to the sd(7) man pages for further details.

Restoring the UFS File System

Use the following procedure to configure a slice on a disk to be used with the UFS file system.

1. Verify that the device label meets your requirements.

   You can use the prtvtoc command to inspect the label for your disk. To modify the label, use the format command. See the prtvtoc(1M) and format(1M) man pages for more information.

2. Verify that the device's partition table satisfies the requirements of the file system(s) you intend to re-create.

   You can use the prtvtoc command to inspect the label for your device. If you need to modify the label, use the format command. Refer to the prtvtoc(1M) and format(1M) man pages for more information.

   For example:

# prtvtoc /dev/rdsk/cwtxdysz

If you have saved a disk partition table using the format utility and the replacement disk type matches the old disk type, then you can use the format utility's partition section to configure the partition table of the replacement disk. See the select and label commands in the partition section.

If the replacement disk is of a different type than the disk it replaced, you can use

the partition size information from the previous disk to set the partition table for the

replacement disk. Refer to the prtvtoc(1M) and format(1M) man pages for more

information.

3. Once you have selected a disk slice for your UFS file system, create a file system

   on the slice:

# newfs /dev/rdsk/cwtxdysz

4. Mount the new file system using the mount command:

# mount mount_point

where: mount_point is the directory on which the faulty disk was mounted.

The new disk is ready to be used. You can now restore data from your

backups.

Sun Enterprise Volume Manager

The following procedure assumes that all user- and application-level processes on all

volumes, plexes, and/or subdisks that are located on the drive to be removed have

been terminated.

---

Caution - These procedures should be performed only by a qualified system

administrator. Performing hot-plug operations on an active disk drive may result in

data loss.

---

Preparing to Replace the Disk Drive

1. Back up your system.

   Refer to the documentation that came with your system for backup details.

2. Identify the disk media name for the disk you intend to replace.

# vxdisk list | grep cwtxdysz

For example, if the disk to be removed is c0t3d0, enter:

# vxdisk list | grep c0t3d0 c0t3d0s2 sliced disk01 rootdg online

The disk media name is the third field in the output above: disk01.

You can use the vxdiskadm utility to prepare the disk for replacement.

3. Type vxdiskadm in a shell.

   For example:

# vxdiskadm

This operation is interactive and requires user confirmation of the operation.

4. Select the "Remove a disk for replacement" option.

   When prompted for a disk name to replace, type the disk media name from Step 2.

   vxdiskadm marks the disk for replacement and saves the subdisk information to be rebuilt on the replacement disk.

   Redundant data is automatically recovered after the replacement disk has been reattached to Volume Manager. Non-redundant data is identified as unusable and must be recreated from backups.

   Refer to the vxdiskadm(1M) man page for further details.

5. Quit the vxdiskadm utility.

Removing the Disk Drive

1. Use the luxadm remove_device command to remove the disk.

   The luxadm remove_device command is interactive. You will be guided through the procedure for removing a device or chain of devices.

# luxadm remove_device /dev/rdsk/c1t3d0 WARNING!!! Please ensure that no filesystems are mounted on these device(s). All data on these devices should have been backed up. The list of devices which will be removed is: 1: Box Name "macs3" rear slot 1 Please enter 'q' to Quit OR <Return> to Continue: offlining: Drive in "macs3" rear slot 1 Hit any key after inserting/removing drives:

2. You must physically remove the disk drive at this time. After the drive is removed, hit any key. The following is displayed:

---

Note - The yellow LED on the designated disk drive(s) should be flashing.

---

Drive in Box Name "macs3" rear slot 1 Removing Logical Nodes: Removing /dev/dsk/c1t3d0s0 Removing /dev/rdsk/c1t3d0s0 Removing /dev/dsk/c1t3d0s1 Removing /dev/rdsk/c1t3d0s1 Removing /dev/dsk/c1t3d0s2 Removing /dev/rdsk/c1t3d0s2 Removing /dev/dsk/c1t3d0s3 Removing /dev/rdsk/c1t3d0s3 Removing /dev/dsk/c1t3d0s4 Removing /dev/rdsk/c1t3d0s4 Removing /dev/dsk/c1t3d0s5 Removing /dev/rdsk/c1t3d0s5 Removing /dev/dsk/c1t3d0s6 Removing /dev/rdsk/c1t3d0s6 Removing /dev/dsk/c1t3d0s7 Removing /dev/rdsk/c1t3d0s7 #

Installing the Disk Drive

1. Use the luxadm insert_device command to add the new device.

   This command is interactive. You will be guided through the procedure for inserting a new device or chain of devices.

# luxadm insert_device Please hit <enter> when you have finished adding device(s):

2. You must physically insert the disk drive at this time. After the drive is installed, hit Return. The following is displayed:

Waiting for Loop Initialization to complete... New Logical Nodes under /dev/es: New Logical Nodes under /dev/dsk and /dev/rdsk : c1t3d0s0 c1t3d0s1 c1t3d0s2 c1t3d0s3 c1t3d0s4 c1t3d0s5 c1t3d0s6 c1t3d0s7 #

The new disk drive is now available for use as a block or character device. Refer to the sd(7) man pages for further details.

Recreating a Volume Manager Configuration on the New Drive

To recreate the replaced disk on the new drive:

1. Configure the Volume Manager to recognize the disk drive.

# vxdctl enable

2. Use the vxdiskadm utility to replace the failed disk.

   Select the "Replace a failed or removed disk" option.

   This operation requires user confirmation. When prompted for a disk name to replace, use the disk media name from Step 2 of "Preparing to Replace the Disk Drive" on page 47.

   vxdiskadm supplies a list of available disks to be used as replacements.

3. Select the replacement drive.

   vxdiskadm automatically configures the replacement drive to match the failed drive.

   Redundant data is recovered automatically. Space for non-redundant data is created and identified. Non-redundant data must be recovered from backing store.

   See the vxdiskadm man pages for further details.

You have now completed the replacement of the failed drive.

Removing a Disk Drive

This chapter contains information on how to configure your system to remove a disk

drive while the power is on and the operating system is running. Use the procedures

in this chapter if you do not intend to replace the disk drive.

The way in which you prepare to remove a disk drive depends on the application

you are using. Each application is different, but requires that you

1. Select the disk drive

2. Remove the disk

3. Reconfigure the operating environment.

In all cases you must select the disk and stop any activity or applications on it,

unmount it, physically remove the drive, and configure the Solaris environment to

recognize that the drive is no longer there. Then you must configure your

application to operate without this device in place.

---

Caution - These procedures should be performed only by a qualified system

administrator. Performing hot-plug operations on an active disk drive may result in

data loss and/or data corruption.

---

Unix File System (UFS)

The following procedure describes how to unconfigure a disk being used by one or

more UFS file systems.

---

Caution - These procedures should be performed only by a qualified system

administrator. Performing hot-plug operations on an active disk drive may result in

data loss.

---

1. Stop any application processes on the file systems to be unconfigured.

2. Back up your system.

3. Determine what file system(s) are on the disk:

# mount | grep cwtxdysz

For example, if the device to be removed is c1t3d0, enter the following:

# mount | grep c1t3d0 /export/home (/dev/dsk/c1t3d0s7 ): 98892 blocks 142713 files /export/home2 (/dev/dsk/c1t3d0s5 ): 153424 blocks 112107 files

4. Unmount any file systems on the disk.

   Note - If the file system(s) are on a disk that is failing or has failed, the umount operation may not complete. A large number of error messages may be displayed in the system console and in the /var directory during the umount operation. If the umount operation does not complete, you may have to restart the system.

   For each file system returned, type:

# umount filesystem

where: filesystem is the first field for each line returned in Step 3.

For example:

# umount /export/home # umount /export/home2

Removing the Disk Drive

1. Use the luxadm remove_device command to remove the disk.

   The luxadm remove_device command is interactive. You will be guided through the procedure for removing a device or chain of devices.

# luxadm remove_device /dev/rdsk/c1t3d0 WARNING!!! Please ensure that no filesystems are mounted on these device(s). All data on these devices should have been backed up. The list of devices which will be removed is: 1: Box Name "macs3" rear slot 1 Please enter 'q' to Quit OR <Return> to Continue: offlining: Drive in "macs3" rear slot 1 Hit any key after inserting/removing drives:

2. You must physically remove the disk drive at this time. After the drive is removed, hit any key. The following is displayed:

---

Note - The yellow LED on the designated disk drive(s) should be flashing.

---

Drive in Box Name "macs3" rear slot 1 Removing Logical Nodes: Removing /dev/dsk/c1t3d0s0 Removing /dev/rdsk/c1t3d0s0 Removing /dev/dsk/c1t3d0s1 Removing /dev/rdsk/c1t3d0s1 Removing /dev/dsk/c1t3d0s2 Removing /dev/rdsk/c1t3d0s2 Removing /dev/dsk/c1t3d0s3 Removing /dev/rdsk/c1t3d0s3 Removing /dev/dsk/c1t3d0s4 Removing /dev/rdsk/c1t3d0s4 Removing /dev/dsk/c1t3d0s5 Removing /dev/rdsk/c1t3d0s5 Removing /dev/dsk/c1t3d0s6 Removing /dev/rdsk/c1t3d0s6 Removing /dev/dsk/c1t3d0s7 Removing /dev/rdsk/c1t3d0s7 #

Sun Enterprise Volume Manager

The following procedure assumes that all user- and application-level processes on all

volumes, plexes, and/or subdisks that are located on the drive to be removed have

been terminated.

---

Caution - These procedures should be performed only by a qualified system

administrator. Performing hot-plug operations on an active disk drive may result in

data loss.

---

Preparing to Remove the Disk Drive

1. Back up your system.

2. Identify the disk media name for the disk you intend to remove.

# vxdisk list | grep cwtxdysz

For example, if the disk to be removed is c0t3d0, enter:

# vxdisk list | grep c0t3d0 c0t3d0s2 sliced disk01 rootdg online

The disk media name is the third field in the output above: disk01.

You can use the vxdiskadm utility to prepare the disk for removal.

3. Type vxdiskadm in a shell.

   For example:

# vxdiskadm

This operation is interactive and requires user confirmation of the operation.

4. Select the "Remove a disk" option.

   When prompted for a disk name to remove, type the disk media name from Step 2.

   vxdiskadm marks the disk to be removed.

   Refer to the vxdiskadm(1M) man page for further details.

Removing the Disk Drive

1. Use the luxadm remove_device command to remove the disk.

   The luxadm remove_device command is interactive. You will be guided through the procedure for removing a device or chain of devices.

# luxadm remove_device /dev/rdsk/c1t3d0 WARNING!!! Please ensure that no filesystems are mounted on these device(s). All data on these devices should have been backed up. The list of devices which will be removed is: 1: Box Name "macs3" rear slot 1 Please enter 'q' to Quit OR <Return> to Continue: offlining: Drive in "macs3" rear slot 1 Hit any key after inserting/removing drives:

2. You must physically remove the disk drive at this time. After the drive is removed, hit any key. The following is displayed:

---

Note - The yellow LED on the designated disk drive(s) should be flashing.

---

Drive in Box Name "macs3" rear slot 1 Removing Logical Nodes: Removing /dev/dsk/c1t3d0s0 Removing /dev/rdsk/c1t3d0s0 Removing /dev/dsk/c1t3d0s1 Removing /dev/rdsk/c1t3d0s1 Removing /dev/dsk/c1t3d0s2 Removing /dev/rdsk/c1t3d0s2 Removing /dev/dsk/c1t3d0s3 Removing /dev/rdsk/c1t3d0s3 Removing /dev/dsk/c1t3d0s4 Removing /dev/rdsk/c1t3d0s4 Removing /dev/dsk/c1t3d0s5 Removing /dev/rdsk/c1t3d0s5 Removing /dev/dsk/c1t3d0s6 Removing /dev/rdsk/c1t3d0s6 Removing /dev/dsk/c1t3d0s7 Removing /dev/rdsk/c1t3d0s7 #