Device Nodes for Meta-Disks

Here are two examples of physical device nodes for disk devices under Solaris 2.x:

/dev/dsk/c0t0d0s0
                   or
/dev/rdsk/c0t0d0s0

where c references the pln port, t is the SCSI bus within the SSA, d is the SCSI target ID of the disk on that bus, and s is the slice number on the disk. These physical device nodes represent a particular physical path to a partition on a disk in the SSA.

Each SSA adapter port has both a port number (such as c0) and a port name (such as pln2). The port name consists of the port type and instance number. See /etc/path_to_inst. When an SSA is connected to two pln ports, it can potentially be accessed from either path via the physical device node, for example /dev/dsk/c0t0d0s0 or /dev/dsk/c1t0d0s0.

The device node for a meta-disk is derived from the physical device node of the primary path for a pathgroup. Here are two examples of meta-disk device nodes:

/dev/ap/dsk/mc0t0d0s0 and /dev/ap/rdsk/mc0t0d0s0

As you can see, an ap directory has been added, and an m (for "meta") is prepended to the device specification. The device node for a meta-disk has the ability to access the underlying physical disk drive from multiple paths.

Creating Disk Pathgroups and Meta-Disks

Before you can create a disk pathgroup, you must decide which two pln ports are to make up the alternate paths for the pathgroup. You can use the apinst(1M) command to display all pln ports (e.g., pln0 and pln1) and their disk device nodes (e.g., /dev/dsk/c1t0d0 etc.). However, you must know your system hardware configuration in order to recognize when two pln ports are connected to the same SSA. You must specify the pln port names, not the device nodes, when you create and delete disk pathgroups. Here is an apinst example:

# apinst pln0 /dev/dsk/c1t0d0 /dev/dsk/c1t1d0 /dev/dsk/c1t2d0 /dev/dsk/c1t3d0 /dev/dsk/c1t4d0 /dev/dsk/c1t5d0 pln1 /dev/dsk/c2t0d0 /dev/dsk/c2t1d0 /dev/dsk/c2t2d0 /dev/dsk/c2t3d0 /dev/dsk/c2t4d0 /dev/dsk/c2t5d0

In this example, it is assumed that the SSA contains six disks and two SSA ports. One SSA port is connected to pln port c1, and the other SSA port is connected to pln port c2.

To create a meta-disk name that is associated with two physical device paths, use the apdisk(1M) command:

# apdisk -c -p pln0 -a pln1 # apconfig -S -u c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 U mc1t4d0 U mc1t3d0 U mc1t2d0 U mc1t1d0 U mc1t0d0 U

This apdisk(1M) command in the listing above creates a meta-disk name, as well as all of the necessary information in the AP database for maintaining the two alternate paths for all six SSA disks. -c specifies that this information is to be created, while -p specifies the alternate path that will be the primary path and -a specifies the other alternate path. Note that you use the pln port name (such as pln0) rather than the pln port number (such as c1) for these options.

The apconfig -S -u command in the listing above lists the uncommitted meta-disks. -S lists SCSI devices only (that is, disks rather than networks). -u lists uncommitted devices only. The U next to each meta-disk name indicates that the meta-disk entry is uncommitted. The R indicates the root disk (which contains the root file system found under the / directory.)

In the listing above, the P next to pln0 indicates that pln0 is the primary path, and the A indicates that pln0 is the active alternate. Although you can change the active alternate, the primary path always remains constant. The significance of the primary path is that it is initially the active alternate, it is used when the meta-disk is named, and it is used to identify the meta-disk. In this case, c1t0d0 from the primary path name becomes part of mc1t0d0 in the meta-disk name.

Assuming that you are satisfied with the resulting meta-disk, the next step is to commit it to all copies of the AP database by using apdb -C:

# apdb -C # apconfig -S c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 mc1t4d0 mc1t3d0 mc1t2d0 mc1t1d0 mc1t0d0

The apdb -C command commits all uncommitted entries in the AP database. apconfig -S lists the committed SCSI entries in the database. As you can see, the listing appears exactly the same as the previous listing, except the U no longer appears after the meta-disk names, indicating that the meta-disk is no longer uncommitted.

The next step is to create the disk nodes with the drvconfig command:

# drvconfig -i ap_dmd # ls /devices/pseudo/ap_dmd* /devices/pseudo/ap_dmd@0:128,blk /devices/pseudo/ap_dmd@0:128,raw /devices/pseudo/ap_dmd@0:129,blk /devices/pseudo/ap_dmd@0:129,raw /devices/pseudo/ap_dmd@0:130,blk /devices/pseudo/ap_dmd@0:130,raw ...

drvconfig rebuilds the devices directory, which represents the device tree in the kernel. The AP disk meta-driver is a pseudo-device. As you can see from the listing, drvconfig created minor nodes for the alternately pathed device.

The next step is to execute apconfig -R as shown below to create symbolic links from /dev/ap/dsk and /dev/ap/rdsk to the meta-disk special files under /devices/pseudo:

# apconfig -R # cd /dev/ap/dsk # ls -l total 8 lrwxrwxrwx 1 root 40 Jul 27 16:47 mc1t0d0s0 -> ../../../devices/pseudo/ap_dmd@0:128,blk lrwxrwxrwx 1 root 40 Jul 27 16:47 mc1t0d0s1 -> ../../../devices/pseudo/ap_dmd@0:129,blk lrwxrwxrwx 1 root 40 Jul 27 16:47 mc1t0d0s2 -> ../../../devices/pseudo/ap_dmd@0:130,blk

The device nodes that you need -- under /dev/ap/dsk as well as /dev/ap/rdsk -- are now ready to be used. Every reference that uses a physical device node (i.e., a path that begins with /dev/dsk or /dev/rdsk) should now be modified to use the corresponding meta-disk device node. You should check /etc/vfstab for such references. If a partition is currently mounted under a physical device path, you must first unmount it, and then mount it again using the meta-disk device node. If you don't wish to unmount a partition, perhaps because it is heavily used, you could delay placing the partition under AP control until you are ready to bring the system down for maintenance and then reboot. In this scenario, you would modify the /etc/vfstab file so that when the system is rebooted, the partition comes up under an AP device. (If you are placing the boot disk under AP control, you would also need to modify /etc/vfstab using apboot(1M) as described in Chapter 5, "Using AP Boot Devices".)

If you are using the VERITAS Volume Manager (VxVM) in conjunction with AP, see Chapter 8, "VxVM / AP Interaction" for additional steps that you must perform.

Switching Disk Pathgroups

Switching from one path to another is easy. First, use the apconfig -S command to view the current configuration:

# apconfig -S c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 mc1t4d0 mc1t3d0 mc1t2d0 mc1t1d0

As you can see, pln0 is the active alternate since it is followed by an A. (It is also the primary path, since it is followed by a P.)

To perform the switch, use the apconfig(1M) command with -P and -a:

# apconfig -P pln0 -a pln1 # apconfig -S c1 pln0 P c2 pln1 A metadiskname(s): mc1t5d0 mc1t4d0 mc1t3d0 mc1t2d0 mc1t1d0

-P specifies the primary path and thereby identifies the pathgroup for which you wish to change the active alternate. Thus, -P pln0 in the example above identifies the pathgroup for which pln0 is the primary path. -a specifies the alternate that you wish to make active. As you can see from the listing produced by apconfig -S, the active alternate has been switched to pln1. (As mentioned earlier, switching the active alternate does not change the primary path.)

You can perform a switch at any time, even while I/O is occurring on the device. You might want to experiment with the switching process to verify that you understand it and that your system is setup properly, rather than wait until a critical situation occurs.

You could perform a switch back to the original configuration as follows:

# apconfig -P pln0 -a pln0 # apconfig -S c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 mc1t4d0 mc1t3d0 mc1t2d0 mc1t1d0

The first apconfig command, above, says to switch the active alternate to pln0 for the pathgroup whose primary adapter is pln0.

---

Warning - When you switch paths, AP does not check to verify that data can be transferred over the path to which you are switching (although it does determine whether or not that path is detached or offline). You may wish to verify the status of the path before switching to it by performing an I/O operation such as prtvtoc(1M). AP does not produce any error or warning messages if you switch to a path that is not functioning properly. If you switch to a non-functioning path for your boot disk, your system may crash if the path is not switched back immediately.

---

Deleting Disk Pathgroups

To delete a disk pathgroup, use apdisk -d, as shown below. Note that all usage of the meta-disks in the pathgroup must be discontinued (e.g., the file systems must be unmounted) before executing this command.

# apdisk -d pln0 # apconfig -S c1 pln0 P A c2 pln1 metadiskname(s): mc1t5d0 D mc1t4d0 D mc1t3d0 D mc1t2d0 D mc1t1d0 D mc1t0d0 D

If the pathgroup that you delete has not been previously committed, the apdisk -d command deletes it from the database. However, if the pathgroup was previously committed, the apdisk -d command simply marks it as deleted, but the deletion is not completed until the next time you commit the entries in the database. In the example above, the pln0 pathgroup was previously committed, so a D indicates that it is marked for deletion. To commit the entries in the database, use apdb -C:

# apdb -C # apconfig -S

You can undo a deletion if the deletion is uncommitted. To undo a deletion, run apdisk -z, specifying the same pln port that you previously specified.

If you are using the VERITAS Volume Manager (VxVM) in conjunction with AP, see Chapter 8, "VxVM / AP Interaction" for more information related to deleting disk pathgroups.