Using Concatenations and Stripes

Concatenating disk partitions enables you to create a single metadevice with large capacity. This provides a solution for the space limitation a single partition poses. The logical disk block addresses are allocated sequentially to concatenated disk partitions.

You can use a concatenated or striped metadevice for any file system with the exception of root, swap, /usr, /var, /opt, or any other file system accessed during a Solaris upgrade or installation.

Striping performs the same function as concatenation, enabling you to create a single metadevice with large capacity. But striping differs from concatenation because the logical disk block addresses are allocated in an interlaced fashion, which can provide improved performance.

The interlace value for striping is user defined. The interlace size is a number (for example, 8, 16, 32) followed by either k for kilobytes, m for megabytes, or b for disk blocks. The suffix can be either uppercase or lowercase. If the interlace size is not specified, the size defaults to 16 Kbytes. A performance gain occurs when the I/O request is larger than the interlace size, because more disk arms are used to retrieve data or multiple requests spread over more disk arms.

DiskSuite supports concatenations consisting of a single component partition. This is supported to allow temporarily defining single component configuration in anticipation of adding more components in the future. Another use of a single component configuration is to enable mirroring of that component. Additional information on mirroring is provided in Chapter 5, "Mirroring."

After the metadevice is configured, it can be used just as if it were a physical partition. As with physical partitions, a file system can be created on the metadevice. Most UNIX disk utilities will work normally on metadevices, with the exception of format(1M).

In addition, all usual file system operations can be performed on a metadevice. The following list offers examples of file system operations:

• mount(1M) the metadevice on a directory
• umount(1M) a mounted metadevice
• Copy files to the metadevice
• Read and write files from and to the metadevice
• ufsdump(1M) and ufsrestore(1M) the metadevice

Defining Metadevice Configurations

There are two methods of defining either a concatenated or striped metadevice:

• Edit the md.tab file to define the concatenation or stripe, then run the metainit(1M)command; or
• Include the definition of the concatenation or stripe on the command line with the metainit(1M)command.

The information that you include either on the command line or in md.tab will differ depending on whether the configuration is a concatenation or a stripe. For a complete discussion of the md.tab file, refer to Appendix A, "Solstice DiskSuite Files."

A drawback of concatenations, stripes, and concatenated stripes is that the loss of a single physical partition can cause the loss of service to the entire metadevice. This can be solved through mirroring and the use of hot spares. These two topics are discussed in subsequent chapters.

The following sections show how to define concatenations, stripes, and concatenated stripes. Additional step-by-step examples are given in this chapter that show the entire procedure for setting up metadevices.

Concatenated Metadevices

Concatenation can help solve the single partition capacity limitation. For example, a single database could exist on a single metadevice spanning multiple physical partitions.

Figure 4-1 shows two physical components and a metadevice. The physical components are /dev/dsk/c0t0d0s2 and /dev/dsk/c1t1d0s2 and the metadevice is /dev/md/dsk/d7. As illustrated in Figure 4-1, a metadevice is a ''virtual disk'' and can be used like a physical disk partition.

Following the illustration are examples of the information that would be required, either on the command line or in your md.tab file, to define a concatenation of the configuration shown.

Figure 4-1 Concatenated Metadevice

To set up a concatenation of the two physical components shown in Figure 4-1, the following would be added to your md.tab file, or specified directly as arguments to metainit on the command line.

d7 2 1 /dev/dsk/c0t0d0s2 1 /dev/dsk/c1t1d0s2

In the above entry, the number 2 represents the two stripes of devices that are being concatenated together to create the metadevice. In this case, each stripe is made up of one component, thus the number 1 must be displayed in front of each component.

Striped Metadevices

When components are configured into stripes, faster I/O throughput can be obtained. The ideal configuration of stripes would have each component on a separate disk controller. This configuration would deliver the best performance.

A configuration that delivers good performance would have the components on separate disks, but perhaps on the same controller.

---

Note - Striping the partitions on a single disk should be avoided. This will hurt performance.

---

Changing the interlace values of an existing stripe is not supported. In order to change the value, the metadevice must be backed up, redefined with a different value, and restored.

Figure 4-2 shows two physical components and a metadevice. The physical components are /dev/dsk/c0t0d0s2 and /dev/dsk/c1t0d0s2 and the metadevice is d1. The 16 Kbyte stripe defined in this example is shown as a dotted rectangle.

Following the illustration are examples of the information that would be required, either in the /etc/opt/SUNWmd/md.tab file or on the command line as arguments to metainit, to define a stripe of the configuration shown.

Figure 4-2 Striped Metadevice

To create a stripe of the two physical components shown in Figure 4-2, enter the following as arguments to the metainit command on the command line, or add the following to your /etc/opt/SUNWmd/md.tab file.

d1 1 2 /dev/dsk/c0t0d0s2 /dev/dsk/c1t0d0s2 -i 16k

In the above entry, the number 1 represents the one row of devices that is being striped to create the metadevice. In this case, the row is made up of the two components, thus the number 2 must be displayed in front of the two components. The optional -i 16k sets the striped interlace value at 16 kilobytes. If the -i and a value are left off, the interlace size defaults to 16 Kbytes.

Metadevices Defined as Concatenated Stripes

Concatenated stripes enable you to expand an existing stripe and provide some improved performance over simple concatenations.

Figure 4-3 shows three physical components and a metadevice. The physical components are /dev/dsk/c0t0d0s2, /dev/dsk/c1t0d0s2, and /dev/dsk/c2t0d0s2. The metadevice is d1.

In Figure 4-3, the three physical components make up a metadevice that is a concatenation of two stripes. The first stripe (shown as a dotted rectangle) is made up of two physical components and the second stripe is one component.

Following the illustration is an example of the information that would be required, either on the command line or in your /etc/opt/SUNWmd/md.tab file, to define the concatenation of stripes shown.

Figure 4-3 Concatenation of Two Stripes Into a Metadevice

To create a concatenation of two stripes of the three physical components shown in Figure 4-3, add the following to your md.tab file, or specify the following on the command line as arguments to metainit.

d1 2 2 /dev/dsk/c0t0d0s2 /dev/dsk/c1t0d0s2 -i 16k \ 1 /dev/dsk/c2t0d0s2

In the above entry, the first number 2 represents the two rows of devices that are being concatenated to create the metadevice. The first row is made up of /dev/dsk/c0t0d0s2 and /dev/dsk/c1t0d0s2, thus the second number 2 is in front of these component names. The second row is made up of /dev/dsk/c2t0d0s2, which is preceded by the number 1. The optional -i 16k sets the striped interlace value at 16 kilobytes for the first row. Because there is only one component in the second row, there is no interlace value.

If there were more than one component, they would have the same interlace value as the previous row, unless a different value is specified.

Replacing Failed Components

A situation at your site could arise that would require removing a component from a concatenation or a stripe (for example, if a component is reporting soft errors). There are two solutions available for replacing a component:

1. If you already have the metadevice mirrored (described in Chapter 5, "Mirroring"), the faulty component can be replaced by running the metareplace(1M) command. metareplace automatically starts a resync on the new component.

2. If you do not have the metadevice mirrored, use the following basic procedure to replace the failing component (if another disk is already attached to the system):

   a. Unmount the file system (umount), if there is one on the metadevice.

   b. Run the ufsdump command on the metadevice.

   c. Run the metaclear command on the metadevice.

d. Edit the md.tab file to change the name of the failing component in the metadevice. (If the metadevice is a stripe, the new component must be the same size as the failed one. If the metadevice is a concatenation, the new component must have at least the same capacity as the failed component.)

e. Run metainit on the metadevice.

f. Run newfs on the metadevice. g. Run ufsrestore to put the data back online.

Clearing Concatenations and Stripes

If you decide to clear (delete) a concatenation or stripe and use the components in a traditional fashion, the procedure requires very little effort.

---

Warning - Any data that is on the concatenation or stripes will be lost when the metadevice is cleared. The data should be backed up to tape or disk.

---

The procedure for clearing concatenations or stripes is described in the following example. In this example, the metadevice that is being cleared is named d8.

1. Back up the data currently on the components that make up the metadevice (either the stripe or concatenation).

2. Stop access to the data on the metadevice.

   For example, if a file system resides on the metadevice, unmount the file system.

# /sbin/umount /dev/md/dsk/d8

3. Clear the metadevice, using the metaclear command. For example:

# /usr/opt/SUNWmd/sbin/metaclear d8

4. You may want to remove the definition from the

   /etc/opt/SUNWmd/md.tab file. For example, you would remove the following line:

d8 3 1 /dev/dsk/c0t0d0s0 1 /dev/dsk/c0t1d0s0 1 /dev/dsk/c0t2d0s0

Hardware and Software Considerations

There are both hardware and software considerations that affect concatenations and stripes. The only software consideration involves the values assigned to the interlace size when building a stripe.

The hardware considerations include mixing different size components, number of controllers, mixing components with different geometry, and the I/O load on the bus.

Assigning Interlace Values

The key to the improved performance using striping is the interlace value. The value is user configurable at the time a metadevice is created. Thereafter, the value cannot be modified.

The interlace value defaults to 16 Kbytes. This is a reasonable value for most applications. In general, a smaller interlace value increases CPU time while a larger interlace value decreases CPU time.

Mixing Different Size Components

When different size disk components are used in a stripe, some disk space will be unused unless you assign the unused portion to another metadevice.

This is because the stripe is limited by the smallest partition in the configuration (n times the smallest component, where n is the number of components in the stripe). For example, if you have two 327-Mbyte partitions and one 661-Mbyte partition in a stripe the stripe, will only use 327 Mbytes of the space on the 661-Mbyte partition.

To assign the unused disk space to another metadevice, the component must be repartitioned (using format(1M)).

Using Components With Different Geometry

All components in a stripe or concatenation should have the same number of sectors and tracks per cylinder. This is referred to as the disk geometry. The geometry is related to the capacity of the drive. Disk geometry varies depending on the manufacturer and type.

The problem with the differing component geometries is that the UFS file system attempts to lay out file blocks in an efficient manner. UNIX counts on a knowledge of the component geometry and uses cylinder groups to attempt to minimize seek distances. If all components do not have the same geometry, the geometry of the first component is what is reported to the file system. This may cause the efficiency of the file system to suffer.

Controllers

Building a stripe with all the component partitions on the same controller will adversely affect I/O performance. Also, creating a stripe of components on different controller types can affect performance, because some controllers are faster than others. The I/O throughput will be limited by the slowest controller.

An example of a controller limiting performance occurs when several devices (for example, 3-Mbyte per second disks) are attached to the same controller. In this instance, the throughput may be limited to the throughput of the controller and not the sum of the devices.

Bus Load

The data traffic load on the VMEbus or SBus can subtract from the performance gains you receive with the addition of controllers. This can be especially noticeable when stripes are defined. For example, a configuration of four disk controllers all connected to the same VMEbus may not give the maximum speed increase (four times the performance). This is because the bandwidth of the VMEbus might limit the throughput.

An example of this would be a VMEbus system with a throughput of approximately 7 Mbytes per second. A configuration on this system that adds four IPI controllers with a throughput of 6 Mbytes per second each would not increase throughput performance with striping.

However, on another VMEbus system with throughput of 22 Mbytes per second, a striping configuration that includes additional controllers would increase performance.

High transfer rate components on the SBus might encounter similar limitations.

Examples

Examples of three types of configurations are provided in this section.

These examples include:

• Striping three components
• Concatenating eight components
• Concatenating stripes of components

Striping Three Components

The following example shows how to define a stripe of three components. The stripe will have an interlace value of 32 Kbytes.

In the following example, the three components being striped are named /dev/dsk/c0t3d0s7, /dev/dsk/c0t1d0s7, and /dev/dsk/c0t2d0s7. The striped metadevice is named d8.

1. Edit the /etc/opt/SUNWmd/md.tab file, adding a line that defines the striped metadevice.

d8 1 3 /dev/dsk/c0t3d0s7 /dev/dsk/c0t1d0s7 \ /dev/dsk/c0t2d0s7 -i 32k

2. Use metainit -n to verify that the information in the /etc/opt/SUNWmd/md.tab file is accurate.

   The -n option enables you to check your entry.

# /usr/opt/SUNWmd/sbin/metainit -n d8

3. If the configuration is accurate, run metainit to begin using the striped metadevice.

# /usr/opt/SUNWmd/sbin/metainit d8

Concatenating Eight Components

The following example shows how to define a concatenation of eight components.

In the following example, the eight components being concatenated are named

/dev/dsk/c0t0d0s2, /dev/dsk/c0t1d0s2, /dev/dsk/c0t2d0s2,
/dev/dsk/c0t3d0s2, /dev/dsk/c2t0d0s2, /dev/dsk/c2t1d0s2,
/dev/dsk/c2t2d0s2, and /dev/dsk/c2t3d0s2. The metadevice being
defined is named d16.

1. Edit the /etc/opt/SUNWmd/md.tab file, adding the names of the
  components and the name of the concatenated metadevice.

d16 8 1 /dev/dsk/c0t0d0s2 1 /dev/dsk/c0t1d0s2 \ 1 /dev/dsk/c0t2d0s2 1 /dev/dsk/c0t3d0s2 \ 1 /dev/dsk/c2t0d0s2 1 /dev/dsk/c2t1d0s2 \ 1 /dev/dsk/c2t2d0s2 1 /dev/dsk/c2t3d0s2

2. Use metainit -n to verify that the information in the /etc/opt/SUNWmd/md.tab file is accurate.

   Using the -n option enables you to check your entry.

# /usr/opt/SUNWmd/sbin/metainit -n d16

3. If the configuration is accurate, run metainit to begin using the concatenated metadevice.

# /usr/opt/SUNWmd/sbin/metainit d16

Concatenating Stripes of Components

The following example shows how to define a concatenation of two stripes. Each stripe consists of four 1-Gbyte components. The interlace value being assigned in this example is 16 Kbytes.

In the following example, the two sets of four components being striped are named /dev/dsk/c0t0d0s7, /dev/dsk/c0t1d0s7, /dev/dsk/c0t2d0s7,

/dev/dsk/c0t3d0s7, /dev/dsk/c2t0d0s2, /dev/dsk/c2t1d0s2,
/dev/dsk/c2t2d0s2, and /dev/dsk/c2t3d0s2. The striped metadevice
being defined is named d24.

1. Edit the /etc/opt/SUNWmd/md.tab file, adding the names of the
  components and the name of the metadevice.

d24 2 4 /dev/dsk/c0t0d0s7 /dev/dsk/c0t1d0s7 \ /dev/dsk/c0t2d0s7 /dev/dsk/c0t3d0s7 -i 16k \ 4 /dev/dsk/c2t0d0s2 /dev/dsk/c2t1d0s2 \ /dev/dsk/c2t2d0s2 /dev/dsk/c2t3d0s2

2. Use metainit -n to verify that the information in the /etc/opt/SUNWmd/md.tab file is accurate.

   The -n option enables you to check your entry.

# /usr/opt/SUNWmd/sbin/metainit -n d24

3. If the configuration is accurate, run metainit to begin using the concatenated metadevice.

# /usr/opt/SUNWmd/sbin/metainit d24