Overview of the State Database Replicas

After you have configured metadevices, the metadevice driver must "remember" this configuration and status information. The metadevice state database is the metadevice driver's long-term memory. The metadevice driver stores all the metadevice configuration information in the state database. This includes the configuration information about metadevices, metamirrors, metatrans devices, and hot spares. This is possibly the same information that exists in the /etc/opt/SUNWmd/md.tab and /etc/opt/SUNWmd/md.cf files, but may differ if the /etc/opt/SUNWmd/md.tab file has not been kept up to date or if failed submirror components have been replaced with hot spares.

If the replicated metadevice state database were lost, the metadevice driver would have no way of knowing any configuration information. This could result in the loss of all data stored on metadevices. To protect against losing the metadevice state database because of hardware failures, multiple replicas (copies) of the state database are kept.

These multiple replicas also protect the state database against corruption that can result from a system crash. Each replica of the state database contains a checksum. When the state database is updated, each replica is modified, one at a time. If a crash occurs while the database is being updated, only one of the replicas will be corrupted. When the system reboots, the metadevice driver uses the checksum embedded in the replicas to determine if a replica has been corrupted. Any replicas that have been corrupted are ignored.

If a disk containing the metadevice state database is turned off, the metadevices remain fully functional because the database is retrieved from one of the replicas still in operation. Changes made to the configuration following the reboot are stored only in those replicas that are running when the system comes back up. If the disk drive that was turned off is later turned back on, the data contained in the replica stored on that disk will be ignored.

Basic State Database Operation

The locator block contains the location and status of all known replicas. The primary information contained in the status is whether a particular replica is up to date. A single commit counter is also contained in the locator block. This commit counter is incremented every time the status of any replica changes. The metadevice driver uses this commit counter to select replicas that have the latest locator block. It then uses the data contained in a replica that is both up to date and has not been corrupted by a system crash.

The following are the basic steps a system goes through when the metadevice state database is accessed:

1. The locator block is read from all database replicas that have been patched into /etc/system or passed in from /etc/opt/SUNWmd/mddb.cf.

2. The locator block is read from any replicas mentioned in previously read locator blocks but not patched into /etc/system or passed in from /etc/opt/SUNWmd/mddb.cf by metainit.

3. One of the locator blocks with a commit counter equal to the highest commit counter is used to select which replicas are up to date.

4. One of the up-to-date replicas is read.

5. If the checksum is correct, the state database is set up.

6. If the checksum is not correct, another of the up-to-date replicas is read and tested for the correctness of the checksum. This continues until a correct replica is found.

If there are no more replicas to test for correctness, DiskSuite gives up and an error message is displayed stating that a usable replica cannot be found. If no replicas can be found that pass the checksum test, the database has been lost and you must start over building the initial state database and redefining all metadevices.

If half or more of the replicas referenced in the locator block were not valid, the metadevices are marked read only. One cause of this situation could be that a component was accidentally turned off or disconnected. If this is the case, you must turn on drives that contain other replicas and reboot the system. The replicas located on these components will be found and checked when the system reboots.

It is also possible that half or more of the replicas were lost due to hardware failure. For instance, if there were five replicas and three were lost, DiskSuite will report that the system cannot boot. The references to the three other replicas can be deleted (using the metadb -d command), and the two remaining replicas would be updated to say they are the only two replicas. The system would then boot.

Planning Locations of Replicas

Planning the location of replicas on a system where DiskSuite has just been loaded involves several considerations. These considerations include:

• Database replicas can reside on any unused partition or on any partition which will also be part of a metadevice or logging device with the exception of root, swap, /usr, or an existing file system.
• If multiple controllers exist, replicas should be spread as evenly as possible across the controllers.
• If multiple disks exist on a controller, at least two of the disks on each controller should store a replica of the metadevice state database.
• At least three replicas should be created. For instance, if you have three components, a replica should be created on each. That way, if one component fails, you will have the necessary two replicas to continue running. When you have less than two replicas, DiskSuite will not function.

---

Note - In a two-component configuration, you should always create two replicas on each component. For example, assume you create two replicas on one component and only one replica on the other. If the component with two replicas fails, DiskSuite will not function because the remaining component only has one replica.

---

• No more than one replica should be placed on a single disk unless that is the only way to reach the minimum number (three) of replicas.

Creating a State Database

You create the initial state database by using the metadb command with both the -a and -f options, followed by the device name where the replica is to reside. For example:

# /usr/opt/SUNWmd/sbin/metadb -a -f /dev/dsk/c0t1d0s0

The -a option specifies that a replica (in this case, the initial) state database should be created. When the -a option is used, the /etc/system file is automatically patched with the new information about the state database or replica location and the /etc/opt/SUNWmd/mddb.cf file is updated. The -f option forces the creation to occur, even though a state database does not exist.

The -a and -f options should only be used together when no state databases exist.

There must be at least two replicas located on the system at any time. Otherwise, if the system crashes, it is possible all metadevice configuration data may be lost.

Creating Replicas

Additional replicas, containing identical information, are necessary to prevent the loss of the configuration information. Losing this information would make operation of the metadevices impossible.

You can add replicas from the command line or by editing the /etc/opt/SUNWmd/md.tab file.

When the /etc/opt/SUNWmd/md.tab is edited, a line of the form:

mddb01 /dev/dsk/c0t2d0s0

is entered, and the metadb -a command is run as follows:

# /usr/opt/SUNWmd/sbin/metadb -a mddb01

To create additional replicas from the command line using the metadb -a command, the following would be entered:

# /usr/opt/SUNWmd/sbin/metadb -a /dev/dsk/c0t2d0s0

In either of the above cases, a replica of the state database is created on /dev/dsk/c0t2d0s0.

All replicas that are located on the same partition of a physical device must be created at the same time. Thus, if additional replicas were to be created on /dev/dsk/c0t2d0s0, they would need to be created when the first one is defined. For example:

# /usr/opt/SUNWmd/sbin/metadb -a -c 2 /dev/dsk/c0t2d0s0

The above command would create two replicas on /dev/dsk/c0t2d0s0.

Removing Replicas

With DiskSuite, you can remove all replicas of the state database. Although all replicas of the state database can be removed, this should never be done if metadevices are still configured. If all the replicas are removed, all metadevices will become inoperable.

Removal is done from the command line using the metadb -d command followed by the name of the replica to be removed. Editing the /etc/opt/SUNWmd/md.tab file and removing the entry for a replica does not remove the replica.

For example, if replicas had been set up on /dev/dsk/c0t2d0s0 and /dev/dsk/c0t1d0s0, they could be removed by entering:

# /usr/opt/SUNWmd/sbin/metadb -d /dev/dsk/c0t2d0s0 \ /dev/dsk/c0t1d0s0

If the replicas, /dev/dsk/c0t2d0s0 and /dev/dsk/c0t1d0s0, had been defined in the /etc/opt/SUNWmd/md.tab file and aliased as mddb01, they could also be removed with the following command:

# /usr/opt/SUNWmd/sbin/metadb -d mddb01

Checking the Status of Replicas

You can use metadb to view the status of all replicas by using the -i option. An example of output from the metadb -i command follows:

example# /usr/opt/SUNWmd/sbin/metadb -i flags first blk block count a m p luo 16 1034 /dev/dsk/c0t3d0s3 a p luo 1050 1034 /dev/dsk/c0t3d0s3 a p luo 16 1034 /dev/dsk/c1t3d0s3 a p luo 1050 1034 /dev/dsk/c1t3d0s3 a p luo 16 1034 /dev/dsk/c0t2d0s3 a p luo 1050 1034 /dev/dsk/c0t2d0s3 o - replica active prior to last mddb configuration change u - replica is up to date l - locator for this replica was read successfully c - replica's location was in /etc/opt/SUNWmd/mddb.cf p - replica's location was patched in kernel m - replica is master, this is replica selected as input W - replica has device write errors a - replica is active, commits are occurring to this replica M - replica had problem with master blocks D - replica had problem with data blocks F - replica had format problems S - replica is to small to hold current data base R - replica had device read errors example#

In the example above, the characters in front of the device name represent the status of the state database. A legend follows the replica status.

Examples

The following two examples show how to set up the initial state database and create replicas. The first example describes the procedure for a new system, and the second describes the procedure for an existing system.

State Database Replicas on a New System

This example shows how to set up an initial state database and several replicas on a new system that has been delivered with four disks for user's data.

This example assumes that DiskSuite is installed. It is also assumed that /dev/dsk/c0t0d0s7 and /dev/dsk/c0t1d0s2 will contain all the user data. The other two disks, /dev/dsk/c1t0d0s7 and /dev/dsk/c1t1d0s2, will mirror the disks that contain the user data.

To set up the initial state database follow these steps:

1. Edit the /etc/opt/SUNWmd/md.tab file and add the following line:

mddb01 /dev/dsk/c0t0d0s7 /dev/dsk/c0t1d0s2 /dev/dsk/c1t0d0s7 \ /dev/dsk/c1t1d0s2

2. Use the metadb command with the -a and -f options to activate the state database.

   For example:

# /usr/opt/SUNWmd/sbin/metadb -a -f mddb01

3. Concatenate the components into two submirrors by adding the following lines to the /etc/opt/SUNWmd/md.tab file:

d10 2 1 /dev/dsk/c0t0d0s7 1 /dev/dsk/c0t1d0s2 d11 2 1 /dev/dsk/c1t0d0s7 1 /dev/dsk/c1t1d0s2 d12 -m d10

4. Use the metainit command to initialize the concatenated submirrors and create the mirror.

# /usr/opt/SUNWmd/sbin/metainit d10 # /usr/opt/SUNWmd/sbin/metainit d11 # /usr/opt/SUNWmd/sbin/metainit d12 # /usr/opt/SUNWmd/sbin/metattach d12 d11

State Databases on Existing Systems

This example shows how to add replicas on new disks that have been connected to a system that is currently running DiskSuite. In this example, the system already has been configured as shown in the previous example.

Two drives, /dev/dsk/c0t2d0s2 and /dev/dsk/c1t1d0s2, are being added to the configuration. Two additional replicas are being added to the existing four replicas.

To add the drives and replicas, follow these steps:

1. Edit the /etc/opt/SUNWmd/md.tab file and add the following line:

mddb02 /dev/dsk/c0t2d0s2 /dev/dsk/c1t1d0s2

2. Use the metadb command with only the -a option to activate the replicas. For example:

# /usr/opt/SUNWmd/sbin/metadb -a mddb02

Or, to add the drives and replicas in one step:

* Type the following on the command line:

# /usr/opt/SUNWmd/sbin/metadb -a \ /dev/dsk/c0t2d0s2 /dev/dsk/c1t1d0s2