Sun and OracleChannel SunHow to BuyLog In日本語

SPARCcluster HA Server Software Planning and Installation Guide
この本のみを検索
検索のヘルプ
内に含ま
その他のドキュメント
サポート リソース
 PDF 文書ファイルをダウンロードする

Setting Up Data Service Classes

10

This chapter gives instructions for setting up the Solstice HA data service classes. Use the following table to locate specific information in this chapter.
Overview of Taskspage 10-1
Setting Up HA-ORACLEpage 10-2
Setting Up HA-NFSpage 10-10
Bringing the Solstice HA Servers On Linepage 10-16

10.1 Overview of Tasks

When the Solstice HA configuration was created by hasetup(1M), you were asked which data services you were going to be running on the configuration. The valid data services available for Solstice HA include:
  • HA-NFS only
  • HA-ORACLE only
  • Both HA-NFS and HA-ORACLE
Note - If you did not configure the desired data services when you were setting up the configuration, you must remove the /etc/opt/SUNWhadf/hadf/hadfconfig file and rerun hasetup. hasetup will exit when you reach the diskset allocation part of the program.
If you are running both HA-NFS and HA-ORACLE in your Solstice HA configuration, the two data services can be set up in any order.
Note - Regardless of what data servers are run in your configuration, you must perform the tasks in Section 10.4, "Bringing the Solstice HA Servers On Line," to start Solstice HA.

10.2 Setting Up HA-ORACLE

Before setting up HA-ORACLE you must have performed the following procedures on each Solstice HA server:
  1. Install the Solaris 2.4 Hardware: 3/95 distribution

  2. Install the Solstice HA packages and patches

  3. Reboot

Since some of the changes made during HA-ORACLE setup require a reboot, you may want to install ORACLE now before rebooting. Alternatively the ORACLE installation may be deferred until just before HA-ORACLE database setup. However, this may require an additional reboot.
CAUTION Caution - In the "How to Prepare Solstice HA Servers for ORACLE Installation" on page 10-2, all the steps should be performed on both Solstice HA servers.

· How to Prepare Solstice HA Servers for ORACLE Installation

You should consult your ORACLE documentation before performing this procedure.
  1. Create a /etc/passwd entry for the ORACLE user ID (oracle_id). This entry is normally oracle for the user ID. For example, add the following line:


  oracle:*:35:520:Oracle Database:/oracle/home:
  1. Update the /etc/shadow file from /etc/passwd by using the pwconv(1M) command. 


  # pwconv
  1. Make a /etc/group entry for the database administrator group. The group is normally named dba. You must ensure that root and oracle are members of the dba group. For example, add the following line:


  dba:*:520:root,oracle
While the name service entries may be made in a network name service (for example, NIS or NIS+) so the information is available to HA-ORACLE clients, entries should also be made in the local /etc files to eliminate dependency on the network name service.
Note - This duplicate information must be kept consistent when changes are made.
  1. Prepare the environment for ORACLE installation.

    Choose a location for the ORACLE_HOME and ORACLE_DOC directories. If possible, the ORACLE distribution should be installed on a separate disk and mounted as /oracle. This is the location where the ORACLE distribution should be installed.

    When they are created, the ORACLE_HOME and ORACLE_DOC directories should be owned by oracle_id and be in the dba_id group.

  2. Requirements for ORACLE installation.

    When first installing ORACLE, you have the option of doing a COMPLETE SOFTWARE/DATABASE FRESH INSTALL or Install/Upgrade/Patch Software Only. You should select the Install/Upgrade/Patch Software Only option. This is necessary because database initialization and configuration files must be modified to reflect the multi-host disks as the location for the database. Having the database reside on the multi-host disks ensures high availability of the database during a cluster reconfiguration.

Note - If you choose the COMPLETE SOFTWARE/DATABASE FRESH INSATALLATION to install the software, you will not have the opportunity to modify the init.ora file before the database is created.
  1. Install the ORACLE software.

    For complete instructions on installing ORACLE, refer to the ORACLE7 Installation and Configuration Guide and the Oracle7 for Sun SPARC Solaris 2.x Installation and Configuration Guide.

Note - If possible, the ORACLE software distribution should occupy its own separate file system. This would prevent ORACLE from being overwritten if the operating system has to be reinstalled.
  1. Remember to create the /opt/bin directory as root. This is the directory where some ORACLE binaries created by the root.sh script will reside. For example:


  # mkdir -p /opt/bin
  1. Remember to run the ORACLE root.sh script. The $ORACLE_HOME/orainst/root.sh script is created during the ORACLE installation procedure. This script makes certain programs setuid and sets necessary permissions for ORACLE products. Log in to the server as oracle, then become root before running the script.


  # $ORACLE_HOME/orainst/root.sh
  1. Set up the SQL*Net V1/TCP listener by making an entry in

    /etc/inet/services on both Solstice HA servers. Modify the /etc/inet/services to include an entry for the SQL*Net V1/TCP listener. An example entry would be:


  oraserv 1525/tcp oracle
Note - This entry is required on Solstice HA servers even if you are using the SQL*Net V2 listener.
  1. Establish means to start the ORACLE SQL*Net V1/TCP listener.

    One method that can be used is to create a script in the /etc/rc3.d directory to start the listener. You must start the listener with the opsrooton option if you want to use Solaris authentication.

10.2.1 Creating an ORACLE Database and Setting Up HA-ORACLE

If Solstice HA is not yet running on your servers, you must now manually mount all file systems before creating an ORACLE database and setting up HA-ORACLE.
Follow both the following procedures to create and set up the initial ORACLE database in a Solstice HA configuration. If you are creating and setting up subsequent databases, only perform the procedure in "How to Create an ORACLE Database and Set Up HA-ORACLE" on page 10-6.

· How to Prepare the Logical Hosts for ORACLE Databases

  1. Take ownership of the disksets.

    hasetup releases the disksets upon completion so before you can perform actions on the disksets you must take ownership.

  2. Set up raw mirrored metadevices on both servers.

    If you will be using raw mirrored metadevices to contain the databases you will want to change the owner, group, and mode of each of the raw mirrored metadevices. If you are not using raw mirrored metadevices, skip to Step 3. Instructions for creating mirrored metadevices are provided in Chapter 9, "Metadevice Creation on Local and Multi-host Disks." If you are creating raw mirrored metadevices, you must run the following commands for each metadevice.


  # chown oracle_id /dev/md/logicalhost/rdsk/dn  
  # chgrp dba_id /dev/md/logicalhost/rdsk/dn  
  # chmod 600 /dev/md/logicalhost/rdsk/dn
CAUTION Caution - While ORACLE supports raw I/O to both raw physical devices and raw metadevices (mirrored or non-mirrored), Solstice HA only supports raw ORACLE I/O on raw mirrored metadevices. You may not use devices such as /dev/rdsk/c1t1d1s2 to contain ORACLE data under Solstice HA.
  1. Configure the logging UFS file systems.

    Regardless of whether the databases will reside on raw mirrored metadevices or on UFS file systems, at least one logging UFS file system must be created to contain the ORACLE configuration files associated with the databases. You may want to create additional logging UFS file systems to contain your ORACLE databases. Each of these UFS file systems must have an entry in the /etc/opt/SUNWhadf/hadf/vfstab.logicalhost file.

    Use the procedure in "How to Set Up a Multi-host UFS File System" on page 10-13 to set up the UFS file systems and vfstab.logicalhost files.

  2. Mount the UFS file systems.

    Use the mount(1M) command to mount all UFS file systems in Step 3.

· How to Create an ORACLE Database and Set Up HA-ORACLE

  1. Prepare database configuration files.

    By default database configuration files are created in $ORACLE_HOME/dbs. It is essential that these files do not reside in the default location; they must reside on the multi-host file system. Configuration file location can be forced during database creation by explicitly assigning the various ORACLE_SID specific files and directories in the init.ora file used to build the database. The same assignments should be used in all init.ora files used with the database (for example, those used for database building and running).

    Within the init.ora file, the assignments include the file name assignments for control_files, background_dump_dest, core_dump_dist, log_archive_dest, and user_dump_dest. The only ORACLE_SID specific file that should reside in $ORACLE_HOME/dbs is the sgadef$ORACLE_SID.dbf file which is recreated at each database start up. HA-ORACLE will start the database using the pfile specified in the haoracle_databases file; this pfile is on the multi-host file system and it in turn assigns the other configuration files to multi-host paths.

Note - If you are using Solaris authentication for database logins, the remote_os_authent variable in the init.ora file must be set to TRUE.
  1. Create the database.

    During creation, ensure that all configuration and database files are placed on the multi-host disks.

    Start the ORACLE installer (orainst) and select the Create New Database Objects option. During the orainst session, place all the database files on the multi-host disks. Be sure to override the default file locations provided by the ORACLE installer.

    Ensure that the file names of the control files you specify match the file names in your configuration files.

    Alternatively, the database can be created through the ORACLE sqldba command. Two points to consider if you are using sqldba are:

  • All of the *.ora files must be placed on the logicalhost.
  • You must also run the catalog scripts that create the v_$sysstat table. This table is used by the Solstice HA fault monitoring scripts.
  1. Make entries for the SIDs of all database instances. The SID of the instance associated with your database must be in the file /var/opt/oracle/oratab on both Solstice HA servers. This file must be kept current on both Solstice HA servers for a failover to succeed (the HA-ORACLE databases, that is SIDs, must exist in /var/opt/oracle/oratab on both servers). This must be done manually as SIDs are added or removed.

    All entries in the /var/opt/oracle/oratab file should have the :N option specified to ensure that the instance will not automatically start.

Note - The HA-ORACLE fault monitoring scripts parse the init.ora files. This script assumes a white space appears on either side of the = sign. While ORACLE does not require the spaces, HA-ORACLE does.
  1. Enable access for the user and password to be used for fault monitoring.

    In the following examples the user is scott and the password is tiger. Note that the user and password pair must agree with those used in Step 6 if you are using ORACLE authentication.

You can enable access by entering the following script into the sqldba(1M) command.

  # sqldba lmode=y <<EOF  
  
     connect internal;  
         grant connect, resource to scott identified by tiger;  
         grant select on v_$sysstat to scott;  
         grant create session to scott;  
         grant create table to scott;  
         alter user scott quota 1m on system;  
     disconnect;  
  
     exit;  
  EOF  
  #
  1. Optionally grant permission for the database to use Solaris authentication.

    The following example entry will enable you to use Solaris authentication.


  # sqldba lmode=y <<EOF  
  
     connect internal;  
        create user ops$root identified by externally;  
        grant select on v_$sysstat to ops$root;  
        grant create session to ops$root;  
        grant create table to ops$root;  
        alter user ops$root quota 1M on system;  
     disconnect;  
  
     exit;  
  EOF  
  #
  1. Update the haoracle_databases(4) file.

    The haoracle_databases files must be updated using the haoracle insert command, after database creation. This file is owned by root and has a mode of 600 to protect the information.


  # haoracle insert $ORACLE_SID logicalhost 60 10 120 300 scott/tiger  \  
            /logicalhost1/.../init$ORACLE_SID.ora
The above command line includes the following:
  • haoracle insert - The command and subcommand
  • $ORACLE_SID - The name of the ORACLE database instance.
  • logicalhost1 - The logical host serving $ORACLE_SID (not the physical host)
  • 60 10 120 300 - These parameters are for probe cycle time of 60 seconds, a connectivity probe cycle count of 10 seconds, a probe time out of 120 seconds, and a restart delay of 300 seconds.
  • scott/tiger - The user and password to be used for fault monitoring. This must agree with the permissions granted in Step 4. To use Solaris authentication enter a / instead of the user name and password.
  • /logicalhost1/.../init$ORACLE_SID.ora - The pfile to be used to start up the database. This must be on a multi-host logging UFS file system. It should not be available via HA-NFS. The init$ORACLE_SID.ora file is the init.ora file referred to in Step 1.
  1. Verify the changes.

    Run the hacheck(1M) command on one of the servers to check the changes. This command will validate the changes made to the haoracle_databases file.


  # hacheck
  1. Bring the HA-ORACLE database into service.

    If the hacheck(1M) command reports no problems you can bring the HA-ORACLE database into service. You can bring the database into service by running haoracle(1M).


  # haoracle start database_instance
CAUTION Caution - From the time the ORACLE database instance is brought into service, the monitoring should be enabled.
If you have not started Solstice HA but have manually taken ownership of the diskset and mounted the file systems, you should unmount the file systems and release the disksets before starting Solstice HA.
HA-ORACLE is now ready to begin servicing HA-ORACLE clients.

10.2.2 HA-ORACLE Client Set Up

No special configuration should be necessary for HA-ORACLE clients.
Note - ORACLE client-server connections are stateful and will not survive an HA-ORACLE switchover. The client application must be prepared to cope with disconnects and reconnect or recover as appropriate. A transaction monitor may simplify the application. Further, HA-ORACLE server recovery time is application dependent.
Clients must always refer to the database using the logical hostname and not the physical hostname. Except for start-up times the database should always be available if the logical host is responding on the network.

10.3 Setting Up HA-NFS

The instructions in this subsection assume you have performed all file system planning (see Chapter 3, "Installation Planning"), created the disksets, and set up the trans (UFS logging) devices. These instructions show how to set up HA-NFS in a configuration where both logical hosts offer HA-NFS services.
Note - When setting up HA-NFS, plan to have an extra multi-host file system that is not shared under NFS. This file system will be used for the Solstice HA locking and status monitor data. This file system must be mounted on /logicalhost and should have at least 10 Mbytes of space, which is sufficient to provide locking for at least 1,000 NFS clients.
You will use the hafstab(1M) command to edit the logical host's vfstab.logicalhost and dfstab.logicalhost files. You run this command on only one server. The files are automatically distributed to the other server.

· How to Share HA-NFS File Systems

  1. Create your multi-host UFS file systems.

    Use the procedure in "How to Set Up a Multi-host UFS File System" on page 10-13 to create the multi-host file system.

  2. Use the hafstab command on one Solstice HA server to edit the dfstab.logicalhost files.

    The dfstab.logicalhost file is in dfstab format. The hafstab command allows editing the dfstab files then distributes the file to both servers in the configuration. The editor defined as the EDITOR environment variable is used. A template of the dfstab file is presented for editing if no version of the file exists.

    Start hafstab with the following argument and respond to the prompts.


  host1# hafstab dfstab.logicalhost1  
  Failed to copy host2 version.  
  Type 'y' to edit host1 version; type 'n' to exit. [y|n] y
In the following example, we will not export the file system on d0.
After the editor begins, the following dfstab template file is displayed:

  #  
  #ident "@(#)dfstab.tmpl   1.10     95/07/26 SMI"  
  #  
  # Copyright 26 Jul 1995 Sun Microsystems, Inc.  All Rights Reserved.  
  #  
  # This file must be replicated on both servers of the HADF configuration;  
  # use hafstab(1M) to edit and distribute.  
  #  
  #       Place the share(1M) commands at the end of this file.  
  #  
  #       share [-F fstype] [ -o options] [-d "<text>"] <pathname> [resource]  
  #       .e.g,  
  #       share  -F nfs  -o rw=engineering  -d "home dirs"  /export/home2  
  #  
  #       Notes: If you use the "rw", "rw=", "ro", and/or "ro=" options to the  
  #       share command, then HA-NFS fault monitoring will work best if you  
  #       grant access to both of the HA server hosts, to ALL of their  
  #       hostnames.  Look at your hadfconfig file,  
  #       /etc/opt/SUNWhadf/hadf/hadfconfig.  For all of the HOSTNAME  
  #       lines, all of the physical hostnames, which are the 2nd column and  
  #       4th column of the HOSTNAME line(s), should be granted access.  
  #       If you use netgroups in the share command (rather than names of  
  #       individual hosts) please add all of those HA server hostnames to  
  #       the appropriate netgroup.  
  #       Also, ideally, both read and write access should be granted  
  #       to all of the HA server hosts' hostnames, to enable the NFS fault  
  #       probes do a more thorough job.  
  #  
  # Example share command lines:  
  #       share  -F nfs  -d "description"  /<pathprefix>/1  
  #       share  -F nfs  -d "description"  /<pathprefix>/2
  1. Make the following changes to the dfstab file:

    a. Remove the comment character (#) from the share lines.

    b. Change the description to be a description of the resources being shared. In this example, the field includes the logical host name, the use (fs), and the mount point name.

    c. Change the <pathprefix> entries to be the logical host name, by convention.

d. Correct the mount point names.
The resulting file will look like the following.

  share  -F nfs  -d "logicalhost1 fs 1" /logicalhost1/1  
  share  -F nfs  -d "logicalhost1 fs 2" /logicalhost1/2
  1. Save this file and quit the editor.

  2. Respond to the following question.

    If you respond with a y, the edited version of the dfstab file will be distributed to the sibling host.


  Would you like to save these changes locally and distribute them  
  to host1 now? [y|n] y  
  
  changes to dfstab.logicalhost1 are saved on host1.  
  Copying dfstab.logicalhost1 to spin ... copy complete.  
  host1#
  1. If you have created a symmetric configuration, repeat all the previous steps for the second logical host.

· How to Set Up a Multi-host UFS File System

  1. Invoke newfs(1M) on all trans devices.

    Run newfs on each diskset in the configuration before Solstice HA is running. Before you can run newfs on the disksets, you must take ownership using the metaset(1M) command. For example, the following list of metatrans devices will be used for file systems that will be exported to clients on logicalhost1. (The metatrans devices were created using hasetup, metatool(1M), or the metainit(1M) command.)


  host1# metaset -s logicalhost1 -t  
  host1# newfs /dev/md/logicalhost1/rdsk/d0  
  host1# newfs /dev/md/logicalhost1/rdsk/d1  
  host1# newfs /dev/md/logicalhost1/rdsk/d2  
  host1# metaset -s logicalhost1 -r
Note - To save time, the newfs commands can be run in parallel.
  1. Use the hafstab(1M)command to edit the vfstab.logicalhost files on each server in the Solstice HA configuration.

Note - hafstab automatically updates files on both servers so it need only be run on one of the Solstice HA servers.
The vfstab.logicalhost file is in vfstab format. The hafstab command allows editing a copy of the vfstab files, performs a limited sanity check on the vfstab files, then distributes the file to both servers in the configuration. The editor defined as the EDITOR environment variable is used. A template of the vfstab file is presented for editing if no version of the file exists.
  1. Start hafstab with the correct argument and respond to the prompts. 


  host1# hafstab vfstab.logicalhost1  
  Failed to copy host2 version.  
  Type 'y' to edit host1 version; type 'n' to exit. [y|n] y
  1. After the editor begins, the following template file is displayed: 


  #  
  #ident "@(#)vfstab.tmpl   1.6     94/07/23 SMI"  
  #  
  # Copyright 26 Jul 1995 Sun Microsystems, Inc.  All Rights Reserved  
  #  
  # This file must be replicated on both servers of the HADF configuration;  
  # use hafstab(1M) to edit and distribute.  
  
  #device                            device                  mount   FS   fsck  mount mount  
  #to mount                          to fsck                 point   type pass  all   options  
  #  
  #/dev/md/<setname>/dsk/d0 /dev/md/<setname>/rdsk/d0 /<pathprefix>   ufs  -  yes   -  
  #/dev/md/<setname>/dsk/d1 /dev/md/<setname>/rdsk/d1 /<pathprefix>/1 ufs  -  yes   -  
  #/dev/md/<setname>/dsk/d2 /dev/md/<setname>/rdsk/d2 /<pathprefix>/2 ufs  -  yes   -
Make the following changes to the vfstab file:
a. Remove the comment character (#) from the last three lines.
b. Change <setname> to be the diskset name to be the diskset name, which must be the same as the logical hostname.
c. Correct the metadevice names (d0, d1, d2) as needed. d. Correct <pathprefix> entries to be the logical hostname, by convention. Also correct the mount points as necessary.
Create additional entries as planned for your configuration.
The resulting file will look like the following:

  #device                              device                 mount   FS   fsck  mount mount  
  #to mount                           to fsck                 point   type pass  all   options  
  #  
  /dev/md/logicalhost1/dsk/d0  /dev/md/logicalhost1/rdsk/d0 /logicalhost1   ufs - yes -  
  /dev/md/logicalhost1/dsk/d1  /dev/md/logicalhost1/rdsk/d1 /logicalhost1/1 ufs - yes -  
  /dev/md/logicalhost1/dsk/d2  /dev/md/logicalhost1/rdsk/d2 /logicalhost1/2 ufs - yes -
  1. Save this file and quit the editor.

  2. Respond to the following question.

    If you respond with a y, the edited version of the vfstab file will be distributed to the sibling host.


  Would you like to save these changes locally and distribute them  
  to host1 now? [y|n] y  
  
  changes to vfstab.logicalhost1 are saved on host1.  
  Copying vfstab.logicalhost1 to spin ... copy complete.  
  host1#
  1. If you have created a symmetric configuration, repeat all the previous steps for the second logical host.

10.4 Bringing the Solstice HA Servers On Line

After you have configured the appropriate data services on your Solstice HA servers, you must create a link in the /etc/rc3.d directory and start the Solstice HA software.
Note - Unless you complete the steps in this subsection, the data services will not start automatically at system boot time.
The steps to follow to bring the Solstice HA servers online are:
  1. On each server, create a hard link from /etc/rc3.d/S20SUNWhadf to /etc/init.d/SUNWhadf.

    This line enables automatic starts Solstice HA at boot time, when the system is brought up multi-user. It will not automatically start Solstice HA when the server is brought up in single user mode. On each server, enter the following:


  # ln /etc/init.d/SUNWhadf /etc/rc3.d/S20SUNWhadf
  1. On each server, start the Solstice HA services.

    Start the services by entering the following on each server:


  # /etc/init.d/SUNWhadf start
CAUTION Caution - The above command must be entered simultaneously on both servers to ensure that each logical host is served by its default master. This is very important for symmetric configurations.