2 Pre-Installation Tasks

2.1 Understanding and Using Cluster Verification Utility

Cluster Verification Utility (CVU) is a tool that performs system checks. This guide provides CVU commands to assist you with confirming that your system is properly configured for Oracle Clusterware and Oracle Real Application Clusters installation.

This section describes the following topics:

• Using CVU to Determine if Installation Prerequisites are Complete

• Entering Cluster Verification Utility Commands

• Using the Cluster Verification Utility Help

• Using Cluster Verification Utility with Oracle 10g Release 1

• Verbose Mode and "Unknown" Output

$ /mountpoint/crs/Disk1/cluvfy/runcluvfy.sh options

/dev/dvdrom/crs/Disk1/cluvfy/runcluvfy.sh comp nodereach -n node1,node2 -verbose

/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh stage -pre crsinst -n node_list 

/dev/dvdrom/crs/Disk1/cluvfy/runcluvfy.sh stage -pre crsinst -n node1,node2,node3

/dev/dvdrom/crs/Disk1/cluvfy/runcluvfy.sh stage -pre crsinst -n node1,node2,node3 -r 10gR1

2.2 Logging In to the System as root

Before you install the Oracle software, you must complete several tasks as the root user. To log in as the root user, complete one of the following procedures:

• If you are installing the software from an X Window System workstation or X terminal, then:

  1. Start a local terminal session, for example, an X terminal (xterm).

  2. If you are not installing the software on the local system, then enter the following command to enable remote hosts to display X applications on the local X server:

     $ xhost + hostname
     
     

     The hostname is the name of the local host.

  3. If you are not installing the software on the local system, then use the ssh, rlogin, or telnet command to connect to the system where you want to install the software:

     $ telnet remote_host
     
     

  4. If you are not logged in as the root user, then enter the following command to switch the user to root:

     $ su - root
     password:
     #
     
     

• If you are installing the software from a PC or other system with X server software installed, then:

  
  

  Note: If necessary, refer to your X server documentation for more information about completing this procedure. Depending on the X server software that you are using, you may need to complete the tasks in a different order.

  
  

  1. Start the X server software.

  2. Configure the security settings of the X server software to permit remote hosts to display X applications on the local system.

  3. Connect to the remote system where you want to install the software and start a terminal session on that system, for example, an X terminal (xterm).

  4. If you are not logged in as the root user on the remote system, then enter the following command to switch user to root:

     $ su - root
     password:
     #
     

2.3 Creating Required Operating System Groups and User

Depending on whether this is the first time Oracle software is being installed on this system and on the products that you are installing, you may need to create several operating system groups and an operating system user account.

The following operating system groups and user are required if you are installing Oracle Database:

• The OSDBA group (typically, dba)

  You must create this group the first time you install Oracle Database software on the system. This group identifies operating system user accounts that have database administrative privileges (the SYSDBA privilege). The default name for this group is dba.

  If you want to specify a group name other than the default dba group, then you must choose the Custom installation type to install the software or start Oracle Universal Installer as a user that is not a member of this group. In this case, Oracle Universal Installer prompts you to specify the name of this group.

• The OSOPER group (typically, oper)

  This is an optional group. Create this group if you want a separate group of operating system users to have a limited set of database administrative privileges (the SYSOPER privilege). By default, members of the OSDBA group also have the SYSOPER privilege.

  If you want to specify a separate OSOPER group, other than the default dba group, then you must choose the Custom installation type to install the software or start Oracle Universal Installer as a user that is not a member of the dba group. In this case, Oracle Universal Installer prompts you to specify the name of this group. The usual name chosen for this group is oper.

• An unprivileged user

  Verify that the unprivileged user nobody exists on the system. The nobody user must own the external jobs (extjob) executable after the installation.

The following operating system group and user are required for all installations:

• The Oracle Inventory group (typically, oinstall)

  You must create this group the first time you install Oracle software on the system. The usual name chosen for this group is oinstall. This group owns the Oracle Inventory, which is a catalog of all Oracle software installed on the system.

  
  

  Note: If Oracle software is already installed on the system, then the existing Oracle Inventory group must be the primary group of the operating system user that you use to install new Oracle software. The following sections describe how to identify an existing Oracle Inventory group.

  
  

• The Oracle software owner user (typically, oracle)

  You must create this user the first time you install Oracle software on the system. This user owns all of the software installed during the installation. The usual name chosen for this user is oracle. This user must have the Oracle Inventory group as its primary group. It must also have the OSDBA and OSOPER groups as secondary groups. In Oracle documentation, when this user account is referred to, it is called the oracle user.

A single Oracle Inventory group is required for all installations of Oracle software on the system. After the first installation of Oracle software, you must use the same Oracle Inventory group for all subsequent Oracle software installations on that system. However, you can choose to create different Oracle software owner users, OSDBA groups, and OSOPER groups (other than oracle, dba, and oper) for separate installations. By using different groups for different installations, members of these different groups have DBA privileges only on the associated databases rather than on all databases on the system.

The following sections describe how to create the required operating system user and groups:.

• Creating the Oracle Inventory Group

• Creating the OSDBA Group

• Creating an OSOPER Group (Optional)

• Creating the Oracle Software Owner User

• Verifying That the User nobody Exists

• Creating Identical Users and Groups on Other Cluster Nodes

• Configuring SSH on All Cluster Nodes

# more /etc/oraInst.loc

inventory_loc=/u01/app/oracle/oraInventory
inst_group=oinstall

# /usr/sbin/groupadd oinstall

# /usr/sbin/groupadd dba

# /usr/sbin/groupadd oper

# id oracle

uid=440(oracle) gid=200(oinstall) groups=201(dba),202(oper)

# /usr/sbin/usermod -g oinstall -G dba[,oper] oracle

$ pgrep sshd

$ man ssh

2.4 Configuring the oracle User's Environment

You run Oracle Universal Installer from the oracle account. However, before you start Oracle Universal Installer you must configure the environment of the oracle user.

To configure the environment, you must:

• Set the default file mode creation mask (umask) to 022 in the shell startup file

• Set the DISPLAY, ORACLE_BASE, and ORACLE_HOME environment variables in preparation for the Oracle Clusterware installation

You should also ensure that the PATH variable contains $ORACLE_HOME/bin before /usr/X11R6/bin

To set the oracle user's environment, follow these steps:

1. Start a new terminal session; for example, start an X terminal (xterm).

2. Enter the following command to ensure that X Window applications can display on this system:

   $ xhost + hostname
   
   

   The hostname is the name of the local host.

3. If you are not already logged in to the system where you want to install the software, then log in to that system as the oracle user.

4. If you are not logged in as the oracle user, then switch user to oracle:

   $ su - oracle
   
   

5. To determine the default shell for the oracle user, enter the following command:

   $ echo $SHELL
   
   

6. Open the oracle user's shell startup file in any text editor:

   
   

   Note: On Red Hat Linux, .bash_profile is the user startup file for the Bash shell.

   
   

   • Bourne shell (sh), Bash shell (bash) or Korn shell (ksh):

     % vi .bash_profile
     
     

   • C shell (csh or tcsh):

     % vi .login
     
     

7. Enter or edit the following line, specifying a value of 022 for the default file mode creation mask:

   umask 022
   
   

8. If the ORACLE_SID, ORACLE_HOME, or ORACLE_BASE environment variable is set in the file, then remove the appropriate lines from the file.

9. Save the file, and exit from the text editor.

10. To run the shell startup script, enter one of the following commands:

    • Bash shell on Red Hat Enterprise Linux:

      $ . ./.bash_profile
      
      

    • Bourne, Bash, or Korn shell:

      $ . ./.profile
      
      

    • C shell:

      % source ./.login
      
      

11. If you are not installing the software on the local system, then enter a command similar to the following to direct X applications to display on the local system:

    • Bourne, Bash, or Korn shell:

      $ DISPLAY=local_host:0.0 ; export DISPLAY
      
      

    • C shell:

      % setenv DISPLAY local_host:0.0
      
      

    In this example, local_host is the host name or IP address of the system that you want to use to display Oracle Universal Installer (your workstation or PC).

12. If you determined that the /tmp directory has less than 400 MB of free disk space, then identify a file system with at least 400 MB of free space and set the TEMP and TMPDIR environment variables to specify a temporary directory on this file system:

    
    

    Note: You cannot use a shared file system as the location of the temporary file directory (typically /tmp) for RAC installation. If you place /tmp on a shared file system, then the installation fails.

    
    

    1. Use the df -h command to identify a suitable file system with sufficient free space.

    2. If necessary, enter commands similar to the following to create a temporary directory on the file system that you identified, and set the appropriate permissions on the directory:

       $ su - root
       # mkdir /mount_point/tmp
       # chmod 775 /mount_point/tmp
       # exit
       
       

    3. Enter commands similar to the following to set the TEMP and TMPDIR environment variables:

       • Bourne, Bash, or Korn shell:

         $ TEMP=/mount_point/tmp
         $ TMPDIR=/mount_point/tmp
         $ export TEMP TMPDIR
         
         

       • C shell:

         % setenv TEMP /mount_point/tmp
         % setenv TMPDIR /mount_point/tmp
         
         

CV_RAW_CHECK_ENABLED=FALSE 

2.5 Checking the Hardware Requirements

Each system must meet the following minimum hardware requirements:

• At least 1 GB of physical RAM

• Swap space equivalent to the multiple of the available RAM, as indicated in the following table:

  Available RAM	Swap Space Required
  Between 1 GB and 2 GB	1.5 times the size of RAM
  More than 2 GB	Equal to the size of RAM

  
  

• 400 MB of disk space in the /tmp directory

• Up to 4 GB of disk space for the Oracle software, depending on the installation type and platform

• 1.2 GB of disk space for a preconfigured database that uses file system storage (optional)

  
  

  Note: The disk space requirement for databases that use Automatic Storage Management or raw device storage is described later in this chapter.

  
  

  Additional disk space, either on a file system or in an Automatic Storage Management disk group, is required for the flash recovery area if you choose to configure automated backups.

To ensure that each system meets these requirements, follow these steps:

1. To determine the physical RAM size, enter the following command:

   # grep MemTotal /proc/meminfo
   
   

   If the size of the physical RAM installed in the system is less than the required size, then you must install more memory before continuing.

2. To determine the size of the configured swap space, enter the following command:

   # grep SwapTotal /proc/meminfo
   
   

   If necessary, refer to your operating system documentation for information about how to configure additional swap space.

3. To determine the amount of disk space available in the /tmp directory, enter the following command:

   • # df -k /tmp
     
     

   If there is less than 400 MB of disk space available in the /tmp directory, then complete one of the following steps:

   • Delete unnecessary files from the /tmp directory to make available the disk space required.

   • Set the TEMP and TMPDIR environment variables when setting the oracle user's environment (described later).

   • Extend the file system that contains the /tmp directory. If necessary, contact your system administrator for information about extending file systems.

4. To determine the amount of free disk space on the system, enter the following command:

   • # df -k
     
     

   The following table shows the approximate disk space requirements for software files for each installation type:

   Installation Type	Requirement for Software Files (GB)
   Enterprise Edition	at least 1.5
   Standard Edition	at least 1.5
   Custom (maximum)	at least 1.5

   
   

5. To determine whether the system architecture can run the software, enter the following command:

   # grep "model name" /proc/cpuinfo
   
   

   
   

   Note: This command displays the processor type. If you intend to install on a 64-bit architecture, then verify that the processor architecture matches the requirement for the Oracle software release that you want to install.

   
   

2.6 Checking the Network Requirements

Check that you have the networking hardware and internet protocol (IP) addresses required for an Oracle Real Application Clusters installation.

http://metalink.oracle.com

Network Hardware Requirements

Each node in the cluster must meet the following requirements:

• Each node must have at least two network adapters: one for the public network interface, and one for the private network interface (the interconnect).

• The public interface names associated with the network adapters for each network must be the same on all nodes, and the private interface names associated with the network adaptors should be the same on all nodes.

  For example: With a two-node cluster, you cannot configure network adapters on node1 with eth0 as the public interface, but on node2 have eth1 as the public interface. Public interface names must be the same, so you must configure eth0 as public on both nodes. You should configure the private interfaces on the same network adapters as well. If eth1 is the private interface for node1, then eth1 should be the private interface for node2.

• For increased reliability, configure redundant public and private network adapters for each node.

• For the public network, each network adapter must support TCP/IP.

• For the private network, the interconnect must support the user datagram protocol (UDP) using high-speed network adapters and switches that support TCP/IP (Gigabit Ethernet or better recommended).

  
  

  Note: UDP is the default interconnect protocol for RAC, and TCP is the interconnect protocol for Oracle Clusterware. Token-Ring is not supported for the interconnect.

  
  

• For the private network, the endpoints of all designated interconnect interfaces must be completely reachable on the network. There should be no node that is not connected to every private network. You can test whether an interconnect interface is reachable using a ping command.

Network Parameter Requirements

If you are using NFS for your shared storage, then you must set the values for the NFS buffer size parameters rsize and wsize to at least 16384. Oracle recommends that you use the value 32768.

For example, if you decide to use rsize and wsize buffer settings with the value 16384, then update the /etc/fstab file on each node with an entry similar to the following:

clusternode:/vol/DATA/oradata  /home/oracle/netapp     nfs     rw,bg,vers=3,tcp,hard,nointr,timeo=600,rsize=32768,wsize=32768,actimeo=0  1 2

IP Address Requirements

Before starting the installation, you must identify or obtain the following IP addresses for each node:

• An IP address and an associated host name registered in the domain name service (DNS) for each public network interface

• One unused virtual IP address (VIP) and an associated virtual host name registered in DNS, or resolved in the hostfile, or both, that you will configure for the primary public network interface

  The VIP address must be in the same subnet as the associated public interface. After installation, you can configure clients to use the virtual host name or IP address. If a node fails, then its VIP address fails over to another node. During installation, OUI pings the VIP, so ensure that the VIP responds to a ping command before starting installation.

• A private IP address and optional host name for each private interface

  Oracle recommends that you use non-routable IP addresses for the private interfaces; for example: 10.*.*.* or 192.168.*.*. You can use the /etc/hosts file on each node to associate private host names with private IP addresses.

In the example described in the following table, the node rac1 has two public interfaces and one private interface. This is a proper configuration of host names and IP addresses. Each node of the cluster on which node rac1 is a member must have similar host names and IP addresses:

Node Time Requirements

Before starting the installation, ensure that each member node of the cluster is set as closely as possible to the same date and time. Oracle strongly recommends using the Network Time Protocol feature of most operating systems for this purpose, with all nodes using the same reference Network Time Protocol server.

Configuring the Network Requirements

To verify that each node meets the requirements, follow these steps:

1. If necessary, install the network adapters for the public and private networks and configure them with either public or private IP addresses.

2. Register the host names and IP addresses for the public network interfaces in DNS.

3. For each node, register one virtual host name and IP address in DNS.

4. For each private interface on every node, add a line similar to the following to the /etc/hosts file on all nodes, specifying the private IP address and associated private host name:

   10.0.0.1     rac1-priv1
   
   

5. To identify the interface name and associated IP address for every network adapter, enter the following command:

   # /sbin/ifconfig
   
   

   From the output, identify the interface name and IP address for all network adapters that you want to specify as public or private network interfaces.

   
   

   Note: When you install Oracle Clusterware and RAC, you will require this information.

   
   

6. To prevent network hangs with failovers from public to virtual IP addresses with RAC databases using NAS devices or NFS mounts, enter the following command as root to enable the Name Service Cache Daemon (nscd):

   # /sbin/service  nscd start
   
   

2.7 Checking the Network Setup with CVU

As the oracle user, enter a command using the following syntax to verify node connectivity among all of the nodes for which your cluster is configured:

/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node_list [-verbose]

In the preceding syntax example, the variable node_list is a comma-separated list of nodes in your cluster. This command detects all the network interfaces available on the cluster nodes, and verifies the connectivity among all the nodes through the network interfaces it finds.

Select the option -verbose to receive progress updates as the CVU performs its system checks, and detailed reporting of the test results.

For example, to verify node connectivity on a two-node cluster with nodes node1 and node2, with the mountpoint /dev/dvdrom, and with updates and a summary of the verification checks the CVU performs, enter the following command:

/dev/dvdrom/crs/Disk1/cluvfy/runcluvfy.sh comp nodecon -n node1,node2 -verbose

2.8 Identifying Software Requirements

Depending on the products that you intend to install, verify that the following software is installed on the system. To check these requirements refer to the section "Checking the Software Requirements", following this section.

The following is the list of supported Linux versions and requirements at the time of release:

• Software Requirements List for x86 (32-bit) Platforms

• Software Requirements List for x86 (64-bit) Platforms

binutils-2.14
compat-db-4.0.14-5
compat-gcc-7.3-2.96.128
compat-gcc-c++-7.3-2.96.128
compat-libstdc++-7.3-2.96.128
compat-libstdc++-devel-7.3-2.96.128
gcc-3.2
glibc-2.3.2-95.27
make-3.79
openmotif-2.2.3
setarch-1.3-1

binutils-2.15.92.0.2-10.EL4
compat-db-4.1.25-9
control-center-2.8.0-12
gcc-3.4.3-9.EL4
gcc-c++-3.4.3-9.EL4
glibc-2.3.4-2
glibc-common-2.3.4-2
gnome-libs-1.4.1.2.90-44.1
libstdc++-3.4.3-9.EL4
libstdc++-devel-3.4.3-9.EL4
make-3.80-5
pdksh-5.2.14-30
sysstat-5.0.5-1
xscreensaver-4.18-5.rhel4.2

gcc-3.3
gcc-c++-3.3.3-43
glibc-2.3.3-98.28
libaio-0.3.98-18
libaio-devel-0.3.98-18
make-3.80
openmotif-libs-2.2.2-519.1

ocfs-support
ocfs-tools
ocfs-kernel_version

http://oss.oracle.com/projects/ocfs/

http://oss.oracle.com/projects/ocfs2/

make-3.79.1-17
compat-db 4.0.14-5.1
control-center-2.2.0.1-13
gcc-3.2.3-47
gcc-c++-3.2.3-47
gdb-6.1post-1.20040607.52
glibc-2.3.2-95.30
glibc-common-2.3.2-95.30
glibc-devel-2.3.2-95.30
glibc-devel-2.3.2-95.20 (32 bit)
compat-db-4.0.14-5
compat-gcc-7.3-2.96.128
compat-gcc-c++-7.3-2.96.128
compat-libstdc++-7.3-2.96.128
compat-libstdc++-devel-7.3-2.96.128
gnome-libs-1.4.1.2.90-34.2 (32 bit)
libstdc++-3.2.3-47
libstdc++-devel-3.2.3-47
openmotif-2.2.3-3.RHEL3
sysstat-5.0.5-5.rhel3
setarch-1.3-1
libaio-0.3.96-3
libaio-devel-0.3.96-3

binutils-2.15.92.0.2-10.EL4
compat-db-4.1.25-9
control-center-2.8.0-12
gcc-3.4.3-9.EL4
gcc-c++-3.4.3-9.EL4
glibc-2.3.4-2
glibc-common-2.3.4-2
gnome-libs-1.4.1.2.90-44.1
libstdc++-3.4.3-9.EL4
libstdc++-devel-3.4.3-9.EL4
make-3.80-5
pdksh-5.2.14-30
sysstat-5.0.5-1
xscreensaver-4.18-5.rhel4.2

binutils-2.15.90.0.1.1-32.5
gcc-3.3.3-43.24
gcc-c++-3.3.3-43.24
glibc-2.3.3-98.28
gnome-libs-1.4.1.7-671.1
libstdc++-3.3.3-43.24
libstdc++-devel-3.3.3-43.24
make-3.80-184.1
pdksh-5.2.14-780.1
sysstat-5.0.1-35.1
xscreensaver-4.16-2.6

ocfs-support
ocfs-tools
ocfs-kernel_version

http://oss.oracle.com/projects/ocfs/

http://oss.oracle.com/projects/ocfs2/

2.9 Checking the Software Requirements

To ensure that the system meets these requirements, follow these steps:

1. To determine which distribution and version of Linux is installed, enter the following command:

   # cat /proc/version
   
   

   
   

   Note: Only the distributions and versions listed in the previous table are supported. Installation on other distributions of Linux are not supported.

   
   

2. To determine whether the required kernel errata is installed, use the following procedure for your Linux distribution:

   Red Hat Enterprise Linux systems

   Enter the following command:

   # uname -r
   2.4.21-27.EL
   
   

   In this example, the output shows the kernel version (2.4.27) and errata level (27.EL) on the system, which is an appropriate value for Red Hat Linux Enterprise Linux 3.0.

   Review the required errata level for your distribution. If the errata level is previous to the required minimum errata update, then install the latest kernel update. Kernel updates are available from Red Hat.

   SUSE Linux Enterprise Server Systems

   To determine whether the required kernel is installed on SUSE Linux Enterprise Server systems, enter the following command:

   # uname -r
   2.6.5-7.201
   
   

   In this example, the output shows the kernel version (2.6.5-7.201) on the system, which is correct for SUSE Linux Enterprise Server 9. If the kernel version is less than the required minimum update for your SUSE distribution, then contact SUSE for information about obtaining and installing the kernel updates.

3. To determine whether the required packages are installed, enter commands similar to the following:

   # rpm -q package_name
   
   

   If a package is not installed, then install it from your Linux distribution media or download the required package version from your Linux vendor's Web site.

4. To determine whether OCFS is installed, enter the following command:

   # rpm -qa | grep ocfs
   
   

   If you want to install the Oracle Database files on an OCFS file system and the packages are not installed, then download them from the following Web site. Follow the instructions listed with the kit to install the packages and configure the file system:

   http://oss.oracle.com/projects/ocfs/
   
   

5. To use Hugepages, or to accommodate the very large memory (VLM) window size on Red Hat Enterprise Linux 4, you must increase the default maximum size of the per-process locked memory. To increase the per-process max locked memory limit, add the following lines to the /etc/security/limits.conf file, where oracle is the user that administers the database:

   oracle soft memlock 3145728
   oracle hard memlock 3145728
   
   

2.10 Configuring Kernel Parameters

Verify that the kernel parameters shown in the following table are set to values greater than or equal to the recommended value shown. The procedure following the table describes how to verify and set the values.

On all cluster nodes, verify that the kernel parameters shown in the following table are set to values greater than or equal to the recommended value shown. The procedure following the table describes how to verify and set the values.

To view the current value specified for these kernel parameters, and to change them if necessary, follow these steps:

1. Enter the commands shown in the following table to view the current values of the kernel parameters:

   
   

   Note: Make a note of the current values and identify any values that you must change.

   
   

   Parameter	Command
   semmsl, semmns, semopm, and semmni	# /sbin/sysctl -a | grep sem This command displays the value of the semaphore parameters in the order listed.
   shmall, shmmax, and shmmni	# /sbin/sysctl -a | grep shm
   file-max	# /sbin/sysctl -a | grep file-max
   ip_local_port_range	# /sbin/sysctl -a | grep ip_local_port_range This command displays a range of port numbers.
   rmem_default, rmem_max, wmem_default, and wmem_max	# /sbin/sysctl -a | grep net.core

   
   

2. If the value of any kernel parameter is different from the recommended value, then complete the following process:

   Using any text editor, create or edit the /etc/sysctl.conf file, and add or edit lines similar to the following:

   
   

   Note: Include lines only for the kernel parameter values that you want to change. For the semaphore parameters (kernel.sem), you must specify all four values. However, if any of the current system parameter values are greater than the recommended values, then keep using the larger values.

   
   

   kernel.shmall = 2097152
   kernel.shmmax = 2147483648
   kernel.shmmni = 4096
   kernel.sem = 250 32000 100 128
   fs.file-max = 65536
   net.ipv4.ip_local_port_range = 1024 65000
   net.core.rmem_default = 262144
   net.core.rmem_max = 262144
   net.core.wmem_default = 262144
   net.core.wmem_max = 262144
   
   

   By specifying the values in the /etc/sysctl.conf file, they persist when you restart the system.

   On Red Hat systems, to have these changes take effect immediately so that you do not have to restart the system, enter the following command:

   /sbin/sysctl -p
   
   

3. Repeat steps 1 and 2 on all other nodes in the cluster.

   On SUSE systems only, enter the following command to cause the system to read the /etc/sysctl.conf file when it restarts:

   # /sbin/chkconfig boot.sysctl on
   
   

Setting Shell Limits for the oracle User

To improve the performance of the software on Linux systems, you must increase the following shell limits for the oracle user:

To increase the shell limits:

1. Add the following lines to the /etc/security/limits.conf file:

   oracle              soft    nproc   2047
   oracle               hard    nproc   16384
   oracle               soft    nofile  1024
   oracle               hard    nofile  65536
   
   

2. Add or edit the following line in the /etc/pam.d/login file, if it does not already exist:

   session    required     /lib/security/pam_limits.so
   
   

3. Depending on the oracle user's default shell, make the following changes to the default shell startup file:

   • For the Bourne, Bash, or Korn shell, add the following lines to the /etc/profile file (or the file on SUSE systems)/etc/profile.local:

     if [ $USER = "oracle" ]; then
             if [ $SHELL = "/bin/ksh" ]; then
                   ulimit -p 16384
                   ulimit -n 65536
             else
                   ulimit -u 16384 -n 65536
             fi
     fi
     
     

   • For the C shell (csh or tcsh), add the following lines to the /etc/csh.login file (or the file on SUSE systems)/etc/csh.login.local:

     if ( $USER == "oracle" ) then
             limit maxproc 16384
             limit descriptors 65536
     endif
     
     

4. Repeat this procedure on all other nodes in the cluster.

Refer to the "Identifying Required Software Directories" section to continue.

2.11 Checking the Hardware and Operating System Setup with CVU

As the oracle user, use the following command syntax to start Cluster Verification Utility (CVU) stage verification to check hardware and operating system setup:

/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node_list [-verbose]

In the preceding syntax example, replace the variable node_list with the names of the nodes in your cluster, separated by commas. For example, to check the hardware and operating system of a two-node cluster with nodes node1 and node2, with the mountpoint /dev/dvdrom/ and with the option to limit the output to the test results, enter the following command:

/dev/dvdrom/crs/Disk1/cluvfy/runcluvfy.sh stage –post hwos –n node1,node2

Select the option -verbose to receive detailed reports of the test results, and progress updates about the system checks performed by Cluster Verification Utility.

2.12 Checking the Operating System Requirements Setup with CVU

As the oracle user, use the following command syntax to check if your system meets the operating system requirement pre-installation tasks:

/mountpoint/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node_list -p {crs|database} -osdba osdba_group -orainv orainv_group -verbose

In the preceding syntax example:

• The variable mountpoint is the mountpoint of the Oracle 10g Release 2 (10.2) installation media

• The variable node_list is the list of nodes in your cluster, separated by commas

• The -p flag identifies either crs or database, and indicates that checks are performed for Oracle Clusterware or Oracle Database system requirements

• The variable osdba_group is the name of your OSDBA group, typically dba

• The variable orainv_group is the name of your Oracle Inventory group, typically oinstall

You can select the option -verbose to receive progress updates as the CVU performs its system checks, and detailed reporting of the test results.

For example, to perform a system check for an Oracle Clusterware installation on a two-node cluster with nodes node1 and node2, with the OSDBA dba and Oracle inventory group oinstall, and with the media mountpoint /dev/dvdrom/, then enter the following command:

/dev/dvdrom/crs/Disk1/cluvfy/runcluvfy.sh comp sys -n node1,node2 -p crs -osdba crs -orainv oinstall

2.13 Identifying Required Software Directories

You must identify or create the following directories for the Oracle software, as follows:

• Oracle Base Directory

• Oracle Inventory Directory

• Oracle Clusterware Home Directory

• Oracle Home Directory

The following subsections describe the requirements for these directories.

Oracle Base Directory

The Oracle base directory acts as a top-level directory for Oracle software installations. Optimal Flexible Architecture (OFA) guidelines recommend that you use a path similar to the following for the Oracle base directory:

/mount_point/app/oracle_sw_owner

In this example:

• mount_point is the mount point directory for the file system that will contain the Oracle software.

  The examples in this guide use /u01 for the mount point directory. However, you could choose another mount point directory, /oracle or /opt/oracle for example.

• oracle_sw_owner is the operating system user name of the Oracle software owner, for example oracle.

You can use the same Oracle base directory for more than one installation or you can create separate Oracle base directories for different installations. If different operating system users install Oracle software on the same system, then each user must create a separate Oracle base directory. The following example Oracle base directories could all exist on the same system:

/u01/app/oracle
/u01/app/orauser
/opt/oracle/app/oracle

The following sections describe how to identify existing Oracle base directories that might be suitable for your installation and how to create an Oracle base directory if necessary.

Regardless of whether you create an Oracle base directory or decide to use an existing one, you must set the ORACLE_BASE environment variable to specify the full path to the Oracle base directory.

Oracle Inventory Directory

The Oracle Inventory directory (oraInventory) stores an inventory of all software installed on the system. It is required by, and shared by, all Oracle software installations on a single system. The first time you install Oracle software on a system, Oracle Universal Installer prompts you to specify the path to this directory. If you are installing the software on a local file system, then Oracle recommends that you choose the following path:

oracle_base/oraInventory

If the Oracle base directory is on a cluster file system, or on an NFS file system on an NAS device, then you must specify a path for the Oracle Inventory directory on a local file system. The Oracle base directory must be on a local file system to enable all of the nodes to have separate inventories.

Oracle Universal Installer creates the directory that you specify, and sets the correct owner, group, and permissions for it. You do not need to create it.

Oracle Clusterware Home Directory

The Oracle Clusterware home directory is the directory where you choose to install the software for Oracle Clusterware. You must install Oracle Clusterware in a separate home directory. When you run Oracle Universal Installer, it prompts you to specify the path to this directory, as well as a name that identifies it. Oracle recommends that you specify a path similar to the following for the Oracle Clusterware home directory:

/u01/app/oracle/product/crs

Oracle Home Directory

The Oracle home directory is the directory where you choose to install the software for a particular Oracle product. You must install different Oracle products, or different releases of the same Oracle product, in separate Oracle home directories. When you run Oracle Universal Installer, it prompts you to specify the path to this directory, as well as a name that identifies it. The directory that you specify must be a subdirectory of the Oracle base directory. Oracle recommends that you specify a path similar to the following for the Oracle home directory:

oracle_base/product/10.2.0/db_1

Oracle Universal Installer creates the directory path that you specify under the Oracle base directory. It also sets the correct owner, group, and permissions on it. You do not need to create this directory.

2.14 Identifying or Creating an Oracle Base Directory

Before starting the installation, you must either identify an existing Oracle base directory or if required, create one. This section contains information about the following:

• Identifying an Existing Oracle Base Directory

• Creating an Oracle Base Directory

Identifying an Existing Oracle Base Directory

Existing Oracle base directories might not have paths that comply with OFA guidelines. However, if you identify an existing Oracle Inventory directory or existing Oracle home directories, you can usually identify the Oracle base directories, as follows:

• Identifying an existing Oracle Inventory directory

  Enter the following command on all nodes in the cluster to view the contents of the oraInst.loc file:

  # more /etc/oraInst.loc
  
  

  If the oraInst.loc file exists, then the output from this command is similar to the following:

  inventory_loc=/u01/app/oracle/oraInventory
  inst_group=oinstall
  
  

  The inventory_loc parameter identifies the Oracle Inventory directory (oraInventory) on that system. The parent directory of the oraInventory directory is typically an Oracle base directory. In the previous example, /u01/app/oracle is an Oracle base directory.

• Identifying existing Oracle home directories

  Enter the following command on all nodes in the cluster to view the contents of the oratab file:

  # more /etc/oratab
  
  

  If the oratab file exists, then it contains lines similar to the following:

  *:/u03/app/oracle/product/10.2.0/db_1:N
  *:/opt/orauser/infra_904:N
  *:/oracle/9.2.0:N
  
  

  The directory paths specified on each line identify Oracle home directories. Directory paths that end with the user name of the Oracle software owner that you want to use are valid choices for an Oracle base directory. If you intend to use the oracle user to install the software, then you could choose one of the following directories from the previous example:

  /u03/app/oracle
  /oracle
  

  
  

  Note: If possible, choose a directory path similar to the first (/u03/app/oracle). This path complies with the OFA guidelines.

  
  

Before deciding to use an existing Oracle base directory for this installation, make sure that it satisfies the following conditions:

• It should not be on the same file system as the operating system.

• It must have an identical path on all nodes in the cluster, or it must be an NFS file system on a certified NAS device.

  If you are not using an NFS file system, then create identical Oracle base directories on the other nodes.

• It must have at least 1.5 GB free disk space on all the nodes in the cluster

  To determine the free disk space on the file system where the Oracle base directory is located, enter the following command:

  # df -h oracle_base_path
  
  

  When you are configuring the oracle user's environment in the section "Creating the Oracle Clusterware Home Directory", set the ORACLE_BASE environment variable to specify the directory you choose.

If an Oracle base directory does not exist on the system, or if you want to create an Oracle base directory, then refer to the following section.

Creating an Oracle Base Directory

Before you create an Oracle base directory, you must identify an appropriate file system. The Oracle base directory requires at least 1.5 GB of free disk space.

To identify an appropriate file system, follow these steps:

1. Use the df -h command to determine the free disk space on each mounted file system.

2. From the display, identify a file system that has appropriate free space.

   
   

   Note: The Oracle base directory can be on a local file system or on an NFS file system on a certified NAS device. Do not create the Oracle base directory on an OCFS version 1 file system.

   
   

   The path to the Oracle base directory must be the same on all nodes.

3. Note the name of the mount point directory for the file system that you identified.

To create the Oracle base directory and specify the correct owner, group, and permissions for it, follow these steps:

1. Enter commands similar to the following to create the recommended subdirectories in the mount point directory that you identified, and to set the appropriate owner, group, and permissions on them:

   # mkdir -p /mount_point/app/oracle_sw_owner
   # chown -R oracle:oinstall /mount_point/app/oracle_sw_owner
   # chmod -R 775 /mount_point/app/oracle_sw_owner
   
   

   For example, if the mount point you identify is /u01, and oracle is the user name of the Oracle software owner, then the recommended Oracle base directory path is as follows:

   /u01/app/oracle
   
   

2. If necessary, repeat the commands listed in the previous step to create the same directory on the other nodes in the cluster.

3. When you configure the oracle user's environment later in this chapter, set the ORACLE_BASE environment variable to specify the Oracle base directory you have created in this task.

2.15 Creating the Oracle Clusterware Home Directory

Oracle Universal Installer (OUI) creates the Oracle Clusterware home directory for you. Ensure before you start the installation that you provide sufficient disk space on a file system for the Oracle Clusterware directory, and the parent directory of the Oracle Clusterware directory space is writable by the oracle user.

To identify an appropriate file system, follow these steps:

1. Use the df -h command to determine the free disk space on each mounted file system.

2. From the display, identify a file system that has at least 120 MB of free disk space.

   If you are using the same file system for the Oracle base directory, then this 120 MB of disk space is additional to the free disk space requirement that you identified previously.

   
   

   Note: The file system can be a local file system or an NFS file system on a certified NAS device. Do not create the Oracle Clusterware home directory on an OCFS file system.

   
   

   The path to the Oracle Clusterware home directory must be the same on all nodes.

3. Note the name of the mount point directory for the file system that you identified.

To create the Oracle Clusterware home directory and specify the correct owner, group, and permissions for it, follow these steps:

1. Enter commands similar to the following to create the recommended subdirectories in the mount point directory that you identified and set the appropriate owner, group, and permissions on them:

   # mkdir -p /mount_point/crs/oracle_sw_owner/product/10/app
   # chown -R root:oinstall /mount_point/crs
   # chmod -R 775 /mount_point/crs/oracle_sw_owner
   
   

   If the mount point you identified is /u01, then the recommended Oracle Clusterware home directory path is as follows:

   /u01/crs/oracle/product/10/crs
   
   

   
   

   Note: After installation, you should change permissions so that only the root user can write to the Oracle Clusterware home directory.

   
   

2. If necessary, repeat the commands listed in the previous step to create the same directory on the other nodes in the cluster.

3. Enter commands similar to the following to set the ORACLE_BASE and ORACLE_HOME environment variable in preparation for the Oracle Clusterware installation:

   • Bourne, Bash, or Korn shell:

     $ ORACLE_BASE=/u01/app/oracle
     $ ORACLE_HOME=/u01/crs/oracle/product/10/app
     $ export ORACLE_BASE
     $ export ORACLE_HOME
     
     

   • C shell:

     % setenv ORACLE_BASE /u01/app/oracle
     % setenv ORACLE_HOME /u01/crs/oracle/product/10/app
     
     

4. Enter the following commands to ensure that the TNS_ADMIN environment variable is not set:

   • Bourne, Bash, or Korn shell:

     $ unset TNS_ADMIN
     
     

   • C shell:

     % unsetenv TNS_ADMIN
     
     

5. To verify that the environment has been set correctly, enter the following commands:

   $ umask
   $ env | more
   
   

   Verify that the umask command displays a value of 22, 022, or 0022 and the environment variables that you set in this section have the correct values.

2.16 Checking the Configuration of the Hangcheck-timer Module

Before installing Oracle Real Application Clusters on Linux systems, verify that the hangcheck-timer module (hangcheck-timer) is loaded and configured correctly. hangcheck-timer monitors the Linux kernel for extended operating system hangs that could affect the reliability of a RAC node and cause a database corruption. If a hang occurs, then the module restarts the node in seconds.

You can use the hangcheck_tick and hangcheck_margin parameters to control the behavior of the module, as follows:

• The hangcheck_tick parameter defines how often, in seconds, the hangcheck-timer checks the node for hangs. The default value is 60 seconds.

• The hangcheck_margin parameter defines how long the timer waits, in seconds, for a response from the kernel. The default value is 180 seconds.

If the kernel fails to respond within the sum of the hangcheck_tick and hangcheck_margin parameter values, the hangcheck-timer module restarts the system. Using the default values, the node would be restarted if the kernel fails to respond within 240 seconds.

To verify that the hangcheck-timer module is running on every node:

1. Enter the following command on each node to determine which kernel modules are loaded:

   # /sbin/lsmod
   
   

2. If the hangcheck-timer module is not listed for any node, enter a command similar to the following to start the module on that node:

   # /sbin/insmod hangcheck-timer hangcheck_tick=30 hangcheck_margin=180
   
   

3. To ensure that the module is loaded every time the system restarts, verify that the local system startup file contains the command shown in the previous step, or add it if necessary:

   • Red Hat:

     On Red Hat Enterprise Linux systems, add the command to the /etc/rc.d/rc.local file.

   • SUSE:

     On SUSE systems, add the command to the /etc/init.d/boot.local file.

2.17 Stopping Existing Oracle Processes for Upgrades or Co-existing Databases

Table 2-3 provides an overview of what you need to do if you have an existing Oracle database on the system where you plan to install Oracle Database 10g Release 2 (10.2). Review the table, and perform tasks as required.

Installing Oracle 10g Release 2 on a System with Oracle 10g Release 1

If your system has an Oracle Database Release 10g Release 10. 1 installation, and you install an Oracle Database 10g Release 2 (10.2) either to coexist with or to upgrade the 10.1, then most installation types configure and start a default Oracle Net listener using TCP/IP port 1521 and the IPC key value EXTPROC. One of the following occurs:

• During a co-existing installation, Database Configuration Assistant (DBCA) automatically migrates the listener and related files from the 10.1 Oracle home to the 10.2 Oracle home.

• During an upgrade, Oracle Database Upgrade Assistant (DBUA) automatically locates the Oracle 10g release 1 (10.1) listener, and migrates it to Oracle 10g release 2.

Proceed to Chapter 3.

Installing Oracle 10g Release 2 on a System with Oracle9i Release 2

This section provides instructions for preparing

Explanation of Tasks If you are installing an Oracle Database 10g Release 2 (10.2) on a system with an existing Oracle9i Release 2 (9.2) database, and the Oracle Net listener process is using the same port or key value as the default used with the Oracle 10g Release 2 (10.2) installation, port 1521, then Oracle Universal Installer can only configure the new listener; it cannot start it. To ensure that the new listener process starts during the installation, you must shut down any existing listeners before starting Oracle Universal Installer. To do this, refer to "Shutting Down the Listener"

You must shut down the Global Services Daemon (GSD), because otherwise, during 10g Release 2 (10.2) installation, the Oracle9i Release 9.2 SRVM shared data is upgraded into an Oracle Cluster Registry that the 9.2 GSD will not be able to use. The 10.2 Oracle Clusterware installation starts a 10g Release 2 (10.2) GSD to serve the Oracle9i 9.2 clients. To do this, refer to "Shutting down the Global Services Daemon"

Shutting Down the Listener To determine whether an existing Oracle9i listener process is running and to shut it down if necessary, follow these steps:

1. Switch user to oracle:

   # su - oracle
   
   

2. Enter the following command to determine whether an Oracle9i listener process is running and to identify its name and the Oracle home directory in which it is installed:

   $ ps -ef | grep tnslsnr
   
   

   This command displays information about the Oracle Net listeners running on the system:

   ... oracle_home1/bin/tnslsnr LISTENER -inherit
   
   

   In this example, oracle_home1 is the Oracle home directory where the listener is installed and LISTENER is the listener name.

   
   

   Note: If no Oracle Net listeners are running, then proceed to Chapter 3.

   
   

3. Set the ORACLE_HOME environment variable to specify the appropriate Oracle home directory for the listener:

   • Bourne, Bash, or Korn shell:

     $ ORACLE_HOME=oracle_home1
     $ export ORACLE_HOME
     
     

   • C or tcsh shell:

     % setenv ORACLE_HOME oracle_home1
     
     

4. Enter the following command to identify the TCP/IP port number and IPC key value that the listener is using:

   $ $ORACLE_HOME/bin/lsnrctl status listenername
   

   
   

   Note: If the listener uses the default name LISTENER, then you do not have to specify the listener name in this command.

   
   

5. Enter a command similar to the following to stop the listener process:

   $ $ORACLE_HOME/bin/lsnrctl stop listenername
   
   

6. Repeat this procedure to stop all listeners running on this system and on all other nodes in the cluster.

Shutting down the Global Services Daemon As the oracle user, on each node of the cluster, use the following syntax to shut down the GSD:

$ cd 92_Oracle_home
$ bin/gsdctl stop

In the preceding syntax example, the variable 92_Oracle_home is the Oracle9i Release 2 (9.2) database home.