8
Adding Nodes and Instances and Deleting Instances in Real Application Clusters

This chapter provides detailed procedures on how to add nodes and instances and how to delete instances in Real Application Clusters database environments. The topics in this chapter are:

Adding Nodes to a Cluster
Overview of Procedures for Adding Nodes
Adding a Node at the Clusterware Layer
Adding a Node at the Oracle Layer
Deleting Instances

See Also:
Oracle9i Real Application Clusters Setup and Configuration for procedures on using the Database Configuration Assistant (DBCA) to create and delete databases

Adding Nodes to a Cluster

This section explains how to dynamically add nodes to Real Application Clusters environments. The topics in this section are:

Overview of Procedures for Adding Nodes
Adding a Node at the Clusterware Layer
Adding a Node at the Oracle Layer

See Also:
"Deleting Instances" for procedures on deleting an instance

Overview of Procedures for Adding Nodes

The procedure for adding nodes to Real Application Clusters comprises two main steps:

The method for adding nodes at the clusterware layer is operating-system dependent as described in this chapter. For the second step, adding a node at the Oracle layer, the procedures in this chapter require using the Oracle Universal Installer (OUI) and the Database Configuration Assistant (DBCA). The procedures for the second step are generic and therefore apply to all platforms.

Adding a Node at the Clusterware Layer

On UNIX, unless you preconfigure your clusterware to accommodate additional nodes, you cannot dynamically add nodes. Therefore, you must stop the clusterware and reconfigure your clusterware to accommodate additional nodes. Then follow the procedures under the heading "Adding a Node at the Clusterware Layer on UNIX".

Note:

You can dynamically add nodes on some platforms without preconfiguring your clusterware.

On Windows NT and Windows 2000, Oracle supplies the cluster software or operating system-dependent (OSD) clusterware. This enables you to dynamically add nodes as described under the heading "Adding a Node at the Clusterware Layer on Windows NT and Windows 2000".

The topics in this section are:

Adding a Node at the Clusterware Layer on UNIX

You can take advantage of the scalability and high availability features of Real Application Clusters by adding new nodes to existing UNIX clusters that uses a Real Application Clusters database. As mentioned, you can add nodes without stopping your database if you have preconfigured your UNIX cluster to accommodate additional nodes.

Oracle Corporation recommends using the following procedures to add new nodes to existing Real Application Clusters in UNIX environments. These procedures assume that there is already a UNIX cluster running your Real Application Clusters database.

To prepare the new node, ensure that it is an exact clone of the existing nodes in terms of vendor operating system, clusterware software, and so on. Then perform the following tasks in the order shown to add a node at the clusterware layer on UNIX:

Connect Node to Cluster

Connect the new node's hardware to the existing cluster. This includes electrical connections, network interconnects, shared disk subsystem connections, and so on. Refer to your hardware vendor documentation for details on this step.

Install Vendor Cluster Software

Install the cluster software by referring to your vendor's installation procedures. Also use your vendor documentation for procedures to make the new node an active member of your cluster.

Create Raw Devices (Optional)

If your platform supports a cluster file system and you not are using raw devices for your cluster database, then you do not need to create raw devices as described in this section. Proceed to "Adding a Node at the Oracle Layer".

As part of the preinstallation steps before installing the Real Application Clusters software on the new node, create at least two new disk volumes to accommodate the redo logs for the new instance. Refer to Oracle9i Real Application Clusters Setup and Configuration for the requirements for these redo logs and the preinstallation steps.

Make the disk volumes for the redo logs the same size as the redo log volumes that you configured for your existing nodes' instances. Refer to your vendor documentation for procedures on creating disk volumes and raw devices.

If the cluster database to which you want to add a new node and instance uses automatic undo management, then create an additional raw volume for the new undo tablespace for the new instance. After completing this step, proceed to "Adding a Node at the Oracle Layer" to add the node at the Oracle layer.

See Also:

Oracle9i Real Application Clusters Setup and Configuration for more information about creating raw devices

Adding a Node at the Clusterware Layer on Windows NT and Windows 2000

You can take advantage of the scalability and high availability features of Real Application Clusters by dynamically adding new nodes to an existing Windows cluster that uses a Real Application Clusters database.

Oracle Corporation recommends using the following procedures to add a node in Windows environments. These procedures assume that there is already a Windows cluster running your Real Application Clusters database and that the new node uses the same operating system that is running on the existing nodes of your cluster. For example, you should have installed Microsoft Windows NT 4.0 or Windows 2000 on all the nodes in your cluster.

Perform the following tasks in the order shown to add a new node at the clusterware layer on Windows NT and Windows 2000:

Connect Node to Cluster

Install Cluster Software

If you are using your operating system vendor supplied clusterware on the existing nodes, also install this vendor software on the new node using your vendor's installation procedures.

However, if you are using Oracle Corporation's reference clusterware/operating system-dependent software on the existing nodes of your cluster, you must also install these Oracle OSD clusterware on the new node using the Cluster Setup Wizard. To do this, perform the following steps from one of the existing nodes before running the wizard:

If you use the Virtual Interface Architecture (VIA) interconnect on the existing cluster, make sure that VIA interconnect is also installed on the new node.
If you have a private interconnect network on the existing cluster, make sure that the private interconnect network is also installed on the new node.
If you are using a cluster file system, then skip this step: Determine the disk drive on which the cluster software was installed on the existing nodes. Make sure that you have at least 2MB of free space on the same drive on the new node to install the OSD clusterware and the Object Link Manager.
Make sure that you can execute the following from each of the existing nodes of your cluster:
```
NET USE \\host_name\C$
```

Where the host_name is the public network name of the new node. You have administrative privileges on each node if the operating system responds with:

Command completed successfully.

To install the Oracle OSD clusterware when the new node already has a version of your vendor OSD clusterware installed, make sure the vendor OSD clusterware is stopped. Otherwise, continue to the next step.
Insert CD number one from the Oracle9i release 2 (9.2) CD set into the new node's CD-ROM drive.
Navigate to the PREINSTALL\CLUSTERSETUP directory, for example:
```
cd \PREINSTALL\CLUSTERSETUP
```
Run the Cluster Setup Wizard by entering the following from the \PREINSTALL\CLUSTERSETUP directory:
```
clustersetup
```
The cluster setup wizard should display its Welcome page. Click Next to proceed.
Check the Node Addition option and click Next.
Provide a public name for the new node you want to add. If the existing cluster also uses a high speed private network, provide a name for the private network interface for the new node and click Next.

Click Finish.

See Also:

Cluster Setup Wizard online help
Oracle9i Database Installation Guide for Windows for more information

Create Disk Partitions

As part of the preinstall requirements before installing the Oracle database software on a node for Oracle Real Application Clusters, create raw disk partitions. Create at least two new disk partitions to accommodate the redo logs for the new instance. Make these disk partitions the same size as the redo log partitions that you configured for the existing nodes' instances. If the database to which you want to add a node uses automatic undo management, then create an additional logical partition for the undo tablespace.

See Also:

Oracle9i Real Application Clusters Setup and Configuration for more information on configuring raw partitions on Windows NT and Windows 2000 platforms

From one of the existing nodes of the cluster:

On Windows NT, start Disk Administrator using the path: Start > Programs > Oracle - [HOME_NAME] > Administrative Tools> Disk Administrator

On Windows 2000, start Disk Management using the path: Start > Programs > Oracle - [HOME_NAME] > Administrative Tools > Computer Management. Then expand the Storage folder to Disk Management. For Windows 2000 only, use a basic disk as an extended partition for creating raw partitions.
Click inside an unallocated part of the extended partition.
Choose Create from the Partition menu. A dialog box appears in which you should enter the size of the partition. Ensure you use the same sizes as those you used on your existing nodes.
Click the newly created partition and select Assign Drive Letter from the Tool menu.
Select Don't Assign Drive Letter, and click OK.
Repeat the last four steps, Steps 2 through 5, for the second and any additional partitions.
Select Commit Changes Now from the Partition menu to save the new partition information.
For Windows NT only, you may need to restart all the nodes in your cluster if you are using a pre-4.0 Service Pack. This makes all of the nodes recognize the new partitions.

Note:
The Pre-5 Service Packs are not Y2K compliant. Oracle Corporation therefore recommends using 6A.

Create Links to Disk Partitions

You do not need to perform these procedures if you have a cluster file system.

Create the links to the partitions that you created in the "Create Disk Partitions" section so the existing nodes recognize these links. In addition, the new node should also recognize the pre-existing symbolic links to logical drives as well as the new links. To create these links, perform the following steps from one of the existing nodes:

Start the Object Link Manager by typing the following from the
%ORACLE_HOME%\bin directory:
```
GUIOracleOBJManager
```

The Object Link Manager automatically detects the symbolic links to the logical drives and displays them in OLM's graphical user interface (GUI).

Recall the disk and partition numbers for the partitions that you created in the previous section "Create Disk Partitions". Look for the disk and partition numbers in the OLM GUI and perform the following tasks:

Right-click next to the box under the New Link column and enter the link name for the first partition.
Repeat step a. for the second and any additional partitions.

For example, if your Real Application Clusters database name is db and it consists of two instances running on two nodes and you are adding a third instance on the third node, your link names for your redo logs are db_redo3_1, db_redo3_2, and so on.

If your existing database uses automatic undo management, then enter the link name for the logical partition for the undo tablespace that you created in the previous section "Create Disk Partitions". For example, if your Real Application Clusters database name is db and it consists of two instances running on two nodes and you are adding a third instance on the third node, then your link name for the undo tablespace should be db_undotbs3.
Select Commit from the Options menu. This creates the new links on the current node.
Select Sync Nodes from the Options menu. This makes the new links visible to the existing nodes of the cluster.
Select Exit from the Options menu to exit the Object Link Manager.

Adding a Node at the Oracle Layer

At this point, you have added the new node to the cluster at the clusterware layer. To add a node to an existing cluster at the Oracle layer, on all platforms perform the following steps:

The following procedures explain these steps in detail.

Install the Oracle Software on the New Node

Follow these steps to install the Oracle software on the new node:

If your platform supports a cluster file system then proceed to the next section titled "Post-Installation Steps".
On the new node, insert the Oracle9i release 2 (9.2) CD-ROM into the new node's CD-ROM drive.
Run the installer using one of the following procedures for your platform.
- On UNIX, run the Installer on by executing the runInstaller command from the root directory of the first Oracle CD. The Installer displays the Welcome page.
- On Windows NT and Windows 2000, run the Installer by executing the setup.exe command. The installer displays the Welcome page.
On the File Locations page in the Path field under the Destination heading, enter the ORACLE_HOME into which you are going to install the Oracle Enterprise Edition and Real Application Clusters software. The ORACLE_HOME you enter should be the same as the ORACLE_HOME that the existing nodes use.
On the Available Products page, select Oracle9i Database.
On the Install Types page, select Enterprise Edition. The Installer displays the Database Configuration page.
Follow the OUI instructions to install the same products and options that you installed on the existing nodes. Or select the Software Only configuration option on the Database Configuration page.
On the Node Selection page, select only the current or new node name. Do not select other pre-existing node names; the Oracle software is already on them.
When the OUI completes the installation of the Oracle software, run the post installation script root.sh. Then exit the Oracle Universal Installer and proceed to the next section, "Post-Installation Steps".

Post-Installation Steps

From the new node, perform the following post-installation steps:

Run the Oracle Net Configuration Assistant from the bin directory in ORACLE_HOME to configure a new listener for the node.
Complete any post-installation steps for your platform by referring to your platform-specific Oracle documentation.

Add a Database Instance on the New Node

To add an instance on the new node, make sure that the Global Services Daemon (GSD) is running on all the existing nodes as well as on the new node. If the GSD is not running, then from any of the nodes execute the gsdctl start command from the following platform-specific locations:

On UNIX from $ORACLE_HOME/bin
On Windows NT and Windows 2000 from %ORACLE_HOME%\bin

Then go to one of the existing nodes and perform the following steps:

Start the Database Configuration Assistant by typing dbca in the bin directory in $ORACLE_HOME on UNIX, or on Windows NT and Windows 2000, choose Start > Programs > Oracle - HOME_NAME > Configuration and Migration Tools > Database Configuration Assistant

The DBCA wizard starts and displays the Welcome page for Real Application Clusters shown in Figure 8-1.

Figure 8-1 DBCA Welcome Page for Real Application Clusters

Text description of welcome.gif follows.

Text description of the illustration welcome.gif

Note:

If the Welcome page for Real Application Clusters does not display, then the DBCA was unable to:

Communicate with the Cluster Manager (CM) software on Windows NT or Windows 2000
Detect the Global Cache Service (GCS) software or the lists of nodes in the cluster on UNIX operating systems

To resolve this, refer to your vendor's operating system-dependent documentation to correct the problem and restart the DBCA.

Select Oracle cluster database and click Next. After you click Next, the DBCA displays the Operations page as shown in Figure 8-2:

Figure 8-2 Database Configuration Assistant Operations Page

Text description of the illustration instmgt.gif
Select Instance Management and click Next.

After you click Next, the DBCA displays the Instance Management page shown in Figure 8-3.

Figure 8-3 Database Configuration Assistant Instance Management Page

Text description of the illustration addinst.gif
Select Add Instance and click Next.

After you click Next, the DBCA displays the List of Cluster Databases page that shows the databases and their current statuses, such as ACTIVE, or INACTIVE, as shown in Figure 8-4.

Figure 8-4 Database Configuration Assistant List of Cluster Databases Page

Text description of the illustration ldbadd.gif
Select an active Real Application Clusters database to which you want to add an instance from the List of Cluster Databases page. If your user ID is not operating-system authenticated, then the DBCA also prompts you for a user ID and password for a database user that has SYSDBA privileges. If the DBCA prompts you, then enter a valid user ID and password and click Next.

After you click Next, the DBCA displays the List of Cluster Database Instances page showing the names of the instances for the Real Application Clusters database that you selected as shown in Figure 8-5.

Figure 8-5 Database Configuration Assistant List of Cluster Database Instances Page

Text description of the illustration lofadd.gif
Click Next.

After you click Next, the DBCA displays the Adding an Instance page shown in Figure 8-6.

Figure 8-6 Database Configuration Assistant Adding an Instance Page

Text description of the illustration addedit.gif
Enter the instance name in the field at the top of this page if the instance name that the DBCA provides does not match your existing instance name sequence. Then select the new node name from the list and click Next.

After you click Next, the DBCA displays the Database Storage page as shown in Figure 8-7.

Figure 8-7 Database Configuration Assistant Database Storage Page

Text description of the illustration instor.gif
If your database uses automatic undo management and your platform supports a cluster file system, then skip this step unless you want to change the default undo tablespace datafile name that the DBCA provides. Otherwise, select the Tablespaces folder and expand it. Then select the undo tablespace storage object and a dialog appears on the right-hand side. Change the default datafile name to the raw device name for the tablespace. You can also change the name to the cluster file system file name if you do not want to use the default name. Oracle Corporation does not recommend that you change the storage properties for the undo tablespace.
If your database uses rollback undo management, then select the Rollback Segment folder to change the rollback segment names if you do not want to use the default rollback segment names that the DBCA provides. Oracle Corporation does not recommend that you change the storage properties for the rollback segments.
If your platform supports a cluster file system, then skip this step unless you want to change the default redo log group file names. Otherwise, select the Redo Log Groups folder and expand it. For each redo log group number you select, the DBCA displays another dialog box. For UNIX platforms, enter the raw device name that you created in the section "Create Raw Devices (Optional)" in the File Name field. On Windows NT or Windows 2000 platforms, enter the symbolic link name that you created in the section, "Create Links to Disk Partitions".
Repeat the previous step for all other redo log groups and click Finish.

After you click Finish, the DBCA displays a Summary dialog similar to the dialog in Figure 8-8.

Figure 8-8 Database Configuration Assistant Summary Dialog

Text description of the illustration summadd.gif
Review the information on the dialog and click OK, or click Cancel to end the instance addition operation.

If you click OK, then the DBCA displays a progress dialog that shows the DBCA performing the instance addition operation. During this operation the DBCA adds the instance and creates the instance's Oracle Net configuration. When the DBCA completes the instance addition operation, the DBCA displays a dialog asking whether you want to perform another operation. Click No to exit the DBCA, or click Yes and the DBCA displays the Operations screen shown in Figure 8-3 .

If you click No and continue the instance addition operation, then at this point you have accomplished the following:
- Created a new database instance on the new node
- Created the new instance's required services for Windows NT or Windows 2000
- Configured Oracle Net
- Started the required services for Windows NT or Windows 2000
- Started the new instance and its listeners
Deleting Instances

The following procedures explain how to delete an instance with the Database Configuration Assistant. Where noted, refer to previous figures in this chapter. To delete an instance:
1. Go to the DBCA Operations Page shown in Figure 8-2, select Instance Management and click Next.
After you click Next, the DBCA displays the Instance Management page shown in Figure 8-9.
1. Select Delete Instance and click Next.
Figure 8-9 DBCA Instance Management Page

Text description of the illustration delinst.gif

After you click Next, the DBCA displays the List of Cluster Databases page shown in Figure 8-10.

Figure 8-10 Database Configuration Assistant List of Cluster Databases

Text description of the illustration lstdbdel.gif
Select a Real Application Clusters database from which to delete an instance. If your user ID is not operating-system authenticated, then the DBCA also prompts you for a user ID and password for a database user that has SYSDBA privileges. If the DBCA prompts you for this, then enter a valid user ID and password. Click Next.

After you click Next, the DBCA displays the List of Cluster Database Instances page shown in Figure 8-11. The List of Cluster Database Instances page shows the instances associated with the Real Application Clusters database that you selected and the status of each instance.

Figure 8-11 Database Configuration Assistant List of Cluster Database Instances

Text description of the illustration listinst.gif
Select the instance to delete and click Finish.

If you select the local instance, then the DBCA displays an Error dialog as shown in Figure 8-12.

Figure 8-12 Database Configuration Assistant Error Dialog

Text description of the illustration dberror.gif
If the DBCA displays this error dialog, then click OK and reselect another instance on the List of Cluster Database Instances page and click Finish.
The DBCA displays the DBCA Summary dialog similar to the dialog in Figure 8-13.

Figure 8-13 Database Configuration Assistant Summary Dialog

Text description of the illustration delsum.gif
Review the information about the instance deletion operation on the Summary dialog and click OK. Otherwise, click Cancel to end the instance deletion operation.

If you click OK, then the DBCA displays a Confirmation dialog similar to the dialog in Figure 8-14.

Figure 8-14 Database Configuration Assistant Confirmation Dialog

Text description of the illustration delconf.gif
Click OK on the Confirmation dialog to proceed with the instance deletion operation.

After you click OK, the DBCA displays a progress dialog showing that the DBCA is performing the instance deletion operation. During this operation the DBCA removes the instance and the instance's Oracle Net configuration. When the DBCA completes this operation, the DBCA displays a dialog asking whether you want to perform another operation. Click No and exit the DBCA or click Yes to perform another operation. If you click Yes, then the DBCA displays the Operations page shown in Figure 8-3 .

At this point, you have accomplished the following:
- Stopped the listeners associated with the selected instance
- Deleted the selected database instance from the instance's configured node
- Deleted the selected instance's services for Windows NT or Windows 2000
- Removed the Oracle Net configuration
- Deleted the Oracle Flexible Architecture directory structure from the instance's configured node