<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Transitional//EN">
<HTML>
<HEAD>
  <!--
    *************************************************************************************
    * Copyright (c) 2005 VERITAS Software Corporation. All rights reserved. *

      For third-party license citations, please see the bottom of this file.
    *************************************************************************************
  -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="TEMPLATEBASE" content="vx_vxhelp_2005c">
<meta name="LASTUPDATED" content="06/15/05 07:50:20">
<meta name="PRODUCTNAME" content="VERITAS Software Documentation">
<meta name="FILENAME" content="13_troubleshoot.fm">
     
<title>Troubleshooting Topics</title>

</HEAD>

<BODY bgcolor="#FFFFFF" link="#000066" vlink="#0000ff" marginheight="0" topmargin="0">
<basefont=1>

<a name="106925"></a>

<div align="left">
<table border="0" cellspacing="0" cellpadding="0" width="100%">
 <tr>
   <td height="2" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#000000">
   </td>
 </tr>
 <tr>
   <td height="15" width="50%" border="0" cellspacing="0" cellpadding="0" bgcolor="#cccccc">
    <img src="images/vxsymbol.gif" height="20" width="31" border="0" cellspacing="0" cellpadding="0" align="left">
     
     
   </td>
   <td align="right" width="50%" bgcolor="#cccccc">

<!-- Header Breadcrumbs -->
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&lt;&nbsp;<a href="13_troubleshoot_sf_ora_rac122_106905.html" title="Previous - Running Scripts for Engineering Support Analysis">Previous</a>&nbsp;&nbsp;|&nbsp;&nbsp;<a href="13_troubleshoot_sf_ora_rac124_123500.html" title="Next - VCSIPC Wait Warning Messages in Oracle Trace/Log Files">Next</a>
&nbsp;&gt;&nbsp;&nbsp;</font>
   </td>
 </tr>

 <tr>
  <td bgcolor="#CCCCCC" colspan="2" height="15" align="right">
  <font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Product:  
<a href="sf_ora_rac_splashpage_docCD_sf_ora_rac1_103036.vxhd" title="Storage Foundation for Oracle RAC Guides title page ">Storage Foundation for Oracle RAC Guides</a>
  &nbsp;&nbsp;
  </font>
  </td>
 </tr>
 <tr>
  <td bgcolor="#CCCCCC" colspan="2" height="15" align="right">
  <font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Manual:  
<a href="sfrac_icg_titlepage_docCD_sf_ora_rac2_73700.vxhd" title="Storage Foundation 4.1 for Oracle RAC Installation and Configuration title page ">Storage Foundation 4.1 for Oracle RAC Installation and Configuration</a>
  &nbsp;&nbsp;
  </font>
  </td>
 </tr>


</table>
</div>

<div align="left">
 <table border="0" width="100%" bgcolor="#FFFFFF">
  <tr>
   <td width="100%" valign="top" bgcolor="#FFFFFF">

<!-- Start Main Topic Content -->
        <a name="startcontent"></a>
	
<h1><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="+1" color="#000000"><a name="106925"></a> Troubleshooting Topics</font>
</h1>
         
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112639"></a>The following troubleshooting topics have headings that indicate likely symptoms or that indicate procedures required for a solution.</font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="131164"></a>Oracle User Must be Able to Read /etc/llttab File </font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="131165"></a>Check the permissions of the <font face="Courier New, Courier, sansserif, sans serif" size="2">/etc/llttab</font> file; the <font face="Courier New, Courier, sansserif, sans serif" size="2">oracle</font> user must be allowed to read it.</font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="131272"></a>Missing Dialog Box During Installation of Oracle9i Release 2</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="131273"></a>During installation of Oracle9i Release 2 using the <font face="Courier New, Courier, sansserif, sans serif" size="2">runInstaller</font> utility, if you choose the Enterprise Edition or Custom Install (with RAC option), a dialog box prompting you about the installation nodes should appear. If the dialog box fails to appear, exit the <font face="Courier New, Courier, sansserif, sans serif" size="2">runInstaller</font> utility and complete the following steps:</font>
</p>

<ol>
<li value="1">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138429"></a>On the cluster node, check for the VERITAS CM library.
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="139631"></a>For PA clusters:</font>
</p>
<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138430"></a> The 32-bit libnmapi2.sl is located in <font face="Courier New, Courier, sansserif, sans serif" size="2">/opt/nmapi/nmapi2/lib/</font>. </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="139626"></a>The 64-bit libnmapi2.sl is located in <font face="Courier New, Courier, sansserif, sans serif" size="2">/opt/nmapi/nmapi2/lib/pa20_64</font>.</font>
</li>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="139653"></a>For IA clusters:</font>
</p>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="139658"></a>The 32-bit libnmapi2.so is located in <font face="Courier New, Courier, sansserif, sans serif" size="2">/opt/nmapi/nmapi2/lib/<br>hpux32/lib/</font>. </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="139670"></a>The 64-bit libnmapi2.so is located in <font face="Courier New, Courier, sansserif, sans serif" size="2">/opt/nmapi/nmapi2/lib/<br>hpux64/lib/</font>.</font>
</li>
</ul>
</p>
<li value="2">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="131280"></a>Start the VCSMM driver on both the nodes by entering: 
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="131281"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>/sbin/vcsmmconfig&nbsp;-c</b></font>
</font>
</ol>



<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="129949"></a>Error When Starting an Oracle Instance</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="106926"></a>If the VCSMM driver (the membership module) is not configured, an error is displayed on starting the Oracle instance that resembles:</font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="106927"></a>ORA-29702:&nbsp;error&nbsp;occurred&nbsp;in&nbsp;Cluster&nbsp;Group&nbsp;Operation
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="106928"></a>To start the driver, enter the following command:</font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="106929"></a>#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>/sbin/vcsmmconfig&nbsp;-c&nbsp;</b></font>
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="106934"></a>The command included in the <font face="Courier New, Courier, sansserif, sans serif" size="2">/etc/vcsmmtab</font> file enables the VCSMM driver to be started at system boot. </font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="108900"></a>Instance Numbers Must be Unique (Error Code 205)</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="108901"></a>If you encounter error code 205 when the <font face="Courier New, Courier, sansserif, sans serif" size="2">skgxnreg</font> function fails (look in the Oracle trace files to find the error returned), make sure there is a unique instance number specified in the <font face="Courier New, Courier, sansserif, sans serif" size="2">$ORACLE_HOME/dbs/init${ORACLE_SID}.ora</font> file on each node.</font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="108946"></a>ORACLE_SID Must be Unique (Error Code 304)</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="108948"></a>If you encounter error code 304 when the <font face="Courier New, Courier, sansserif, sans serif" size="2">skgxnreg</font> function fails (look in the Oracle trace file to find the error returned), make sure that the ORACLE_SID environment variable specified during Oracle startup is unique on each node in your cluster. Also, make sure that the <font face="Courier New, Courier, sansserif, sans serif" size="2">SID</font> attribute for the Oracle resource in the <font face="Courier New, Courier, sansserif, sans serif" size="2">main.cf</font> is specified locally and is unique.</font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="106937"></a>Oracle Log Files Show Shutdown Called Even When Not Shutdown Manually</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="106938"></a>The Oracle enterprise agent calls shutdown if monitoring of the Oracle/Netlsnr resources fails for some reason. On all cluster nodes, look at the following VCS and Oracle agent log files for any errors or status:</font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="106939"></a>/var/VRTSvcs/log/engine_A.log
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="106940"></a>/var/VRTSvcs/log/Oracle_A.log
</font>



<h3><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000"><a name="138743"></a>Set MLOCK Privilege for DBA User</font>
</h3>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138766"></a>If ASYNCH_IO errors occur during select and update queries on the Oracle database, the workaround involves setting the MLOCK privilege for the dba user:</font>
</p>

<ol>
<li value="1">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138776"></a>Give the MLOCK privilege to the dba group:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="138796"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>setprivgrp&nbsp;dba&nbsp;MLOCK</b></font>
</font>
<li value="2">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138791"></a>Create the <font face="Courier New, Courier, sansserif, sans serif" size="2">/etc/privgroup</font> file and add the line:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="138809"></a>&nbsp;&nbsp;dba&nbsp;MLOCK
</font>
<li value="3">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138813"></a>Verify the availability of MLOCK privilege for the dba group:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="138817"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>/usr/bin/getprivgrp&nbsp;dba</b></font>
</font>
</ol>



<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="112711"></a>File System Configured Incorrectly for ODM Shuts Down Oracle</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112712"></a>Linking Oracle9i with the VERITAS ODM libraries provides the best file system performance. See <a href="04_oracle_install_sf_ora_rac45_263235.html#263235">Linking to VERITAS Libraries</a> for instructions on creating the link and confirming that Oracle uses the libraries. Shared file systems in RAC clusters without ODM Libraries linked to Oracle9i may exhibit slow performance and are <em>not</em> supported. </font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112716"></a>If ODM cannot find the resources it needs to provide support for cluster file systems, it does not allow Oracle to identify cluster files and causes Oracle to fail at startup. Run the following command:</font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="112717"></a>#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>cat&nbsp;/dev/odm/cluster</b></font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="112718"></a>cluster&nbsp;status:&nbsp;enabled
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112719"></a>If the status is &quot;enabled,&quot; ODM is supporting cluster files. Any other cluster status indicates that ODM is not supporting cluster files. Other possible values include:
<Table width="100%" border="0" cellpadding="1" cellspacing="0" >
<tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112722"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">pending</font></font>
</p>
</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112724"></a>ODM cannot yet communicate with its peers, but anticipates being able to eventually.</font>
</p>
</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112726"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">failed</font></font>
</p>
</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112728"></a>ODM cluster support has failed to initialize properly. Check console logs.</font>
</p>
</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112730"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">disabled</font></font>
</p>
</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112732"></a>ODM is not supporting cluster files. If you think it should, check:</font>
</p>
<ul compact><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112733"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">/dev/odm</font> mount options in <font face="Courier New, Courier, sansserif, sans serif" size="2">/etc/vfstab</font>. If the &quot;<font face="Courier New, Courier, sansserif, sans serif" size="2">nocluster</font>&quot; option is being used, it can force the &quot;<font face="Courier New, Courier, sansserif, sans serif" size="2">disabled</font>&quot; cluster support state.</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112734"></a>Make sure the <font face="Courier New, Courier, sansserif, sans serif" size="2">VRTSgms</font> (group messaging service) package is installed. Run the following command:</font>
</li>
</ul><ul compact><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112735"></a>   # <font face="Courier New, Courier, sansserif, sans serif" size="2"><b>/opt/VRTSvcs/bin/chk_sfrac_pkgs</b></font></font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112736"></a>The utility reports any missing required packages.</font>
</li>
</ul></td></tr>
</Table>


<Table width="100%" border="0" cellpadding="1" cellspacing="0">
<tr><td>
</Table></font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="112705"></a>If <font face="Courier New, Courier, sansserif, sans serif" size="2">/dev/odm</font> is not mounted, no status can be reported.</font>
</p>


<!-- End Main Topic Content -->

   </td>
  </tr>
 </table>
</div>

<div>
<table border="0" cellspacing="0" cellpadding="0" width="100%">
 <tr>
  <td height="18" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#FFFFFF">
  </td>
 </tr>
  <td align="left"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&nbsp;&#94;&nbsp;<a href="13_troubleshoot_sf_ora_rac123_106925.html" title="Troubleshooting Topics">Return to Top</a></font>

  </td>
  <td align="right"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&lt;&nbsp;<a href="13_troubleshoot_sf_ora_rac122_106905.html" title="Previous - Running Scripts for Engineering Support Analysis">Previous</a>&nbsp;&nbsp;|&nbsp;&nbsp;<a href="13_troubleshoot_sf_ora_rac124_123500.html" title="Next - VCSIPC Wait Warning Messages in Oracle Trace/Log Files">Next</a>
&nbsp;&gt;&nbsp;&nbsp;</font>
  </td>
 <tr>
   <td height="9" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#FFFFFF">
   </td>
 </tr>

<!-- Footer Breadcrumbs -->
<tr>
<td bgcolor="#FFFFFF" colspan="2" height="15" align="right">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Product:  
<a href="sf_ora_rac_splashpage_docCD_sf_ora_rac1_103036.vxhd" title="Storage Foundation for Oracle RAC Guides title page ">Storage Foundation for Oracle RAC Guides</a>&nbsp;&nbsp;</font></td>
</tr>

<tr>
<td bgcolor="#FFFFFF" colspan="2" height="15" align="right">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Manual:  
<a href="sfrac_icg_titlepage_docCD_sf_ora_rac2_73700.vxhd" title="Storage Foundation 4.1 for Oracle RAC Installation and Configuration title page ">Storage Foundation 4.1 for Oracle RAC Installation and Configuration</a>&nbsp;&nbsp;</font></td>
</tr>


 <tr>
  <td colspan="2" align="left"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" color="#000000" align="right" size="-2">
<em>VERITAS Software Corporation
</em></font><br>

<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">www.veritas.com</font><br>
  </td>
 </tr>
</table>
</div>

</BODY>
  <!--
    *************************************************************************************
    * Copyright (c) 2005 VERITAS Software Corporation. All rights reserved. *

    VERITAS, VERITAS SOFTWARE, the VERITAS Logo, and VERITAS Software Documentation 
    are trademarks or registered trademarks of VERITAS Software Corporation 
    or its affiliates in the U.S. and other countries. Other names may be 
    trademarks of their respective owners. 

    Apache Jakarta Lucene:
    Apache Software License
    Version 1.1

    Copyright (c) 2000 The Apache Software Foundation. All rights reserved.

    Redistribution and use in source and binary forms, with or without modification, 
    are permitted provided that the following conditions are met:

    1. Redistributions of source code must retain the above copyright notice, this 
    list of conditions and the following disclaimer.

    2. Redistributions in binary form must reproduce the above copyright notice, this 
    list of conditions and the following disclaimer in the documentation and/or other 
    materials provided with the distribution.

    3. The end-user documentation included with the redistribution, if any, must include 
    the following acknowledgment: "This product includes software developed by the Apache 
    Software Foundation (http://www.apache.org/)." Alternately, this acknowledgment may 
    appear in the software itself, if and wherever such third-party acknowledgments 
    normally appear.

    4. The names "Apache" and "Apache Software Foundation" must not be used to endorse or 
    promote products derived from this software without prior written permission. For 
    written permission, please contact apache@apache.org.

    5. Products derived from this software may not be called "Apache", nor may "Apache" 
    appear in their name, without prior written permission of the Apache Software Foundation.

    THIS SOFTWARE IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, 
    BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A 
    PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION 
    OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, 
    OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 
    OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 
    ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 
    NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF 
    ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    This software consists of voluntary contributions made by many individuals on behalf 
    of the Apache Software Foundation. For more information on the Apache Software Foundation, 
    please see <http://www.apache.org/>. Portions of this software are based upon public domain 
    software originally written at the National Center for Supercomputing Applications, University 
    of Illinois, Urbana-Champaign.
    
    This product includes WebWindow Software developed by Horst Heistermann.
    *************************************************************************************
  -->
</HTML>
