<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Transitional//EN">
<HTML>
<HEAD>
  <!--
    *************************************************************************************
    * Copyright (c) 2005 VERITAS Software Corporation. All rights reserved. *

      For third-party license citations, please see the bottom of this file.
    *************************************************************************************
  -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="TEMPLATEBASE" content="vx_vxhelp_2005c">
<meta name="LASTUPDATED" content="06/15/05 07:50:20">
<meta name="PRODUCTNAME" content="VERITAS Software Documentation">
<meta name="FILENAME" content="13_troubleshoot.fm">
     
<title>System Panics to Prevent Potential Data Corruption</title>

</HEAD>

<BODY bgcolor="#FFFFFF" link="#000066" vlink="#0000ff" marginheight="0" topmargin="0">
<basefont=1>

<a name="107004"></a>

<div align="left">
<table border="0" cellspacing="0" cellpadding="0" width="100%">
 <tr>
   <td height="2" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#000000">
   </td>
 </tr>
 <tr>
   <td height="15" width="50%" border="0" cellspacing="0" cellpadding="0" bgcolor="#cccccc">
    <img src="images/vxsymbol.gif" height="20" width="31" border="0" cellspacing="0" cellpadding="0" align="left">
     
     
   </td>
   <td align="right" width="50%" bgcolor="#cccccc">

<!-- Header Breadcrumbs -->
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&lt;&nbsp;<a href="13_troubleshoot_sf_ora_rac126_121380.html" title="Previous - Removing Existing Keys From Disks">Previous</a>&nbsp;&nbsp;|&nbsp;&nbsp;<a href="0A_SampleMain_sf_ora_rac128_62674.html" title="Next - Appendix:  Sample Configuration Files">Next</a>
&nbsp;&gt;&nbsp;&nbsp;</font>
   </td>
 </tr>

 <tr>
  <td bgcolor="#CCCCCC" colspan="2" height="15" align="right">
  <font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Product:  
<a href="sf_ora_rac_splashpage_docCD_sf_ora_rac1_103036.vxhd" title="Storage Foundation for Oracle RAC Guides title page ">Storage Foundation for Oracle RAC Guides</a>
  &nbsp;&nbsp;
  </font>
  </td>
 </tr>
 <tr>
  <td bgcolor="#CCCCCC" colspan="2" height="15" align="right">
  <font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Manual:  
<a href="sfrac_icg_titlepage_docCD_sf_ora_rac2_73700.vxhd" title="Storage Foundation 4.1 for Oracle RAC Installation and Configuration title page ">Storage Foundation 4.1 for Oracle RAC Installation and Configuration</a>
  &nbsp;&nbsp;
  </font>
  </td>
 </tr>


</table>
</div>

<div align="left">
 <table border="0" width="100%" bgcolor="#FFFFFF">
  <tr>
   <td width="100%" valign="top" bgcolor="#FFFFFF">

<!-- Start Main Topic Content -->
        <a name="startcontent"></a>
	
<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="107004"></a> System Panics to Prevent Potential Data Corruption</font>
</h2>

        
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107006"></a>When a node experiences a split brain condition and is ejected from the cluster, it panics and displays the following console message:</font>
</p>

<a name="107007"></a><br><font face="Courier New, Courier, sansserif, sans serif" size="2">VXFEN:vxfen_plat_panic: Local cluster node ejected from cluster to prevent potential data corruption.</font>



<h3><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000"><a name="107008"></a>How vxfen Driver Checks for Pre-existing Split Brain Condition</font>
</h3>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107009"></a>The <font face="Courier New, Courier, sansserif, sans serif" size="2">vxfen</font> driver functions to prevent an ejected node from rejoining the cluster after the failure of the private network links and before the private network links are repaired. </font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107010"></a>For example, suppose the cluster of system 1 and system 2 is functioning normally when the private network links are broken. Also suppose system 1 is the ejected system. When system 1 restarts before the private network links are restored, its membership configuration does not show system 2; however, when it attempts to register with the coordinator disks, it discovers system 2 is registered with them. Given this conflicting information about system 2, system 1 does not join the cluster and returns an error from <font face="Courier New, Courier, sansserif, sans serif" size="2">vxfenconfig</font> that resembles:</font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107011"></a>vxfenconfig:&nbsp;ERROR:&nbsp;There&nbsp;exists&nbsp;the&nbsp;potential&nbsp;for&nbsp;a&nbsp;preexisting
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107012"></a>&nbsp;&nbsp;split-brain.&nbsp;The&nbsp;coordinator&nbsp;disks&nbsp;list&nbsp;no&nbsp;nodes&nbsp;which&nbsp;are&nbsp;in&nbsp;the
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107013"></a>&nbsp;&nbsp;current&nbsp;membership.&nbsp;However,&nbsp;they&nbsp;also&nbsp;list&nbsp;nodes&nbsp;which&nbsp;are&nbsp;not
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107014"></a>&nbsp;&nbsp;in&nbsp;the&nbsp;current&nbsp;membership.
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107016"></a>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107017"></a>I/O&nbsp;Fencing&nbsp;Disabled!
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107019"></a>Also, the following information is displayed on the console:</font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107020"></a>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;vxfen:&nbsp;WARNING:&nbsp;Potentially&nbsp;a&nbsp;preexisting
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107021"></a>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;split-brain.
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107022"></a>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;Dropping&nbsp;out&nbsp;of&nbsp;cluster.
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107023"></a>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;Refer&nbsp;to&nbsp;user&nbsp;documentation&nbsp;for&nbsp;steps
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107024"></a>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;required&nbsp;to&nbsp;clear&nbsp;preexisting&nbsp;split-brain.
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107025"></a>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107026"></a>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;I/O&nbsp;Fencing&nbsp;DISABLED!
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107027"></a>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107028"></a>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;gab:&nbsp;GAB:20032:&nbsp;Port&nbsp;b&nbsp;closed
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107029"></a>However, the same error can occur when the private network links are working and both systems go down, system 1 restarts, and system 2 fails to come back up. From the view of the cluster from system 1, system 2 may still have the registrations on the coordinator disks.</font>
</p>




<h3><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000"><a name="107030"></a>Case 1: System 2 Up, System 1 Ejected (Actual Potential Split Brain)</font>
</h3>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107031"></a>Determine if system1 is up or not. If it is up and running, shut it down and repair the private network links to remove the split brain condition. restart system 1.</font>
</p>




<h3><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000"><a name="107033"></a>Case 2: System 2 Down, System 1 Ejected (Apparent Potential Split Brain)</font>
</h3>
<ol>
<li value="1">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107034"></a>Physically verify that system 2 is down.
</font>
<li value="2">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107035"></a>Verify the systems currently registered with the coordinator disks. Use the following command:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="107036"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>vxfenadm&nbsp;-g&nbsp;all&nbsp;-f&nbsp;/etc/vxfentab</b></font>
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107037"></a>The output of this command identifies the keys registered with the coordinator disks.</font>
</p>
<li value="3">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107041"></a>Clear the keys on the coordinator disks as well as the data disks using the command <font face="Courier New, Courier, sansserif, sans serif" size="2">/opt/VRTSvcs/rac/bin/vxfenclearpre</font>. See <a href="13_troubleshoot_sf_ora_rac127_107004.html#109953">Using vxfenclearpre Command to Clear Keys After Split Brain</a>.
</font>
<li value="4">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="107042"></a>Make any necessary repairs to system 2 and restart.
</font>
</ol>



<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="109953"></a>Using vxfenclearpre Command to Clear Keys After Split Brain</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="109955"></a>When you have encountered a split brain condition, use the <font face="Courier New, Courier, sansserif, sans serif" size="2">vxfenclearpre</font> command to remove SCSI-3 registrations and reservations on the coordinator disks as well as on the data disks in all shared disk groups.</font>
</p>

<ol>
<li value="1">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="109967"></a>Shut down all other nodes in the cluster that have access to the shared storage. This prevents data corruption.
</font>
<li value="2">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="109970"></a>Start the script:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="109978"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>cd&nbsp;/opt/VRTSvcs/vxfen/bin</b></font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="109979"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>./vxfenclearpre</b></font>
</font>
<li value="3">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="109984"></a>Read the script's introduction and warning. Then, you can choose to let the script run.
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="109994"></a>&nbsp;&nbsp;Do&nbsp;you&nbsp;still&nbsp;want&nbsp;to&nbsp;continue:&nbsp;[y/n]&nbsp;(default&nbsp;:&nbsp;n)
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="109997"></a>&nbsp;&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>y</b></font>
</font>
<hr noshade size="2" width="100%" color="#000000">
<img src="images/note.gif" alt="Note " height="16" width="16">&nbsp;
<font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000">
<a name="110501"></a><b>Note  &nbsp;&nbsp;</b>
</font>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2" color="#000000">		Informational messages resembling the following may appear on the console of one of the nodes in the cluster when a node is ejected from a disk/LUN:<br><br><font face="Courier New, Courier, sansserif, sans serif" size="2">&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;scsi:&nbsp;WARNING:&nbsp;/sbus@3,0/lpfs@0,0/sd@0,1(sd91):<br>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;Error&nbsp;for&nbsp;Command:&nbsp;&lt;undecoded&nbsp;cmd&nbsp;0x5f&gt;&nbsp;Error&nbsp;Level:&nbsp;Informational<br>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;scsi:&nbsp;Requested&nbsp;Block:&nbsp;0&nbsp;Error&nbsp;Block&nbsp;0<br>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;scsi:&nbsp;Vendor:&nbsp;&lt;vendor&gt;&nbsp;Serial&nbsp;Number:&nbsp;0400759B006E<br>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;scsi:&nbsp;Sense&nbsp;Key:&nbsp;Unit&nbsp;Attention<br>&lt;date&gt;&nbsp;&lt;system&nbsp;name&gt;&nbsp;scsi:&nbsp;ASC:&nbsp;0x2a&nbsp;(&lt;vendor&nbsp;unique&nbsp;code&nbsp;0x2a&gt;),&nbsp;ASCQ:&nbsp;0x4,&nbsp;FRU:&nbsp;0x0<br></font><br>These informational messages may be ignored.
</font>
<hr noshade size="2" width="100%" color="#000000">
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110510"></a>&nbsp;&nbsp;Cleaning&nbsp;up&nbsp;the&nbsp;coordinator&nbsp;disks...
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110515"></a>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110520"></a>&nbsp;&nbsp;Cleaning&nbsp;up&nbsp;the&nbsp;data&nbsp;disks&nbsp;for&nbsp;all&nbsp;shared&nbsp;disk&nbsp;groups...
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110534"></a>&nbsp;&nbsp;
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110535"></a>&nbsp;&nbsp;Successfully&nbsp;removed&nbsp;SCSI-3&nbsp;persistent&nbsp;registration&nbsp;and
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110536"></a>&nbsp;&nbsp;reservations&nbsp;from&nbsp;the&nbsp;coordinator&nbsp;disks&nbsp;as&nbsp;well&nbsp;as&nbsp;the&nbsp;shared
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110538"></a>&nbsp;&nbsp;data&nbsp;disks.
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110105"></a>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110540"></a>&nbsp;&nbsp;Reboot&nbsp;the&nbsp;server&nbsp;to&nbsp;proceed&nbsp;with&nbsp;normal&nbsp;cluster&nbsp;startup...
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="110541"></a>&nbsp;&nbsp;#
</font>
<li value="4">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="110005"></a>Restart all nodes in the cluster.
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="109378"></a></font>
</p>
</ol>



<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="132935"></a>Removing or Adding Coordinator Disks </font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132936"></a>This section describes how to:</font>
</p>

<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132945"></a>Replace coordinator disk in the coordinator disk group</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132950"></a>Destroy a coordinator disk group</font>
</li>
<hr noshade size="2" width="100%" color="#000000">
<img src="images/note.gif" alt="Note " height="16" width="16">&nbsp;
<font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000">
<a name="132937"></a><b>Note  &nbsp;&nbsp;</b>
</font>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2" color="#000000">Adding or removing coordinator disks requires all services be shut down.
</font>
<hr noshade size="2" width="100%" color="#000000">
</ul>
</p>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132938"></a>Note the following about the procedure:</font>
</p>

<ul><p><img src="images/check.gif" alt="Checkmark">&nbsp;
<a name="132755"></a><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2">A coordinator disk group requires an odd number (three minimum) of disks/LUNs. </font>
</p><p><img src="images/check.gif" alt="Checkmark">&nbsp;
<a name="132758"></a><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2">When adding a disk, add the disk to the disk group <font face="Courier New, Courier, sansserif, sans serif" size="2">vxfencoorddg</font> and retest the group for support of SCSI-3 persistent reservations. </font>
</p><p><img src="images/check.gif" alt="Checkmark">&nbsp;
<a name="132760"></a><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2">You can destroy the coordinator disk group such that no registration keys remain on the disks. The disks can then be used elsewhere.</font>
</p></ul></ol>
<p><img src="images/stepintro.gif" height="16" width="16">
<font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000"><a name="132762"></a>&nbsp;&nbsp;<b>To remove and replace a disk in the coordinator disk group</b></font>
</p>
<ol>
<li value="1">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132763"></a>Log in as root user on one of the cluster nodes.
</font>
<li value="2">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132764"></a>If VCS is running, shut it down:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="132765"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>hastop&nbsp;-all</b></font>
</font>
<li value="3">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="139305"></a>Stop the VCSMM driver on each node:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="139306"></a>&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>/sbin/vcsmmconfig&nbsp;-U</b></font>
</font>
<li value="4">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="139308"></a>Stop I/O fencing on all nodes:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="133535"></a>&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>/sbin/init.d/vxfen&nbsp;stop</b></font>
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138622"></a>This removes any registration keys on the disks.</font>
</p>
<li value="5">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138623"></a>Import the coordinator disk group. The file <font face="Courier New, Courier, sansserif, sans serif" size="2">/etc/vxfendg</font> includes the name of the disk group (typically, <font face="Courier New, Courier, sansserif, sans serif" size="2">vxfencoorddg</font>) that contains the coordinator disks, so use the command:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="132774"></a>&nbsp;&nbsp;#<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>&nbsp;vxdg&nbsp;-tfC&nbsp;import&nbsp;'cat&nbsp;/etc/vxfendg'</b></font>
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132775"></a>where: </font>
</p>
<ul><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132776"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">-t</font> specifies that the disk group is imported only until the node restarts. </font>
</p>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132777"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">-f</font> specifies that the import is to be done forcibly, which is necessary if one or more disks is not accessible. </font>
</p>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132778"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">-C</font> specifies that any import blocks are removed.</font>
</p>
</ul><li value="6">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132883"></a>To remove disks from the disk group, use the VxVM disk administrator utility, <font face="Courier New, Courier, sansserif, sans serif" size="2">vxdiskadm</font>. 
</font>
<hr noshade size="2" width="100%" color="#000000">
<img src="images/note.gif" alt="Note " height="16" width="16">&nbsp;
<font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000">
<a name="132971"></a><b>Note  &nbsp;&nbsp;</b>
</font>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2" color="#000000">You may also destroy the existing coordinator disk group. For example:<br><br>   				# <font face="Courier New, Courier, sansserif, sans serif" size="2"><b>vxdg destroy vxfencoorddg</b></font>
</font>
<hr noshade size="2" width="100%" color="#000000">
<li value="7">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="133020"></a>Add the new disk to the node, initialize it as a VxVM disk, and add it to the <font face="Courier New, Courier, sansserif, sans serif" size="2">vxfencoorddg</font> disk group. Refer to <a href="03_installsfrac_sf_ora_rac32_361367.html#319237">Creating the vxfencoorddg Disk Group</a>
</font>
<li value="8">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138637"></a>Test the recreated disk group for SCSI-3 persistent reservations compliance. Refer to <a href="03_installsfrac_sf_ora_rac32_361367.html#319270">Requirements for Testing the Coordinator Disk Group</a>.
</font>
<li value="9">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138642"></a>After replacing disks in a coordinator disk group, deport the disk group:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="138643"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>vxdg&nbsp;deport&nbsp;'cat&nbsp;/etc/vxfendg'</b></font>
</font>
<li value="10">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138614"></a>On each node, start the I/O fencing driver:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="138615"></a>#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>/sbin/init.d/vxfen&nbsp;start</b></font>
</font>
<li value="11">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="138662"></a>On each node, start the VCSMM driver:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="138610"></a>#&nbsp;/<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>sbin/init.d/vcsmm&nbsp;start</b></font>
</font>
<li value="12">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="132797"></a>If necessary, restart VCS on each node:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="132798"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>hastart</b></font>
</font>
</ol>

<!-- End Main Topic Content -->

   </td>
  </tr>
 </table>
</div>

<div>
<table border="0" cellspacing="0" cellpadding="0" width="100%">
 <tr>
  <td height="18" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#FFFFFF">
  </td>
 </tr>
  <td align="left"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&nbsp;&#94;&nbsp;<a href="13_troubleshoot_sf_ora_rac127_107004.html" title="System Panics to Prevent Potential Data Corruption">Return to Top</a></font>

  </td>
  <td align="right"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&lt;&nbsp;<a href="13_troubleshoot_sf_ora_rac126_121380.html" title="Previous - Removing Existing Keys From Disks">Previous</a>&nbsp;&nbsp;|&nbsp;&nbsp;<a href="0A_SampleMain_sf_ora_rac128_62674.html" title="Next - Appendix:  Sample Configuration Files">Next</a>
&nbsp;&gt;&nbsp;&nbsp;</font>
  </td>
 <tr>
   <td height="9" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#FFFFFF">
   </td>
 </tr>

<!-- Footer Breadcrumbs -->
<tr>
<td bgcolor="#FFFFFF" colspan="2" height="15" align="right">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Product:  
<a href="sf_ora_rac_splashpage_docCD_sf_ora_rac1_103036.vxhd" title="Storage Foundation for Oracle RAC Guides title page ">Storage Foundation for Oracle RAC Guides</a>&nbsp;&nbsp;</font></td>
</tr>

<tr>
<td bgcolor="#FFFFFF" colspan="2" height="15" align="right">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Manual:  
<a href="sfrac_icg_titlepage_docCD_sf_ora_rac2_73700.vxhd" title="Storage Foundation 4.1 for Oracle RAC Installation and Configuration title page ">Storage Foundation 4.1 for Oracle RAC Installation and Configuration</a>&nbsp;&nbsp;</font></td>
</tr>


 <tr>
  <td colspan="2" align="left"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" color="#000000" align="right" size="-2">
<em>VERITAS Software Corporation
</em></font><br>

<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">www.veritas.com</font><br>
  </td>
 </tr>
</table>
</div>

</BODY>
  <!--
    *************************************************************************************
    * Copyright (c) 2005 VERITAS Software Corporation. All rights reserved. *

    VERITAS, VERITAS SOFTWARE, the VERITAS Logo, and VERITAS Software Documentation 
    are trademarks or registered trademarks of VERITAS Software Corporation 
    or its affiliates in the U.S. and other countries. Other names may be 
    trademarks of their respective owners. 

    Apache Jakarta Lucene:
    Apache Software License
    Version 1.1

    Copyright (c) 2000 The Apache Software Foundation. All rights reserved.

    Redistribution and use in source and binary forms, with or without modification, 
    are permitted provided that the following conditions are met:

    1. Redistributions of source code must retain the above copyright notice, this 
    list of conditions and the following disclaimer.

    2. Redistributions in binary form must reproduce the above copyright notice, this 
    list of conditions and the following disclaimer in the documentation and/or other 
    materials provided with the distribution.

    3. The end-user documentation included with the redistribution, if any, must include 
    the following acknowledgment: "This product includes software developed by the Apache 
    Software Foundation (http://www.apache.org/)." Alternately, this acknowledgment may 
    appear in the software itself, if and wherever such third-party acknowledgments 
    normally appear.

    4. The names "Apache" and "Apache Software Foundation" must not be used to endorse or 
    promote products derived from this software without prior written permission. For 
    written permission, please contact apache@apache.org.

    5. Products derived from this software may not be called "Apache", nor may "Apache" 
    appear in their name, without prior written permission of the Apache Software Foundation.

    THIS SOFTWARE IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, 
    BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A 
    PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION 
    OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, 
    OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 
    OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 
    ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 
    NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF 
    ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    This software consists of voluntary contributions made by many individuals on behalf 
    of the Apache Software Foundation. For more information on the Apache Software Foundation, 
    please see <http://www.apache.org/>. Portions of this software are based upon public domain 
    software originally written at the National Center for Supercomputing Applications, University 
    of Illinois, Urbana-Champaign.
    
    This product includes WebWindow Software developed by Horst Heistermann.
    *************************************************************************************
  -->
</HTML>
