<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Transitional//EN">
<HTML>
<HEAD>
  <!--
    *************************************************************************************
    * Copyright (c) 2005 VERITAS Software Corporation. All rights reserved. *

      For third-party license citations, please see the bottom of this file.
    *************************************************************************************
  -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="TEMPLATEBASE" content="vx_vxhelp_2005c">
<meta name="LASTUPDATED" content="06/15/05 07:49:47">
<meta name="PRODUCTNAME" content="VERITAS Software Documentation">
<meta name="FILENAME" content="03_installsfrac.fm">
     
<title>Setting Up Shared Storage and I/O Fencing for SFRAC</title>

</HEAD>

<BODY bgcolor="#FFFFFF" link="#000066" vlink="#0000ff" marginheight="0" topmargin="0">
<basefont=1>

<a name="157546"></a>

<div align="left">
<table border="0" cellspacing="0" cellpadding="0" width="100%">
 <tr>
   <td height="2" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#000000">
   </td>
 </tr>
 <tr>
   <td height="15" width="50%" border="0" cellspacing="0" cellpadding="0" bgcolor="#cccccc">
    <img src="images/vxsymbol.gif" height="20" width="31" border="0" cellspacing="0" cellpadding="0" align="left">
     
     
   </td>
   <td align="right" width="50%" bgcolor="#cccccc">

<!-- Header Breadcrumbs -->
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&lt;&nbsp;<a href="03_installsfrac_sf_ora_rac30_395309.html" title="Previous - Using the installsfrac -configure Script to Configure SFRAC">Previous</a>&nbsp;&nbsp;|&nbsp;&nbsp;<a href="03_installsfrac_sf_ora_rac32_361367.html" title="Next - Testing Disks Using vxfentsthdw">Next</a>
&nbsp;&gt;&nbsp;&nbsp;</font>
   </td>
 </tr>

 <tr>
  <td bgcolor="#CCCCCC" colspan="2" height="15" align="right">
  <font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Product:  
<a href="sf_ora_rac_splashpage_docCD_sf_ora_rac1_103036.vxhd" title="Storage Foundation for Oracle RAC Guides title page ">Storage Foundation for Oracle RAC Guides</a>
  &nbsp;&nbsp;
  </font>
  </td>
 </tr>
 <tr>
  <td bgcolor="#CCCCCC" colspan="2" height="15" align="right">
  <font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Manual:  
<a href="sfrac_icg_titlepage_docCD_sf_ora_rac2_73700.vxhd" title="Storage Foundation 4.1 for Oracle RAC Installation and Configuration title page ">Storage Foundation 4.1 for Oracle RAC Installation and Configuration</a>
  &nbsp;&nbsp;
  </font>
  </td>
 </tr>


</table>
</div>

<div align="left">
 <table border="0" width="100%" bgcolor="#FFFFFF">
  <tr>
   <td width="100%" valign="top" bgcolor="#FFFFFF">

<!-- Start Main Topic Content -->
        <a name="startcontent"></a>
	
<h1><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="+1" color="#000000"><a name="157546"></a>Setting Up Shared Storage and I/O Fencing for SFRAC</font>
</h1>
 
  
  
  

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="362945"></a>The shared storage for VERITAS Storage Foundation <em>for Oracle RAC</em> must support SCSI-3 persistent reservations to enable I/O fencing (see <a href="01_oview_rep_sf_ora_rac19_269179.html#269179">I/O Fencing</a>). SFRAC involves two types of shared storage: data disks to store shared data, and coordinator disks, which are small LUNs (typically three per cluster), to control access to data disks by the nodes.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="326621"></a>Setting up shared storage and I/O fencing involve the following tasks:</font>
</p>

<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="326622"></a>Adding disks. See <a href="03_installsfrac_sf_ora_rac31_157546.html#192348">Adding and Initializing Disks</a>.</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="326483"></a>Testing data disks. See <a href="03_installsfrac_sf_ora_rac32_361367.html#361367">Testing Disks Using vxfentsthdw</a>.</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="326486"></a>Setting up coordinator disks. See <a href="03_installsfrac_sf_ora_rac32_361367.html#149851">Setting Up Coordinator Disks</a>.</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="334291"></a>Editing the VCS configuration file for I/O fencing. See <a href="03_installsfrac_sf_ora_rac32_361367.html#363576">Editing the UseFence Attribute in VCS Configuration</a>.</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="334295"></a>Stopping and starting each node to bring up the cluster configuration with I/O fencing enabled. See <a href="03_installsfrac_sf_ora_rac32_361367.html#202808">Starting VCS, CVM, and CFS on All Nodes</a>.</font>
</li>
</ul>
</p>



<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="192348"></a>Adding and Initializing Disks</font>
</h2>
 
<hr noshade size="2" width="100%" color="#000000">
<img src="images/note.gif" alt="Note " height="16" width="16">&nbsp;
<font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000">
<a name="361332"></a><b>Note  &nbsp;&nbsp;</b>
</font>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2" color="#000000">Install the driver and HBA card. Refer to the documentation from the vendor for instructions.
</font>
<hr noshade size="2" width="100%" color="#000000">
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="192349"></a>After you physically add shared disks to the nodes, you must initialize them as VxVM disks. Use the following examples; see the <em>VERITAS Volume Manager Administrator's Guide</em> for more information on adding and configuring disks.</font>
</p>

<ol>
<li value="1">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="374577"></a>Use the <font face="Courier New, Courier, sansserif, sans serif" size="2">ioscan</font> and <font face="Courier New, Courier, sansserif, sans serif" size="2">insf</font> commands on each node to make the new disks recognizable:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="374578"></a>#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>ioscan&nbsp;-C&nbsp;disk</b></font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="374579"></a>#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>insf&nbsp;-e</b></font>
</font>
<hr noshade size="2" width="100%" color="#000000">
<img src="images/caution.gif" alt="Caution " height="16" width="16">&nbsp;<font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000"><a name="374580"></a><b>Caution  &nbsp;&nbsp;</b></font>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2" color="#000000"> The HP-UX man page for the <font face="Courier New, Courier, sansserif, sans serif" size="2">insf</font> command instructs you to run the command in single-user mode only. Running <font face="Courier New, Courier, sansserif, sans serif" size="2">insf -e</font> in multi-user mode is acceptable assuming no other user is accessing any of the device files. This command can change the mode, owner, or group of an existing special (device) file, or unlink and recreate a file. Special files that are currently open may be left in an indeterminate state.</font>
<hr noshade size="2" width="100%" color="#000000">
<li value="2">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="374725"></a>Verify that the Array Support Library (ASL) for the disk array is installed on each of the nodes. Run the following command on each node and examine the output to verify the installation of ASL:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="374582"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>vxddladm&nbsp;listsupport&nbsp;all</b></font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="374583"></a>&nbsp;&nbsp;LIB_NAME&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;ARRAY_TYPE&nbsp;&nbsp;&nbsp;VID&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;PID
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="374584"></a>&nbsp;&nbsp;==============================================================&nbsp;
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="374585"></a>&nbsp;&nbsp;.
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="374586"></a>&nbsp;&nbsp;libvxemc.sl&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;A/A&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;EMC&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;SYMMETRIX
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="374587"></a>&nbsp;&nbsp;.
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="374588"></a>If the ASL for the array you are adding is not installed, obtain and install it on each node before proceeding. The ASL for the supported storage device you are adding is available from the disk array vendor or VERITAS technical support.</font>
</p>
<li value="3">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="374752"></a>Scan all disk drives and their attributes, update the VxVM device list, and reconfigure DMP with the new devices. Type:
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="379295"></a>&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>vxdisk&nbsp;scandisks</b></font>
</font>
<li value="4">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="379296"></a>To initialize the disks as VxVM disks, use one of the following methods:
</font>
<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="312277"></a>Use the interactive <font face="Courier New, Courier, sansserif, sans serif" size="2">vxdiskadm</font> utility to initialize the disks as VxVM disks. </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="313712"></a>Use the <font face="Courier New, Courier, sansserif, sans serif" size="2">vxdisksetup</font> command to initialize a disk as a VxVM disk. The example that follows specifies the CDS format:</font>
</li>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="313713"></a>&nbsp;&nbsp;&nbsp;&nbsp;vxdisksetup&nbsp;-i&nbsp;<em>device_name</em>&nbsp;format=cdsdisk
</font>
<ul><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="313714"></a>For example:</font>
</p>
</ul><br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="312283"></a>&nbsp;&nbsp;&nbsp;&nbsp;#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>vxdisksetup&nbsp;-i&nbsp;c2t0d2&nbsp;format=cdsdisk</b></font>
</font>
</ul>
</p>
</ol>



<h3><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000"><a name="181195"></a>Verifying the Nodes See the Same Disk</font>
</h3>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="184388"></a>To confirm whether a disk (or LUN) supports SCSI-3 persistent reservations, two nodes must simultaneously have access to the same disks. Because a shared disk is likely to have a different name on each node, check the serial number to verify the identity of the disk. Use the <font face="Courier New, Courier, sansserif, sans serif" size="2">vxfenadm</font> command with the <font face="Courier New, Courier, sansserif, sans serif" size="2">-i</font> option to verify that the same serial number for the LUN is returned on all paths to the LUN.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="181197"></a>For example, an EMC array is accessible by the <font face="Courier New, Courier, sansserif, sans serif" size="2">/dev/rdsk/c2t13d0</font> path on node A and by the <font face="Courier New, Courier, sansserif, sans serif" size="2">/dev/rdsk/c2t11d0</font> path on node B. From node A, enter:</font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181198"></a>#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>vxfenadm&nbsp;-i&nbsp;/dev/rdsk/c2t13d0</b></font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181199"></a>Vendor&nbsp;id&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;EMC&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181200"></a>Product&nbsp;id&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;SYMMETRIX&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181201"></a>Revision&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;5567
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181202"></a>Serial&nbsp;Number&nbsp;&nbsp;&nbsp;:&nbsp;42031000a
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="181203"></a>The same serial number information should appear when you enter the equivalent command on node B using the <font face="Courier New, Courier, sansserif, sans serif" size="2">/dev/rrdsk/c2t11d0</font> path.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="181204"></a>On a disk from another manufacturer, Hitachi Data Systems, the output is different and may resemble:</font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181205"></a>#&nbsp;<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>vxfenadm&nbsp;-i&nbsp;/dev/rdsk/c2t0d2</b></font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181206"></a>Vendor&nbsp;id&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;HITACHI&nbsp;
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181207"></a>Product&nbsp;id&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;OPEN-3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;-HP
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181208"></a>Revision&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;:&nbsp;0117
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="181209"></a>Serial&nbsp;Number&nbsp;&nbsp;&nbsp;:&nbsp;0401EB6F0002
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="174249"></a>Refer to the <font face="Courier New, Courier, sansserif, sans serif" size="2">vxfenadm</font>(1M) manual page.</font>
</p>


<!-- End Main Topic Content -->

   </td>
  </tr>
 </table>
</div>

<div>
<table border="0" cellspacing="0" cellpadding="0" width="100%">
 <tr>
  <td height="18" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#FFFFFF">
  </td>
 </tr>
  <td align="left"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&nbsp;&#94;&nbsp;<a href="03_installsfrac_sf_ora_rac31_157546.html" title="Setting Up Shared Storage and I/O Fencing for SFRAC">Return to Top</a></font>

  </td>
  <td align="right"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&lt;&nbsp;<a href="03_installsfrac_sf_ora_rac30_395309.html" title="Previous - Using the installsfrac -configure Script to Configure SFRAC">Previous</a>&nbsp;&nbsp;|&nbsp;&nbsp;<a href="03_installsfrac_sf_ora_rac32_361367.html" title="Next - Testing Disks Using vxfentsthdw">Next</a>
&nbsp;&gt;&nbsp;&nbsp;</font>
  </td>
 <tr>
   <td height="9" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#FFFFFF">
   </td>
 </tr>

<!-- Footer Breadcrumbs -->
<tr>
<td bgcolor="#FFFFFF" colspan="2" height="15" align="right">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Product:  
<a href="sf_ora_rac_splashpage_docCD_sf_ora_rac1_103036.vxhd" title="Storage Foundation for Oracle RAC Guides title page ">Storage Foundation for Oracle RAC Guides</a>&nbsp;&nbsp;</font></td>
</tr>

<tr>
<td bgcolor="#FFFFFF" colspan="2" height="15" align="right">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Manual:  
<a href="sfrac_icg_titlepage_docCD_sf_ora_rac2_73700.vxhd" title="Storage Foundation 4.1 for Oracle RAC Installation and Configuration title page ">Storage Foundation 4.1 for Oracle RAC Installation and Configuration</a>&nbsp;&nbsp;</font></td>
</tr>


 <tr>
  <td colspan="2" align="left"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" color="#000000" align="right" size="-2">
<em>VERITAS Software Corporation
</em></font><br>

<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">www.veritas.com</font><br>
  </td>
 </tr>
</table>
</div>

</BODY>
  <!--
    *************************************************************************************
    * Copyright (c) 2005 VERITAS Software Corporation. All rights reserved. *

    VERITAS, VERITAS SOFTWARE, the VERITAS Logo, and VERITAS Software Documentation 
    are trademarks or registered trademarks of VERITAS Software Corporation 
    or its affiliates in the U.S. and other countries. Other names may be 
    trademarks of their respective owners. 

    Apache Jakarta Lucene:
    Apache Software License
    Version 1.1

    Copyright (c) 2000 The Apache Software Foundation. All rights reserved.

    Redistribution and use in source and binary forms, with or without modification, 
    are permitted provided that the following conditions are met:

    1. Redistributions of source code must retain the above copyright notice, this 
    list of conditions and the following disclaimer.

    2. Redistributions in binary form must reproduce the above copyright notice, this 
    list of conditions and the following disclaimer in the documentation and/or other 
    materials provided with the distribution.

    3. The end-user documentation included with the redistribution, if any, must include 
    the following acknowledgment: "This product includes software developed by the Apache 
    Software Foundation (http://www.apache.org/)." Alternately, this acknowledgment may 
    appear in the software itself, if and wherever such third-party acknowledgments 
    normally appear.

    4. The names "Apache" and "Apache Software Foundation" must not be used to endorse or 
    promote products derived from this software without prior written permission. For 
    written permission, please contact apache@apache.org.

    5. Products derived from this software may not be called "Apache", nor may "Apache" 
    appear in their name, without prior written permission of the Apache Software Foundation.

    THIS SOFTWARE IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, 
    BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A 
    PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION 
    OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, 
    OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 
    OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 
    ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 
    NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF 
    ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    This software consists of voluntary contributions made by many individuals on behalf 
    of the Apache Software Foundation. For more information on the Apache Software Foundation, 
    please see <http://www.apache.org/>. Portions of this software are based upon public domain 
    software originally written at the National Center for Supercomputing Applications, University 
    of Illinois, Urbana-Champaign.
    
    This product includes WebWindow Software developed by Horst Heistermann.
    *************************************************************************************
  -->
</HTML>
