<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Transitional//EN">
<HTML>
<HEAD>
  <!--
    *************************************************************************************
    * Copyright (c) 2005 VERITAS Software Corporation. All rights reserved. *

      For third-party license citations, please see the bottom of this file.
    *************************************************************************************
  -->
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<meta name="TEMPLATEBASE" content="vx_vxhelp_2005c">
<meta name="LASTUPDATED" content="06/13/05 10:36:40">
<meta name="PRODUCTNAME" content="VERITAS Software Documentation">
<meta name="FILENAME" content="oracle_hp_ch14tune.fm">
     
<title>Tuning VxFS</title>

</HEAD>

<BODY bgcolor="#FFFFFF" link="#000066" vlink="#0000ff" marginheight="0" topmargin="0">
<basefont=1>

<a name="1026390"></a>

<div align="left">
<table border="0" cellspacing="0" cellpadding="0" width="100%">
 <tr>
   <td height="2" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#000000">
   </td>
 </tr>
 <tr>
   <td height="15" width="50%" border="0" cellspacing="0" cellpadding="0" bgcolor="#cccccc">
    <img src="images/vxsymbol.gif" height="20" width="31" border="0" cellspacing="0" cellpadding="0" align="left">
     
     
   </td>
   <td align="right" width="50%" bgcolor="#cccccc">

<!-- Header Breadcrumbs -->
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&lt;&nbsp;<a href="oracle_hp_ch14tune_sf_dbag135_1028690.html" title="Previous - Tuning VxVM">Previous</a>&nbsp;&nbsp;|&nbsp;&nbsp;<a href="oracle_hp_ch14tune_sf_dbag137_1023086.html" title="Next - Tuning Oracle Databases">Next</a>
&nbsp;&gt;&nbsp;&nbsp;</font>
   </td>
 </tr>

 <tr>
  <td bgcolor="#CCCCCC" colspan="2" height="15" align="right">
  <font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Product:  
<a href="sf_dbag_splashpage_docCD_sf_dbag1_103036.vxhd" title="Storage Foundation for Databases Guides title page ">Storage Foundation for Databases Guides</a>
  &nbsp;&nbsp;
  </font>
  </td>
 </tr>
 <tr>
  <td bgcolor="#CCCCCC" colspan="2" height="15" align="right">
  <font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Manual:  
<a href="oracle_hp_dbag_titlepage_docCD_sf_dbag2_73700.vxhd" title="Storage Foundation 4.1 for Oracle Administrator's Guide title page ">Storage Foundation 4.1 for Oracle Administrator's Guide</a>
  &nbsp;&nbsp;
  </font>
  </td>
 </tr>


</table>
</div>

<div align="left">
 <table border="0" width="100%" bgcolor="#FFFFFF">
  <tr>
   <td width="100%" valign="top" bgcolor="#FFFFFF">

<!-- Start Main Topic Content -->
        <a name="startcontent"></a>
	
<h1><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="+1" color="#000000"><a name="1026390"></a>Tuning VxFS</font>
</h1>
 
  
  
  

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028177"></a>VERITAS File System provides a set of tuning options to optimize file system performance for different application workloads. VxFS provides a set of tunable I/O parameters that control some of its behavior. These I/O parameters help the file system adjust to striped or RAID-5 volumes that could yield performance far superior to a single disk. Typically, data streaming applications that access large files see the largest benefit from tuning the file system. </font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028180"></a>Most of these tuning options have little or no impact on database performance when using Quick I/O, with the exception of the <font face="Courier New, Courier, sansserif, sans serif" size="2">max_thread_proc</font> parameter. Other than setting the <font face="Courier New, Courier, sansserif, sans serif" size="2">max_thread_proc</font> parameter (see <a href="oracle_hp_ch14tune_sf_dbag138_1028834.html#1028839">max_thread_proc</a>), use the general VxFS defaults when creating a VxFS file system for databases. However, you can gather file system performance data when using Quick I/O, and use this information to adjust the system configuration to make the most efficient use of system resources.</font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="1028195"></a>Monitoring Free Space</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028196"></a>In general, VxFS works best if the percentage of free space in the file system is greater than 10 percent. This is because file systems with 10 percent or more of free space have less fragmentation and better extent allocation. Regular use of the <font face="Courier New, Courier, sansserif, sans serif" size="2">df</font> command to monitor free space is desirable. Full file systems may have an adverse effect on file system performance. Full file systems should therefore have some files removed or should be expanded. See the <font face="Courier New, Courier, sansserif, sans serif" size="2">fsadm_vxfs</font>(1M) manual page for a description of online file system expansion.</font>
</p>




<h3><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="2" color="#000000"><a name="1028203"></a>Monitoring Fragmentation</font>
</h3>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028204"></a>Fragmentation reduces performance and availability. Regular use of <font face="Courier New, Courier, sansserif, sans serif" size="2">fsadm</font>'s fragmentation reporting and reorganization facilities is therefore advisable.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028208"></a>The easiest way to ensure that fragmentation does not become a problem is to schedule regular defragmentation runs using the <font face="Courier New, Courier, sansserif, sans serif" size="2">cron</font> command. </font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028211"></a>Defragmentation scheduling should range from weekly (for frequently used file systems) to monthly (for infrequently used file systems). Extent fragmentation should be monitored with <font face="Courier New, Courier, sansserif, sans serif" size="2">fsadm</font> or the <font face="Courier New, Courier, sansserif, sans serif" size="2">df&nbsp;-o</font> <font face="Courier New, Courier, sansserif, sans serif" size="2">s</font> commands. There are three factors that can be used to determine the degree of fragmentation: </font>
</p>

<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028213"></a>Percentage of free space in extents that are less than eight blocks in length </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028214"></a>Percentage of free space in extents that are less than 64 blocks in length </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028215"></a>Percentage of free space in extents that are 64 or more blocks in length</font>
</li>
</ul>
</p>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028216"></a>An unfragmented file system will have the following characteristics: </font>
</p>

<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028217"></a>Less than 1 percent of free space in extents that are less than eight blocks in length </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028218"></a>Less than 5 percent of free space in extents that are less than 64 blocks in length </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028219"></a>More than 5 percent of the total file system size available as free extents that are 64 or more blocks in length</font>
</li>
</ul>
</p>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028220"></a>A badly fragmented file system will have one or more of the following characteristics: </font>
</p>

<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028221"></a>More than 5 percent of free space in extents that are less than 8 blocks in length </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028222"></a>More than 50 percent of free space in extents that are less than 64 blocks in length </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028223"></a>Less than 5 percent of the total file system size available as free extents that are 64 or more blocks in length</font>
</li>
</ul>
</p>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028225"></a>The optimal period for scheduling extent reorganization runs can be determined by choosing a reasonable interval, scheduling <font face="Courier New, Courier, sansserif, sans serif" size="2">fsadm</font> runs at the initial interval, and running the extent fragmentation report feature of <font face="Courier New, Courier, sansserif, sans serif" size="2">fsadm</font> before and after the reorganization. </font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028228"></a>The &quot;before&quot; result is the degree of fragmentation prior to the reorganization. If the degree of fragmentation approaches the percentages for bad fragmentation, reduce the interval between <font face="Courier New, Courier, sansserif, sans serif" size="2">fsadm</font>. If the degree of fragmentation is low, increase the interval between <font face="Courier New, Courier, sansserif, sans serif" size="2">fsadm</font> runs.</font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="1028232"></a>Tuning VxFS I/O Parameters</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028234"></a>VxFS provides a set of tunable I/O parameters that control some of its behavior. These I/O parameters are useful to help the file system adjust to striped or RAID-5 volumes that could yield performance far superior to a single disk. Typically, data streaming applications that access large files see the biggest benefit from tuning the file system. </font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028235"></a>If VxFS is being used with VERITAS Volume Manager, the file system queries VxVM to determine the geometry of the underlying volume and automatically sets the I/O parameters. VxVM is queried by <font face="Courier New, Courier, sansserif, sans serif" size="2">mkfs</font> when the file system is created to automatically align the file system to the volume geometry. If the default alignment from <font face="Courier New, Courier, sansserif, sans serif" size="2">mkfs</font> is not acceptable, the <font face="Courier New, Courier, sansserif, sans serif" size="2">-o</font> <font face="Courier New, Courier, sansserif, sans serif" size="2">align=n</font> option can be used to override alignment information obtained from VxVM. The <font face="Courier New, Courier, sansserif, sans serif" size="2">mount</font> command also queries VxVM when the file system is mounted and downloads the I/O parameters. </font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028237"></a>If the default parameters are not acceptable or the file system is being used without VxVM, then the <font face="Courier New, Courier, sansserif, sans serif" size="2">/etc/vx/tunefstab</font> file can be used to set values for I/O parameters. The <font face="Courier New, Courier, sansserif, sans serif" size="2">mount</font> command reads the <font face="Courier New, Courier, sansserif, sans serif" size="2">/etc/vx/tunefstab</font> file and downloads any parameters specified for a file system. The <font face="Courier New, Courier, sansserif, sans serif" size="2">tunefstab</font> file overrides any values obtained from VxVM. While the file system is mounted, any I/O parameters can be changed using the <font face="Courier New, Courier, sansserif, sans serif" size="2">vxtunefs</font> command, which can have tunables specified on the command line or can read them from the <font face="Courier New, Courier, sansserif, sans serif" size="2">/etc/vx/tunefstab</font> file. For more details, see the <font face="Courier New, Courier, sansserif, sans serif" size="2">vxtunefs</font>(1M) and <font face="Courier New, Courier, sansserif, sans serif" size="2">tunefstab</font>(4) manual pages. The <font face="Courier New, Courier, sansserif, sans serif" size="2">vxtunefs</font> command can be used to print the current values of the I/O parameters.</font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="1028241"></a>Tunable VxFS I/O Parameters</font>
</h2>
 
<a name="no_figure_caption0"> </a>
<Table width="100%" border="0" cellpadding="1" cellspacing="0" >
<tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028245"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">read_pref_io</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028247"></a>The preferred read request size. The file system uses this parameter in conjunction with the <font face="Courier New, Courier, sansserif, sans serif" size="2">read_nstream</font> value to determine how much data to read ahead. The default value is 64K.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028250"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">write_pref_io</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028252"></a>The preferred write request size. The file system uses this parameter in conjunction with the <font face="Courier New, Courier, sansserif, sans serif" size="2">write_nstream</font> value to determine how to do flush behind on writes. The default value is 64K.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028255"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">read_nstream</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028257"></a>The number of parallel read requests of size <font face="Courier New, Courier, sansserif, sans serif" size="2">read_pref_io</font> that you can have outstanding at one time. The file system uses the product of <font face="Courier New, Courier, sansserif, sans serif" size="2">read_nstream</font> multiplied by <font face="Courier New, Courier, sansserif, sans serif" size="2">read_pref_io</font> to determine its read ahead size. The default value for <font face="Courier New, Courier, sansserif, sans serif" size="2">read_nstream</font> is 1.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028260"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">write_nstream</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028262"></a>The number of parallel write requests of size <font face="Courier New, Courier, sansserif, sans serif" size="2">write_pref_io</font> that you can have outstanding at one time. The file system uses the product of <font face="Courier New, Courier, sansserif, sans serif" size="2">write_nstream</font> multiplied by <font face="Courier New, Courier, sansserif, sans serif" size="2">write_pref_io</font> to determine when to do flush behind on writes. The default value for <font face="Courier New, Courier, sansserif, sans serif" size="2">write_nstream</font> is 1.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028271"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">discovered_direct_iosz</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028274"></a>Any file I/O requests larger than the <font face="Courier New, Courier, sansserif, sans serif" size="2">discovered_direct_iosz</font> are handled as discovered direct I/O. A discovered direct I/O is unbuffered similar to direct I/O, but does not require a synchronous commit of the inode when the file is extended or blocks are allocated. For larger I/O requests, the CPU time for copying the data into the page cache and the cost of using memory to buffer the I/O data becomes more expensive than the cost of doing the disk I/O. For these I/O requests, using discovered direct I/O is more efficient than regular I/O. The default value of this parameter is 256K.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028276"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">initial_extent_<br>size</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028279"></a>Changes the default initial extent size. VxFS determines the size of the first extent to be allocated to the file based on the first write to a new file. Normally, the first extent is the smallest power of 2 that is larger than the size of the first write. If that power of 2 is less than 8K, the first extent allocated is 8K. After the initial extent, the file system increases the size of subsequent extents (see <font face="Courier New, Courier, sansserif, sans serif" size="2">max_seqio_extent_size</font>) with each allocation. Since most applications write to files using a buffer size of 8K or less, the increasing extents start doubling from a small initial extent. <font face="Courier New, Courier, sansserif, sans serif" size="2">initial_extent_size</font> can change the default initial extent size to be larger, so the doubling policy will start from a much larger initial size and the file system will not allocate a set of small extents at the start of file. Use this parameter only on file systems that will have a very large average file size. On these file systems, it will result in fewer extents per file and less fragmentation. <font face="Courier New, Courier, sansserif, sans serif" size="2">initial_extent_size</font> is measured in file system blocks.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028282"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">max_buf_data_size</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028284"></a>The maximum buffer size allocated for file data; either 8K bytes or 64K bytes. Use the larger value for workloads where large reads/writes are performed sequentially. Use the smaller value on workloads where the I/O is random or is done in small chunks. The default value is 8K bytes.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028287"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">max_direct_iosz</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028289"></a>The maximum size of a direct I/O request that will be issued by the file system. If a larger I/O request comes in, then it is broken up into <font face="Courier New, Courier, sansserif, sans serif" size="2">max_direct_iosz</font> chunks. This parameter defines how much memory an I/O request can lock at once, so it should not be set to more than 20 percent of memory.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028292"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">max_diskq</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028294"></a>Limits the maximum disk queue generated by a single file. When the file system is flushing data for a file and the number of pages being flushed exceeds <font face="Courier New, Courier, sansserif, sans serif" size="2">max_diskq</font>, processes will block until the amount of data being flushed decreases. Although this doesn't limit the actual disk queue, it prevents flushing processes from making the system unresponsive. The default value is 1MB.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028296"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">max_seqio_extent_size</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028299"></a>Increases or decreases the maximum size of an extent. When the file system is following its default allocation policy for sequential writes to a file, it allocates an initial extent that is large enough for the first write to the file. When additional extents are allocated, they are progressively larger (the algorithm tries to double the size of the file with each new extent) so each extent can hold several writes' worth of data. This is done to reduce the total number of extents in anticipation of continued sequential writes. When the file stops being written, any unused space is freed for other files to use. Normally, this allocation stops increasing the size of extents at 2048 blocks, which prevents one file from holding too much unused space. <font face="Courier New, Courier, sansserif, sans serif" size="2">max_seqio_extent_size</font> is measured in file system blocks.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028302"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">qio_cache_enable</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028304"></a>Enables or disables caching on Quick I/O files. The default behavior is to disable caching. To enable caching, set <font face="Courier New, Courier, sansserif, sans serif" size="2">qio_cache_enable</font> to 1. On systems with large memories, the database cannot always use all of the memory as a cache. By enabling file system caching as a second level cache, performance may be improved. If the database is performing sequential scans of tables, the scans may run faster by enabling file system caching so the file system will perform aggressive read-ahead on the files.</font>
</p>

</td></tr><tr valign="top"><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028326"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">write_throttle</font></font>
</p>

</td><td><p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028328"></a>The <font face="Courier New, Courier, sansserif, sans serif" size="2">write_throttle</font> parameter is useful in special situations where a computer system has a combination of a lot of memory and slow storage devices. In this configuration, sync operations (such as <font face="Courier New, Courier, sansserif, sans serif" size="2">fsync()</font>) may take so long to complete that the system appears to hang. This behavior occurs because the file system is creating <em>dirty pages</em> (in-memory updates) faster than they can be asynchronously flushed to disk without slowing system performance.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028329"></a>Lowering the value of <font face="Courier New, Courier, sansserif, sans serif" size="2">write_throttle</font> limits the number of dirty pages per file that a file system will generate before flushing the pages to disk. After the number of dirty pages for a file reaches the  <font face="Courier New, Courier, sansserif, sans serif" size="2">write_throttle</font> threshold, the file system starts flushing pages to disk even if free memory is still available. The default value of <font face="Courier New, Courier, sansserif, sans serif" size="2">write_throttle</font> typically generates a lot of dirty pages, but maintains fast user writes. Depending on the speed of the storage device, if you lower <font face="Courier New, Courier, sansserif, sans serif" size="2">write_throttle</font>, user write performance may suffer, but the number of dirty pages is limited, so sync operations will complete much faster.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028330"></a>Because lowering <font face="Courier New, Courier, sansserif, sans serif" size="2">write_throttle</font> can delay write requests (for example, lowering <font face="Courier New, Courier, sansserif, sans serif" size="2">write_throttle</font> may increase the file disk queue to the  <font face="Courier New, Courier, sansserif, sans serif" size="2">max_diskq</font> value, delaying user writes until the disk queue decreases), it is recommended that you avoid changing the value of <font face="Courier New, Courier, sansserif, sans serif" size="2">write_throttle</font> unless your system has a a large amount of physical memory and slow storage devices.</font>
</p>

</td></tr>
</Table>


<Table width="100%" border="0" cellpadding="1" cellspacing="0">
<tr><td>
</Table>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028333"></a>If the file system is being used with VxVM, it is recommended that you set the VxFS I/O parameters to default values based on the volume geometry.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028334"></a>If the file system is being used with a hardware disk array or volume manager other than VxVM, align the parameters to match the geometry of the logical disk. With striping or RAID-5, it is common to set <font face="Courier New, Courier, sansserif, sans serif" size="2">read_pref_io</font> to the stripe unit size and <font face="Courier New, Courier, sansserif, sans serif" size="2">read_nstream</font> to the number of columns in the stripe. For striping arrays, use the same values for <font face="Courier New, Courier, sansserif, sans serif" size="2">write_pref_io</font> and <font face="Courier New, Courier, sansserif, sans serif" size="2">write_nstream</font>, but for RAID-5 arrays, set <font face="Courier New, Courier, sansserif, sans serif" size="2">write_pref_io</font> to the full stripe size and <font face="Courier New, Courier, sansserif, sans serif" size="2">write_nstream</font> to 1.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028335"></a>For an application to do efficient disk I/O, it should issue read requests that are equal to the product of <font face="Courier New, Courier, sansserif, sans serif" size="2">read_nstream</font> multiplied by <font face="Courier New, Courier, sansserif, sans serif" size="2">read_pref_io</font>. Generally, any multiple or factor of <font face="Courier New, Courier, sansserif, sans serif" size="2">read_nstream</font> multiplied by <font face="Courier New, Courier, sansserif, sans serif" size="2">read_pref_io</font> should be a good size for performance. For writing, the same rule of thumb applies to the <font face="Courier New, Courier, sansserif, sans serif" size="2">write_pref_io</font> and <font face="Courier New, Courier, sansserif, sans serif" size="2">write_nstream</font> parameters. When tuning a file system, the best thing to do is try out the tuning parameters under a real-life workload.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028336"></a>If an application is doing sequential I/O to large files, it should issue requests larger than the <font face="Courier New, Courier, sansserif, sans serif" size="2">discovered_direct_iosz</font>. This causes the I/O requests to be performed as discovered direct I/O requests, which are unbuffered like direct I/O but do not require synchronous inode updates when extending the file. If the file is too large to fit in the cache, then using unbuffered I/O avoids throwing useful data out of the cache and lessons CPU overhead.</font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="1028338"></a>Obtaining File I/O Statistics using the Quick I/O Interface</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028340"></a>The <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat</font> command provides access to activity information on Quick I/O files on VxFS file systems. The command reports statistics on the activity levels of files from the time the files are first opened using their Quick I/O interface. The accumulated <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat</font> statistics are reset once the last open reference to the Quick I/O file is closed.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028345"></a>The <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat</font> command displays the following I/O statistics:</font>
</p>

<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028346"></a>Number of read and write operations </font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028347"></a>Number of data blocks (sectors) transferred</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028348"></a>Average time spent on read and write operations</font>
</li>
</ul>
</p>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028350"></a>When Cached Quick I/O is used, <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat</font> also displays the caching statistics when the <font face="Courier New, Courier, sansserif, sans serif" size="2">-l</font> (the long format) option is selected. </font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028351"></a>The following is an example of <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat</font> output: </font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028352"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OPERATIONS&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;FILE&nbsp;BLOCKS&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AVG&nbsp;TIME(ms)</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028353"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;FILENAME&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;READ&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WRITE&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;READ&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WRITE&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;READ&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WRITE</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028354"></a>&nbsp;/db01/file1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028355"></a>&nbsp;/db01/file2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028356"></a>&nbsp;/db01/file3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;73017&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;181735&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;718528&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1114227&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;26.8&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;27.9
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028357"></a>&nbsp;/db01/file4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;13197&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;20252&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;105569&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;162009&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;25.8&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;397.0&nbsp;
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028358"></a>&nbsp;/db01/file5&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028360"></a>For detailed information on available options, see the <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat</font>(1M) manual page.</font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="1028408"></a>Using I/O Statistics Data</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028410"></a>Once you gather the file I/O performance data, you can use it to adjust the system configuration to make the most efficient use of system resources. There are three primary statistics to consider: </font>
</p>

<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028411"></a>file I/O activity</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028412"></a>volume I/O activity</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028413"></a>raw disk I/O activity</font>
</li>
</ul>
</p>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028414"></a>If your database is using one file system on a striped volume, you may only need to pay attention to the file I/O activity statistics. If you have more than one file system, you may need to monitor volume I/O activity as well.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028416"></a>First, use the <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat -r</font> command to clear all existing statistics. After clearing the statistics, let the database run for a while during a typical database workload period. For example, if you are monitoring a database with many users, let the statistics accumulate for a few hours during prime working time before displaying the accumulated I/O statistics.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028417"></a>To display active file I/O statistics, use the <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat</font> command and specify an interval (using <font face="Courier New, Courier, sansserif, sans serif" size="2">-i</font>) for displaying the statistics for a period of time. This command displays a list of statistics such as: </font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028418"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;OPERATIONS&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;FILE&nbsp;BLOCKS&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;AVG&nbsp;TIME(ms)</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028419"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;FILENAME&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;READ&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WRITE&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;READ&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WRITE&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;READ&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WRITE</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028420"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/cust1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;218&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;36&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;872&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;144&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;22.8&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;55.6</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028421"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/hist1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;10.0</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028422"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/nord1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;10&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;14&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;40&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;56&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;21.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;75.0</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028423"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/ord1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;19&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;16&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;76&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;64&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;17.4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;56.2</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028424"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/ordl1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;189&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;41&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;756&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;164&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;21.1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;50.0</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028425"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/roll1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;50&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;200&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;49.0</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028426"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/stk1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;1614&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;238&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;6456&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;952&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;19.3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;46.5</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028427"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/sys1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028428"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/temp1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028429"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/db01/ware1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;14&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;12&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;56&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;23.3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;44.3</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028430"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/logs/log1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0</font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028431"></a><font face="Courier New, Courier, sansserif, sans serif" size="2">&nbsp;&nbsp;&nbsp;&nbsp;/logs/log2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;217&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;2255&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;6.8</font>
</font>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028433"></a>File I/O statistics help identify files with an unusually large number of operations or excessive read or write times. When this happens, try moving the &quot;hot&quot; files or busy file systems to different disks or changing the layout to balance the I/O load.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028448"></a></font>
</p>




<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="1028449"></a>Obtaining File I/O Statistics using VERITAS Extension for Oracle Disk Manager</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028452"></a>The <font face="Courier New, Courier, sansserif, sans serif" size="2">odmstat</font> command provides access to activity information on Oracle Disk Manager files on VxFS systems. Refer to the <font face="Courier New, Courier, sansserif, sans serif" size="2">odmstat</font>(1M) manual page for more information. The command reports statistics on the activity from the time that the files were opened by the Oracle Disk Manager interface. The command has an option for zeroing the statistics. When the file is closed, the statistics are discarded. </font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028460"></a>The <font face="Courier New, Courier, sansserif, sans serif" size="2">odmstat</font> command displays the following I/O statistics:</font>
</p>

<p>
<ul type="disc"><li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028461"></a>Number of read and write operations</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028462"></a>Number of data blocks read and written</font>
</li>
<li><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028463"></a>Average time spent on read and write operations</font>
</li>
</ul>
</p>
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028464"></a>The following is an example of <font face="Courier New, Courier, sansserif, sans serif" size="2">odmstat</font> output:</font>
</p>

<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028465"></a>#<font face="Courier New, Courier, sansserif, sans serif" size="2"><b>&nbsp;/opt/VRTS/bin/odmstat&nbsp;-i&nbsp;5&nbsp;/mnt/odmfile*</b></font>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028466"></a>&nbsp;&nbsp;OPERATIONS&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;FILE&nbsp;BLOCKS&nbsp;&nbsp;&nbsp;&nbsp;AVG&nbsp;TIME(ms)
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028467"></a>FILE&nbsp;NAME&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;READ&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WRITE&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;READ&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WRITE&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;READ&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;WRITE
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028468"></a>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028469"></a>Mon&nbsp;May&nbsp;11&nbsp;16:21:10&nbsp;2015
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028470"></a>/db/cust.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028471"></a>/db/system.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028472"></a>Mon&nbsp;May&nbsp;11&nbsp;16:21:15&nbsp;2015
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028473"></a>/db/cust.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;371&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;371&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028474"></a>/db/system.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;371&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;371&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.7
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028475"></a>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028476"></a>Mon&nbsp;May&nbsp;11&nbsp;16:21:20&nbsp;2015
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028477"></a>/db/cust.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;813&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;813&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028478"></a>/db/system.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;813&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;813&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.5
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028479"></a>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028480"></a>Mon&nbsp;May&nbsp;11&nbsp;16:21:25&nbsp;2015
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028481"></a>/db/cust.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;816&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;816&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028482"></a>/db/system.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;816&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;816&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;5.3
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028483"></a>
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028484"></a>Mon&nbsp;May&nbsp;11&nbsp;16:21:30&nbsp;2015
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028485"></a>/db/cust.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0
</font>
<br>
<font face="Courier New, Courier, sansserif, sans serif" size="2"><a name="1028486"></a>/db/system.dbf&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;0.0
</font>



<h2><font face="Arial, Helvetica, Geneva, sansserif, sans serif, Verdana" size="3" color="#000000"><a name="1028487"></a>Interpreting I/O Statistics</font>
</h2>
 
<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028488"></a>When running your database through the file system, the read-write lock on each file allows only one active write per file. When you look at the disk statistics using <font face="Courier New, Courier, sansserif, sans serif" size="2">iostat</font>, the disk reports queueing time and service time. The service time is the time that I/O spends on the disk, and the queueing time is how long it waits for all of the other I/Os ahead of it. At the volume level or the file system level, there is no queueing, so <font face="Courier New, Courier, sansserif, sans serif" size="2">vxstat</font> and <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat</font> do not show queueing time.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028489"></a>For example, if you send 100 I/Os at the same time and each takes 10 milliseconds, the disk reports an average of 10 milliseconds of service and 490 milliseconds of queueing time. The <font face="Courier New, Courier, sansserif, sans serif" size="2">vxstat</font>, <font face="Courier New, Courier, sansserif, sans serif" size="2">odmstat</font>, and <font face="Courier New, Courier, sansserif, sans serif" size="2">qiostat</font> report an average of 500 milliseconds service time.</font>
</p>

<p>
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="2"><a name="1028490"></a></font>
</p>


<!-- End Main Topic Content -->

   </td>
  </tr>
 </table>
</div>

<div>
<table border="0" cellspacing="0" cellpadding="0" width="100%">
 <tr>
  <td height="18" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#FFFFFF">
  </td>
 </tr>
  <td align="left"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&nbsp;&#94;&nbsp;<a href="oracle_hp_ch14tune_sf_dbag136_1026390.html" title="Tuning VxFS">Return to Top</a></font>

  </td>
  <td align="right"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">&lt;&nbsp;<a href="oracle_hp_ch14tune_sf_dbag135_1028690.html" title="Previous - Tuning VxVM">Previous</a>&nbsp;&nbsp;|&nbsp;&nbsp;<a href="oracle_hp_ch14tune_sf_dbag137_1023086.html" title="Next - Tuning Oracle Databases">Next</a>
&nbsp;&gt;&nbsp;&nbsp;</font>
  </td>
 <tr>
   <td height="9" border="0" cellspacing="0" cellpadding="0" colspan="2" bgcolor="#FFFFFF">
   </td>
 </tr>

<!-- Footer Breadcrumbs -->
<tr>
<td bgcolor="#FFFFFF" colspan="2" height="15" align="right">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Product:  
<a href="sf_dbag_splashpage_docCD_sf_dbag1_103036.vxhd" title="Storage Foundation for Databases Guides title page ">Storage Foundation for Databases Guides</a>&nbsp;&nbsp;</font></td>
</tr>

<tr>
<td bgcolor="#FFFFFF" colspan="2" height="15" align="right">
<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="1">Manual:  
<a href="oracle_hp_dbag_titlepage_docCD_sf_dbag2_73700.vxhd" title="Storage Foundation 4.1 for Oracle Administrator's Guide title page ">Storage Foundation 4.1 for Oracle Administrator's Guide</a>&nbsp;&nbsp;</font></td>
</tr>


 <tr>
  <td colspan="2" align="left"><font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" color="#000000" align="right" size="-2">
<em>VERITAS Software Corporation
</em></font><br>

<font face="Verdana, Arial, Helvetica, Geneva, sansserif, sans serif" size="-2">www.veritas.com</font><br>
  </td>
 </tr>
</table>
</div>

</BODY>
  <!--
    *************************************************************************************
    * Copyright (c) 2005 VERITAS Software Corporation. All rights reserved. *

    VERITAS, VERITAS SOFTWARE, the VERITAS Logo, and VERITAS Software Documentation 
    are trademarks or registered trademarks of VERITAS Software Corporation 
    or its affiliates in the U.S. and other countries. Other names may be 
    trademarks of their respective owners. 

    Apache Jakarta Lucene:
    Apache Software License
    Version 1.1

    Copyright (c) 2000 The Apache Software Foundation. All rights reserved.

    Redistribution and use in source and binary forms, with or without modification, 
    are permitted provided that the following conditions are met:

    1. Redistributions of source code must retain the above copyright notice, this 
    list of conditions and the following disclaimer.

    2. Redistributions in binary form must reproduce the above copyright notice, this 
    list of conditions and the following disclaimer in the documentation and/or other 
    materials provided with the distribution.

    3. The end-user documentation included with the redistribution, if any, must include 
    the following acknowledgment: "This product includes software developed by the Apache 
    Software Foundation (http://www.apache.org/)." Alternately, this acknowledgment may 
    appear in the software itself, if and wherever such third-party acknowledgments 
    normally appear.

    4. The names "Apache" and "Apache Software Foundation" must not be used to endorse or 
    promote products derived from this software without prior written permission. For 
    written permission, please contact apache@apache.org.

    5. Products derived from this software may not be called "Apache", nor may "Apache" 
    appear in their name, without prior written permission of the Apache Software Foundation.

    THIS SOFTWARE IS PROVIDED "AS IS" AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, 
    BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A 
    PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE APACHE SOFTWARE FOUNDATION 
    OR ITS CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, 
    OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 
    OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 
    ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 
    NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF 
    ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

    This software consists of voluntary contributions made by many individuals on behalf 
    of the Apache Software Foundation. For more information on the Apache Software Foundation, 
    please see <http://www.apache.org/>. Portions of this software are based upon public domain 
    software originally written at the National Center for Supercomputing Applications, University 
    of Illinois, Urbana-Champaign.
    
    This product includes WebWindow Software developed by Horst Heistermann.
    *************************************************************************************
  -->
</HTML>
