A
Troubleshooting

This appendix explains how to manage trace files and how to contact Oracle Support Services. The topics in this appendix are:

Using Trace Files
Contacting Oracle Support Services

See Also:
Oracle9i Real Application Clusters Setup and Configuration for information on troubleshooting Oracle Enterprise Manager Service Discovery failures.

Using Trace Files

This section discusses the following trace file subjects:

Background Thread Trace Files

Real Application Clusters background threads use trace files to record occurrences and exceptions of database operations as well as errors. These detailed trace logs are helpful to Oracle support to debug problems in your cluster database configuration. Background thread trace files are created regardless of whether the BACKGROUND_DUMP_DEST parameter is set in the initdbname.ora initialization parameter file. If you set BACKGROUND_DUMP_DEST, the trace files are stored in the directory specified. If you do not set the parameter, the trace files are stored in:

$ORACLE_HOME/admin/db_name/bdump on UNIX operating systems
%ORACLE_HOME%\admin\db_name\bdump on Windows NT and Windows 2000 operating systems

The Oracle database creates a different trace file for each background thread. The name of the trace file contains the name of the background thread followed by the extension .trc, such as:

siddbwr.trc
sidsmon.trc

Trace information is reported in the trace files shown in Table A-1:

Table A-1 Background Thread Trace Files

Trace File	Description
sid`lckn.trc`	Trace file for the Global Cache Service Processes (LMSn). This trace file shows lock requests for other background processes.
sid`lmdn.trc`	Trace file for the LMDn process. This trace file shows lock requests.
sid`lmon.trc`	Trace file for the LMON process. This trace file shows the status of the cluster.
sid`p00n.trc`	Trace file for the parallel execution processes.

User Thread Trace Files

Trace files are also created for user threads if the USER_DUMP_DEST parameter is set in the initialization parameter file. The trace files for the user threads have the form oraxxxxx.trc, where xxxxx is a 5-digit number indicating the process identifier (PID) on UNIX or the thread number on Windows NT and Windows 2000.

Alert File

The alert files, sidalrt.log, contains important information about error messages and exceptions that occur during database operations. Each instance has one alert file; information is appended to the file each time you start the instance. All process threads can write to the alert file.

The sidalrt.log file is in the directory specified by the BACKGROUND_DUMP_DEST parameter in the initdb_name.ora initialization parameter file. If you do not set the BACKGROUND_DUMP_DEST parameter, the sidalrt.log file is generated in:

$ORACLE_BASE/admin/db_name/bdump on UNIX operating systems
%ORACLE_HOME%\admin\db_name\bdump on Windows NT and Windows 2000 operating systems

Error Call Trace Stack

Oracle Worldwide Support might ask you to create an error call trace stack for a particular trace file. An error call trace stack provides a program trace of specific background or user threads in the database.

To create an error call trace stack:

Obtain the Oracle process ID for the background processes:

SELECT pid "Oracle Process Id", 
       name 
    from v$PROCESS, V$BGPROCESS 
    where V$PROCESS.addr = V$BGPROCESS.paddr;

The output displayed should resemble the following:

Oracle Pro NAME 
---------- ----- 
         2 PMON 
         3 LMON 
         4 DBW0 
         5 LGWR 
         6 CKPT 
         7 SMON 
         8 RECO 
         9 SNP0 
        10 SNP1 
        11 GCS0

Dump the trace stack to the trace file. For example, to dump the trace stack of LMON, enter:
1. Set the Oracle process ID to LMON. The process ID is 3 in this example:
```
ORADEBUG setorapid 3 
```
2. Dump the error stack to sidlmon.trc:
```
ORADEBUG dump errorstack 3 
```

Contacting Oracle Support Services

If after reading this appendix, you still cannot resolve your problems, submit a TAR by way of Metalink to report the error to Oracle Support Services. Please include the following information:

Your cluster hardware specifications, for example, a two-node cluster of Dell PowerEdge 6100 servers
Your operating system version
All five digits of the release number of your Oracle RDBMS (for example, 9.2.0.0.1)
All five digits in the release number of your Oracle9i Real Application Clusters database
On Windows NT and Windows 2000, the contents of HKEY_LOCAL_MACHINE\SOFTWARE\ORACLE\OSD key
Cluster OSD upgrades from vendor
Information about the particular operation that failed, for example, database startup or query
A list of steps that reproduce the problem

Severe Errors

If an ORA-600 error appears, it will appear in the sidalrt.log file. If an ORA-600 error or any other severe errors appear in the sidalrt.log file, then provide all files in:

$ORACLE_HOME/admin/db_name/bdump on UNIX operating systems
%ORACLE_HOME%\admin\db_name\bdump on Windows NT and Windows 2000 operating systems

A Troubleshooting