tag:blogger.com,1999:blog-86928810744492328492024-03-08T00:39:48.691-08:00Oracle DBA ScratchpadThis blog is Created to post the various Oracle Topics and scripts.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.comBlogger63125tag:blogger.com,1999:blog-8692881074449232849.post-43815688254179233892009-10-06T09:34:00.001-07:002009-10-06T09:34:18.212-07:00Virtual IndexesVirtual Indexes are another undocumented feature used by Oracle. Virtual indexes, as the name suggests are pseudo-indexes that will not behave the same way that normal indexes behave, and are meant for a very specific purpose. <br />
A virtual index is created in a slightly different manner than the normal indexes. A virtual index has no segment pegged to it, i.e., the DBA_SEGMENTS view will not show an entry for this. Oracle handles such indexes internally and few required dictionary tables are updated so that the optimizer can be made aware of its presence and generate an execution plan considering such indexes.<br />
As per Oracle, this functionality is not intended for standalone usage. It is part of the Oracle Enterprise Manger Tuning Pack (Virtual Index Wizard). The virtual index wizard functionality allows the user to test a potential new index prior to actually building the new index in the database. It allows the CBO to evaluate the potential new index for a selected SQL statement by building an explain plan that is aware of the potential new index. This allows the user to determine if the optimizer would use the index, once implemented.<br />
Therefore, the feature is here to be supported from Enterprise Manager and not for standalone usage. I went a bit further and actually tested it using SQL*Plus, basically, trying to use the same feature but without the enterprise manager.<br />
I do not see much use of Virtual Indexes in a development area where we can create and drop indexes while testing. However, this feature could prove handy if a query or group of queries have to be tested in production (for want of simulation or urgency!), to determine if a new index will improve the performance, without impacting existing or new sessions.<br />
Below are some attributes of the Virtual Indexes.<br />
1. These are permanent and continue to exist unless we drop them.<br />
2. Their creation will not affect existing and new sessions. Only sessions marked for Virtual Index usage will become aware of their existence.<br />
3. Such indexes will be used only when the hidden parameter "_use_nosegment_indexes" is set to true.<br />
4. The Rule based optimizer did not recognize Virtual Indexes when I <br />
tested, however, CBO recognizes them. In all of my examples, I have used CBO. However, I did not carry out intensive testing in RBO and you may come across exceptions to this view.<br />
5. Dictionary view DBA_SEGMENTS will not show an entry for Virtual Indexes. The table DBA_INDEXES and DBA_OBJECTS will have an entry for them in Oracle 8i; in Oracle 9i onwards, DBA_INDEXES no longer show Virtual Indexes.<br />
6. Virtual Indexes cannot be altered and throw a "fake index" error!<br />
7. Virtual Indexes can be analyzed, using the ANALYZE command or DBMS_STATS package, but the statistics cannot be viewed (in Oracle 8i, DBA_INDEXES will not show this either). Oracle may be generating artificial statistics and storing it somewhere for referring it later.<br />
Creating Virtual Index<br />
Creating a Virtual Index can be achieved by using the NOSEGMENT clause with the CREATE INDEX command.<br />
e.g.:<br />
SQL> create unique index am304_u1 on am304(col2) nosegment;<br />
<br />
Index created.<br />
Parameter _USE_NOSEGMENT_INDEXES<br />
This is a hidden/internal parameter and therefore undocumented. Such parameters should not be altered for Oracle databases unless Oracle Support either advises or recommends that you do so. In our case, we make an exception (!), but only to be set at session level. Do not set it for the complete instance. <br />
Setting the "_use_nosegment_indexes" parameter enables the optimizer to use virtual indexes.<br />
Examples:<br />
Creating the virtual index:<br />
SQL> create index am301_n1 on am301(col1) nosegment;<br />
<br />
Index created.<br />
Checking some dictionary tables:<br />
SQL> select segment_name, segment_type, bytes<br />
2 from dba_segments<br />
3 where segment_name = 'AM301_N1';<br />
<br />
no rows selected<br />
<br />
<br />
SQL> select object_name, object_type, status<br />
2 from dba_objects<br />
3 where object_name = 'AM301_N1';<br />
<br />
OBJECT_NAME |OBJECT_TYPE |STATUS<br />
------------------|-----------------|---------------<br />
AM301_N1 |INDEX |VALID<br />
<br />
<br />
SQL> select index_name, index_type, status<br />
2 from dba_indexes<br />
3 where index_name = 'AM301_N1';<br />
<br />
INDEX_NAME |INDEX_TYPE |STATUS<br />
------------------------------|------------|---------------<br />
AM301_N1 |NORMAL |VALID<br />
Working with the Virtual indexes:<br />
SQL> create table am301<br />
2 (col1 number, col2 varchar2(20));<br />
<br />
Table created.<br />
<br />
SQL> insert into am301 values(dbms_random.random, dbms_random.string('A', 20));<br />
<br />
1 row created.<br />
<br />
SQL> insert into am301 values(dbms_random.random, dbms_random.string('A', 20));<br />
<br />
1 row created.<br />
<br />
SQL> insert into am301 values(dbms_random.random, dbms_random.string('A', 20));<br />
<br />
1 row created.<br />
<br />
SQL> insert into am301 values(dbms_random.random, dbms_random.string('A', 20));<br />
<br />
1 row created.<br />
<br />
SQL> select * from am301;<br />
<br />
COL1 COL2<br />
---------- --------------------<br />
-512599724 aCR_PdFVdSGJLCOLCjJQ<br />
-2.049E+09 qiVUlonc^p^R_X_W_^Xn<br />
-189009085 prTNPqPUod^miAnLXrMA<br />
2082093299 Bq_icbmcpNFNUKDRdMi] <br />
<br />
--Though inserting alpha-numeric, Oracle also inserted<br />
--special characters in col2. This one is a bug and is<br />
--resolved in 9i+.<br />
<br />
SQL> insert into am301 select * from am301;<br />
<br />
4 rows created.<br />
<br />
SQL> insert into am301 select * from am301;<br />
<br />
8 rows created.<br />
<br />
SQL> insert into am301 select * from am301;<br />
<br />
16 rows created.<br />
<br />
SQL> alter session set optimizer_mode=first_rows;<br />
<br />
Session altered.<br />
<br />
SQL> create index am301_n1 on am301(col1) nosegment;<br />
<br />
Index created.<br />
<br />
SQL> analyze table am301 compute statistics;<br />
<br />
Table analyzed.<br />
<br />
SQL> analyze index am301_n1 compute statistics;<br />
<br />
Index analyzed.<br />
<br />
--It is recommended to use dbms_stats package to <br />
--generate statistics. Analyze is used here as an<br />
--example only.<br />
<br />
SQL> set autotrace on<br />
<br />
SQL> select * from am301 where col1 = 2082093299;<br />
<br />
COL1 COL2<br />
---------- --------------------<br />
2082093299 Bq_icbmcpNFNUKDRdMi]<br />
2082093299 Bq_icbmcpNFNUKDRdMi]<br />
..<br />
<br />
32 rows selected.<br />
<br />
Execution Plan<br />
----------------------------------------------------------<br />
0 SELECT STATEMENT Optimizer=FIRST_ROWS (Cost=1 Card=32 Bytes=864)<br />
1 0 TABLE ACCESS (FULL) OF 'AM301' (Cost=1 Card=32 Bytes=864)<br />
<br />
<br />
SQL> alter session set "_use_nosegment_indexes" = true; <br />
--set only for the session testing it.<br />
<br />
Session altered.<br />
<br />
SQL> select * from am301 where col1 = 2082093299;<br />
<br />
COL1 COL2<br />
---------- --------------------<br />
2082093299 Bq_icbmcpNFNUKDRdMi]<br />
2082093299 Bq_icbmcpNFNUKDRdMi]<br />
<br />
32 rows selected.<br />
<br />
Execution Plan<br />
----------------------------------------------------------<br />
0 SELECT STATEMENT Optimizer=FIRST_ROWS (Cost=207 Card=32 Bytes=864)<br />
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'AM301' (Cost=207 Card=32 Bytes=864)<br />
2 1 INDEX (RANGE SCAN) OF 'AM301_N1' (NON-UNIQUE) (Cost=7 Card=32)<br />
Trying to alter the virual index:<br />
SQL> alter index am301_n1 rebuild;<br />
alter index am301_n1 rebuild<br />
*<br />
ERROR at line 1:<br />
ORA-08114: can not alter a fake index<br />
<br />
<br />
SQL> alter index am301_n1 storage(pctincrease 0);<br />
alter index am301_n1 storage(pctincrease 0)<br />
*<br />
ERROR at line 1:<br />
ORA-08114: can not alter a fake index<br />
Trying to re-create the index as a real one:<br />
SQL> create index am301_n1 on am301(col1);<br />
create index am301_n1 on am301(col1)<br />
*<br />
ERROR at line 1:<br />
ORA-00955: name is already used by an existing object<br />
As the Virtual Index has an entry in some of the dictionary tables, it will prevent the creation of an object with the same name. The alternative is to drop and recreate the Virtual Index as a real index.<br />
Dropping and re-creating again as real index:<br />
SQL> drop index am301_n1;<br />
<br />
Index dropped.<br />
<br />
SQL> create index am301_n1 on am301(col1);<br />
<br />
Index created.<br />
However, a Virtual Index will not prevent the creation of an index with the same column(s). <br />
In the example below, a Virtual Index is created with name DUMMY, afterwards a new index with a different name is created with the same column and structure. Both of the indexes will show in the DBA_OBJECTS listing.<br />
SQL> create index dummy on am310(col1, col2, col3) nosegment;<br />
<br />
Index created.<br />
<br />
SQL> create index am310_n1 on am310(col1, col2, col3);<br />
<br />
Index created.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-2531245973789799632009-10-06T09:31:00.001-07:002009-10-06T09:31:42.650-07:00Tuning PL/SQL Applications for PerformanceHow PL/SQL Optimizes Your Programs<br />
In releases prior to 10g, the PL/SQL compiler translated your code to machine code without applying many changes for performance. Now, PL/SQL uses an optimizing compiler that can rearrange code for better performance.<br />
You do not need to do anything to get the benefits of this new optimizer. It is enabled by default. In rare cases, if the overhead of the optimizer makes compilation of very large applications take too long, you might lower the optimization by setting the initialization parameter PLSQL_OPTIMIZE_LEVEL=1 instead of its default value 2. In even rarer cases, you might see a change in exception behavior, either an exception that is not raised at all, or one that is raised earlier than expected. Setting PL_SQL_OPTIMIZE_LEVEL=0 prevents the code from being rearranged at all.<br />
<br />
When to Tune PL/SQL Code<br />
The information in this chapter is especially valuable if you are responsible for:<br />
" Programs that do a lot of mathematical calculations. You will want to investigate the datatypes PLS_INTEGER, BINARY_FLOAT, and BINARY_DOUBLE.<br />
" Functions that are called from PL/SQL queries, where the functions might be executed millions of times. You will want to look at all performance features to make the function as efficient as possible, and perhaps a function-based index to precompute the results for each row and save on query time.<br />
" Programs that spend a lot of time processing INSERT, UPDATE, or DELETE statements, or looping through query results. You will want to investigate the FORALL statement for issuing DML, and the BULK COLLECT INTO and RETURNING BULK COLLECT INTO clauses for queries.<br />
" Older code that does not take advantage of recent PL/SQL language features. (With the many performance improvements in Oracle Database 10g, any code from earlier releases is a candidate for tuning.)<br />
" Any program that spends a lot of time doing PL/SQL processing, as opposed to issuing DDL statements like CREATE TABLE that are just passed directly to SQL. You will want to investigate native compilation. Because many built-in database features use PL/SQL, you can apply this tuning feature to an entire database to improve performance in many areas, not just your own code.<br />
Before starting any tuning effort, benchmark the current system and measure how long particular subprograms take. PL/SQL in Oracle Database 10g includes many automatic optimizations, so you might see performance improvements without doing any tuning.<br />
<br />
<br />
Guidelines for Avoiding PL/SQL Performance Problems<br />
When a PL/SQL-based application performs poorly, it is often due to badly written SQL statements, poor programming practices, inattention to PL/SQL basics, or misuse of shared memory.<br />
<br />
Avoiding CPU Overhead in PL/SQL Code<br />
<br />
Make SQL Statements as Efficient as Possible<br />
PL/SQL programs look relatively simple because most of the work is done by SQL statements. Slow SQL statements are the main reason for slow execution.<br />
If SQL statements are slowing down your program:<br />
" Make sure you have appropriate indexes. There are different kinds of indexes for different situations. Your index strategy might be different depending on the sizes of various tables in a query, the distribution of data in each query, and the columns used in the WHERE clauses.<br />
" Make sure you have up-to-date statistics on all the tables, using the subprograms in the DBMS_STATS package.<br />
" Analyze the execution plans and performance of the SQL statements, using:<br />
" EXPLAIN PLAN statement<br />
" SQL Trace facility with TKPROF utility<br />
" Oracle Trace facility<br />
" Rewrite the SQL statements if necessary. For example, query hints can avoid problems such as unnecessary full-table scans.<br />
Some PL/SQL features also help improve the performance of SQL statements:<br />
" If you are running SQL statements inside a PL/SQL loop, look at the FORALL statement as a way to replace loops of INSERT, UPDATE, and DELETE statements.<br />
" If you are looping through the result set of a query, look at the BULK COLLECT clause of the SELECT INTO statement as a way to bring the entire result set into memory in a single operation.<br />
<br />
Make Function Calls as Efficient as Possible<br />
Badly written subprograms (for example, a slow sort or search function) can harm performance. Avoid unnecessary calls to subprograms, and optimize their code:<br />
" If a function is called within a SQL query, you can cache the function value for each row by creating a function-based index on the table in the query. The CREATE INDEX statement might take a while, but queries can be much faster.<br />
" If a column is passed to a function within an SQL query, the query cannot use regular indexes on that column, and the function might be called for every row in a (potentially very large) table. Consider nesting the query so that the inner query filters the results to a small number of rows, and the outer query calls the function only a few times:<br />
BEGIN<br />
-- Inefficient, calls my_function for every row.<br />
FOR item IN (SELECT DISTINCT(SQRT(department_id)) col_alias FROM employees)<br />
LOOP<br />
dbms_output.put_line(item.col_alias);<br />
END LOOP;<br />
<br />
-- Efficient, only calls function once for each distinct value.<br />
FOR item IN<br />
( SELECT SQRT(department_id) col_alias FROM<br />
( SELECT DISTINCT department_id FROM employees)<br />
)<br />
LOOP<br />
dbms_output.put_line(item.col_alias);<br />
END LOOP;<br />
END;<br />
/<br />
If you use OUT or IN OUT parameters, PL/SQL adds some performance overhead to ensure correct behavior in case of exceptions (assigning a value to the OUT parameter, then exiting the subprogram because of an unhandled exception, so that the OUT parameter keeps its original value).<br />
If your program does not depend on OUT parameters keeping their values in such situations, you can add the NOCOPY keyword to the parameter declarations, so the parameters are declared OUT NOCOPY or IN OUT NOCOPY.<br />
This technique can give significant speedup if you are passing back large amounts of data in OUT parameters, such as collections, big VARCHAR2 values, or LOBs.<br />
This technique also applies to member subprograms of object types. If these subprograms modify attributes of the object type, all the attributes are copied when the subprogram ends. To avoid this overhead, you can explicitly declare the first parameter of the member subprogram as SELF IN OUT NOCOPY, instead of relying on PL/SQL's implicit declaration SELF IN OUT.<br />
<br />
Make Loops as Efficient as Possible<br />
Because PL/SQL applications are often built around loops, it is important to optimize the loop itself and the code inside the loop:<br />
" Move initializations or computations outside the loop if possible.<br />
" To issue a series of DML statements, replace loop constructs with FORALL statements.<br />
" To loop through a result set and store the values, use the BULK COLLECT clause on the query to bring the query results into memory in one operation.<br />
" If you have to loop through a result set more than once, or issue other queries as you loop through a result set, you can probably enhance the original query to give you exactly the results you want. Some query operators to explore include UNION, INTERSECT, MINUS, and CONNECT BY.<br />
" You can also nest one query inside another (known as a subselect) to do the filtering and sorting in multiple stages. For example, instead of calling a PL/SQL function in the inner WHERE clause (which might call the function once for each row of the table), you can filter the result set to a small set of rows in the inner query, and call the function in the outer query.<br />
<br />
Don't Duplicate Built-in String Functions<br />
PL/SQL provides many highly optimized string functions such as REPLACE, TRANSLATE, SUBSTR, INSTR, RPAD, and LTRIM. The built-in functions use low-level code that is more efficient than regular PL/SQL.<br />
If you use PL/SQL string functions to search for regular expressions, consider using the built-in regular expression functions, such as REGEXP_SUBSTR.<br />
<br />
Reorder Conditional Tests to Put the Least Expensive First<br />
PL/SQL stops evaluating a logical expression as soon as the result can be determined (known as short-circuit evaluation).<br />
When evaluating multiple conditions separated by AND or OR, put the least expensive ones first. For example, check the values of PL/SQL variables before testing function return values, because PL/SQL might be able to skip calling the functions.<br />
<br />
Minimize Datatype Conversions<br />
At run time, PL/SQL converts between different datatypes automatically. For example, assigning a PLS_INTEGER variable to a NUMBER variable results in a conversion because their internal representations are different.<br />
Avoiding implicit conversions can improve performance. Use literals of the appropriate types: character literals in character expressions, decimal numbers in number expressions, and so on.<br />
In the example below, the integer literal 15 must be converted to an Oracle NUMBER before the addition. The floating-point literal 15.0 is represented as a NUMBER, avoiding the need for a conversion.<br />
DECLARE<br />
n NUMBER;<br />
c CHAR(5);<br />
BEGIN<br />
n := n + 15; -- converted implicitly; slow<br />
n := n + 15.0; -- not converted; fast<br />
c := 25; -- converted implicitly; slow<br />
c := TO_CHAR(25); -- converted explicitly; still slow<br />
c := '25'; -- not converted; fast<br />
END;<br />
/<br />
Minimizing conversions might mean changing the types of your variables, or even working backward and designing your tables with different datatypes. Or, you might convert data once (such as from an INTEGER column to a PLS_INTEGER variable) and use the PL/SQL type consistently after that.<br />
<br />
Use PLS_INTEGER or BINARY_INTEGER for Integer Arithmetic<br />
When you need to declare a local integer variable, use the datatype PLS_INTEGER, which is the most efficient integer type. PLS_INTEGER values require less storage than INTEGER or NUMBER values, and PLS_INTEGER operations use machine arithmetic.<br />
The BINARY_INTEGER datatype is just as efficient as PLS_INTEGER for any new code, but if you are running the same code on Oracle9i or Oracle8i databases, PLS_INTEGER is faster.<br />
The datatype NUMBER and its subtypes are represented in a special internal format, designed for portability and arbitrary scale and precision, not performance. Even the subtype INTEGER is treated as a floating-point number with nothing after the decimal point. Operations on NUMBER or INTEGER variables require calls to library routines.<br />
Avoid constrained subtypes such as INTEGER, NATURAL, NATURALN, POSITIVE, POSITIVEN, and SIGNTYPE in performance-critical code. Variables of these types require extra checking at run time, each time they are used in a calculation.<br />
<br />
Use BINARY_FLOAT and BINARY_DOUBLE for Floating-Point Arithmetic<br />
The datatype NUMBER and its subtypes are represented in a special internal format, designed for portability and arbitrary scale and precision, not performance. Operations on NUMBER or INTEGER variables require calls to library routines.<br />
The BINARY_FLOAT and BINARY_DOUBLE types can use native machine arithmetic instructions, and are more efficient for number-crunching applications such as scientific processing. They also require less space in the database.<br />
These types do not always represent fractional values precisely, and handle rounding differently than the NUMBER types. These types are less suitable for financial code where accuracy is critical.<br />
<br />
Avoiding Memory Overhead in PL/SQL Code<br />
<br />
Be Generous When Declaring Sizes for VARCHAR2 Variables<br />
You might need to allocate large VARCHAR2 variables when you are not sure how big an expression result will be. You can actually conserve memory by declaring VARCHAR2 variables with large sizes, such as 32000, rather than estimating just a little on the high side, such as by specifying a size such as 256 or 1000. PL/SQL has an optimization that makes it easy to avoid overflow problems and still conserve memory. Specify a size of 2000 or more characters for the VARCHAR2 variable; PL/SQL waits until you assign the variable, then only allocates as much storage as needed.<br />
<br />
Group Related Subprograms into Packages<br />
When you call a packaged subprogram for the first time, the whole package is loaded into the shared memory pool. Subsequent calls to related subprograms in the package require no disk I/O, and your code executes faster. If the package is aged out of memory, it must be reloaded if you reference it again.<br />
You can improve performance by sizing the shared memory pool correctly. Make sure it is large enough to hold all frequently used packages but not so large that memory is wasted.<br />
<br />
Pin Packages in the Shared Memory Pool<br />
You can "pin" frequently accessed packages in the shared memory pool, using the supplied package DBMS_SHARED_POOL. When a package is pinned, it is not aged out by the least recently used (LRU) algorithm that Oracle normally uses. The package remains in memory no matter how full the pool gets or how frequently you access the package.<br />
Profiling and Tracing PL/SQL Programs<br />
As you develop larger and larger PL/SQL applications, it becomes more difficult to isolate performance problems. PL/SQL provides a Profiler API to profile run-time behavior and to help you identify performance bottlenecks. PL/SQL also provides a Trace API for tracing the execution of programs on the server. You can use Trace to trace the execution by subprogram or exception.<br />
<br />
Using The Profiler API: Package DBMS_PROFILER<br />
The Profiler API is implemented as PL/SQL package DBMS_PROFILER, which provides services for gathering and saving run-time statistics. The information is stored in database tables, which you can query later. For example, you can learn how much time was spent executing each PL/SQL line and subprogram.<br />
To use the Profiler, you start the profiling session, run your application long enough to get adequate code coverage, flush the collected data to the database, then stop the profiling session.<br />
The Profiler traces the execution of your program, computing the time spent at each line and in each subprogram. You can use the collected data to improve performance. For instance, you might focus on subprograms that run slowly.<br />
<br />
Analyzing the Collected Performance Data<br />
The next step is to determine why more time was spent executing certain code segments or accessing certain data structures. Find the problem areas by querying the performance data. Focus on the subprograms and packages that use up the most execution time, inspecting possible performance bottlenecks such as SQL statements, loops, and recursive functions.<br />
<br />
Using Trace Data to Improve Performance<br />
Use the results of your analysis to rework slow algorithms. For example, due to an exponential growth in data, you might need to replace a linear search with a binary search. Also, look for inefficiencies caused by inappropriate data structures, and, if necessary, replace those data structures.<br />
<br />
Using The Trace API: Package DBMS_TRACE<br />
With large, complex applications, it becomes difficult to keep track of calls between subprograms. By tracing your code with the Trace API, you can see the order in which subprograms execute. The Trace API is implemented as PL/SQL package DBMS_TRACE, which provides services for tracing execution by subprogram or exception.<br />
To use Trace, you start the tracing session, run your application, then stop the tracing session. As the program executes, trace data is collected and stored in database tables.<br />
<br />
Controlling the Trace<br />
Tracing large applications can produce huge amounts of data that are difficult to manage. Before starting Trace, you can optionally limit the volume of data collected by selecting specific subprograms for trace data collection.<br />
In addition, you can choose a tracing level. For example, you can choose to trace all subprograms and exceptions, or you can choose to trace selected subprograms and exceptions.<br />
Reducing Loop Overhead for DML Statements and Queries (FORALL, BULK COLLECT)<br />
PL/SQL sends SQL statements such as DML and queries to the SQL engine for execution, and SQL returns the result data to PL/SQL. You can minimize the performance overhead of this communication between PL/SQL and SQL by using the PL/SQL language features known collectively as bulk SQL. The FORALL statement sends INSERT, UPDATE, or DELETE statements in batches, rather than one at a time. The BULK COLLECT clause brings back batches of results from SQL. If the DML statement affects four or more database rows, the use of bulk SQL can improve performance considerably.<br />
The assigning of values to PL/SQL variables in SQL statements is called binding. PL/SQL binding operations fall into three categories:<br />
" in-bind When a PL/SQL variable or host variable is stored in the database by an INSERT or UPDATE statement.<br />
" out-bind When a database value is assigned to a PL/SQL variable or a host variable by the RETURNING clause of an INSERT, UPDATE, or DELETE statement.<br />
" define When a database value is assigned to a PL/SQL variable or a host variable by a SELECT or FETCH statement.<br />
Bulk SQL uses PL/SQL collections, such as varrays or nested tables, to pass large amounts of data back and forth in a single operation. This process is known as bulk binding. If the collection has 20 elements, bulk binding lets you perform the equivalent of 20 SELECT, INSERT, UPDATE, or DELETE statements using a single operation. Queries can pass back any number of results, without requiring a FETCH statement for each row.<br />
To speed up INSERT, UPDATE, and DELETE statements, enclose the SQL statement within a PL/SQL FORALL statement instead of a loop construct.<br />
To speed up SELECT statements, include the BULK COLLECT INTO clause in the SELECT statement instead of using INTO.<br />
Using the FORALL Statement<br />
The keyword FORALL lets you run multiple DML statements very efficiently. It can only repeat a single DML statement, unlike a general-purpose FOR loop.<br />
The SQL statement can reference more than one collection, but FORALL only improves performance where the index value is used as a subscript.<br />
Usually, the bounds specify a range of consecutive index numbers. If the index numbers are not consecutive, such as after you delete collection elements, you can use the INDICES OF or VALUES OF clause to iterate over just those index values that really exist.<br />
The INDICES OF clause iterates over all of the index values in the specified collection, or only those between a lower and upper bound.<br />
The VALUES OF clause refers to a collection that is indexed by BINARY_INTEGER or PLS_INTEGER and whose elements are of type BINARY_INTEGER or PLS_INTEGER. The FORALL statement iterates over the index values specified by the elements of this collection.<br />
<br />
Example 11-1 Issuing DELETE Statements in a Loop<br />
This FORALL statement sends all three DELETE statements to the SQL engine at once:<br />
CREATE TABLE employees2 AS SELECT * FROM employees;<br />
DECLARE<br />
TYPE NumList IS VARRAY(20) OF NUMBER;<br />
depts NumList := NumList(10, 30, 70); -- department numbers<br />
BEGIN<br />
FORALL i IN depts.FIRST..depts.LAST<br />
DELETE FROM employees2 WHERE department_id = depts(i);<br />
COMMIT;<br />
END;<br />
/<br />
DROP TABLE employees2;<br />
<br />
Example 11-2 Issuing INSERT Statements in a Loop<br />
The following example loads some data into PL/SQL collections. Then it inserts the collection elements into a database table twice: first using a FOR loop, then using a FORALL statement. The FORALL version is much faster.<br />
CREATE TABLE parts1 (pnum INTEGER, pname VARCHAR2(15));<br />
CREATE TABLE parts2 (pnum INTEGER, pname VARCHAR2(15));<br />
DECLARE<br />
TYPE NumTab IS TABLE OF parts1.pnum%TYPE INDEX BY PLS_INTEGER;<br />
TYPE NameTab IS TABLE OF parts1.pname%TYPE INDEX BY PLS_INTEGER;<br />
pnums NumTab;<br />
pnames NameTab;<br />
iterations CONSTANT PLS_INTEGER := 500;<br />
t1 INTEGER; t2 INTEGER; t3 INTEGER;<br />
BEGIN<br />
FOR j IN 1..iterations LOOP -- load index-by tables<br />
pnums(j) := j;<br />
pnames(j) := 'Part No. ' || TO_CHAR(j);<br />
END LOOP;<br />
t1 := dbms_utility.get_time;<br />
FOR i IN 1..iterations LOOP -- use FOR loop<br />
INSERT INTO parts1 VALUES (pnums(i), pnames(i));<br />
END LOOP;<br />
t2 := dbms_utility.get_time;<br />
FORALL i IN 1..iterations -- use FORALL statement<br />
INSERT INTO parts2 VALUES (pnums(i), pnames(i));<br />
t3 := dbms_utility.get_time;<br />
dbms_output.put_line('Execution Time (secs)');<br />
dbms_output.put_line('---------------------');<br />
dbms_output.put_line('FOR loop: ' || TO_CHAR((t2 - t1)/100));<br />
dbms_output.put_line('FORALL: ' || TO_CHAR((t3 - t2)/100));<br />
COMMIT;<br />
END;<br />
/<br />
DROP TABLE parts1;<br />
DROP TABLE parts2;<br />
Executing this block should show that the loop using FORALL is much faster.<br />
<br />
Example 11-3 Using FORALL with Part of a Collection<br />
The bounds of the FORALL loop can apply to part of a collection, not necessarily all the elements:<br />
CREATE TABLE employees2 AS SELECT * FROM employees;<br />
DECLARE<br />
TYPE NumList IS VARRAY(10) OF NUMBER;<br />
depts NumList := NumList(5,10,20,30,50,55,57,60,70,75);<br />
BEGIN<br />
FORALL j IN 4..7 -- use only part of varray<br />
DELETE FROM employees2 WHERE department_id = depts(j);<br />
COMMIT;<br />
END;<br />
/<br />
DROP TABLE employees2;<br />
<br />
Example 11-4 Using FORALL with Non-Consecutive Index Values<br />
You might need to delete some elements from a collection before using the collection in a FORALL statement. The INDICES OF clause processes sparse collections by iterating through only the remaining elements.<br />
You might also want to leave the original collection alone, but process only some elements, process the elements in a different order, or process some elements more than once. Instead of copying the entire elements into new collections, which might use up substantial amounts of memory, the VALUES OF clause lets you set up simple collections whose elements serve as "pointers" to elements in the original collection.<br />
The following example creates a collection holding some arbitrary data, a set of table names. Deleting some of the elements makes it a sparse collection that would not work in a default FORALL statement. The program uses a FORALL statement with the INDICES OF clause to insert the data into a table. It then sets up two more collections, pointing to certain elements from the original collection. The program stores each set of names in a different database table using FORALL statements with the VALUES OF clause.<br />
-- Create empty tables to hold order details<br />
CREATE TABLE valid_orders (cust_name VARCHAR2(32), amount NUMBER(10,2));<br />
CREATE TABLE big_orders AS SELECT * FROM valid_orders WHERE 1 = 0;<br />
CREATE TABLE rejected_orders AS SELECT * FROM valid_orders WHERE 1 = 0;<br />
<br />
DECLARE<br />
-- Make collections to hold a set of customer names and order amounts.<br />
<br />
SUBTYPE cust_name IS valid_orders.cust_name%TYPE;<br />
TYPE cust_typ IS TABLe OF cust_name;<br />
cust_tab cust_typ;<br />
<br />
SUBTYPE order_amount IS valid_orders.amount%TYPE;<br />
TYPE amount_typ IS TABLE OF NUMBER;<br />
amount_tab amount_typ;<br />
<br />
-- Make other collections to point into the CUST_TAB collection.<br />
TYPE index_pointer_t IS TABLE OF PLS_INTEGER;<br />
big_order_tab index_pointer_t := index_pointer_t();<br />
rejected_order_tab index_pointer_t := index_pointer_t();<br />
<br />
PROCEDURE setup_data IS BEGIN<br />
-- Set up sample order data, including some invalid orders and some 'big' orders.<br />
cust_tab := cust_typ('Company 1','Company 2','Company 3','Company 4', 'Company 5');<br />
amount_tab := amount_typ(5000.01, 0, 150.25, 4000.00, NULL);<br />
END;<br />
<br />
BEGIN<br />
setup_data();<br />
<br />
dbms_output.put_line('--- Original order data ---');<br />
FOR i IN 1..cust_tab.LAST LOOP<br />
dbms_output.put_line('Customer #' || i || ', ' || cust_tab(i) || ': $' || amount_tab(i));<br />
END LOOP;<br />
<br />
-- Delete invalid orders (where amount is null or 0).<br />
FOR i IN 1..cust_tab.LAST LOOP<br />
IF amount_tab(i) is null or amount_tab(i) = 0 THEN<br />
cust_tab.delete(i);<br />
amount_tab.delete(i);<br />
END IF;<br />
END LOOP;<br />
<br />
dbms_output.put_line('--- Data with invalid orders deleted ---');<br />
FOR i IN 1..cust_tab.LAST LOOP<br />
IF cust_tab.EXISTS(i) THEN<br />
dbms_output.put_line('Customer #' || i || ', ' || cust_tab(i) || ': $' || amount_tab(i));<br />
END IF;<br />
END LOOP;<br />
<br />
-- Since the subscripts of our collections are not consecutive, we use<br />
-- FORALL...INDICES OF to iterate through the actual subscripts, rather than 1..COUNT.<br />
FORALL i IN INDICES OF cust_tab<br />
INSERT INTO valid_orders(cust_name, amount) VALUES(cust_tab(i), amount_tab(i));<br />
<br />
-- Now let's process the order data differently. We'll extract 2 subsets<br />
-- and store each subset in a different table.<br />
<br />
setup_data(); -- Initialize the CUST_TAB and AMOUNT_TAB collections again.<br />
<br />
FOR i IN cust_tab.FIRST .. cust_tab.LAST LOOP<br />
IF amount_tab(i) IS NULL OR amount_tab(i) = 0 THEN<br />
rejected_order_tab.EXTEND; -- Add a new element to this collection.<br />
rejected_order_tab(rejected_order_tab.LAST) := i; -- And record the subscript from the original collection.<br />
END IF;<br />
IF amount_tab(i) > 2000 THEN<br />
big_order_tab.EXTEND; -- Add a new element to this collection.<br />
big_order_tab(big_order_tab.LAST) := i; -- And record the subscript from the original collection.<br />
END IF;<br />
END LOOP;<br />
<br />
-- Now it's easy to run one DML statement on one subset of elements, and another DML statement on a different subset.<br />
<br />
FORALL i IN VALUES OF rejected_order_tab<br />
INSERT INTO rejected_orders VALUES (cust_tab(i), amount_tab(i));<br />
<br />
FORALL i IN VALUES OF big_order_tab<br />
INSERT INTO big_orders VALUES (cust_tab(i), amount_tab(i));<br />
<br />
COMMIT;<br />
END;<br />
/<br />
-- Verify that the correct order details were stored.<br />
SELECT cust_name "Customer", amount "Valid order amount" FROM valid_orders;<br />
SELECT cust_name "Customer", amount "Big order amount" FROM big_orders;<br />
SELECT cust_name "Customer", amount "Rejected order amount" FROM rejected_orders;<br />
<br />
DROP TABLE valid_orders;<br />
DROP TABLE big_orders;<br />
DROP TABLE rejected_orders;<br />
<br />
How FORALL Affects Rollbacks<br />
In a FORALL statement, if any execution of the SQL statement raises an unhandled exception, all database changes made during previous executions are rolled back. However, if a raised exception is caught and handled, changes are rolled back to an implicit savepoint marked before each execution of the SQL statement. Changes made during previous executions are not rolled back. For example, suppose you create a database table that stores department numbers and job titles, as follows. Then, you change the job titles so that they are longer. The second UPDATE fails because the new value is too long for the column. Because we handle the exception, the first UPDATE is not rolled back and we can commit that change.<br />
CREATE TABLE emp2 (deptno NUMBER(2), job VARCHAR2(18));<br />
DECLARE<br />
TYPE NumList IS TABLE OF NUMBER;<br />
depts NumList := NumList(10, 20, 30);<br />
BEGIN<br />
INSERT INTO emp2 VALUES(10, 'Clerk');<br />
INSERT INTO emp2 VALUES(20, 'Bookkeeper'); -- Lengthening this job title causes an exception.<br />
INSERT INTO emp2 VALUES(30, 'Analyst');<br />
COMMIT;<br />
<br />
FORALL j IN depts.FIRST..depts.LAST -- Run 3 UPDATE statements.<br />
UPDATE emp2 SET job = job || ' (Senior)' WHERE deptno = depts(j);<br />
-- raises a "value too large" exception<br />
EXCEPTION<br />
WHEN OTHERS THEN<br />
dbms_output.put_line('Problem in the FORALL statement.');<br />
COMMIT; -- Commit results of successful updates.<br />
END;<br />
/<br />
DROP TABLE emp2;<br />
<br />
<br />
Counting Rows Affected by FORALL with the %BULK_ROWCOUNT Attribute<br />
The cursor attributes SQL%FOUND, SQL%ISOPEN, SQL%NOTFOUND, and SQL%ROWCOUNT, return useful information about the most recently executed DML statement.<br />
The SQL cursor has one composite attribute, %BULK_ROWCOUNT, for use with the FORALL statement. This attribute works like an associative array: SQL%BULK_ROWCOUNT(i) stores the number of rows processed by the ith execution of an INSERT, UPDATE or DELETE statement. For example:<br />
CREATE TABLE emp2 AS SELECT * FROM employees;<br />
DECLARE<br />
TYPE NumList IS TABLE OF NUMBER;<br />
depts NumList := NumList(30, 50, 60);<br />
BEGIN<br />
FORALL j IN depts.FIRST..depts.LAST<br />
DELETE FROM emp2 WHERE department_id = depts(j);<br />
-- How many rows were affected by each DELETE statement?<br />
FOR i IN depts.FIRST..depts.LAST<br />
LOOP<br />
dbms_output.put_line('Iteration #' || i || ' deleted ' ||<br />
SQL%BULK_ROWCOUNT(i) || ' rows.');<br />
END LOOP;<br />
END;<br />
/<br />
DROP TABLE emp2;<br />
The FORALL statement and %BULK_ROWCOUNT attribute use the same subscripts. For example, if FORALL uses the range 5..10, so does %BULK_ROWCOUNT. If the FORALL tatement uses the INDICES OF clause to process a sparse collection, %BULK_ROWCOUNT has corresponding sparse subscripts. If the FORALL statement uses the VALUES OF clause to process a subset of elements, %BULK_ROWCOUNT has subscripts corresponding to the values of the elements in the index collection. If the index collection contains duplicate elements, so that some DML statements are issued multiple times using the same subscript, then the corresponding elements of %BULK_ROWCOUNT represent the sum of all rows affected by the DML statement using that subscript. (For examples showing how to interpret %BULK_ROWCOUNT when using the INDICES OF and VALUES OF clauses, see the PL/SQL sample programs at http://otn.oracle.com/tech/pl_sql/.)<br />
%BULK_ROWCOUNT is usually equal to 1 for inserts, because a typical insert operation affects only a single row. For the INSERT ... SELECT construct, %BULK_ROWCOUNT might be greater than 1. For example, the FORALL statement below inserts an arbitrary number of rows for each iteration. After each iteration, %BULK_ROWCOUNT returns the number of items inserted:<br />
CREATE TABLE emp_by_dept AS SELECT employee_id, department_id<br />
FROM employees WHERE 1 = 0;<br />
DECLARE<br />
TYPE dept_tab IS TABLE OF departments.department_id%TYPE;<br />
deptnums dept_tab;<br />
BEGIN<br />
SELECT department_id BULK COLLECT INTO deptnums FROM departments;<br />
<br />
FORALL i IN 1..deptnums.COUNT<br />
INSERT INTO emp_by_dept<br />
SELECT employee_id, department_id FROM employees<br />
WHERE department_id = deptnums(i);<br />
<br />
FOR i IN 1..deptnums.COUNT LOOP<br />
-- Count how many rows were inserted for each department; that is,<br />
-- how many employees are in each department.<br />
dbms_output.put_line('Dept '||deptnums(i)||': inserted '||<br />
SQL%BULK_ROWCOUNT(i)||' records');<br />
END LOOP;<br />
<br />
dbms_output.put_line('Total records inserted =' || SQL%ROWCOUNT);<br />
END;<br />
/<br />
DROP TABLE emp_by_dept;<br />
You can also use the scalar attributes %FOUND, %NOTFOUND, and %ROWCOUNT after running a FORALL statement. For example, %ROWCOUNT returns the total number of rows processed by all executions of the SQL statement.<br />
%FOUND and %NOTFOUND refer only to the last execution of the SQL statement. You can use %BULK_ROWCOUNT to infer their values for individual executions. For example, when %BULK_ROWCOUNT(i) is zero, %FOUND and %NOTFOUND are FALSE and TRUE, respectively.<br />
Handling FORALL Exceptions with the %BULK_EXCEPTIONS Attribute<br />
PL/SQL provides a mechanism to handle exceptions raised during the execution of a FORALL statement. This mechanism enables a bulk-bind operation to save information about exceptions and continue processing.<br />
To have a bulk bind complete despite errors, add the keywords SAVE EXCEPTIONS to your FORALL statement after the bounds, before the DML statement.<br />
All exceptions raised during the execution are saved in the cursor attribute %BULK_EXCEPTIONS, which stores a collection of records. Each record has two fields:<br />
" %BULK_EXCEPTIONS(i).ERROR_INDEX holds the "iteration" of the FORALL statement during which the exception was raised.<br />
" %BULK_EXCEPTIONS(i).ERROR_CODE holds the corresponding Oracle error code.<br />
The values stored by %BULK_EXCEPTIONS always refer to the most recently executed FORALL statement. The number of exceptions is saved in %BULK_EXCEPTIONS.COUNT. Its subscripts range from 1 to COUNT.<br />
You might need to work backward to determine which collection element was used in the iteration that caused an exception. For example, if you use the INDICES OF clause to process a sparse collection, you must step through the elements one by one to find the one corresponding to %BULK_EXCEPTIONS(i).ERROR_INDEX. If you use the VALUES OF clause to process a subset of elements, you must find the element in the index collection whose subscript matches %BULK_EXCEPTIONS(i).ERROR_INDEX, and then use that element's value as the subscript to find the erroneous element in the original collection. (For examples showing how to find the erroneous elements when using the INDICES OF and VALUES OF clauses, see the PL/SQL sample programs at http://otn.oracle.com/tech/pl_sql/.)<br />
If you omit the keywords SAVE EXCEPTIONS, execution of the FORALL statement stops when an exception is raised. In that case, SQL%BULK_EXCEPTIONS.COUNT returns 1, and SQL%BULK_EXCEPTIONS contains just one record. If no exception is raised during execution, SQL%BULK_EXCEPTIONS.COUNT returns 0.<br />
<br />
Example 11-5 Bulk Operation That Continues Despite Exceptions<br />
The following example shows how you can perform a number of DML operations, without stopping if some operations encounter errors:<br />
CREATE TABLE emp2 AS SELECT * FROM employees;<br />
DECLARE<br />
TYPE NumList IS TABLE OF NUMBER;<br />
-- The zeros in this list will cause divide-by-zero errors.<br />
num_tab NumList := NumList(10,0,11,12,30,0,20,199,2,0,9,1);<br />
errors NUMBER;<br />
dml_errors EXCEPTION;<br />
PRAGMA exception_init(dml_errors, -24381);<br />
BEGIN<br />
-- SAVE EXCEPTIONS means don't stop if some DELETEs fail.<br />
FORALL i IN num_tab.FIRST..num_tab.LAST SAVE EXCEPTIONS<br />
DELETE FROM emp2 WHERE salary > 500000/num_tab(i);<br />
-- If any errors occurred during the FORALL SAVE EXCEPTIONS,<br />
-- a single exception is raised when the statement completes.<br />
EXCEPTION<br />
WHEN dml_errors THEN -- Now we figure out what failed and why.<br />
errors := SQL%BULK_EXCEPTIONS.COUNT;<br />
dbms_output.put_line('Number of DELETE statements that failed: ' || errors);<br />
FOR i IN 1..errors LOOP<br />
dbms_output.put_line('Error #' || i || ' occurred during '||<br />
'iteration #' || SQL%BULK_EXCEPTIONS(i).ERROR_INDEX);<br />
dbms_output.put_line('Error message is ' ||<br />
SQLERRM(-SQL%BULK_EXCEPTIONS(i).ERROR_CODE));<br />
END LOOP;<br />
END;<br />
/<br />
DROP TABLE emp2;<br />
In this example, PL/SQL raised the predefined exception ZERO_DIVIDE when i equaled 2, 6, 10. After the FORALL statement, SQL%BULK_EXCEPTIONS.COUNT returned 3, and the contents of SQL%BULK_EXCEPTIONS were (2,1476), (6,1476), and (10,1476). To get the Oracle error message (which includes the code), we negated the value of SQL%BULK_EXCEPTIONS(i).ERROR_CODE and passed the result to the error-reporting function SQLERRM, which expects a negative number. Here is the output:<br />
<br />
Number of errors is 3<br />
Error 1 occurred during iteration 2<br />
Oracle error is ORA-01476: divisor is equal to zero<br />
Error 2 occurred during iteration 6<br />
Oracle error is ORA-01476: divisor is equal to zero<br />
Error 3 occurred during iteration 10<br />
Oracle error is ORA-01476: divisor is equal to zero <br />
<br />
Retrieving Query Results into Collections with the BULK COLLECT Clause<br />
Using the keywords BULK COLLECT with a query is a very efficient way to retrieve the result set. Instead of looping through each row, you store the results in one or more collections, in a single operation. You can use these keywords in the SELECT INTO and FETCH INTO statements, and the RETURNING INTO clause.<br />
With the BULK COLLECT clause, all the variables in the INTO list must be collections. The table columns can hold scalar or composite values, including object types. The following example loads two entire database columns into nested tables:<br />
DECLARE<br />
TYPE NumTab IS TABLE OF employees.employee_id%TYPE;<br />
TYPE NameTab IS TABLE OF employees.last_name%TYPE;<br />
enums NumTab; -- No need to initialize the collections.<br />
names NameTab; -- Values will be filled in by the SELECT INTO.<br />
PROCEDURE print_results IS<br />
BEGIN<br />
dbms_output.put_line('Results:');<br />
FOR i IN enums.FIRST .. enums.LAST<br />
LOOP<br />
dbms_output.put_line(' Employee #' || enums(i) || ': ' ||<br />
names(i));<br />
END LOOP;<br />
END;<br />
BEGIN<br />
SELECT employee_id, last_name -- Retrieve data for 10 arbitrary employees.<br />
BULK COLLECT INTO enums, names<br />
FROM employees WHERE ROWNUM < 11;
-- The data has all been brought into memory by BULK COLLECT.
-- No need to FETCH each row from the result set.
print_results;
SELECT employee_id, last_name -- Retrieve approximately 20% of all rows
BULK COLLECT INTO enums, names
FROM employees SAMPLE (20);
print_results;
END;
/
The collections are initialized automatically. Nested tables and associative arrays are extended to hold as many elements as needed. If you use varrays, all the return values must fit in the varray's declared size. Elements are inserted starting at index 1, overwriting any existing elements.
Since the processing of the BULK COLLECT INTO clause is similar to a FETCH loop, it does not raise a NO_DATA_FOUND exception if no rows match the query. You must check whether the resulting nested table or varray is null, or if the resulting associative array has no elements.
To prevent the resulting collections from expanding without limit, you can use the pseudocolumn ROWNUM to limit the number of rows processed. Or, you can use the SAMPLE clause to retrieve a random sample of rows.
DECLARE
TYPE SalList IS TABLE OF emp.sal%TYPE;
sals SalList;
BEGIN
-- Limit the number of rows to 100.
SELECT sal BULK COLLECT INTO sals FROM emp
WHERE ROWNUM <= 100;
-- Retrieve 10% (approximately) of the rows in the table.
SELECT sal BULK COLLECT INTO sals FROM emp SAMPLE 10;
END;
/
You can process very large result sets by fetching a specified number of rows at a time from a cursor, as shown in the following sections.
Examples of Bulk-Fetching from a Cursor
Example 11-6 Bulk-Fetching from a Cursor Into One or More Collections
You can fetch from a cursor into one or more collections:
DECLARE
TYPE NameList IS TABLE OF employees.last_name%TYPE;
TYPE SalList IS TABLE OF employees.salary%TYPE;
CURSOR c1 IS SELECT last_name, salary FROM employees WHERE salary > 10000;<br />
names NameList;<br />
sals SalList;<br />
TYPE RecList IS TABLE OF c1%ROWTYPE;<br />
recs RecList;<br />
<br />
PROCEDURE print_results IS<br />
BEGIN<br />
dbms_output.put_line('Results:');<br />
IF names IS NULL OR names.COUNT = 0 THEN<br />
RETURN; -- Don't print anything if collections are empty.<br />
END IF;<br />
FOR i IN names.FIRST .. names.LAST<br />
LOOP<br />
dbms_output.put_line(' Employee ' || names(i) || ': $' ||<br />
sals(i));<br />
END LOOP;<br />
END;<br />
BEGIN<br />
dbms_output.put_line('--- Processing all results at once ---');<br />
OPEN c1;<br />
FETCH c1 BULK COLLECT INTO names, sals;<br />
CLOSE c1;<br />
print_results;<br />
<br />
dbms_output.put_line('--- Processing 7 rows at a time ---');<br />
OPEN c1;<br />
LOOP<br />
FETCH c1 BULK COLLECT INTO names, sals LIMIT 7;<br />
EXIT WHEN c1%NOTFOUND;<br />
print_results;<br />
END LOOP;<br />
-- Loop exits when fewer than 7 rows are fetched. Have to<br />
-- process the last few. Need extra checking inside PRINT_RESULTS<br />
-- in case it is called when the collection is empty.<br />
print_results;<br />
CLOSE c1;<br />
<br />
dbms_output.put_line('--- Fetching records rather than columns ---');<br />
OPEN c1;<br />
FETCH c1 BULK COLLECT INTO recs;<br />
FOR i IN recs.FIRST .. recs.LAST<br />
LOOP<br />
-- Now all the columns from the result set come from a single record.<br />
dbms_output.put_line(' Employee ' || recs(i).last_name || ': $'<br />
|| recs(i).salary);<br />
END LOOP;<br />
END;<br />
/<br />
<br />
Example 11-7 Bulk-Fetching from a Cursor Into a Collection of Records<br />
You can fetch from a cursor into a collection of records:<br />
DECLARE<br />
TYPE DeptRecTab IS TABLE OF dept%ROWTYPE;<br />
dept_recs DeptRecTab;<br />
CURSOR c1 IS<br />
SELECT deptno, dname, loc FROM dept WHERE deptno > 10;<br />
BEGIN<br />
OPEN c1;<br />
FETCH c1 BULK COLLECT INTO dept_recs;<br />
END;<br />
/<br />
<br />
Limiting the Rows for a Bulk FETCH Operation with the LIMIT Clause<br />
The optional LIMIT clause, allowed only in bulk FETCH statements, limits the number of rows fetched from the database.<br />
In the example below, with each iteration of the loop, the FETCH statement fetches ten rows (or less) into index-by table empnos. The previous values are overwritten.<br />
DECLARE<br />
TYPE NumTab IS TABLE OF NUMBER INDEX BY BINARY_INTEGER;<br />
CURSOR c1 IS SELECT empno FROM emp;<br />
empnos NumTab;<br />
rows NATURAL := 10;<br />
BEGIN<br />
OPEN c1;<br />
LOOP<br />
/* The following statement fetches 10 rows (or less). */<br />
FETCH c1 BULK COLLECT INTO empnos LIMIT rows;<br />
EXIT WHEN c1%NOTFOUND;<br />
...<br />
END LOOP;<br />
CLOSE c1;<br />
END;<br />
/<br />
<br />
Retrieving DML Results into a Collection with the RETURNING INTO Clause<br />
You can use the BULK COLLECT clause in the RETURNING INTO clause of an INSERT, UPDATE, or DELETE statement:<br />
CREATE TABLE emp2 AS SELECT * FROM employees;<br />
DECLARE<br />
TYPE NumList IS TABLE OF employees.employee_id%TYPE;<br />
enums NumList;<br />
TYPE NameList IS TABLE OF employees.last_name%TYPE;<br />
names NameList;<br />
BEGIN<br />
DELETE FROM emp2 WHERE department_id = 30<br />
RETURNING employee_id, last_name BULK COLLECT INTO enums, names;<br />
dbms_output.put_line('Deleted ' || SQL%ROWCOUNT || ' rows:');<br />
FOR i IN enums.FIRST .. enums.LAST<br />
LOOP<br />
dbms_output.put_line('Employee #' || enums(i) || ': ' || names(i));<br />
END LOOP;<br />
END;<br />
/<br />
DROP TABLE emp2;<br />
<br />
Using FORALL and BULK COLLECT Together<br />
You can combine the BULK COLLECT clause with a FORALL statement. The output collections are built up as the FORALL statement iterates.<br />
In the following example, the EMPNO value of each deleted row is stored in the collection ENUMS. The collection DEPTS has 3 elements, so the FORALL statement iterates 3 times. If each DELETE issued by the FORALL statement deletes 5 rows, then the collection ENUMS, which stores values from the deleted rows, has 15 elements when the statement completes:<br />
CREATE TABLE emp2 AS SELECT * FROM employees;<br />
DECLARE<br />
TYPE NumList IS TABLE OF NUMBER;<br />
depts NumList := NumList(10,20,30);<br />
TYPE enum_t IS TABLE OF employees.employee_id%TYPE;<br />
TYPE dept_t IS TABLE OF employees.department_id%TYPE;<br />
e_ids enum_t;<br />
d_ids dept_t;<br />
BEGIN<br />
FORALL j IN depts.FIRST..depts.LAST<br />
DELETE FROM emp2 WHERE department_id = depts(j)<br />
RETURNING employee_id, department_id BULK COLLECT INTO e_ids, d_ids;<br />
dbms_output.put_line('Deleted ' || SQL%ROWCOUNT || ' rows:');<br />
FOR i IN e_ids.FIRST .. e_ids.LAST<br />
LOOP<br />
dbms_output.put_line('Employee #' || e_ids(i) || ' from dept #' || d_ids(i));<br />
END LOOP;<br />
END;<br />
/<br />
DROP TABLE emp2;<br />
The column values returned by each execution are added to the values returned previously. If you use a FOR loop instead of the FORALL statement, the set of returned values is overwritten by each DELETE statement.<br />
You cannot use the SELECT ... BULK COLLECT statement in a FORALL statement.<br />
<br />
Using Host Arrays with Bulk Binds<br />
Client-side programs can use anonymous PL/SQL blocks to bulk-bind input and output host arrays. This is the most efficient way to pass collections to and from the database server.<br />
Host arrays are declared in a host environment such as an OCI or a Pro*C program and must be prefixed with a colon to distinguish them from PL/SQL collections. In the example below, an input host array is used in a DELETE statement. At run time, the anonymous PL/SQL block is sent to the database server for execution.<br />
DECLARE<br />
...<br />
BEGIN<br />
-- assume that values were assigned to the host array<br />
-- and host variables in the host environment<br />
FORALL i IN :lower..:upper<br />
DELETE FROM employees WHERE department_id = :depts(i);<br />
COMMIT;<br />
END;<br />
/<br />
<br />
<br />
Writing Computation-Intensive Programs in PL/SQL<br />
The BINARY_FLOAT and BINARY_DOUBLE datatypes make it practical to write PL/SQL programs to do number-crunching, for scientific applications involving floating-point calculations. These datatypes behave much like the native floating-point types on many hardware systems, with semantics derived from the IEEE-754 floating-point standard.<br />
The way these datatypes represent decimal data make them less suitable for financial applications, where precise representation of fractional amounts is more important than pure performance.<br />
The PLS_INTEGER and BINARY_INTEGER datatypes are PL/SQL-only datatypes that are more efficient than the SQL datatypes NUMBER or INTEGER for integer arithmetic. You can use PLS_INTEGER to write pure PL/SQL code for integer arithmetic, or convert NUMBER or INTEGER values to PLS_INTEGER for manipulation by PL/SQL.<br />
In previous releases, PLS_INTEGER was more efficient than BINARY_INTEGER. Now, they have similar performance, but you might still prefer PLS_INTEGER if your code might be run under older database releases.<br />
Within a package, you can write overloaded versions of procedures and functions that accept different numeric parameters. The math routines can be optimized for each kind of parameter (BINARY_FLOAT, BINARY_DOUBLE, NUMBER, PLS_INTEGER), avoiding unnecessary conversions.<br />
The built-in math functions such as SQRT, SIN, COS, and so on already have fast overloaded versions that accept BINARY_FLOAT and BINARY_DOUBLE parameters. You can speed up math-intensive code by passing variables of these types to such functions, and by calling the TO_BINARY_FLOAT or TO_BINARY_DOUBLE functions when passing expressions to such functions.<br />
Tuning Dynamic SQL with EXECUTE IMMEDIATE and Cursor Variables<br />
Some programs (a general-purpose report writer for example) must build and process a variety of SQL statements, where the exact text of the statement is unknown until run time. Such statements probably change from execution to execution. They are called dynamic SQL statements.<br />
Formerly, to execute dynamic SQL statements, you had to use the supplied package DBMS_SQL. Now, within PL/SQL, you can execute any kind of dynamic SQL statement using an interface called native dynamic SQL. The main PL/SQL features involved are the EXECUTE IMMEDIATE statement and cursor variables (also known as REF CURSORs).<br />
Native dynamic SQL code is more compact and much faster than calling the DBMS_SQL package. The following example declares a cursor variable, then associates it with a dynamic SELECT statement:<br />
DECLARE<br />
TYPE EmpCurTyp IS REF CURSOR;<br />
emp_cv EmpCurTyp;<br />
my_ename VARCHAR2(15);<br />
my_sal NUMBER := 1000;<br />
table_name VARCHAR2(30) := 'employees';<br />
BEGIN<br />
OPEN emp_cv FOR 'SELECT last_name, salary FROM ' || table_name ||<br />
' WHERE salary > :s' USING my_sal;<br />
CLOSE emp_cv;<br />
END;<br />
/<br />
Tuning PL/SQL Procedure Calls with the NOCOPY Compiler Hint<br />
By default, OUT and IN OUT parameters are passed by value. The values of any IN OUT parameters are copied before the subprogram is executed. During subprogram execution, temporary variables hold the output parameter values. If the subprogram exits normally, these values are copied to the actual parameters. If the subprogram exits with an unhandled exception, the original parameters are unchanged.<br />
When the parameters represent large data structures such as collections, records, and instances of object types, this copying slows down execution and uses up memory. In particular, this overhead applies to each call to an object method: temporary copies are made of all the attributes, so that any changes made by the method are only applied if the method exits normally.<br />
To avoid this overhead, you can specify the NOCOPY hint, which allows the PL/SQL compiler to pass OUT and IN OUT parameters by reference. If the subprogram exits normally, the behavior is the same as normal. If the subprogram exits early with an exception, the values of OUT and IN OUT parameters (or object attributes) might still change. To use this technique, ensure that the subprogram handles all exceptions.<br />
The following example asks the compiler to pass IN OUT parameter MY_STAFF by reference, to avoid copying the varray on entry to and exit from the subprogram:<br />
DECLARE<br />
TYPE Staff IS VARRAY(200) OF Employee;<br />
PROCEDURE reorganize (my_staff IN OUT NOCOPY Staff) IS ...<br />
BEGIN<br />
NULL;<br />
END;<br />
/<br />
The following example loads 25,000 records into a local nested table, which is passed to two local procedures that do nothing. A call to the procedure that uses NOCOPY takes much less time.<br />
DECLARE<br />
TYPE EmpTabTyp IS TABLE OF employees%ROWTYPE;<br />
emp_tab EmpTabTyp := EmpTabTyp(NULL); -- initialize<br />
t1 NUMBER;<br />
t2 NUMBER;<br />
t3 NUMBER;<br />
PROCEDURE get_time (t OUT NUMBER) IS<br />
BEGIN t := dbms_utility.get_time; END;<br />
PROCEDURE do_nothing1 (tab IN OUT EmpTabTyp) IS<br />
BEGIN NULL; END;<br />
PROCEDURE do_nothing2 (tab IN OUT NOCOPY EmpTabTyp) IS<br />
BEGIN NULL; END;<br />
BEGIN<br />
SELECT * INTO emp_tab(1) FROM employees WHERE employee_id = 100;<br />
emp_tab.EXTEND(49999, 1); -- copy element 1 into 2..50000<br />
get_time(t1);<br />
do_nothing1(emp_tab); -- pass IN OUT parameter<br />
get_time(t2);<br />
do_nothing2(emp_tab); -- pass IN OUT NOCOPY parameter<br />
get_time(t3);<br />
dbms_output.put_line('Call Duration (secs)');<br />
dbms_output.put_line('--------------------');<br />
dbms_output.put_line('Just IN OUT: ' || TO_CHAR((t2 - t1)/100.0));<br />
dbms_output.put_line('With NOCOPY: ' || TO_CHAR((t3 - t2))/100.0);<br />
END;<br />
/<br />
<br />
Restrictions on NOCOPY<br />
The use of NOCOPY increases the likelihood of parameter aliasing. For more information, see "Understanding Subprogram Parameter Aliasing".<br />
Remember, NOCOPY is a hint, not a directive. In the following cases, the PL/SQL compiler ignores the NOCOPY hint and uses the by-value parameter-passing method; no error is generated:<br />
" The actual parameter is an element of an associative array. This restriction does not apply if the parameter is an entire associative array.<br />
" The actual parameter is constrained, such as by scale or NOT NULL. This restriction does not apply to size-constrained character strings. This restriction does not extend to constrained elements or attributes of composite types.<br />
" The actual and formal parameters are records, one or both records were declared using %ROWTYPE or %TYPE, and constraints on corresponding fields in the records differ.<br />
" The actual and formal parameters are records, the actual parameter was declared (implicitly) as the index of a cursor FOR loop, and constraints on corresponding fields in the records differ.<br />
" Passing the actual parameter requires an implicit datatype conversion.<br />
" The subprogram is called through a database link or as an external procedure.<br />
<br />
<br />
Compiling PL/SQL Code for Native Execution<br />
You can speed up PL/SQL procedures by compiling them into native code residing in shared libraries. The procedures are translated into C code, then compiled with your usual C compiler and linked into the Oracle process.<br />
You can use this technique with both the supplied Oracle packages, and procedures you write yourself. Procedures compiled this way work in all server environments, such as the shared server configuration (formerly known as multi-threaded server) and Oracle Real Application Clusters.<br />
<br />
Before You Begin<br />
If you are a first-time user of native PL/SQL compilation, try it first with a test database, before proceeding to a production environment.<br />
Always back up your database before configuring the database for PL/SQL native compilation. If you find that the performance benefit is outweighed by extra compilation time, it might be faster to restore from a backup than to recompile everything in interpreted mode.<br />
Some of the setup steps require DBA authority. You must change the values of some initialization parameters, and create a new directory on the database server, preferably near the data files for the instance. The database server also needs a C compiler; on a cluster, the compiler is needed on each node. Even if you can test out these steps yourself on a development machine, you will generally need to consult with a DBA and enlist their help to use native compilation on a production server.<br />
Contact your system administrator to ensure that you have the required C compiler on your operating system, and find the path for its location. Use a text editor such as vi to open the file $ORACLE_HOME/plsql/spnc_commands, and make sure the command templates are correct. Generally, you should not need to make any changes here, just confirm that the setup is correct.<br />
<br />
Determining Whether to Use PL/SQL Native Compilation<br />
PL/SQL native compilation provides the greatest performance gains for computation-intensive procedural operations. Examples of such operations are data warehouse applications, and applications with extensive server-side transformations of data for display. In such cases, expect speed increases of up to 30%.<br />
Because this technique cannot do much to speed up SQL statements called from PL/SQL, it is most effective for compute-intensive PL/SQL procedures that do not spend most of their time executing SQL. You can test to see how much performance gain you can get by enabling PL/SQL native compilation.<br />
It takes longer to compile program units with native compilation than to use the default interpreted mode. You might turn off native compilation during the busiest parts of the development cycle, where code is being frequently recompiled.<br />
When you have decided that you will have significant performance gains in database operations using PL/SQL native compilation, Oracle Corporation recommends that you compile the whole database using the NATIVE setting. Compiling all the PL/SQL code in the database means you see the speedup in your own code, and in calls to all the built-in PL/SQL packages.<br />
<br />
How PL/SQL Native Compilation Works<br />
If you do not use native compilation, each PL/SQL program unit is compiled into an intermediate form, machine-readable code (m-code). The m-code is stored in the database dictionary and interpreted at run time.<br />
With PL/SQL native compilation, the PL/SQL statements are turned into C code that bypasses all the runtime interpretation, giving faster runtime performance.<br />
PL/SQL uses the command file $ORACLE_HOME/plsql/spnc_commands, and the supported operating system C compiler and linker, to compile and link the resulting C code into shared libraries. The shared libraries are stored inside the data dictionary, so that they can be backed up automatically and are protected from being deleted. These shared library files are copied to the filesystem and are loaded and run when the PL/SQL subprogram is invoked. If the files are deleted from the filesystem while the database is shut down, or if you change the directory that holds the libraries, they are extracted again automatically.<br />
Although PL/SQL program units that just call SQL statements might see little or no speedup, natively compiled PL/SQL is always at least as fast as the corresponding interpreted code. The compiled code makes the same library calls as the interpreted code would, so its behavior is exactly the same.<br />
<br />
Format of the spnc_commands File<br />
The spnc_commands file, in the $ORACLE_HOME/plsql directory, contains the templates for the commands to compile and link each program. Some special names such as %(src) are predefined, and are replaced by the corresponding filename. The variable $(ORACLE_HOME) is replaced by the location of the Oracle home directory. You can include comment lines, starting with a # character. The file contains comments that explain all the special notation.<br />
The spnc_commands file contains a predefined path for the C compiler, depending on the particular operating system. (One specific compiler is supported on each operating system.) In most cases, you should not need to change this path, but you might if you the system administrator has installed it in another location.<br />
<br />
System-Level Initialization Parameters for PL/SQL Native Compilation<br />
The following table lists the initialization parameters you must set before using PL/SQL native compilation. They can be set only at the system level, not by an ALTER SESSION command. You cannot use variables such as ORACLE_HOME in the values; use the full path instead.<br />
Note:<br />
The examples in this section for setting system parameters for PL/SQL native compilation assume a system using a server parameter file (SPFILE). <br />
If you use a text initialization parameter file (PFILE, or initsid.ora), ensure that you change parameters in your initialization parameter file, as indicated in the following table.<br />
<br />
Parameter Characteristics<br />
PLSQL_NATIVE_LIBRARY_DIR The full path and directory name used to store the shared libraries that contain natively compiled PL/SQL code. <br />
In accordance with optimal flexible architecture (OFA) rules, Oracle Corporation recommends that you create the shared library directory as a subdirectory where the data files are located.<br />
For security reasons, only the users oracle and root should have write privileges for this directory.<br />
PLSQL_NATIVE_LIBRARY_SUBDIR_COUNT The number of subdirectories in the directory specified by the parameter PLSQL_NATIVE_LIBRARY_DIR. <br />
Optional; use if the number of natively compiled program units exceeds 15000. If you need to set this option, Oracle Database Reference for complete details about the initialization parameters and data dictionary views.<br />
" <br />
Setting Up and Using PL/SQL Native Compilation<br />
To speed up one or more subprograms through native compilation:<br />
Set up the PLSQL_NATIVE_LIBRARY_DIR initialization parameter, and optionally the PLSQL_NATIVE_LIBRARY_SUBDIR_COUNT initialization parameter, as described above.<br />
Use the ALTER SYSTEM or ALTER SESSION command, or update your initialization file, to set the parameter PLSQL_CODE_TYPE to the value NATIVE.<br />
Compile one or more subprograms, using one of these methods:<br />
" Use CREATE OR REPLACE to create or recompile the subprogram.<br />
" Use the ALTER PROCEDURE, ALTER FUNCTION, or ALTER PACKAGE command with the COMPILE option to recompile the subprogram or the entire package. (You can also use the clause PLSQL_CODE_TYPE = NATIVE with the ALTER statements to affect specific subprograms without changing the initialization parameter for the whole session.)<br />
" Drop the subprogram and create it again.<br />
" Run one of the SQL*Plus scripts that creates a set of Oracle-supplied packages.<br />
" Create a database using a preconfigured initialization file with PLSQL_CODE_TYPE=NATIVE. During database creation, the UTLIRP script is run to compile all the Oracle-supplied packages.<br />
To be sure that the process worked, you can query the data dictionary to see that a procedure is compiled for native execution. To check whether an existing procedure is compiled for native execution or not, you can query the data dictionary views USER_PLSQL_OBJECT_SETTINGS, DBA_PLSQL_OBJECT_SETTINGS, and ALL_PLSQL_OBJECT_SETTINGS. For example, to check the status of the procedure MY_PROC, you could enter:<br />
CREATE OR REPLACE PROCEDURE my_proc AS BEGIN NULL; END;<br />
/<br />
SELECT plsql_code_type FROM user_plsql_object_settings WHERE name = 'MY_PROC';<br />
DROP PROCEDURE my_proc;<br />
The CODE_TYPE column has a value of NATIVE for procedures that are compiled for native execution, and INTERPRETED otherwise.<br />
After the procedures are compiled and turned into shared libraries, they are automatically linked into the Oracle process. You do not need to restart the database, or move the shared libraries to a different location. You can call back and forth between stored procedures, whether they are all interpreted, all compiled for native execution, or a mixture of both.<br />
<br />
Dependencies, Invalidation and Revalidation<br />
This recompilation happens automatically invalidated, such as when a table that it depends on is re-created.<br />
If an object on which a natively compiled PL/SQL subprogram depends changes, the subprogram is invalidated. The next time the same subprogram is called, the database recompiles the subprogram automatically. Because the PLSQL_CODE_TYPE setting is stored inside the library unit for each subprogram, the automatic recompilation uses this stored setting for code type.<br />
The stored settings are only used when recompiling as part of revalidation. If a PL/SQL subprogram is explicitly compiled through the SQL commands "create or replace" or "alter...compile", the current session setting is used.<br />
The generated shared libraries are stored in the database, in the SYSTEM tablespace. The first time a natively compiled procedure is executed, the corresponding shared library is copied from the database to the directory specified by the initialization parameter PLSQL_NATIVE_LIBRARY_DIR.<br />
<br />
Setting Up Databases for PL/SQL Native Compilation<br />
Use the procedures in this section to set up an entire database for PL/SQL native compilation. The performance benefits apply to all the built-in PL/SQL packages, which are used for many database operations.<br />
<br />
Creating a New Database for PL/SQL Native Compilation<br />
If you use Database Configuration Assistant, use it to set the initialization parameters required for PL/SQL native compilation, as described in the preceding section, "System-Level Initialization Parameters for PL/SQL Native Compilation".<br />
To find the supported C compiler on your operating system. refer to the table "Precompilers and Tools Restrictions and Requirements" in the installation guide for your operating system. Determine from your system administrator where it is located on your system. You will need to check that this path is correct in the spnc_commands file.<br />
Determine if you have so many PL/SQL program units that you need to set the initialization parameter PLSQL_NATIVE_DIR_SUBDIR_COUNT, and create PL/SQL native library subdirectories if necessary. By default, PL/SQL program units are kept in one directory. If the number of program units exceeds 15,000, the operating system begins to impose performance limits. To work around this problem, Oracle Corporation recommends that you spread the PL/SQL program units among subdirectories.<br />
If you have set up a test database, use this SQL query to determine how many PL/SQL program units you are using:<br />
select count (*) from DBA_PLSQL_OBJECTS;<br />
If the count returned by this query is greater than 15,000, complete the procedure described in the section "Setting Up PL/SQL Native Library Subdirectories".<br />
<br />
Modifying an Existing Database for PL/SQL Native Compilation<br />
To natively compile an existing database, complete the following procedure:<br />
Contact your system administrator to ensure that you have the required C compiler on your operating system, and find the path for its location. Use a text editor such as vi to open the file spnc_commands, and make sure the command templates are correct.<br />
As the oracle user, create the PL/SQL native library directory for each Oracle database.<br />
Note:<br />
You must set up PL/SQL libraries for each Oracle database. Shared libraries (.so and .dll files) are logically connected to the database. They cannot be shared between databases. If you set up PL/SQL libraries to be shared, the databases will be corrupted. <br />
Create a directory in a secure place, in accordance with OFA rules, to prevent .so and .dll files from unauthorized access.<br />
In addition, ensure that the compiler executables used for PL/SQL native compilation are writable only by a properly secured user.<br />
The original copies of the shared libraries are stored inside the database, so they are backed up automatically with the database.<br />
Using SQL, set the initialization parameter PLSQL_NATIVE_LIBRARY_DIR to the full path to the PL/SQL native library.<br />
For example, if the path to the PL/SQL native library directory is /oracle/oradata/mydb/natlib, enter the following:<br />
alter system set plsql_native_library_dir='/oracle/oradata/mydb/natlib'<br />
Determine if you need to set the initialization parameter PLSQL_NATIVE_DIR_SUBDIR_COUNT, and create PL/SQL native library subdirectories if necessary.<br />
By default, PL/SQL program units are kept in one directory. However, if the number of program units exceeds 15000, then the operating system begins to impose performance limits. To work around this problem, Oracle Corporation recommends that you spread the PL/SQL program units in subdirectories.<br />
If you have an existing database that you will migrate to the new installation, or if you have set up a test database, use the following SQL query to determine how many PL/SQL program units you are using:<br />
select count (*) from DBA_PLSQL_OBJECTS;<br />
Create the following stored procedure to confirm that PL/SQL native compilation is enabled:<br />
CREATE OR REPLACE PROCEDURE Hello AS<br />
BEGIN<br />
dbms_output.put_line ( 'This output is from a natively compiled procedure.' );<br />
END Hello;<br />
/<br />
Run the stored procedure:<br />
CALL Hello();<br />
If the program does not return the expected output, contact Oracle Support for assistance. (Remember to SET SERVEROUTPUT ON in SQL*Plus before running the procedure.)<br />
Recompile all the PL/SQL subprograms in the database. The script $ORACLE_HOME/admin/utlirp.sql is typically used here.<br />
<br />
Setting Up PL/SQL Native Library Subdirectories<br />
If you need to set up PL/SQL native library subdirectories, use the following procedure:<br />
Create subdirectories sequentially in the form of d0, d1, d2, d3...dx, where x is the total number of directories. Oracle Corporation recommends that you use a script for this task. For example, you might run a PL/SQL block like the following, save its output to a file, then run that file as a shell script:<br />
BEGIN<br />
FOR j IN 0..999<br />
LOOP<br />
dbms_output.put_line ( 'mkdir d' || TO_CHAR(j) );<br />
END LOOP;<br />
END;<br />
/<br />
Set the initialization parameter PLSQL_NATIVE_DIR_COUNT to the number of subdirectories you have created. For example, if you created 1000 subdirectories, enter the following SQL statement in SQL*Plus:<br />
alter system set plsql_native_library_subdir_count=1000;<br />
<br />
Example 11-8 Compiling a PL/SQL Procedure for Native Execution<br />
alter session set plsql_code_type='NATIVE';<br />
CREATE OR REPLACE PROCEDURE hello_native<br />
AS<br />
BEGIN<br />
dbms_output.put_line('Hello world.');<br />
dbms_output.put_line('Today is ' || TO_CHAR(SYSDATE) || '.');<br />
END;<br />
/<br />
select plsql_code_type from user_plsql_object_settings<br />
where name = 'HELLO_NATIVE';<br />
alter session set plsql_code_type='INTERPRETED';<br />
The procedure is immediately available to call, and runs as a shared library directly within the Oracle process. If any errors occur during compilation, you can see them using the USER_ERRORS view or the SHOW ERRORS command in SQL*Plus.<br />
<br />
Limitations of Native Compilation<br />
" Debugging tools for PL/SQL do not handle procedures compiled for native execution.<br />
" When many procedures and packages (typically, over 15000) are compiled for native execution, the large number of shared objects in a single directory might affect system performance. <br />
<br />
Real Application Clusters and PL/SQL Native Compilation<br />
Because any node might need to compile a PL/SQL subprogram, each node in the cluster needs a C compiler and correct settings and paths in the $ORACLE_HOME/plsql/spnc_commands file.<br />
When you use PLSQL native compilation in a Real Application Clusters (RAC) environment, the original copies of the shared library files are stored in the databases, and these files are automatically propagated to all nodes in the cluster. You do not need to do any copying of libraries for this feature to work.<br />
The reason for using a server parameter file (SPFILE) in the examples in this section, is to make sure that all nodes of a RAC cluster use the same settings for the parameters that control PL/SQL native compilation.<br />
<br />
<br />
Setting Up Transformation Pipelines with Table Functions<br />
This section describes how to chain together special kinds of functions known as table functions. These functions are used in situations such as data warehousing to apply multiple transformations to data.<br />
Overview of Table Functions<br />
Table functions are functions that produce a collection of rows (either a nested table or a varray) that can be queried like a physical database table or assigned to a PL/SQL collection variable. You can use a table function like the name of a database table, in the FROM clause of a query, or like a column name in the SELECT list of a query.<br />
A table function can take a collection of rows as input. An input collection parameter can be either a collection type (such as a VARRAY or a PL/SQL table) or a REF CURSOR.<br />
Execution of a table function can be parallelized, and returned rows can be streamed directly to the next process without intermediate staging. Rows from a collection returned by a table function can also be pipelined-that is, iteratively returned as they are produced instead of in a batch after all processing of the table function's input is completed.<br />
Streaming, pipelining, and parallel execution of table functions can improve performance:<br />
" By enabling multi-threaded, concurrent execution of table functions<br />
" By eliminating intermediate staging between processes<br />
" By improving query response time: With non-pipelined table functions, the entire collection returned by a table function must be constructed and returned to the server before the query can return a single result row. Pipelining enables rows to be returned iteratively, as they are produced. This also reduces the memory that a table function requires, as the object cache does not need to materialize the entire collection.<br />
" By iteratively providing result rows from the collection returned by a table function as the rows are produced instead of waiting until the entire collection is staged in tables or memory and then returning the entire collection.<br />
<br />
Example 11-9 Example: Querying a Table Function<br />
The following example shows a table function GetBooks that takes a CLOB as input and returns an instance of the collection type BookSet_t. The CLOB column stores a catalog listing of books in some format (either proprietary or following a standard such as XML). The table function returns all the catalogs and their corresponding book listings.<br />
The collection type BookSet_t is defined as:<br />
CREATE TYPE Book_t AS OBJECT ( name VARCHAR2(100), author VARCHAR2(30), abstract VARCHAR2(1000));<br />
/<br />
CREATE TYPE BookSet_t AS TABLE OF Book_t;<br />
/<br />
-- The CLOBs are stored in a table Catalogs:<br />
CREATE TABLE Catalogs ( name VARCHAR2(30), cat CLOB );<br />
Function GetBooks is defined as follows:<br />
CREATE FUNCTION GetBooks(a CLOB) RETURN BookSet_t;<br />
/<br />
The query below returns all the catalogs and their corresponding book listings.<br />
SELECT c.name, Book.name, Book.author, Book.abstract<br />
FROM Catalogs c, TABLE(GetBooks(c.cat)) Book;<br />
<br />
Example 11-10 Example: Assigning the Result of a Table Function<br />
The following example shows how you can assign the result of a table function to a PL/SQL collection variable. Because the table function is called from the SELECT list of the query, you do not need the TABLE keyword.<br />
CREATE TYPE numset_t AS TABLE OF NUMBER;<br />
/<br />
<br />
CREATE FUNCTION f1(x number) RETURN numset_t PIPELINED IS<br />
BEGIN<br />
FOR i IN 1..x LOOP<br />
PIPE ROW(i);<br />
END LOOP;<br />
RETURN;<br />
END;<br />
/<br />
<br />
-- pipelined function in from clause<br />
select * from table(f1(3));<br />
<br />
Using Pipelined Table Functions for Transformations<br />
A pipelined table function can accept any argument that regular functions accept. A table function that accepts a REF CURSOR as an argument can serve as a transformation function. That is, it can use the REF CURSOR to fetch the input rows, perform some transformation on them, and then pipeline the results out.<br />
For example, the following code sketches the declarations that define a StockPivot function. This function converts a row of the type (Ticker, OpenPrice, ClosePrice) into two rows of the form (Ticker, PriceType, Price). Calling StockPivot for the row ("ORCL", 41, 42) generates two rows: ("ORCL", "O", 41) and ("ORCL", "C", 42).<br />
Input data for the table function might come from a source such as table StockTable:<br />
CREATE TABLE StockTable (<br />
ticker VARCHAR(4),<br />
open_price NUMBER,<br />
close_price NUMBER<br />
);<br />
<br />
-- Create the types for the table function's output collection<br />
-- and collection elements<br />
CREATE TYPE TickerType AS OBJECT<br />
(<br />
ticker VARCHAR2(4),<br />
PriceType VARCHAR2(1),<br />
price NUMBER<br />
);<br />
/<br />
<br />
CREATE TYPE TickerTypeSet AS TABLE OF TickerType;<br />
/<br />
<br />
-- Define the ref cursor type<br />
<br />
CREATE PACKAGE refcur_pkg IS<br />
TYPE refcur_t IS REF CURSOR RETURN StockTable%ROWTYPE;<br />
END refcur_pkg;<br />
/<br />
<br />
-- Create the table function<br />
<br />
CREATE FUNCTION StockPivot(p refcur_pkg.refcur_t) RETURN TickerTypeSet<br />
PIPELINED ... ;<br />
/<br />
Here is an example of a query that uses the StockPivot table function:<br />
SELECT * FROM TABLE(StockPivot(CURSOR(SELECT * FROM StockTable)));<br />
In the query above, the pipelined table function StockPivot fetches rows from the CURSOR subquery SELECT * FROM StockTable, performs the transformation, and pipelines the results back to the user as a table. The function produces two output rows (collection elements) for each input row.<br />
Note that when a CURSOR subquery is passed from SQL to a REF CURSOR function argument as in the example above, the referenced cursor is already open when the function begins executing.<br />
<br />
<br />
Writing a Pipelined Table Function<br />
You declare a pipelined table function by specifying the PIPELINED keyword. This keyword indicates that the function returns rows iteratively. The return type of the pipelined table function must be a collection type, such as a nested table or a varray. You can declare this collection at the schema level or inside a package. Inside the function, you return individual elements of the collection type.<br />
For example, here are declarations for two pipelined table functions. (The function bodies are shown in later examples.)<br />
CREATE FUNCTION GetBooks(cat CLOB) RETURN BookSet_t<br />
PIPELINED IS ...;<br />
/<br />
<br />
CREATE FUNCTION StockPivot(p refcur_pkg.refcur_t) RETURN TickerTypeSet<br />
PIPELINED IS...;<br />
/<br />
<br />
<br />
Returning Results from Table Functions<br />
In PL/SQL, the PIPE ROW statement causes a table function to pipe a row and continue processing. The statement enables a PL/SQL table function to return rows as soon as they are produced. (For performance, the PL/SQL runtime system provides the rows to the consumer in batches.) For example:<br />
CREATE FUNCTION StockPivot(p refcur_pkg.refcur_t) RETURN TickerTypeSet<br />
PIPELINED IS<br />
out_rec TickerType := TickerType(NULL,NULL,NULL);<br />
in_rec p%ROWTYPE;<br />
BEGIN<br />
LOOP<br />
FETCH p INTO in_rec;<br />
EXIT WHEN p%NOTFOUND;<br />
-- first row<br />
out_rec.ticker := in_rec.Ticker;<br />
out_rec.PriceType := 'O';<br />
out_rec.price := in_rec.OpenPrice;<br />
PIPE ROW(out_rec);<br />
-- second row<br />
out_rec.PriceType := 'C';<br />
out_rec.Price := in_rec.ClosePrice;<br />
PIPE ROW(out_rec);<br />
END LOOP;<br />
CLOSE p;<br />
RETURN;<br />
END;<br />
/<br />
In the example, the PIPE ROW(out_rec) statement pipelines data out of the PL/SQL table function. out_rec is a record, and its type matches the type of an element of the output collection.<br />
The PIPE ROW statement may be used only in the body of pipelined table functions; an error is raised if it is used anywhere else. The PIPE ROW statement can be omitted for a pipelined table function that returns no rows.<br />
A pipelined table function must have a RETURN statement that does not return a value. The RETURN statement transfers the control back to the consumer and ensures that the next fetch gets a NO_DATA_FOUND exception.<br />
Because table functions pass control back and forth to a calling routine as rows areproduced, there is a restriction on combining table functions and PRAGMA AUTONOMOUS_TRANSACTION. If a table function is part of an autonomous transaction, it must COMMIT or ROLLBACK before each PIPE ROW statement, to avoid an error in the calling subprogram.<br />
Oracle has three special SQL datatypes that enable you to dynamically encapsulate and access type descriptions, data instances, and sets of data instances of any other SQL type, including object and collection types. You can also use these three special types to create anonymous (that is, unnamed) types, including anonymous collection types. The types are SYS.ANYTYPE, SYS.ANYDATA, and SYS.ANYDATASET. The SYS.ANYDATA type can be useful in some situations as a return value from table functions.<br />
Pipelining Data Between PL/SQL Table Functions<br />
With serial execution, results are pipelined from one PL/SQL table function to another using an approach similar to co-routine execution. For example, the following statement pipelines results from function g to function f:<br />
SELECT * FROM TABLE(f(CURSOR(SELECT * FROM TABLE(g()))));<br />
Parallel execution works similarly except that each function executes in a different process (or set of processes).<br />
<br />
Querying Table Functions<br />
Pipelined table functions are used in the FROM clause of SELECT statements. The result rows are retrieved by Oracle iteratively from the table function implementation. For example:<br />
SELECT x.Ticker, x.Price<br />
FROM TABLE(StockPivot( CURSOR(SELECT * FROM StockTable))) x<br />
WHERE x.PriceType='C';<br />
Note:<br />
A table function returns a collection. In some cases, such as when the top-level query uses SELECT * and the query refers to a PL/SQL variable or a bind variable, you may need a CAST operator around the table function to specify the exact return type.<br />
<br />
Optimizing Multiple Calls to Table Functions<br />
Multiple invocations of a table function, either within the same query or in separate queries result in multiple executions of the underlying implementation. By default, there is no buffering or reuse of rows.<br />
For example,<br />
SELECT * FROM TABLE(f(...)) t1, TABLE(f(...)) t2<br />
WHERE t1.id = t2.id;<br />
<br />
SELECT * FROM TABLE(f());<br />
SELECT * FROM TABLE(f());<br />
If the function always produces the same result value for each combination of values passed in, you can declare the function DETERMINISTIC, and Oracle automatically buffers rows for it. If the function is not really deterministic, results are unpredictable.<br />
<br />
Fetching from the Results of Table Functions<br />
PL/SQL cursors and ref cursors can be defined for queries over table functions. For example:<br />
OPEN c FOR SELECT * FROM TABLE(f(...));<br />
Cursors over table functions have the same fetch semantics as ordinary cursors. REF CURSOR assignments based on table functions do not have any special semantics.<br />
However, the SQL optimizer will not optimize across PL/SQL statements. For example:<br />
DECLARE<br />
r SYS_REFCURSOR;<br />
BEGIN<br />
OPEN r FOR SELECT * FROM TABLE(f(CURSOR(SELECT * FROM tab)));<br />
SELECT * BULK COLLECT INTO rec_tab FROM TABLE(g(r));<br />
END;<br />
/<br />
does not execute as well as:<br />
SELECT * FROM TABLE(g(CURSOR(SELECT * FROM<br />
TABLE(f(CURSOR(SELECT * FROM tab))))));<br />
This is so even ignoring the overhead associated with executing two SQL statements and assuming that the results can be pipelined between the two statements.<br />
<br />
Passing Data with Cursor Variables<br />
You can pass a set of rows to a PL/SQL function in a REF CURSOR parameter. For example, this function is declared to accept an argument of the predefined weakly typed REF CURSOR type SYS_REFCURSOR:<br />
FUNCTION f(p1 IN SYS_REFCURSOR) RETURN ... ;<br />
Results of a subquery can be passed to a function directly:<br />
SELECT * FROM TABLE(f(CURSOR(SELECT empno FROM tab)));<br />
In the example above, the CURSOR keyword is required to indicate that the results of a subquery should be passed as a REF CURSOR parameter.<br />
A predefined weak REF CURSOR type SYS_REFCURSOR is also supported. With SYS_REFCURSOR, you do not need to first create a REF CURSOR type in a package before you can use it.<br />
To use a strong REF CURSOR type, you still must create a PL/SQL package and declare a strong REF CURSOR type in it. Also, if you are using a strong REF CURSOR type as an argument to a table function, then the actual type of the REF CURSOR argument must match the column type, or an error is generated. Weak REF CURSOR arguments to table functions can only be partitioned using the PARTITION BY ANY clause. You cannot use range or hash partitioning for weak REF CURSOR arguments.<br />
<br />
Example 11-11 Example: Using Multiple REF CURSOR Input Variables<br />
PL/SQL functions can accept multiple REF CURSOR input variables:<br />
CREATE FUNCTION g(p1 pkg.refcur_t1, p2 pkg.refcur_t2) RETURN...<br />
PIPELINED ... ;<br />
/<br />
Function g can be invoked as follows:<br />
SELECT * FROM TABLE(g(CURSOR(SELECT employee_id FROM tab),<br />
CURSOR(SELECT * FROM employees));<br />
You can pass table function return values to other table functions by creating a REF CURSOR that iterates over the returned data:<br />
SELECT * FROM TABLE(f(CURSOR(SELECT * FROM TABLE(g(...)))));<br />
<br />
Example 11-12 Example: Explicitly Opening a REF CURSOR for a Query<br />
You can explicitly open a REF CURSOR for a query and pass it as a parameter to a table function:<br />
DECLARE<br />
r SYS_REFCURSOR;<br />
rec ...;<br />
BEGIN<br />
OPEN r FOR SELECT * FROM TABLE(f(...));<br />
-- Must return a single row result set.<br />
SELECT * INTO rec FROM TABLE(g(r));<br />
END;<br />
/<br />
In this case, the table function closes the cursor when it completes, so your program should not explicitly try to close the cursor.<br />
<br />
Example 11-13 Example: Using a Pipelined Table Function as an Aggregate Function<br />
A table function can compute aggregate results using the input ref cursor. The following example computes a weighted average by iterating over a set of input rows.<br />
DROP TABLE gradereport;<br />
CREATE TABLE gradereport (student VARCHAR2(30), subject VARCHAR2(30), weight NUMBER, grade NUMBER);<br />
<br />
INSERT INTO gradereport VALUES('Mark', 'Physics', 4, 4);<br />
INSERT INTO gradereport VALUES('Mark','Chemistry', 4,3);<br />
INSERT INTO gradereport VALUES('Mark','Maths', 3,3);<br />
INSERT INTO gradereport VALUES('Mark','Economics', 3,4);<br />
<br />
CREATE OR replace TYPE gpa AS TABLE OF NUMBER;<br />
/<br />
<br />
CREATE OR replace FUNCTION weighted_average(input_values<br />
sys_refcursor)<br />
RETURN gpa PIPELINED IS<br />
grade NUMBER;<br />
total NUMBER := 0;<br />
total_weight NUMBER := 0;<br />
weight NUMBER := 0;<br />
BEGIN<br />
-- The function accepts a ref cursor and loops through all the input rows.<br />
LOOP<br />
FETCH input_values INTO weight, grade;<br />
EXIT WHEN input_values%NOTFOUND;<br />
-- Accumulate the weighted average.<br />
total_weight := total_weight + weight;<br />
total := total + grade*weight;<br />
END LOOP;<br />
PIPE ROW (total / total_weight);<br />
-- The function returns a single result.<br />
RETURN;<br />
END;<br />
/<br />
show errors;<br />
<br />
-- The result comes back as a nested table with a single row.<br />
-- COLUMN_VALUE is a keyword that returns the contents of a nested table.<br />
select weighted_result.column_value from<br />
table( weighted_average( cursor(select weight,grade from gradereport) ) ) weighted_result;<br />
<br />
<br />
Performing DML Operations Inside Table Functions<br />
To execute DML statements, declare a table function with the AUTONOMOUS_TRANSACTION pragma, which causes the function to execute in a new transaction not shared by other processes:<br />
CREATE FUNCTION f(p SYS_REFCURSOR) return CollType PIPELINED IS<br />
PRAGMA AUTONOMOUS_TRANSACTION;<br />
BEGIN NULL; END;<br />
/<br />
During parallel execution, each instance of the table function creates an independent transaction.<br />
<br />
<br />
Performing DML Operations on Table Functions<br />
Table functions cannot be the target table in UPDATE, INSERT, or DELETE statements. For example, the following statements will raise an error:<br />
UPDATE F(CURSOR(SELECT * FROM tab)) SET col = value;<br />
INSERT INTO f(...) VALUES ('any', 'thing');<br />
However, you can create a view over a table function and use INSTEAD OF triggers to update it. For example:<br />
CREATE VIEW BookTable AS<br />
SELECT x.Name, x.Author<br />
FROM TABLE(GetBooks('data.txt')) x;<br />
The following INSTEAD OF trigger is fired when the user inserts a row into the BookTable view:<br />
CREATE TRIGGER BookTable_insert<br />
INSTEAD OF INSERT ON BookTable<br />
REFERENCING NEW AS n<br />
FOR EACH ROW<br />
BEGIN<br />
...<br />
END;<br />
/<br />
INSERT INTO BookTable VALUES (...);<br />
INSTEAD OF triggers can be defined for all DML operations on a view built on a table function.<br />
<br />
Handling Exceptions in Table Functions<br />
Exception handling in table functions works just as it does with regular functions.<br />
Some languages, such as C and Java, provide a mechanism for user-supplied exception handling. If an exception raised within a table function is handled, the table function executes the exception handler and continues processing. Exiting the exception handler takes control to the enclosing scope. If the exception is cleared, execution proceeds normally.<br />
An unhandled exception in a table function causes the parent transaction to roll back.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-57064256244817780572009-10-06T09:28:00.001-07:002009-10-06T09:28:43.811-07:00Dbms_Profiler ScriptsSELECT u.unit_name,<br />
d.total_occur,<br />
d.total_time / 1000 microsec,<br />
substr(s.text, 1, 60) plsql_code<br />
FROM plsql_profiler_units u<br />
INNER JOIN plsql_profiler_data d ON (u.runid = d.runid AND<br />
u.unit_number = d.unit_number)<br />
INNER JOIN all_source s ON (s.owner = u.unit_owner AND<br />
s.type = u.unit_type AND s.name = u.unit_name AND<br />
s.line = d.line#)<br />
WHERE u.runid = 53<br />
AND u.unit_name LIKE 'PARSE_CSV%'<br />
ORDER BY u.unit_number, d.line#;<br />
<br />
SELECT runid, run_date, run_comment, run_total_time<br />
FROM plsql_profiler_runs<br />
ORDER BY runid;<br />
<br />
SELECT u.runid,<br />
u.unit_number,<br />
u.unit_type,<br />
u.unit_owner,<br />
u.unit_name,<br />
d.line#,<br />
d.total_occur,<br />
d.total_time,<br />
d.min_time,<br />
d.max_time<br />
FROM plsql_profiler_units u<br />
JOIN plsql_profiler_data d ON u.runid = d.runid<br />
AND u.unit_number = d.unit_number<br />
WHERE u.runid = 1<br />
ORDER BY u.unit_number, d.line#;<br />
<br />
SELECT line || ' : ' || text<br />
FROM all_source<br />
WHERE owner = 'MY_SCHEMA'<br />
AND type = 'PROCEDURE'<br />
AND name = 'DO_SOMETHING';<br />
<br />
SELECT dbms_profiler.internal_version_check FROM dual;<br />
<br />
DECLARE<br />
i PLS_INTEGER;<br />
BEGIN<br />
i := dbms_profiler.flush_data;<br />
i := dbms_profiler.pause_profiler;<br />
i := dbms_profiler.resume_profiler;<br />
END;<br />
<br />
SELECT dbms_profiler.pause_profiler FROM dual;<br />
<br />
SELECT dbms_profiler.resume_profiler FROM dual;<br />
<br />
ROLLUP_RUN<br />
<br />
CREATE OR REPLACE PROCEDURE proc1 IS vd VARCHAR2(5);<br />
BEGIN<br />
FOR i IN 1 .. 100 LOOP<br />
SELECT dummy INTO vd FROM dual;<br />
END LOOP;<br />
END proc1;<br />
/<br />
<br />
DECLARE v_run NUMBER;<br />
BEGIN<br />
dbms_profiler.start_profiler('test', 'test1', v_run);<br />
proc1;<br />
dbms_profiler.stop_profiler;<br />
dbms_profiler.rollup_run(v_run);<br />
END;<br />
/<br />
<br />
ROLLUP_UNIT dbms_profiler.rollup_unit(run_number IN NUMBER, unit IN NUMBER);<br />
<br />
-- executes the following code<br />
UPDATE plsql_profiler_units<br />
SET total_time = (SELECT SUM(total_time)<br />
FROM plsql_profiler_data<br />
WHERE runid = run_number<br />
AND unit_number = unit);<br />
<br />
SELECT * FROM plsql_profiler_units;<br />
<br />
exec dbms_profiler.rollup_unit(8, 3);<br />
<br />
<br />
SELECT u.runid, u.unit_type, u.unit_name, sum(d.total_time) / 1000 microsec<br />
FROM plsql_profiler_units u<br />
INNER JOIN plsql_profiler_data d ON (u.runid = d.runid AND<br />
u.unit_number = d.unit_number)<br />
WHERE u.runid between 53 and 57<br />
AND u.unit_name LIKE 'PARSE_CSV%'<br />
GROUP BY u.runid, u.unit_type, u.unit_name<br />
ORDER BY u.runid, u.unit_name;<br />
<br />
http://radino.eu/tag/dbms_profiler/<br />
<br />
Statement_Tracer_for_Oracle : http://www.aboves.com/downloads/ -- ask IT Team to get Installed. ( free tool )<br />
<br />
http://www.orafaq.com/wiki/Scripts<br />
<br />
Your DBA may have to install the profiler in your database. The procedure for <br />
installing this package is simple:<br />
<br />
ć cd $ORACLE_HOME/rdbms/admin<br />
ć Using SVRMGRL you would connect as SYS or INTERNAL<br />
ć Run profload.sql<br />
<br />
In order to actually use the profiler after that, you will need to have the profiling <br />
tables installed. You can install these once per database but I recommend each developer <br />
would have their own copy. Fortunately, the DBMS_PROFILER package is built with invokers <br />
rights and unqualified tablenames so that we can install the tables in each schema and <br />
the profiler package will use them. The reason you each want your own tables is so that <br />
you only see the results of your profiling runs, not those of your coworkers. In order <br />
to get a copy of the profiling tables in your schema, you would run <br />
$ORACLE_HOME\rdbms\admin\proftab in SQLPlus. After you run proftab.sql, you¡¦ll need to <br />
run profrep.sql as well. This script creates views and packages to operate on the <br />
profiler tables in order to generate reports. This script is found in <br />
$ORACLE_HOME\plsql\demo\profrep.sql. You should run this in your schema as well after <br />
creating the tables.<br />
<br />
I like to keep a small script around to reset these tables (clear them out). After I¡¦ve <br />
done a run or two and have analyzed the results ¡V I run this script to reset the tables. <br />
I have the following in a script I call profreset.sql:<br />
<br />
delete from plsql_profiler_data;<br />
delete from plsql_profiler_units;<br />
delete from plsql_profiler_runs;<br />
<br />
finally, and to answer the question, you will find $ORACLE_HOME/plsql/demo/profsum.sql. <br />
that is what generates the report.<br />
<br />
set echo off<br />
set linesize 5000<br />
set trimspool on<br />
set serveroutput on<br />
set termout off<br />
<br />
column owner format a11<br />
column unit_name format a14<br />
column text format a21 word_wrapped<br />
column runid format 9999<br />
column secs format 999.99<br />
column hsecs format 999.99<br />
column grand_total format 9999.99<br />
column run_comment format a11 word_wrapped<br />
column line# format 99999<br />
column pct format 999.9<br />
column unit_owner format a11<br />
<br />
spool profsum.out<br />
<br />
/* Clean out rollup results, and recreate */<br />
update plsql_profiler_units set total_time = 0;<br />
<br />
execute prof_report_utilities.rollup_all_runs;<br />
<br />
prompt =<br />
prompt =<br />
prompt ====================<br />
prompt Total time<br />
select grand_total/1000000000 as grand_total<br />
from plsql_profiler_grand_total;<br />
<br />
prompt =<br />
prompt =<br />
prompt ====================<br />
prompt Total time spent on each run<br />
select runid,<br />
substr(run_comment,1, 30) as run_comment,<br />
run_total_time/1000000000 as secs<br />
from (select a.runid, sum(a.total_time) run_total_time, b.run_comment<br />
from plsql_profiler_units a, plsql_profiler_runs b<br />
where a.runid = b.runid group by a.runid, b.run_comment )<br />
where run_total_time > 0<br />
order by runid asc;<br />
<br />
<br />
prompt =<br />
prompt =<br />
prompt ====================<br />
prompt Percentage of time in each module, for each run separately<br />
<br />
select p1.runid,<br />
substr(p2.run_comment, 1, 20) as run_comment,<br />
p1.unit_owner,<br />
decode(p1.unit_name, '', '<anonymous>',<br />
substr(p1.unit_name,1, 20)) as unit_name,<br />
p1.total_time/1000000000 as secs,<br />
TO_CHAR(100*p1.total_time/p2.run_total_time, '999.9') as percentage<br />
from plsql_profiler_units p1,<br />
(select a.runid, sum(a.total_time) run_total_time, b.run_comment<br />
from plsql_profiler_units a, plsql_profiler_runs b<br />
where a.runid = b.runid group by a.runid, b.run_comment ) p2<br />
where p1.runid=p2.runid<br />
and p1.total_time > 0<br />
and p2.run_total_time > 0<br />
and (p1.total_time/p2.run_total_time) >= .01<br />
order by p1.runid asc, p1.total_time desc;<br />
<br />
column secs form 9.99<br />
prompt =<br />
prompt =<br />
prompt ====================<br />
prompt Percentage of time in each module, summarized across runs<br />
select p1.unit_owner,<br />
decode(p1.unit_name, '', '<anonymous>', substr(p1.unit_name,1, 25)) as unit_name,<br />
p1.total_time/1000000000 as secs,<br />
TO_CHAR(100*p1.total_time/p2.grand_total, '99999.99') as percentage<br />
from plsql_profiler_units_cross_run p1,<br />
plsql_profiler_grand_total p2<br />
order by p1.total_time DESC;<br />
<br />
<br />
prompt =<br />
prompt =<br />
prompt ====================<br />
prompt Lines taking more than 1% of the total time, each run separate<br />
select p1.runid as runid,<br />
p1.total_time/10000000 as Hsecs,<br />
p1.total_time/p4.grand_total*100 as pct,<br />
substr(p2.unit_owner, 1, 20) as owner,<br />
decode(p2.unit_name, '', '<anonymous>', substr(p2.unit_name,1, 20)) as unit_name,<br />
p1.line#,<br />
( select p3.text<br />
from all_source p3<br />
where p3.owner = p2.unit_owner and<br />
p3.line = p1.line# and<br />
p3.name=p2.unit_name and<br />
p3.type not in ( 'PACKAGE', 'TYPE' )) text<br />
from plsql_profiler_data p1,<br />
plsql_profiler_units p2,<br />
plsql_profiler_grand_total p4<br />
where (p1.total_time >= p4.grand_total/100)<br />
AND p1.runID = p2.runid<br />
and p2.unit_number=p1.unit_number<br />
order by p1.total_time desc;<br />
<br />
prompt =<br />
prompt =<br />
prompt ====================<br />
prompt Most popular lines (more than 1%), summarize across all runs<br />
select p1.total_time/10000000 as hsecs,<br />
p1.total_time/p4.grand_total*100 as pct,<br />
substr(p1.unit_owner, 1, 20) as unit_owner,<br />
decode(p1.unit_name, '', '<anonymous>',<br />
substr(p1.unit_name,1, 20)) as unit_name,<br />
p1.line#,<br />
( select p3.text from all_source p3<br />
where (p3.line = p1.line#) and<br />
(p3.owner = p1.unit_owner) AND<br />
(p3.name = p1.unit_name) and<br />
(p3.type not in ( 'PACKAGE', 'TYPE' ) ) ) text<br />
from plsql_profiler_lines_cross_run p1,<br />
plsql_profiler_grand_total p4<br />
where (p1.total_time >= p4.grand_total/100)<br />
order by p1.total_time desc;<br />
<br />
execute prof_report_utilities.rollup_all_runs;<br />
<br />
prompt =<br />
prompt =<br />
prompt ====================<br />
prompt Number of lines actually executed in different units (by unit_name)<br />
<br />
select p1.unit_owner,<br />
p1.unit_name,<br />
count( decode( p1.total_occur, 0, null, 0)) as lines_executed ,<br />
count(p1.line#) as lines_present,<br />
count( decode( p1.total_occur, 0, null, 0))/count(p1.line#) *100<br />
as pct<br />
from plsql_profiler_lines_cross_run p1<br />
where (p1.unit_type in ( 'PACKAGE BODY', 'TYPE BODY',<br />
'PROCEDURE', 'FUNCTION' ) )<br />
group by p1.unit_owner, p1.unit_name;<br />
<br />
<br />
prompt =<br />
prompt =<br />
prompt ====================<br />
prompt Number of lines actually executed for all units<br />
select count(p1.line#) as lines_executed<br />
from plsql_profiler_lines_cross_run p1<br />
where (p1.unit_type in ( 'PACKAGE BODY', 'TYPE BODY',<br />
'PROCEDURE', 'FUNCTION' ) )<br />
AND p1.total_occur > 0;<br />
<br />
<br />
prompt =<br />
prompt =<br />
prompt ====================<br />
prompt Total number of lines in all units<br />
select count(p1.line#) as lines_present<br />
from plsql_profiler_lines_cross_run p1<br />
where (p1.unit_type in ( 'PACKAGE BODY', 'TYPE BODY',<br />
'PROCEDURE', 'FUNCTION' ) );<br />
<br />
spool off<br />
set termout on<br />
edit profsum.out<br />
set linesize 131 <br />
<br />
SELECT s.line line,<br />
DECODE(p.occured,<br />
0,<br />
'<FONT
color=#FF00Ff>' || substr(s.text, 1, length(s.text) - 1) ||<br />
'</FONT>',<br />
s.text) text<br />
FROM (SELECT d.TOTAL_OCCUR occured, u.unit_type, d.line#<br />
FROM plsql_profiler_units u, plsql_profiler_data d<br />
WHERE d.RUNID = u.runid<br />
AND d.UNIT_NUMBER = u.unit_number<br />
--AND d.TOTAL_OCCUR > 0<br />
AND u.runid = :A3<br />
AND u.unit_name = :A5) p,<br />
user_source s<br />
WHERE p.line#(+) = NVL(s.line, NULL)<br />
AND p.unit_type(+) = NVL(s.TYPE, NULL)<br />
AND s.NAME = :A5<br />
ORDER BY s.TYPE, s.NAME, s.line<br />
<br />
select s.line "Line",<br />
p.total_occur "Occur",<br />
p.total_time "Msec",<br />
s.text "Text"<br />
from all_source s,<br />
(select u.unit_owner,<br />
u.unit_name,<br />
u.unit_type,<br />
d.line#,<br />
d.total_occur,<br />
d.total_time / 1000000 total_time<br />
from plsql_profiler_data d, plsql_profiler_units u<br />
where u.runid = 5<br />
and u.runid = d.runid<br />
and u.unit_number = d.unit_number) p<br />
where s.owner = p.unit_owner(+)<br />
and s.name = p.unit_name(+)<br />
and s.type = p.unit_type(+)<br />
and s.line = p.line#(+)<br />
and s.name = 'DO_SOMETHING'<br />
and s.owner = 'USER'<br />
order by s.line;<br />
<br />
CREATE OR REPLACE TRIGGER bri_prof_memory<br />
BEFORE INSERT<br />
ON plsql_profiler_data<br />
REFERENCING NEW AS NEW OLD AS OLD<br />
FOR EACH ROW<br />
DECLARE<br />
v_used_mem NUMBER;<br />
BEGIN<br />
SELECT p.pga_used_mem<br />
INTO v_used_mem<br />
FROM SYS.v_$process p, SYS.v_$session s<br />
WHERE p.addr = s.paddr AND s.audsid = USERENV ('sessionid');<br />
<br />
:new.spare1:=v_used_mem;<br />
<br />
END bri_prof_memory;KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com1tag:blogger.com,1999:blog-8692881074449232849.post-80754070326094694092009-10-06T09:23:00.001-07:002009-10-06T09:27:10.473-07:00DBMS_ADVISOR Process1. Gather a number of SQL statements that will form the tuning workload.<br />
<br />
2. Check that the user running DBMS_ADVISOR has the ADVISOR privilege, and has SELECT access to the tables and views referenced by the SQL statements.<br />
<br />
3. Define a task using DBMS_ADVISOR.CREATE_TASK, to create a container for your tuning exercise:<br />
<br />
VARIABLE task_id NUMBER;<br />
VARIABLE my_first_task varhcar2(300);<br />
EXECUTE DBMS_ADVISOR.CREATE_TASK ('SQL Access Advisor', :task_id, :my_first_task);<br />
PRINT my_first_task <br />
<br />
Then define a workload object, into which we will later load individual SQL statements:<br />
<br />
VARIABLE my_first_workload varhcar2(300);<br />
EXECUTE DBMS_ADVISOR.CREATE_SQLWKLD(:my_first_workload,'This is my first workload');<br />
PRINT my_first_workload<br />
<br />
Then, link your task and workload objects:<br />
<br />
EXECUTE DBMS_ADVISOR.ADD_SQLWKLD_REF('my_first_task','my_first_workload');<br />
<br />
4. Statements can then be manually added into the workload object:<br />
<br />
EXECUTE DBMS_ADVISOR.ADD_SQLWKLD_STATEMENT ('my_first_workload', 'MONTHLY', 'ROLLUP', priority=>1, executions=>20, username => 'DEMO', sql_text => ‘SELECT SUM(sales) FROM sales);<br />
<br />
Alternatively, they can be loaded in from a table of SQL statements you create, an SQL Tuning Set, an SQL Cache workload, an Oracle9i Summary Advisor workload; otherwise, a hypothetical workload can be generated from a set of table and view definitions in a schema.<br />
<br />
5. Generate recommendations for this task’s workload:<br />
<br />
EXECUTE DBMS_ADVISOR.EXECUTE_TASK('my_first_task');<br />
<br />
Each recommendation generated by the SQL Access Advisor can be viewed using catalog views such as USER_ADVISOR_RECOMMENDATIONS. <br />
In addition, the procedure GET_TASK_SCRIPT generates an executable SQL script that contains the CREATE, DROP, and ALTER statements to implement the advisor recommendations:<br />
<br />
EXECUTE DBMS_ADVISOR.CREATE_FILE(DBMS_ADVISOR.GET_TASK_SCRIPT('my_first_task'),'ADVISOR_RESULTS', 'script.sql'); <br />
<br />
As an alternative to generating recommendations using the DBMS_ADVISOR package, the SQL Access Advisor functionality is also available as a wizard within the Enterprise Manager 10g Web site. <br />
<br />
If you just want to tune an individual statement you can use the QUICK_TUNE procedure as follows:<br />
BEGIN<br />
DBMS_ADVISOR.quick_tune(<br />
advisor_name => DBMS_ADVISOR.SQLACCESS_ADVISOR, <br />
task_name => 'emp_quick_tune',<br />
attr1 => 'SELECT e.* FROM emp e WHERE UPPER(e.ename) = ''SMITH''');<br />
END;<br />
/<br />
Any recommendations can then be displayed using the previous query with the correct task name specified.<br />
<br />
<br />
· DBA_ADVISOR_TASKS - Basic information about existing tasks. <br />
· DBA_ADVISOR_LOG - Status information about existing tasks. <br />
· DBA_ADVISOR_FINDINGS - Findings identified for an existing task. <br />
· DBA_ADVISOR_RECOMMENDATIONS - Recommendations for the problems identified by an existing task<br />
<br />
<br />
DBMS_ADVISOR.TUNE_MVIEW <br />
As well as containing functionality that recommends suitable materialized views and indexes, DBMS_ADVISOR contains a procedure called TUNE_MVIEW that is used when working with the query rewrite mechanism.<br />
Oracle9i came with two packages, DBMS_MVIEW.EXPLAIN_MVIEW and DBMS_MVIEW.EXPLAIN_REWRITE that could be used to diagnose why a materialized view wasn't being used for query rewrite. However, although these packages told you why rewrite hadn't happened, they left it down to you to work out how to alter your CREATE MATERIALIZED VIEW statement to ensure that rewrite happened correctly. Oracle 10g’s DBMS_ADVISOR.TUNE_MVIEW builds on this functionality and tells the DBA what changes to make to a materialized view to make it suitable for query rewrite, taking as its input a CREATE MATERIALIZED VIEW statement and outputting a corrected version that supports query rewrite and features such as fast refresh.<br />
Tuning a materialized view follows a similar process to that used with the SQL Access Advisor. <br />
1. First, create a task and supply a CREATE MATERIALIZED VIEW statement to DBMS_ADVISOR.TUNE_MVIEW<br />
EXECUTE DBMS_ADVISOR.TUNE_MVIEW(‘my_first_mv’, ' -<br />
CREATE MATERIALIZED VIEW sales_mv -<br />
REFRESH FAST -<br />
disable QUERY REWRITE AS - <br />
select category, country, sum(sales) -<br />
from product p, geography g, sales s -<br />
where s.product_id = p.product_id -<br />
and s.geography_id = p.geography_id -<br />
group by p.category, g.country<br />
');<br />
2. You can then directly query the resulting tuned CREATE MATERIALIZED VIEW statement from the USER_TUNE_MVIEW view:<br />
SELECT * FROM USER_TUNE_MVIEW <br />
WHERE TASK_NAME= ‘my_first_mv’ AND <br />
SCRIPT_TYPE='IMPLEMENTATION';<br />
3. Or you can output the tuned statement to the filesystem using the DBMS_ADVISOR.GET_TASK_SCRIPT procedure.<br />
SELECT * FROM USER_TUNE_MVIEW <br />
WHERE TASK_NAME= ‘my_first_mv’ AND <br />
SCRIPT_TYPE='IMPLEMENTATION';<br />
The DBMS_ADVISOR.TUNE_MVIEW procedure has the capability to add additional aggregate columns and materialized view logs to the view definition so that it becomes fast refreshable, and it can restate the view definition (sometimes breaking it into separate, simpler definitions) so that it is more likely to satisfy query rewrite restrictions.<br />
<br />
DBA Advisor Views:<br />
dba_advisor_journal<br />
dba_advisor_objects<br />
dba_advisor_parameters<br />
dba_advisor_rationale<br />
dba_advisor_recommendations<br />
dba_advisor_sqlw_journal<br />
dba_advisor_tasks<br />
dba_advisor_usage<br />
<br />
DBMS_ADVISOR, <br />
<br />
SQLAcess Advisor and SQLWorkload Advisor.<br />
<br />
<br />
DECLARE<br />
taskname varchar2(30) := 'SQLACCESS3638195';<br />
task_desc varchar2(256) := 'SQL Access Advisor';<br />
task_or_template varchar2(30) := 'SQLACCESS_EMTASK';<br />
task_id number := 0;<br />
num_found number;<br />
sts_name varchar2(256) := 'SQLACCESS3638195_sts';<br />
sts_cursor dbms_sqltune.sqlset_cursor;<br />
BEGIN<br />
/* Create Task */<br />
dbms_advisor.create_task(DBMS_ADVISOR.SQLACCESS_ADVISOR,<br />
task_id,<br />
taskname,<br />
task_desc,<br />
task_or_template);<br />
<br />
/* Reset Task */<br />
dbms_advisor.reset_task(taskname);<br />
<br />
/* Delete Previous STS Workload Task Link */<br />
select count(*)<br />
into num_found<br />
from user_advisor_sqla_wk_map<br />
where task_name = taskname<br />
and workload_name = sts_name;<br />
IF num_found > 0 THEN<br />
dbms_advisor.delete_sqlwkld_ref(taskname,sts_name);<br />
END IF;<br />
<br />
/* Delete Previous STS */<br />
select count(*)<br />
into num_found<br />
from user_advisor_sqlw_sum<br />
where workload_name = sts_name;<br />
IF num_found > 0 THEN<br />
dbms_sqltune.delete_sqlset(sts_name);<br />
END IF;<br />
<br />
/* Create STS */<br />
dbms_sqltune.create_sqlset(sts_name, 'Obtain workload from cursor cache');<br />
<br />
/* Select all statements in the cursor cache. */<br />
OPEN sts_cursor FOR<br />
SELECT VALUE(P)<br />
FROM TABLE(dbms_sqltune.select_cursor_cache) P;<br />
<br />
/* Load the statements into STS. */<br />
dbms_sqltune.load_sqlset(sts_name, sts_cursor);<br />
CLOSE sts_cursor;<br />
<br />
/* Link STS Workload to Task */<br />
dbms_advisor.add_sqlwkld_ref(taskname,sts_name);<br />
<br />
/* Set STS Workload Parameters */<br />
dbms_advisor.set_task_parameter(taskname,'VALID_ACTION_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'VALID_MODULE_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'SQL_LIMIT','25');<br />
dbms_advisor.set_task_parameter(taskname,'VALID_USERNAME_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'VALID_TABLE_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'INVALID_TABLE_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'INVALID_ACTION_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'INVALID_USERNAME_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'INVALID_MODULE_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'VALID_SQLSTRING_LIST',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'INVALID_SQLSTRING_LIST','"@!"');<br />
<br />
/* Set Task Parameters */<br />
dbms_advisor.set_task_parameter(taskname,'ANALYSIS_SCOPE','ALL');<br />
dbms_advisor.set_task_parameter(taskname,'RANKING_MEASURE','PRIORITY,OPTIMIZER_COST');<br />
dbms_advisor.set_task_parameter(taskname,'DEF_PARTITION_TABLESPACE',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'TIME_LIMIT',10000);<br />
dbms_advisor.set_task_parameter(taskname,'MODE','LIMITED');<br />
dbms_advisor.set_task_parameter(taskname,'STORAGE_CHANGE',DBMS_ADVISOR.ADVISOR_UNLIMITED);<br />
dbms_advisor.set_task_parameter(taskname,'DML_VOLATILITY','TRUE');<br />
dbms_advisor.set_task_parameter(taskname,'WORKLOAD_SCOPE','PARTIAL');<br />
dbms_advisor.set_task_parameter(taskname,'DEF_INDEX_TABLESPACE',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'DEF_INDEX_OWNER',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'DEF_MVIEW_TABLESPACE',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'DEF_MVIEW_OWNER',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'DEF_MVLOG_TABLESPACE',DBMS_ADVISOR.ADVISOR_UNUSED);<br />
dbms_advisor.set_task_parameter(taskname,'CREATION_COST','TRUE');<br />
dbms_advisor.set_task_parameter(taskname,'JOURNALING','4');<br />
dbms_advisor.set_task_parameter(taskname,'DAYS_TO_EXPIRE','30');<br />
<br />
/* Execute Task */<br />
dbms_advisor.execute_task(taskname);<br />
END;<br />
<br />
<br />
Other Way Go : <br />
Recommendations From a User-Defined Workload<br />
<br />
The following example imports workload from a user-defined table, SH.USER_WORKLOAD. It then creates a task called MYTASK, sets the storage budget to 100 MB and runs the task. The recommendations are printed out using a PL/SQL procedure. Finally, it generates a script, which can be used to implement the recommendations.<br />
<br />
Step 1 Prepare the USER_WORKLOAD table<br />
<br />
The USER_WORKLOAD table is loaded with SQL statements as follows:<br />
<br />
CONNECT SH/SH;<br />
-- aggregation with selection<br />
INSERT INTO user_workload (username, module, action, priority, sql_text)<br />
VALUES ('SH', 'Example1', 'Action', 2, <br />
'SELECT t.week_ending_day, p.prod_subcategory, <br />
SUM(s.amount_sold) AS dollars, s.channel_id, s.promo_id<br />
FROM sales s, times t, products p WHERE s.time_id = t.time_id<br />
AND s.prod_id = p.prod_id AND s.prod_id > 10 AND s.prod_id < 50
GROUP BY t.week_ending_day, p.prod_subcategory,
s.channel_id, s.promo_id')
/
-- aggregation with selection
INSERT INTO user_workload (username, module, action, priority, sql_text)
VALUES ('SH', 'Example1', 'Action', 2,
'SELECT t.calendar_month_desc, SUM(s.amount_sold) AS dollars
FROM sales s , times t
WHERE s.time_id = t.time_id
AND s.time_id between TO_DATE(''01-JAN-2000'', ''DD-MON-YYYY'')
AND TO_DATE(''01-JUL-2000'', ''DD-MON-YYYY'')
GROUP BY t.calendar_month_desc')
/
--Load all SQL queries.
INSERT INTO user_workload (username, module, action, priority, sql_text)
VALUES ('SH', 'Example1', 'Action', 2,
'SELECT ch.channel_class, c.cust_city, t.calendar_quarter_desc,
SUM(s.amount_sold) sales_amount
FROM sales s, times t, customers c, channels ch
WHERE s.time_id = t.time_id AND s.cust_id = c.cust_id
AND s.channel_id = ch.channel_id AND c.cust_state_province = ''CA''
AND ch.channel_desc IN (''Internet'',''Catalog'')
AND t.calendar_quarter_desc IN (''1999-Q1'',''1999-Q2'')
GROUP BY ch.channel_class, c.cust_city, t.calendar_quarter_desc')
/
-- order by
INSERT INTO user_workload (username, module, action, priority, sql_text)
VALUES ('SH', 'Example1', 'Action', 2,
'SELECT c.country_id, c.cust_city, c.cust_last_name
FROM customers c WHERE c.country_id IN (52790, 52789)
ORDER BY c.country_id, c.cust_city, c.cust_last_name')
/
COMMIT;
CONNECT SH/SH;
set serveroutput on;
VARIABLE task_id NUMBER;
VARIABLE task_name VARCHAR2(255);
VARIABLE workload_name VARCHAR2(255);
Step 2 Create a SQL Tuning Set named MYWORKLOAD
EXECUTE :workload_name := 'MYWORKLOAD';
EXECUTE DBMS_SQLTUNE.CREATE_SQLSET(:workload_name, 'test purposeV);
Step 3 Load the SQL Tuning Set from the user-defined table SH.USER_WORKLOAD
DECLARE
sqlset_cur DBMS_SQLTUNE.SQLSET_CURSOR; /*a sqlset cursor variable*/
BEGIN
OPEN sqlset_cur FOR
SELECT
SQLSET_ROW(null, sql_text, null, null, username, null,
null, 0,0,0,0,0,0,0,0,0,null, 0,0,0,0)
AS ROW
FROM USER_WORKLOAD;
DBMS_SQLTUNE.LOAD_SQLSET(:workload_name, sqlset_cur);
END;
Step 4 Create a task named MYTASK
EXECUTE :task_name := 'MYTASK';
EXECUTE DBMS_ADVISOR.CREATE_TASK('SQL Access Advisor', :task_id, :task_name);
Step 5 Set task parameters
EXECUTE DBMS_ADVISOR.SET_TASK_PARAMETER(:task_name, 'STORAGE_CHANGE', 100);
EXECUTE DBMS_ADVISOR.SET_TASK_PARAMETER(:task_name, 'ANALYSIS_SCOPE, INDEX');
Step 6 Create a link between the SQL Tuning Set and the task
EXECUTE DBMS_ADVISOR.ADD_STS_REF(:task_name, :workload_name);
Step 7 Execute the task
EXECUTE DBMS_ADVISOR.EXECUTE_TASK(:task_name);
Step 8 View the recommendations
-- See the number of recommendations and the status of the task.
SELECT rec_id, rank, benefit
FROM user_advisor_recommendations WHERE task_name = :task_name;
See "Viewing Recommendations" or "Generating SQL Scripts" for further details.
-- See recommendation for each query.
SELECT sql_id, rec_id, precost, postcost,
(precost-postcost)*100/precost AS percent_benefit
FROM user_advisor_sqla_wk_stmts
WHERE task_name = :task_name AND workload_name = :workload_name;
-- See the actions for each recommendations.
SELECT rec_id, action_id, SUBSTR(command,1,30) AS command
FROM user_advisor_actions
WHERE task_name = :task_name
ORDER BY rec_id, action_id;
-- See what the actions are using sample procedure.
SET SERVEROUTPUT ON SIZE 99999
EXECUTE show_recm(:task_name);
Generate Recommendations Using a Task Template
The following example creates a template and then uses it to create a task. It then uses this task to generate recommendations from a user-defined table, similar to "Recommendations From a User-Defined Workload".
CONNECT SH/SH;
VARIABLE template_id NUMBER;
VARIABLE template_name VARCHAR2(255);
Step 1 Create a template called MY_TEMPLATE
EXECUTE :template_name := 'MY_TEMPLATE';
EXECUTE DBMS_ADVISOR.CREATE_TASK ( -
'SQL Access Advisor',:template_id, :template_name, is_template=>'TRUE');<br />
<br />
Step 2 Set template parameters<br />
<br />
Set naming conventions for recommended indexes and materialized views.<br />
<br />
EXECUTE DBMS_ADVISOR.SET_TASK_PARAMETER ( -<br />
:template_name, 'INDEX_NAME_TEMPLATE', 'SH_IDX$$_<SEQ>');<br />
EXECUTE DBMS_ADVISOR.SET_TASK_PARAMETER ( -<br />
:template_name, 'MVIEW_NAME_TEMPLATE', 'SH_MV$$_<SEQ>');<br />
<br />
--Set default owners for recommended indexes/materialized views.<br />
EXECUTE DBMS_ADVISOR.SET_TASK_PARAMETER ( -<br />
:template_name, 'DEF_INDEX_OWNER', 'SH');<br />
EXECUTE DBMS_ADVISOR.SET_TASK_PARAMETER ( -<br />
:template_name, 'DEF_MVIEW_OWNER', 'SH');<br />
<br />
--Set default tablespace for recommended indexes/materialized views.<br />
EXECUTE DBMS_ADVISOR.SET_TASK_PARAMETER ( -<br />
:template_name, 'DEF_INDEX_TABLESPACE', 'SH_INDEXES');<br />
EXECUTE DBMS_ADVISOR.SET_TASK_PARAMETER ( -<br />
:template_name, 'DEF_MVIEW_TABLESPACE', 'SH_MVIEWS');<br />
<br />
Step 3 Create a task using the template<br />
<br />
VARIABLE task_id NUMBER;<br />
VARIABLE task_name VARCHAR2(255);<br />
EXECUTE :task_name := 'MYTASK';<br />
EXECUTE DBMS_ADVISOR.CREATE_TASK ( -<br />
'SQL Access Advisor', :task_id, :task_name, template => 'MY_TEMPLATE');<br />
<br />
--See the parameter settings for task<br />
SELECT parameter_name, parameter_value<br />
FROM user_advisor_parameters<br />
WHERE task_name = :task_name AND (parameter_name LIKE '%MVIEW%' <br />
OR parameter_name LIKE '%INDEX%');<br />
<br />
Step 4 Create a SQL Tuning Set named MYWORKLOAD<br />
<br />
EXECUTE :workload_name := 'MYWORKLOAD';<br />
EXECUTE DBMS_SQLTUNE.CREATE_SQLSET(:workload_name, 'test_purpose');<br />
<br />
Step 5 Load the SQL Tuning Set from the user-defined table SH.USER_WORKLOAD<br />
<br />
DECLARE<br />
sqlset_cur DBMS_SQLTUNE.SQLSET_CURSOR; /*a sqlset cursor variable*/<br />
BEGIN<br />
OPEN sqlset_cur FOR<br />
SELECT<br />
SQLSET_ROW(null,sql_text,null,null,username, null, null, 0,0,0,0,0,0,0,0,0,<br />
null,0,0,00) AS row<br />
FROM user_workload;<br />
DBMS_SQLTUNE.LOAD_SQLSET(:workload_name, sqlsetcur);<br />
END;<br />
<br />
Step 6 Create a link between the workload and the task<br />
<br />
EXECUTE DBMS_ADVISOR.ADD_STS_REF(:task_name, :workload_name);<br />
<br />
Step 7 Execute the task<br />
<br />
EXECUTE DBMS_ADVISOR.EXECUTE_TASK(:task_name);<br />
<br />
Step 8 Generate a script<br />
<br />
EXECUTE DBMS_ADVISOR.CREATE_FILE(DBMS_ADVISOR.GET_TASK_SCRIPT(:task_name),-<br />
'ADVISOR_RESULTS', 'Example2_script.sql');<br />
<br />
Evaluate Current Usage of Indexes and Materialized Views<br />
<br />
This example illustrates how the SQL Access Advisor can be used to evaluate the utilization of existing indexes and materialized views. We assume the workload is loaded into USER_WORKLOAD table as in "Recommendations From a User-Defined Workload". The indexes and materialized views that are being currently used (by the given workload) will appear as RETAIN actions in the SQL Access Advisor recommendations.<br />
<br />
VARIABLE task_id NUMBER;<br />
VARIABLE task_name VARCHAR2(255);<br />
VARIABLE workload_name VARCHAR2(255);<br />
<br />
Step 1 Create a SQL Tuning Set named WORKLOAD<br />
<br />
EXECUTE :workload_name := 'MYWORKLOAD';<br />
EXECUTE DBMS_SQLTUNE.CREATE_SQLSET(:workload_name, 'test_purpose');<br />
<br />
Step 2 Load the SQL Tuning Set from the user-defined table SH.USER_WORKLOAD<br />
<br />
DECLARE<br />
sqlset_cur DBMS_SQLTUNE.SQLSET_CURSOR; /*a sqlset cursor variable*/<br />
BEGIN<br />
OPEN sqlset_cur FOR<br />
SELECT<br />
SQLSET_ROW(null,sql_text,null,null,username, null, null, 0,0,0,0,0,0,0,0,0,<br />
null, 0,0,0,0)<br />
AS ROW<br />
FROM user_workload;<br />
DBMS_SQLTUNE.LOAD_SQLSET(:workload_name, :sqlsetcur);<br />
END;<br />
<br />
Step 3 Create a task named MY_EVAL_TASK<br />
<br />
EXECUTE :task_name := 'MY_EVAL_TASK';<br />
EXECUTE DBMS_ADVISOR.CREATE_TASK ('SQL Access Advisor', :task_id, :task_name);<br />
<br />
Step 4 Create a link between workload and task<br />
<br />
EXECUTE DBMS_ADVISOR.ADD_STS_REF(:task_name, :workload_name);<br />
<br />
Step 5 Set task parameters to indicate EVALUATION ONLY task<br />
<br />
EXECUTE DBMS_ADVISOR.SET_TASK_PARAMETER (:task_name, 'EVALUATION_ONLY', 'TRUE');<br />
<br />
Step 6 Execute the task<br />
<br />
EXECUTE DBMS_ADVISOR.EXECUTE_TASK(:task_name);<br />
<br />
Step 7 View evaluation results<br />
<br />
--See the number of recommendations and the status of the task.<br />
SELECT rec_id, rank, benefit<br />
FROM user_advisor_recommendations WHERE task_name = :task_name;<br />
<br />
--See the actions for each recommendation.<br />
SELECT rec_id, action_id, SUBSTR(command,1,30) AS command, attr1 AS name<br />
FROM user_advisor_actions WHERE task_name = :task_name<br />
ORDER BY rec_id, action_id;<br />
<br />
Tuning Materialized Views for Fast Refresh and Query Rewrite<br />
The TUNE_MVIEW procedure takes two input parameters: task_name and mv_create_stmt. task_name is a user-provided task identifier used to access the output results. mv_create_stmt is a complete CREATE MATERIALIZED VIEW statement that is to be tuned. If the input CREATE MATERIALIZED VIEW statement does not have the clauses of REFRESH FAST or ENABLE QUERY REWRITE, or both, TUNE_MVIEW will use the default clauses REFRESH FORCE and DISABLE QUERY REWRITE to tune the statement to be fast refreshable if possible or only complete refreshable otherwise.<br />
The TUNE_MVIEW procedure handles a broad range of CREATE MATERIALIZED VIEW statements that can have arbitrary defining queries in them. The defining query could be a simple SELECT statement or a complex query with set operators or inline views. When the defining query of the materialized view contains the clause REFRESH FAST, TUNE_MVIEW analyzes the query and checks to see if it is fast refreshable. If it is already fast refreshable, the procedure will return a message saying "the materialized view is already optimal and cannot be further tuned". Otherwise, the TUNE_MVIEW procedure will start the tuning work on the given statement.<br />
The TUNE_MVIEW procedure can generate the output statements that correct the defining query by adding extra columns such as required aggregate columns or fix the materialized view logs so that FAST REFRESH is possible. In the case of a complex defining query, the TUNE_MVIEW procedure may decompose the query and generates two or more fast refreshable materialized views or will restate the materialized view in a way to fulfill fast refresh requirements as much as possible. The TUNE_MVIEW procedure supports defining queries with the following complex query constructs:<br />
Set operators (UNION, UNION ALL, MINUS, and INTERSECT)<br />
COUNT DISTINCT<br />
SELECT DISTINCT<br />
Inline views<br />
When the ENABLE QUERY REWRITE clause is specified, TUNE_MVIEW will also fix the statement using a process similar to REFRESH FAST, that will redefine the materialized view so that as many of the advanced forms of query rewrite are possible.<br />
The TUNE_MVIEW procedure generates two sets of output results as executable statements. One set of the output (IMPLEMENTATION) is for implementing materialized views and required components such as materialized view logs or rewrite equivalences to achieve fast refreshability and query rewritablity as much as possible. The other set of the output (UNDO) is for dropping the materialized views and the rewrite equivalences in case you decide they are not required.<br />
The output statements for the IMPLEMENTATION process include:<br />
CREATE MATERIALIZED VIEW LOG statements: creates any missing materialized view logs required for fast refresh.<br />
ALTER MATERIALIZED VIEW LOG FORCE statements: fixes any materialized view log related requirements such as missing filter columns, sequence, and so on, required for fast refresh.<br />
One or more CREATE MATERIALIZED VIEW statements: In the case of one output statement, the original defining query is directly restated and transformed. Simple query transformation could be just adding required columns. For example, add rowid column for materialized join view and add aggregate column for materialized aggregate view. In the case of decomposition, multiple CREATE MATERIALIZED VIEW statements are generated and form a nested materialized view hierarchy in which one or more submaterialized views are referenced by a new top-level materialized view modified from the original statement. This is to achieve fast refresh and query rewrite as much as possible. Submaterialized views are often fast refreshable.<br />
BUILD_SAFE_REWRITE_EQUIVALENCE statement: enables the rewrite of top-level materialized views using submaterialized views. It is required to enable query rewrite when a composition occurs.<br />
Note that the decomposition result implies no sharing of submaterialized views. That is, in the case of decomposition, the TUNE_MVIEW output will always contain new submaterialized view and it will not reference existing materialized views.<br />
The output statements for the UNDO process include:<br />
DROP MATERIALIZED VIEW statements to reverse the materialized view creations (including submaterialized views) in the IMPLEMENTATION process.<br />
DROP_REWRITE_EQUIVALENCE statement to remove the rewrite equivalence relationship built in the IMPLEMENTATION process if needed.<br />
Note that the UNDO process does not include statement to drop materialized view logs. This is because materialized view logs can be shared by many different materialized views, some of which may reside on remote Oracle instances.<br />
Accessing TUNE_MVIEW Output Results<br />
There are two ways to access TUNE_MVIEW output results:<br />
Script generation using DBMS_ADVISOR.GET_TASK_SCRIPT function and DBMS_ADVISOR.CREATE_FILE procedure.<br />
Use USER_TUNE_MVIEW or DBA_TUNE_MVIEW views.<br />
Script Generation DBMS_ADVISOR Function and Procedure<br />
The most straightforward method for generating the execution scripts for a recommendation is to use the procedure DBMS_ADVISOR.GET_TASK_SCRIPT. The following is a simple example. First, a directory must be defined which is where the results will be stored:<br />
CREATE DIRECTORY TUNE_RESULTS AS '/tmp/script_dir';<br />
GRANT READ, WRITE ON DIRECTORY TUNE_RESULTS TO PUBLIC;<br />
Now generate both the implementation and undo scripts and place them in /tmp/script_dir/mv_create.sql and /tmp/script_dir/mv_undo.sql, respectively.<br />
EXECUTE DBMS_ADVISOR.CREATE_FILE(DBMS_ADVISOR.GET_TASK_SCRIPT(:task_name),- <br />
'TUNE_RESULTS', 'mv_create.sql'); <br />
EXECUTE DBMS_ADVISOR.CREATE_FILE(DBMS_ADVISOR.GET_TASK_SCRIPT(:task_name, - <br />
'UNDO'), 'TUNE_RESULTS', 'mv_undo.sql');<br />
Optimizing the Defining Query for Fast Refresh<br />
This example shows how TUNE_MVIEW changes the defining query to be fast refreshable. A CREATE MATERIALIZED VIEW statement is defined in variable create_mv_ddl, which includes a FAST REFRESH clause. Its defining query contains a single query block in which an aggregate column, SUM(s.amount_sold), does not have the required aggregate columns to support fast refresh. If you execute the TUNE_MVIEW statement with this MATERIALIZED VIEW CREATE statement, the resulting materialized view recommendation will be fast refreshable:<br />
VARIABLE task_cust_mv VARCHAR2(30);<br />
VARIABLE create_mv_ddl VARCHAR2(4000);<br />
EXECUTE :task_cust_mv := 'cust_mv';<br />
<br />
EXECUTE :create_mv_ddl := '<br />
CREATE MATERIALIZED VIEW cust_mv<br />
REFRESH FAST<br />
DISABLE QUERY REWRITE AS<br />
SELECT s.prod_id, s.cust_id, SUM(s.amount_sold) sum_amount<br />
FROM sales s, customers cs<br />
WHERE s.cust_id = cs.cust_id<br />
GROUP BY s.prod_id, s.cust_id';<br />
<br />
EXECUTE DBMS_ADVISOR.TUNE_MVIEW(:task_cust_mv, :create_mv_ddl);<br />
<br />
Access IMPLEMENTATION Output Through USER_TUNE_MVIEW View<br />
SELECT STATEMENT FROM USER_TUNE_MVIEW<br />
WHERE TASK_NAME= :task_cust_mv AND SCRIPT_TYPE='IMPLEMENTATION';<br />
<br />
Save IMPLEMENTATION Output in a Script File<br />
CREATE DIRECTORY TUNE_RESULTS AS '/myscript'<br />
GRANT READ, WRITE ON DIRECTORY TUNE_RESULTS TO PUBLIC;<br />
<br />
EXECUTE DBMS_ADVISOR.CREATE_FILE(DBMS_ADVISOR.GET_TASK_SCRIPT(:task_cust_mv), -<br />
'TUNE_RESULTS', 'mv_create.sql');<br />
<br />
Access IMPLEMENTATION Output Through USER_TUNE_MVIEW View<br />
SELECT * FROM USER_TUNE_MVIEW<br />
WHERE TASK_NAME='cust_mv2'<br />
AND SCRIPT_TYPE='IMPLEMENTATION';<br />
<br />
Save IMPLEMENTATION Output in a Script File<br />
The following statements save the IMPLEMENTATION output in a script file located at /myscript/mv_create2.sql:<br />
CREATE DIRECTORY TUNE_RESULTS AS '/myscript'<br />
GRANT READ, WRITE ON DIRECTRY TUNE_RESULTS TO PUBLIC;<br />
EXECUTE DBMS_ADVISOR.CREATE_FILE(DBMS_ADVISOR.GET_TASK_SCRIPT('cust_mv2'),<br />
'TUNE_RESULTS', 'mv_create2.sql');<br />
<br />
Fast Refreshable with Optimized Sub-Materialized View<br />
The example illustrates how TUNE_MVIEW can optimize the materialized view so that fast refresh is possible. In the example, the materialized view's defining query with set operators is transformed into one sub-materialized view and one top-level materialized view. The subselect queries in the original defining query are of similar shape and their predicate expressions are combined.<br />
The materialized view defining query contains a UNION set-operator so that the materialized view itself is not fast-refreshable. However, two subselect queries in the materialized view defining query can be combined as one single query.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-2304886555371098262009-10-06T09:20:00.001-07:002009-10-06T09:20:29.396-07:00What to do when your query is too slow?First of all, you have to know why it is slow. What is the real cause of your problem. If the reason why is not known, suggesting to rewrite the query, or hinting the query, suggesting parallellization et cetera is not very productive. Once in a while you may get lucky. But even then you have to realize that if your problem seems "solved", but you don't know why, nothing guarantees you that the problem won't come back tomorrow. So the first step should always be to investigate the root cause.<br />
<br />
The tools at your disposal are, among more:<br />
- dbms_profiler<br />
- explain plan<br />
- SQL*Trace / tkprof<br />
- statspack<br />
<br />
Use dbms_profiler if you want to know where time is being spent in PL/SQL code. Statspack is a must if you are a dba and want to know what is going on in your entire database. For a single query or a small process, explain plan and SQL*Trace and tkprof are your tools.<br />
<br />
explain plan<br />
<br />
in SQL*Plus you have to type:<br />
explain plan for <your query>;<br />
select * from table(dbms_xplan.display);<br />
<br />
<br />
When you get error messages or a message complaining about an old version of plan_table, make sure you run the script utlxplan.sql.<br />
<br />
The output you get here basically shows you what the cost based optimizer expects. It gives you an idea on why the cost based optimizer chooses an access path.<br />
<br />
SQL*Trace/tkprof<br />
<br />
For this you have to type in SQL*Plus:<br />
- alter session set sql_trace true;<br />
- <run your query><br />
- disconnect (this step is important, because it ensures all cursors get closed, and "row source operation" is generated)<br />
- identify your trace file in the server directory as specified in the parameter user_dump_dest<br />
- on your operating system: tkprof <trace file> a.txt sys=no sort=prsela exeela fchela<br />
<br />
The file a.txt will now give you valuable information on what has actually happened. No predictions but the truth.<br />
<br />
By comparing the output from explain plan with the output from tkprof, you are able to identify the possible problem areas.<br />
<br />
So before rushing into possible solutions, always post the output of explain plan and tkprof with your question and don't forget to post them between the tags [pre] and [/pre] for readability.<br />
<br />
PS: I am fully aware that this text is only a tiny fraction of what can be done, and that other people may choose different tools and actions, but the above gives you a very reasonable start at solving your performance problem.<br />
<br />
Oracle bitmap indexes are very different from standard b-tree indexes. In bitmap structures, a two-dimensional array is created with one column for every row in the table being indexed. Each column represents a distinct value within the bitmapped index. This two-dimensional array represents each value within the index multiplied by the number of rows in the table. At row retrieval time, Oracle decompresses the bitmap into the RAM data buffers so it can be rapidly scanned for matching values. These matching values are delivered to Oracle in the form of a Row-ID list, and these Row-ID values may directly access the required information.<br />
<br />
The real benefit of bitmapped indexing occurs when one table includes multiple bitmapped indexes. Each individual column may have low cardinality. The creation of multiple bitmapped indexes provides a very powerful method for rapidly answering difficult SQL queries.<br />
<br />
For example, assume there is a motor vehicle database with numerous low-cardinality columns such as car_color, car_make, car_model, and car_year. Each column contains less than 100 distinct values by themselves, and a b-tree index would be fairly useless in a database of 20 million vehicles. However, combining these indexes together in a query can provide blistering response times a lot faster than the traditional method of reading each one of the 20 million rows in the base table. For example, assume we wanted to find old blue Toyota Corollas manufactured in 1981:<br />
<br />
select<br />
license_plat_nbr<br />
from<br />
vehicle<br />
where<br />
color = ‘blue’<br />
and<br />
make = ‘toyota’<br />
and<br />
year = 1981;<br />
<br />
Oracle uses a specialized optimizer method called a bitmapped index merge to service this query. In a bitmapped index merge, each Row-ID, or RID, list is built independently by using the bitmaps, and a special merge routine is used in order to compare the RID lists and find the intersecting values. <br />
<br />
Using this methodology, Oracle can provide sub-second response time when working against multiple low-cardinality columns:<br />
<br />
Oracle Bitmap indexes are a very powerful Oracle feature, but they can be tricky!<br />
You will want a bitmap index when: <br />
1 - Table column is low cardinality - As a ROUGH guide, consider a bitmap for any index with less than 100 distinct values<br />
<br />
select region, count(*) from sales group by region; <br />
2 - The table has LOW DML - You must have low insert./update/delete activity. Updating bitmapped indexes take a lot of resources, and bitmapped indexes are best for largely read-only tables and tables that are batch updated nightly. <br />
3 - Multiple columns - Your SQL queries reference multiple, low cardinality values in there where clause. Oracle cost-based SQL optimizer (CBO) will scream when you have bitmap indexes on . <br />
Troubleshooting Oracle bitmap indexes: <br />
Some of the most common problems when implementing bitmap indexes include: <br />
1. Small table - The CBO may force a full-table scan if your table is small! <br />
2. Bad stats - Make sure you always analyze the bitmap with dbms_stats right after creation: <br />
CREATE BITMAP INDEX <br />
emp_bitmap_idx<br />
ON index_demo (gender);<br />
<br />
exec dbms_stats.gather_index_stats(OWNNAME=>'SCOTT', INDNAME=>'EMP_BITMAP_IDX');<br />
3. Test with a hint - To force the use of your new bitmap index, just use a Oracle INDEX hint: <br />
select /*+ index(emp emp_bitmap_idx) */ <br />
count(*)<br />
from <br />
emp, dept<br />
where <br />
emp.deptno = dept.deptno;KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-62703907093848730422009-10-06T09:19:00.001-07:002009-10-06T09:19:00.263-07:00Generating SQL trace filesThere are numerous ways to enable, disable and vary the contents of this trace. The following methods have been available for several versions of the database.<br />
-- All versions.<br />
SQL> ALTER SESSION SET sql_trace=TRUE;<br />
SQL> ALTER SESSION SET sql_trace=FALSE;<br />
<br />
SQL> EXEC DBMS_SESSION.set_sql_trace(sql_trace => TRUE);<br />
SQL> EXEC DBMS_SESSION.set_sql_trace(sql_trace => FALSE);<br />
<br />
SQL> ALTER SESSION SET EVENTS '10046 trace name context forever, level 8';<br />
SQL> ALTER SESSION SET EVENTS '10046 trace name context off';<br />
<br />
SQL> EXEC DBMS_SYSTEM.set_sql_trace_in_session(sid=>123, serial#=>1234, sql_trace=>TRUE);<br />
SQL> EXEC DBMS_SYSTEM.set_sql_trace_in_session(sid=>123, serial#=>1234, sql_trace=>FALSE);<br />
<br />
SQL> EXEC DBMS_SYSTEM.set_ev(si=>123, se=>1234, ev=>10046, le=>8, nm=>' ');<br />
SQL> EXEC DBMS_SYSTEM.set_ev(si=>123, se=>1234, ev=>10046, le=>0, nm=>' ');<br />
<br />
-- Available from SQL*Plus since 8i (commandline utility prior to this.<br />
SQL> CONN sys/password AS SYSDBA; -- User must have SYSDBA.<br />
SQL> ORADEBUG SETMYPID; -- Debug current session.<br />
SQL> ORADEBUG SETOSPID 1234; -- Debug session with the specified OS process.<br />
SQL> ORADEBUG SETORAPID 123456; -- Debug session with the specified Oracle process ID.<br />
<br />
SQL> ORADEBUG EVENT 10046 TRACE NAME CONTEXT FOREVER, LEVEL 12;<br />
SQL> ORADEBUG TRACEFILE_NAME; -- Display the current trace file.<br />
SQL> ORADEBUG EVENT 10046 TRACE NAME CONTEXT OFF;<br />
<br />
-- All versions, requires DBMS_SUPPORT package to be loaded.<br />
SQL> EXEC DBMS_SUPPORT.start_trace(waits=>TRUE, binds=>FALSE);<br />
SQL> EXEC DBMS_SUPPORT.stop_trace;<br />
<br />
SQL> EXEC DBMS_SUPPORT.start_trace_in_session(sid=>123, serial=>1234, waits=>TRUE, binds=>FALSE);<br />
SQL> EXEC DBMS_SUPPORT.stop_trace_in_session(sid=>123, serial=>1234);<br />
The DBMS_SUPPORT package is not present by default, but can be loaded as the SYS user by executing the @$ORACLE_HOME/rdbms/admin/dbmssupp.sql script.<br />
<br />
For methods that require tracing levels the following are valid values:<br />
0 - No trace. Like switching sql_trace off. <br />
2 - The equivalent of regular sql_trace. <br />
4 - The same as 2, but with the addition of bind variable values. <br />
8 - The same as 2, but with the addition of wait events. <br />
12 - The same as 2, but with both bind variable values and wait events. <br />
The same combinations are possible for those methods with Boolean parameters for waits and binds.<br />
With the advent of Oracle 10g the SQL tracing options have been centralized and extended using the DBMS_MONITOR package. The examples below show just a few possible variations for enabling and disabling SQL trace in Oracle 10g.<br />
-- Oracle 10g<br />
SQL> EXEC DBMS_MONITOR.session_trace_enable;<br />
SQL> EXEC DBMS_MONITOR.session_trace_enable(waits=>TRUE, binds=>FALSE);<br />
SQL> EXEC DBMS_MONITOR.session_trace_disable;<br />
<br />
SQL> EXEC DBMS_MONITOR.session_trace_enable(session_id=>1234, serial_num=>1234);<br />
SQL> EXEC DBMS_MONITOR.session_trace_enable(session_id =>1234, serial_num=>1234, waits=>TRUE, binds=>FALSE);<br />
SQL> EXEC DBMS_MONITOR.session_trace_disable(session_id=>1234, serial_num=>1234);<br />
<br />
SQL> EXEC DBMS_MONITOR.client_id_trace_enable(client_id=>'tim_hall');<br />
SQL> EXEC DBMS_MONITOR.client_id_trace_enable(client_id=>'tim_hall', waits=>TRUE, binds=>FALSE);<br />
SQL> EXEC DBMS_MONITOR.client_id_trace_disable(client_id=>'tim_hall');<br />
<br />
SQL> EXEC DBMS_MONITOR.serv_mod_act_trace_enable(service_name=>'db10g', module_name=>'test_api', action_name=>'running');<br />
SQL> EXEC DBMS_MONITOR.serv_mod_act_trace_enable(service_name=>'db10g', module_name=>'test_api', action_name=>'running', -<br />
> waits=>TRUE, binds=>FALSE);<br />
SQL> EXEC DBMS_MONITOR.serv_mod_act_trace_disable(service_name=>'db10g', module_name=>'test_api', action_name=>'running');<br />
The package provides the conventional session level tracing along with two new variations. First, tracing can be enabled on multiple sessions based on the value of the client_identifier column of the V$SESSION view, set using the DBMS_SESSION package. Second, trace can be activated for multiple sessions based on various combinations of the service_name, module, action columns in the V$SESSION view, set using the DBMS_APPLICATION_INFO package, along with the instance_name in RAC environments. With all the possible permutations and default values this provides a high degree of flexibility.<br />
trcsess<br />
Activating trace on multiple sessions means that trace information is spread throughout many trace files. For this reason Oracle 10g introduced the trcsess utility, which allows trace information from multiple trace files to be identified and consolidated into a single trace file. The trcsess usage is listed below. <br />
trcsess [output=<output file name >] [session=<session ID>] [clientid=<clientid>]<br />
[service=<service name>] [action=<action name>] [module=<module name>] <trace file names><br />
<br />
output=<output file name> output destination default being standard output.<br />
session=<session Id> session to be traced.<br />
Session id is a combination of session Index & session serial number e.g. 8.13.<br />
clientid=<clientid> clientid to be traced.<br />
service=<service name> service to be traced.<br />
action=<action name> action to be traced.<br />
module=<module name> module to be traced.<br />
<trace_file_names> Space separated list of trace files with wild card '*' supported.<br />
With all these options the consolidated trace file can be as broad or as specific as needed.<br />
tkprof<br />
The SQL trace files produced by the methods discussed previously can be read in their raw form, or they can be translated by the tkprof utility into a more human readable form. The output below lists the usage notes from the tkprof utility in Oracle 10g.<br />
$ tkprof<br />
Usage: tkprof tracefile outputfile [explain= ] [table= ]<br />
[print= ] [insert= ] [sys= ] [sort= ]<br />
table=schema.tablename Use 'schema.tablename' with 'explain=' option.<br />
explain=user/password Connect to ORACLE and issue EXPLAIN PLAN.<br />
print=integer List only the first 'integer' SQL statements.<br />
aggregate=yes|no<br />
insert=filename List SQL statements and data inside INSERT statements.<br />
sys=no TKPROF does not list SQL statements run as user SYS.<br />
record=filename Record non-recursive statements found in the trace file.<br />
waits=yes|no Record summary for any wait events found in the trace file.<br />
sort=option Set of zero or more of the following sort options:<br />
prscnt number of times parse was called<br />
prscpu cpu time parsing<br />
prsela elapsed time parsing<br />
prsdsk number of disk reads during parse<br />
prsqry number of buffers for consistent read during parse<br />
prscu number of buffers for current read during parse<br />
prsmis number of misses in library cache during parse<br />
execnt number of execute was called<br />
execpu cpu time spent executing<br />
exeela elapsed time executing<br />
exedsk number of disk reads during execute<br />
exeqry number of buffers for consistent read during execute<br />
execu number of buffers for current read during execute<br />
exerow number of rows processed during execute<br />
exemis number of library cache misses during execute<br />
fchcnt number of times fetch was called<br />
fchcpu cpu time spent fetching<br />
fchela elapsed time fetching<br />
fchdsk number of disk reads during fetch<br />
fchqry number of buffers for consistent read during fetch<br />
fchcu number of buffers for current read during fetch<br />
fchrow number of rows fetched<br />
userid userid of user that parsed the cursor<br />
<br />
$<br />
The waits parameter was only added in Oracle 9i, so prior to this version wait information had to be read from the raw trace file. The values of bind variables must be read from the raw files as they are not displayed in the tkprof output.<br />
<br />
The following section shows an example of gathering SQL trace for a session to give you an idea of the whole process.<br />
Trace Example<br />
The sql_trace_test.sql script creates a test table and a procedure that populates it. This procedure will be the subject of our example trace.<br />
<br />
sql_trace_test.sql<br />
CREATE TABLE sql_trace_test (<br />
id NUMBER,<br />
description VARCHAR2(50)<br />
);<br />
<br />
EXEC DBMS_STATS.gather_table_stats('test', 'sql_trace_test');<br />
<br />
CREATE OR REPLACE PROCEDURE populate_sql_trace_test (p_loops IN NUMBER) AS<br />
l_number NUMBER;<br />
BEGIN<br />
FOR i IN 1 .. p_loops LOOP<br />
INSERT INTO sql_trace_test (id, description)<br />
VALUES (i, 'Description for ' || i);<br />
END LOOP;<br />
<br />
SELECT COUNT(*)<br />
INTO l_number<br />
FROM sql_trace_test;<br />
<br />
COMMIT;<br />
<br />
DBMS_OUTPUT.put_line(l_number || ' rows inserted.');<br />
END;<br />
/<br />
SHOW ERRORS<br />
Gathering the statistics on the empty table may seem odd, but this prevents any dynamic sampling being added to the trace file contents, which would only serve to complicate the file.<br />
Before we start to trace the procedure we should identify the trace file that will be produced. The name of the trace file is based on the current value of the user_dump_dest, the session's process id and the instance name. The identify_trace_file.sql script combines these values to produce the expected trace file name.<br />
<br />
identify_trace_file.sql<br />
SET LINESIZE 100<br />
COLUMN trace_file FORMAT A60<br />
<br />
SELECT s.sid,<br />
s.serial#,<br />
pa.value || '/' || LOWER(SYS_CONTEXT('userenv','instance_name')) || <br />
'_ora_' || p.spid || '.trc' AS trace_file<br />
FROM v$session s,<br />
v$process p,<br />
v$parameter pa<br />
WHERE pa.name = 'user_dump_dest'<br />
AND s.paddr = p.addr<br />
AND s.audsid = SYS_CONTEXT('USERENV', 'SESSIONID');<br />
If you are using a windows environment you will need to alter the "/" to a "" in the concatenated string. The expected output from this script is displayed below.<br />
SQL> @identify_trace_file.sql<br />
<br />
SID SERIAL# TRACE_FILE<br />
---------- ---------- ------------------------------------------------------------<br />
130 26739 /u01/app/oracle/admin/DEV/udump/dev1_ora_367660.trc<br />
<br />
1 row selected.<br />
<br />
SQL><br />
Now we know the name of the trace file we can enable tracing, execute the procedure and disable tracing.<br />
SQL> ALTER SESSION SET EVENTS '10046 trace name context forever, level 8';<br />
<br />
Session altered.<br />
<br />
SQL> SET SERVEROUTPUT ON<br />
SQL> EXEC populate_sql_trace_test(p_loops => 5);<br />
5 rows inserted.<br />
<br />
PL/SQL procedure successfully completed.<br />
<br />
SQL> ALTER SESSION SET EVENTS '10046 trace name context off';<br />
<br />
Session altered.<br />
<br />
SQL><br />
The contents of the file are listed below. The output looks a little confusing, but you can identify the individual statements and their associated waits and statistics.<br />
/u01/app/oracle/admin/DEV/udump/dev1_ora_367660.trc<br />
Oracle Database 10g Enterprise Edition Release 10.1.0.3.0 - Production<br />
With the Partitioning, Oracle Label Security, OLAP and Data Mining options<br />
ORACLE_HOME = /u01/app/oracle/product/10.1.0/db_1<br />
System name: OSF1<br />
Node name: dbserver.oracle-base.com<br />
Release: V5.1<br />
Version: 2650<br />
Machine: alpha<br />
Instance name: DEV1<br />
Redo thread mounted by this instance: 1<br />
Oracle process number: 16<br />
Unix process pid: 367660, image: oracleDEV1@dbserver.oracle-base.com<br />
<br />
*** 2005-04-05 09:46:51.499<br />
*** ACTION NAME:() 2005-04-05 09:46:51.499<br />
*** MODULE NAME:(SQL*Plus) 2005-04-05 09:46:51.499<br />
*** SERVICE NAME:(SYS$USERS) 2005-04-05 09:46:51.499<br />
*** SESSION ID:(130.26739) 2005-04-05 09:46:51.499<br />
=====================<br />
PARSING IN CURSOR #29 len=68 dep=0 uid=180 oct=42 lid=180 tim=11746409761792 hv=3847243385 ad='2bb57798'<br />
ALTER SESSION SET EVENTS '10046 trace name context forever, level 8'<br />
END OF STMT<br />
EXEC #29:c=0,e=1024,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=11746409761792<br />
WAIT #29: nam='SQL*Net message to client' ela= 0 p1=1413697536 p2=1 p3=0<br />
WAIT #29: nam='SQL*Net message from client' ela= 7358464 p1=1413697536 p2=1 p3=0<br />
=====================<br />
PARSING IN CURSOR #4 len=36 dep=0 uid=180 oct=47 lid=180 tim=11746417122304 hv=3425213768 ad='2c631cd8'<br />
BEGIN DBMS_OUTPUT.ENABLE(2000); END;<br />
END OF STMT<br />
PARSE #4:c=0,e=1024,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=11746417122304<br />
EXEC #4:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=1,dep=0,og=1,tim=11746417122304<br />
WAIT #4: nam='SQL*Net message to client' ela= 0 p1=1413697536 p2=1 p3=0<br />
WAIT #4: nam='SQL*Net message from client' ela= 6924288 p1=1413697536 p2=1 p3=0<br />
=====================<br />
PARSING IN CURSOR #18 len=51 dep=0 uid=180 oct=47 lid=180 tim=11746424051712 hv=2083693016 ad='27ecb338'<br />
BEGIN populate_sql_trace_test(p_loops => 5); END;<br />
END OF STMT<br />
PARSE #18:c=0,e=3072,p=0,cr=0,cu=0,mis=1,r=0,dep=0,og=1,tim=11746424051712<br />
=====================<br />
PARSING IN CURSOR #19 len=86 dep=1 uid=180 oct=2 lid=180 tim=11746424052736 hv=3247833140 ad='28fa57f8'<br />
INSERT INTO SQL_TRACE_TEST (ID, DESCRIPTION) VALUES (:B1 , 'Description for ' || :B1 )<br />
END OF STMT<br />
PARSE #19:c=0,e=1024,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=1,tim=11746424052736<br />
=====================<br />
PARSING IN CURSOR #23 len=69 dep=2 uid=0 oct=3 lid=0 tim=11746424053760 hv=1471956217 ad='2e7591f0'<br />
select con#,obj#,rcon#,enabled,nvl(defer,0) from cdef$ where robj#=:1<br />
END OF STMT<br />
PARSE #23:c=0,e=1024,p=0,cr=0,cu=0,mis=1,r=0,dep=2,og=4,tim=11746424053760<br />
EXEC #23:c=0,e=5120,p=0,cr=0,cu=0,mis=1,r=0,dep=2,og=4,tim=11746424059904<br />
FETCH #23:c=0,e=0,p=0,cr=2,cu=0,mis=0,r=0,dep=2,og=4,tim=11746424059904<br />
=====================<br />
PARSING IN CURSOR #25 len=146 dep=2 uid=0 oct=3 lid=0 tim=11746424065024 hv=2107929772 ad='2e7b4b08'<br />
select con#,type#,condlength,intcols,robj#,rcon#,match#,refact,nvl(enabled,0),rowid,cols,nvl(defer,0)<br />
,mtime,nvl(spare1,0) from cdef$ where obj#=:1<br />
END OF STMT<br />
PARSE #25:c=0,e=5120,p=0,cr=0,cu=0,mis=1,r=0,dep=2,og=4,tim=11746424065024<br />
=====================<br />
PARSING IN CURSOR #27 len=210 dep=3 uid=0 oct=3 lid=0 tim=11746424102912 hv=864012087 ad='2e351f08'<br />
select /*+ rule */ bucket_cnt, row_cnt, cache_cnt, null_cnt, timestamp#, sample_size, minimum, maximum,<br />
distcnt, lowval, hival, density, col#, spare1, spare2, avgcln from hist_head$ where obj#=:1 and intcol#=:2<br />
END OF STMT<br />
PARSE #27:c=50000,e=36864,p=0,cr=0,cu=0,mis=1,r=0,dep=3,og=3,tim=11746424102912<br />
EXEC #27:c=16667,e=4096,p=0,cr=0,cu=0,mis=1,r=0,dep=3,og=3,tim=11746424108032<br />
FETCH #27:c=0,e=1024,p=0,cr=3,cu=0,mis=0,r=1,dep=3,og=3,tim=11746424109056<br />
EXEC #27:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=3,og=3,tim=11746424109056<br />
FETCH #27:c=0,e=0,p=0,cr=3,cu=0,mis=0,r=1,dep=3,og=3,tim=11746424109056<br />
EXEC #27:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=3,og=3,tim=11746424110080<br />
FETCH #27:c=0,e=0,p=0,cr=3,cu=0,mis=0,r=1,dep=3,og=3,tim=11746424110080<br />
=====================<br />
PARSING IN CURSOR #28 len=121 dep=3 uid=0 oct=3 lid=0 tim=11746424119296 hv=3150898423 ad='2e347db8'<br />
select /*+ rule */ bucket, endpoint, col#, epvalue from histgrm$ where obj#=:1 and intcol#=:2 and row#=:3<br />
order by bucket<br />
END OF STMT<br />
PARSE #28:c=0,e=9216,p=0,cr=0,cu=0,mis=1,r=0,dep=3,og=3,tim=11746424119296<br />
EXEC #28:c=0,e=1024,p=0,cr=0,cu=0,mis=1,r=0,dep=3,og=3,tim=11746424121344<br />
FETCH #28:c=0,e=1024,p=0,cr=3,cu=0,mis=0,r=3,dep=3,og=3,tim=11746424122368<br />
EXEC #27:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=3,og=3,tim=11746424122368<br />
FETCH #27:c=0,e=0,p=0,cr=3,cu=0,mis=0,r=1,dep=3,og=3,tim=11746424122368<br />
EXEC #27:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=3,og=3,tim=11746424123392<br />
FETCH #27:c=0,e=0,p=0,cr=3,cu=0,mis=0,r=1,dep=3,og=3,tim=11746424123392<br />
EXEC #25:c=83334,e=61440,p=0,cr=18,cu=0,mis=1,r=0,dep=2,og=4,tim=11746424126464<br />
FETCH #25:c=0,e=0,p=0,cr=2,cu=0,mis=0,r=0,dep=2,og=4,tim=11746424126464<br />
=====================<br />
PARSING IN CURSOR #12 len=169 dep=2 uid=0 oct=3 lid=0 tim=11746424128512 hv=1173719687 ad='2e7c2c50'<br />
select col#, grantee#, privilege#,max(mod(nvl(option$,0),2)) from objauth$ where obj#=:1 and col# is not null<br />
group by privilege#, col#, grantee# order by col#, grantee#<br />
END OF STMT<br />
PARSE #12:c=0,e=1024,p=0,cr=0,cu=0,mis=1,r=0,dep=2,og=4,tim=11746424128512<br />
EXEC #12:c=0,e=3072,p=0,cr=0,cu=0,mis=1,r=0,dep=2,og=4,tim=11746424131584<br />
FETCH #12:c=0,e=0,p=0,cr=2,cu=0,mis=0,r=0,dep=2,og=4,tim=11746424131584<br />
=====================<br />
PARSING IN CURSOR #26 len=151 dep=2 uid=0 oct=3 lid=0 tim=11746424131584 hv=4139184264 ad='2e7cc608'<br />
select grantee#,privilege#,nvl(col#,0),max(mod(nvl(option$,0),2))from objauth$ where obj#=:1 group by grantee#,<br />
privilege#,nvl(col#,0) order by grantee#<br />
END OF STMT<br />
PARSE #26:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=4,tim=11746424131584<br />
EXEC #26:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=4,tim=11746424131584<br />
FETCH #26:c=0,e=1024,p=0,cr=2,cu=0,mis=0,r=0,dep=2,og=4,tim=11746424132608<br />
=====================<br />
PARSING IN CURSOR #7 len=175 dep=2 uid=0 oct=3 lid=0 tim=11746424132608 hv=1729330152 ad='2f3597a8'<br />
select u.name,o.name, t.update$, t.insert$, t.delete$, t.enabled from obj$ o,user$ u,trigger$ t where <br />
t.baseobject=:1 and t.obj#=o.obj# and o.owner#=u.user# order by o.obj#<br />
END OF STMT<br />
PARSE #7:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=4,tim=11746424132608<br />
EXEC #7:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=2,og=4,tim=11746424132608<br />
FETCH #7:c=0,e=0,p=0,cr=2,cu=0,mis=0,r=0,dep=2,og=4,tim=11746424132608<br />
STAT #7 id=1 cnt=0 pid=0 pos=1 obj=0 op='SORT ORDER BY (cr=2 pr=0 pw=0 time=0 us)'<br />
STAT #7 id=2 cnt=0 pid=1 pos=1 obj=0 op='NESTED LOOPS (cr=2 pr=0 pw=0 time=0 us)'<br />
STAT #7 id=3 cnt=0 pid=2 pos=1 obj=0 op='NESTED LOOPS (cr=2 pr=0 pw=0 time=0 us)'<br />
STAT #7 id=4 cnt=0 pid=3 pos=1 obj=84 op='TABLE ACCESS BY INDEX ROWID TRIGGER$ (cr=2 pr=0 pw=0 time=0 us)'<br />
STAT #7 id=5 cnt=0 pid=4 pos=1 obj=128 op='INDEX RANGE SCAN I_TRIGGER1 (cr=2 pr=0 pw=0 time=0 us)'<br />
STAT #7 id=6 cnt=0 pid=3 pos=2 obj=18 op='TABLE ACCESS BY INDEX ROWID OBJ$ (cr=0 pr=0 pw=0 time=0 us)'<br />
STAT #7 id=7 cnt=0 pid=6 pos=1 obj=36 op='INDEX UNIQUE SCAN I_OBJ1 (cr=0 pr=0 pw=0 time=0 us)'<br />
STAT #7 id=8 cnt=0 pid=2 pos=2 obj=22 op='TABLE ACCESS CLUSTER USER$ (cr=0 pr=0 pw=0 time=0 us)'<br />
STAT #7 id=9 cnt=0 pid=8 pos=1 obj=11 op='INDEX UNIQUE SCAN I_USER# (cr=0 pr=0 pw=0 time=0 us)'<br />
EXEC #19:c=83334,e=86016,p=0,cr=29,cu=21,mis=1,r=1,dep=1,og=1,tim=11746424138752<br />
EXEC #19:c=0,e=0,p=0,cr=0,cu=1,mis=0,r=1,dep=1,og=1,tim=11746424138752<br />
EXEC #19:c=0,e=0,p=0,cr=0,cu=1,mis=0,r=1,dep=1,og=1,tim=11746424138752<br />
EXEC #19:c=0,e=0,p=0,cr=0,cu=1,mis=0,r=1,dep=1,og=1,tim=11746424139776<br />
EXEC #19:c=0,e=0,p=0,cr=0,cu=1,mis=0,r=1,dep=1,og=1,tim=11746424139776<br />
=====================<br />
PARSING IN CURSOR #29 len=35 dep=1 uid=180 oct=3 lid=180 tim=11746424140800 hv=3788777626 ad='2c84eb58'<br />
SELECT COUNT(*) FROM SQL_TRACE_TEST<br />
END OF STMT<br />
PARSE #29:c=0,e=1024,p=0,cr=0,cu=0,mis=1,r=0,dep=1,og=1,tim=11746424140800<br />
EXEC #29:c=0,e=1024,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=1,tim=11746424141824<br />
FETCH #29:c=0,e=0,p=0,cr=7,cu=0,mis=0,r=1,dep=1,og=1,tim=11746424141824<br />
=====================<br />
PARSING IN CURSOR #4 len=6 dep=1 uid=180 oct=44 lid=180 tim=11746424141824 hv=255718823 ad='2e5363f0'<br />
COMMIT<br />
END OF STMT<br />
PARSE #4:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=1,og=1,tim=11746424141824<br />
XCTEND rlbk=0, rd_only=0<br />
EXEC #4:c=0,e=0,p=0,cr=0,cu=1,mis=0,r=0,dep=1,og=1,tim=11746424141824<br />
EXEC #18:c=83334,e=91136,p=0,cr=36,cu=26,mis=0,r=1,dep=0,og=1,tim=11746424142848<br />
WAIT #18: nam='SQL*Net message to client' ela= 0 p1=1413697536 p2=1 p3=0<br />
WAIT #18: nam='SQL*Net message from client' ela= 1024 p1=1413697536 p2=1 p3=0<br />
=====================<br />
PARSING IN CURSOR #7 len=52 dep=0 uid=180 oct=47 lid=180 tim=11746424143872 hv=1029988163 ad='2c2ec1b8'<br />
BEGIN DBMS_OUTPUT.GET_LINES(:LINES, :NUMLINES); END;<br />
END OF STMT<br />
PARSE #7:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=11746424143872<br />
WAIT #7: nam='SQL*Net message to client' ela= 0 p1=1413697536 p2=1 p3=0<br />
EXEC #7:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=1,dep=0,og=1,tim=11746424144896<br />
WAIT #7: nam='SQL*Net message from client' ela= 8560640 p1=1413697536 p2=1 p3=0<br />
=====================<br />
PARSING IN CURSOR #12 len=55 dep=0 uid=180 oct=42 lid=180 tim=11746432706560 hv=2655499671 ad='2a24eab8'<br />
ALTER SESSION SET EVENTS '10046 trace name context off'<br />
END OF STMT<br />
PARSE #12:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=11746432705536<br />
EXEC #12:c=0,e=0,p=0,cr=0,cu=0,mis=0,r=0,dep=0,og=1,tim=11746432706560<br />
The following command uses the tkprof utility to translate the trace file, placing the translated output in the translated.txt file. The explain and table parameters have been set to allow execution plans to be displayed and the sys parameter prevents recursive SQL being displayed.<br />
$ cd /u01/app/oracle/admin/DEV/udump/<br />
$ tkprof dev1_ora_367660.trc translated.txt explain=test/test table=sys.plan_table sys=no waits=yes<br />
<br />
TKPROF: Release 10.1.0.3.0 - Production on Tue Apr 5 09:22:43 2005<br />
<br />
Copyright (c) 1982, 2004, Oracle. All rights reserved.<br />
<br />
<br />
$<br />
The contents of the translated file are displayed below.<br />
TKPROF: Release 10.1.0.3.0 - Production on Tue Apr 5 09:53:50 2005<br />
<br />
Copyright (c) 1982, 2004, Oracle. All rights reserved.<br />
<br />
Trace file: dev1_ora_367660.trc<br />
Sort options: default<br />
<br />
********************************************************************************<br />
count = number of times OCI procedure was executed<br />
cpu = cpu time in seconds executing <br />
elapsed = elapsed time in seconds executing<br />
disk = number of physical reads of buffers from disk<br />
query = number of buffers gotten for consistent read<br />
current = number of buffers gotten in current mode (usually for update)<br />
rows = number of rows processed by the fetch or execute call<br />
********************************************************************************<br />
<br />
ALTER SESSION SET EVENTS '10046 trace name context forever, level 8'<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 0 0.00 0.00 0 0 0 0<br />
Execute 1 0.00 0.00 0 0 0 0<br />
Fetch 0 0.00 0.00 0 0 0 0<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 1 0.00 0.00 0 0 0 0<br />
<br />
Misses in library cache during parse: 0<br />
Optimizer mode: ALL_ROWS<br />
Parsing user id: 180 (TEST)<br />
<br />
Elapsed times include waiting on following events:<br />
Event waited on Times Max. Wait Total Waited<br />
---------------------------------------- Waited ---------- ------------<br />
SQL*Net message to client 1 0.00 0.00<br />
SQL*Net message from client 1 7.35 7.35<br />
********************************************************************************<br />
<br />
BEGIN DBMS_OUTPUT.ENABLE(2000); END;<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.00 0 0 0 0<br />
Execute 1 0.00 0.00 0 0 0 1<br />
Fetch 0 0.00 0.00 0 0 0 0<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 2 0.00 0.00 0 0 0 1<br />
<br />
Misses in library cache during parse: 0<br />
Optimizer mode: ALL_ROWS<br />
Parsing user id: 180 (TEST)<br />
<br />
Elapsed times include waiting on following events:<br />
Event waited on Times Max. Wait Total Waited<br />
---------------------------------------- Waited ---------- ------------<br />
SQL*Net message to client 1 0.00 0.00<br />
SQL*Net message from client 1 6.92 6.92<br />
********************************************************************************<br />
<br />
BEGIN populate_sql_trace_test(p_loops => 5); END;<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.00 0 0 0 0<br />
Execute 1 0.00 0.00 0 0 0 1<br />
Fetch 0 0.00 0.00 0 0 0 0<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 2 0.00 0.00 0 0 0 1<br />
<br />
Misses in library cache during parse: 1<br />
Optimizer mode: ALL_ROWS<br />
Parsing user id: 180 (TEST)<br />
<br />
Elapsed times include waiting on following events:<br />
Event waited on Times Max. Wait Total Waited<br />
---------------------------------------- Waited ---------- ------------<br />
SQL*Net message to client 1 0.00 0.00<br />
SQL*Net message from client 1 0.00 0.00<br />
********************************************************************************<br />
<br />
INSERT INTO SQL_TRACE_TEST (ID, DESCRIPTION) <br />
VALUES<br />
(:B1 , 'Description for ' || :B1 )<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.00 0 0 0 0<br />
Execute 5 0.00 0.00 0 1 25 5<br />
Fetch 0 0.00 0.00 0 0 0 0<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 6 0.00 0.00 0 1 25 5<br />
<br />
Misses in library cache during parse: 1<br />
Misses in library cache during execute: 1<br />
Optimizer mode: ALL_ROWS<br />
Parsing user id: 180 (TEST) (recursive depth: 1)<br />
<br />
Rows Execution Plan<br />
------- ---------------------------------------------------<br />
0 INSERT STATEMENT MODE: ALL_ROWS<br />
<br />
********************************************************************************<br />
<br />
SELECT COUNT(*) <br />
FROM<br />
SQL_TRACE_TEST<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.00 0 0 0 0<br />
Execute 1 0.00 0.00 0 0 0 0<br />
Fetch 1 0.00 0.00 0 7 0 1<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 3 0.00 0.00 0 7 0 1<br />
<br />
Misses in library cache during parse: 1<br />
Optimizer mode: ALL_ROWS<br />
Parsing user id: 180 (TEST) (recursive depth: 1)<br />
<br />
Rows Execution Plan<br />
------- ---------------------------------------------------<br />
0 SELECT STATEMENT MODE: ALL_ROWS<br />
0 SORT (AGGREGATE)<br />
0 TABLE ACCESS MODE: ANALYZED (FULL) OF 'SQL_TRACE_TEST' <br />
(TABLE)<br />
<br />
********************************************************************************<br />
<br />
COMMIT<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.00 0 0 0 0<br />
Execute 1 0.00 0.00 0 0 1 0<br />
Fetch 0 0.00 0.00 0 0 0 0<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 2 0.00 0.00 0 0 1 0<br />
<br />
Misses in library cache during parse: 0<br />
Optimizer mode: ALL_ROWS<br />
Parsing user id: 180 (TEST) (recursive depth: 1)<br />
********************************************************************************<br />
<br />
BEGIN DBMS_OUTPUT.GET_LINES(:LINES, :NUMLINES); END;<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.00 0 0 0 0<br />
Execute 1 0.00 0.00 0 0 0 1<br />
Fetch 0 0.00 0.00 0 0 0 0<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 2 0.00 0.00 0 0 0 1<br />
<br />
Misses in library cache during parse: 0<br />
Optimizer mode: ALL_ROWS<br />
Parsing user id: 180 (TEST)<br />
<br />
Elapsed times include waiting on following events:<br />
Event waited on Times Max. Wait Total Waited<br />
---------------------------------------- Waited ---------- ------------<br />
SQL*Net message to client 1 0.00 0.00<br />
SQL*Net message from client 1 8.56 8.56<br />
********************************************************************************<br />
<br />
ALTER SESSION SET EVENTS '10046 trace name context off'<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.00 0 0 0 0<br />
Execute 1 0.00 0.00 0 0 0 0<br />
Fetch 0 0.00 0.00 0 0 0 0<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 2 0.00 0.00 0 0 0 0<br />
<br />
Misses in library cache during parse: 0<br />
Optimizer mode: ALL_ROWS<br />
Parsing user id: 180 (TEST)<br />
<br />
<br />
<br />
********************************************************************************<br />
<br />
OVERALL TOTALS FOR ALL NON-RECURSIVE STATEMENTS<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 4 0.00 0.00 0 0 0 0<br />
Execute 5 0.00 0.00 0 0 0 3<br />
Fetch 0 0.00 0.00 0 0 0 0<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 9 0.00 0.00 0 0 0 3<br />
<br />
Misses in library cache during parse: 1<br />
<br />
Elapsed times include waiting on following events:<br />
Event waited on Times Max. Wait Total Waited<br />
---------------------------------------- Waited ---------- ------------<br />
SQL*Net message to client 4 0.00 0.00<br />
SQL*Net message from client 4 8.56 22.84<br />
<br />
<br />
OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 10 0.05 0.05 0 0 0 0<br />
Execute 18 0.03 0.03 0 1 26 5<br />
Fetch 12 0.00 0.00 0 35 0 9<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 40 0.08 0.08 0 36 26 14<br />
<br />
Misses in library cache during parse: 7<br />
Misses in library cache during execute: 6<br />
<br />
8 user SQL statements in session.<br />
7 internal SQL statements in session.<br />
15 SQL statements in session.<br />
2 statements EXPLAINed in this session.<br />
********************************************************************************<br />
Trace file: dev1_ora_367660.trc<br />
Trace file compatibility: 10.01.00<br />
Sort options: default<br />
<br />
1 session in tracefile.<br />
8 user SQL statements in trace file.<br />
7 internal SQL statements in trace file.<br />
15 SQL statements in trace file.<br />
15 unique SQL statements in trace file.<br />
2 SQL statements EXPLAINed using schema:<br />
sys.plan_table<br />
Schema was specified.<br />
Existing table was used.<br />
146 lines in trace file.<br />
22 elapsed seconds in trace file.<br />
For each statement executed by the session, the file contains a record of the parse, execute and fetch statistics, an execution plan where necessary and a list of session waits.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-67212767457278220192009-10-06T09:16:00.001-07:002009-10-06T09:16:39.958-07:00Performance tuning tips for database developersPerformance tuning is not easy and there aren't any silver bullets, but you can go a surprisingly long way with a few basic guidelines. <br />
In theory, performance tuning is done by a DBA. But in practice, the DBA is not going to have time to scrutinize every change made to a stored procedure. Learning to do basic tuning might save you from reworking code late in the game.<br />
Below is my list of the top 15 things I believe developers should do as a matter of course to tune performance when coding. These are the low hanging fruit of SQL Server performance – they are easy to do and often have a substantial impact. Doing these won't guarantee lightening fast performance, but it won't be slow either.<br />
1. Create a primary key on each table you create and unless you are really knowledgeable enough to figure out a better plan, make it the clustered index (note that if you set the primary key in Enterprise Manager it will cluster it by default). <br />
2. Create an index on any column that is a foreign key. If you know it will be unique, set the flag to force the index to be unique. <br />
3. Don't index anything else (yet). <br />
4. Unless you need a different behaviour, always owner qualify your objects when you reference them in TSQL. Use dbo.sysdatabases instead of just sysdatabases. <br />
5. Use set nocount on at the top of each stored procedure (and set nocount off) at the bottom. <br />
6. Think hard about locking. If you're not writing banking software, would it matter that you take a chance on a dirty read? You can use the NOLOCK hint, but it's often easier to use SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED at the top of the procedure, then reset to READ COMMITTED at the bottom. <br />
7. I know you've heard it a million times, but only return the columns and the rows you need. <br />
8. Use transactions when appropriate, but allow zero user interaction while the transaction is in progress. I try to do all my transactions inside a stored procedure. <br />
9. Avoid temp tables as much as you can, but if you need a temp table, create it explicitly using Create Table #temp. <br />
10. Avoid NOT IN, instead use a left outer join - even though it's often easier to visualize the NOT IN. <br />
11. If you insist on using dynamic sql (executing a concatenated string), use named parameters and sp_executesql (rather than EXEC) so you have a chance of reusing the query plan. While it's simplistic to say that stored procedures are always the right answer, it's also close enough that you won't go wrong using them. <br />
12. Get in the habit of profiling your code before and after each change. While you should keep in mind the depth of the change, if you see more than a 10-15% increase in CPU, Reads, or Writes it probably needs to be reviewed.<br />
13. Look for every possible way to reduce the number of round trips to the server. Returning multiple resultsets is one way to do this. <br />
14. Avoid index and join hints. <br />
15. When you're done coding, set Profiler to monitor statements from your machine only, then run through the application from start to finish once. Take a look at the number of reads and writes, and the number of calls to the server. See anything that looks unusual? It's not uncommon to see calls to procedures that are no longer used, or to see duplicate calls. Impress your DBA by asking him to review those results with you.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-9529099006024779632009-10-06T09:14:00.001-07:002009-10-06T09:14:46.531-07:00Tuning ScriptsOracle Database Tuning Statistics<br />
· Cache Hit Ratio<br />
· Sorts in Memory<br />
· Shared Pool<br />
· Library Cache Hit Ratios<br />
· Recursive Calls/Total Calls<br />
· Short/Total Table Scans<br />
· Redo Activity<br />
· Table Contention<br />
· CPU parse overhead<br />
· Latches<br />
· Rollback Segment Contention<br />
Cache Hit Ratios<br />
There are two main figures<br />
Buffer Cache Hit Ratio<br />
select round((1-(pr.value/(bg.value+cg.value)))*100,2)<br />
from v$sysstat pr, v$sysstat bg, v$sysstat cg<br />
where pr.name='physical reads'<br />
and bg.name='db block gets'<br />
and cg.name='consistent gets'<br />
The buffer cache hit ratio is a measure of the proportion of requests for data which is satisfied by data already in the buffer cache. Higher ratios are better as access to data in memory is speedier than an IO operation to disk. There comes a point of diminishing returns when increasing the size of the database buffer. Also, remember that this is part of the SGA and it may be more important to use additional memory for other parts of the SGA. It is vital that the whole SGA fits within main memory, as paging of the SGA is disastrous for performance.<br />
<br />
Optimum High <br />
init.ora parameter DB_BLOCK_BUFFERS <br />
Dictionary Cache Hit Ratio<br />
select sum(gets-getmisses)*100/sum(gets) <br />
from v$rowcache<br />
The dictionary cache hit ratio is a measure of the proportion of requests for information from the data dictionary, the collection of database tables and views containing reference information about the database, its structures, and its users. On instance startup, the data dictionary cache contains no data, so any SQL statement issued is likely to result in cache misses. As more data is read into the cache, the likelihood of cache misses should decrease. Eventually the database should reach a "steady state" in which the most frequently used dictionary data is in the cache.<br />
The dictionary cache resides within the Shared Pool, part of the SGA, so increasing the shared pool size should improve the dictionary cacge hit ratio.<br />
<br />
Optimum High <br />
init.ora parameter SHARED_POOL_SIZE <br />
Sorts in Memory<br />
select round((mem.value/(mem.value+dsk.value))*100,2)<br />
from v$sysstat mem, v$sysstat dsk<br />
where mem.name='sorts (memory)'<br />
and dsk.name='sorts (disk)'<br />
This is a measure of the proportion of data sorts which occur within memory rather than on disk. Sorts on disk make use of the user's tempoary table space. The maximum size of sort which will occur in memory is defined by the sort area size, which is the size within the PGA which will be used. Each Oracle process which sorts will allocate this much memory, though it may not need all of it. Use of memory for this purpose reduces that available to the SGA.<br />
<br />
Optimum High <br />
init.ora parameter SORT_AREA_SIZE <br />
Shared Pool<br />
There are two important figures<br />
Shared Pool Free<br />
select round((sum(decode(name,'free memory',bytes,0))/sum(bytes))*100,2)<br />
from v$sgastat<br />
The percentage of the shared pool not currently in use. If a large proportion of the shared pool is always free, it is likely that the size of the shared pool can be reduced. Low free values are not a cause for concern unless other factors also indicate problems, e.g. a poordictionary cache hit ratio or large proportion of reloads occurring.<br />
<br />
Optimum Small but non-zero <br />
init.ora parameter SHARED_POOL_SIZE <br />
Shared Pool Reloads<br />
select round(sum(reloads)/sum(pins)*100,2)<br />
from v$librarycache<br />
where namespace in ('SQL AREA','TABLE/PROCEDURE','BODY','TRIGGER')<br />
This is similar to a Library Cache Miss Ratio, but is specific to SQL and PL/SQL blocks. Shared pool reloads occur when Oracle has to implicitly reparse SQL or PL/SQL at the point when it attempts to execute it. A larger shared pool wil reduce the number of times that code needs to be reloaded. Also, ensuring that similar pieces of SQL are written identically will increase sharing of code.<br />
To take advantage of additional memory available for shared SQL areas, you may also need to increase the number of cursors permitted for a session. You can do this by increasing the value of the initialization parameter OPEN_CURSORS.<br />
<br />
Optimum Low <br />
init.ora parameter SHARED_POOL_SIZE <br />
init.ora parameter OPEN_CURSORS <br />
Library Cache Hit Ratios<br />
These values are closely related to shared pool reloads . Improving these figures is carried out in the same manner as discussed in that section. There are two figures<br />
Library Cache Get Hit Ratio<br />
The proportion of requests for a lock on an object which were satisfied by finding that object's handle already in memory.<br />
select round(sum(gethits)/sum(gets)*100,2)<br />
from v$librarycache<br />
<br />
Optimum High <br />
init.ora parameter SHARED_POOL_SIZE <br />
init.ora parameter OPEN_CURSORS <br />
Library Cache Pin Hit Ratio<br />
select round(sum(pinhits)/sum(pins)*100,2)<br />
from v$librarycache<br />
The proportion of attempts to pin an object which were satisfied by finding all the pieces of that object already in memory.<br />
<br />
Optimum High <br />
init.ora parameter SHARED_POOL_SIZE <br />
init.ora parameter OPEN_CURSORS <br />
Recursive Calls/Total Calls<br />
select round((rcv.value/(rcv.value+usr.value))*100,2)<br />
from v$sysstat rcv, v$sysstat usr<br />
where rcv.name='recursive calls'<br />
and usr.name='user calls'<br />
A high ratio of recursive calls to total calls may indicate any of the following:<br />
· Dynamic extension of tables due to poor sizing<br />
· Growing and shrinking of rollback segments due to unsuitable OPTIMAL settings<br />
· Large amounts of sort to disk resulting in creation and deletion of temporary segments<br />
· Data dictionary misses<br />
· Complex triggers, integrity constraints, procedures, functions and/or packages<br />
<br />
Optimum Low <br />
Short/Total Table Scans<br />
select round((shrt.value/(shrt.value+lng.value))*100,2)<br />
from v$sysstat shrt, v$sysstat lng<br />
where shrt.name='table scans (short tables)'<br />
and lng.name='table scans (long tables)'<br />
This is the proportion of full table scans which are occurring on short tables. Short tables may be scanned by Oracle when this is quicker than using an index. Full table scans of long tables is generally bad for overall performance. Low figures for this graph may indicate lack of indexes on large tables or poorly written SQL which fails to use existing indexes or is returning a large percentage of the table.<br />
<br />
Optimum High <br />
Redo Activity<br />
The redo log buffer is a circular buffer in the SGA that holds information about changes made to the database. This graph gives an indication of the level of contention occuring for redos. It is made up of two figures<br />
Redo Space Wait Ratio<br />
select round((req.value/wrt.value)*100,2)<br />
from v$sysstat req, v$sysstat wrt<br />
where req.name= 'redo log space requests' <br />
and wrt.name= 'redo writes' <br />
A redo space wait is when there is insufficient space in the redo buffer for a transaction to write redo information. It is an indication that the redo buffer is too small given the rate of transactions occurring in relation to the rate at which the log writer is writing data to the redo logs.<br />
<br />
Optimum Very Low <br />
init.ora parameter LOG_BUFFER <br />
Redo Log Allocation Contention<br />
There are two latches:<br />
The Redo Allocation Latch<br />
select round(greatest(<br />
(sum(decode(ln.name,'redo allocation',misses,0))<br />
/greatest(sum(decode(ln.name,'redo allocation',gets,0)),1)),<br />
(sum(decode(ln.name,'redo allocation',immediate_misses,0))<br />
/greatest(sum(decode(ln.name,'redo allocation',immediate_gets,0))<br />
+sum(decode(ln.name,'redo allocation',immediate_misses,0)),1))<br />
)*100,2)<br />
from v$latch l,v$latchname ln<br />
where l.latch#=ln.latch#<br />
The redo allocation latch controls the allocation of space for redo entries in the redo log buffer. To allocate space in the buffer, an Oracle user process must obtain the redo allocation latch. Since there is only one redo allocation latch, only one user process can allocate space in the buffer at a time. The single redo allocation latch enforces the sequential nature of the entries in the buffer.<br />
After allocating space for a redo entry, the user process may copy the entry into the buffer. This is called "copying on the redo allocation latch". A process may only copy on the redo allocation latch if the redo entry is smaller than a threshold size.<br />
The maximum size of a redo entry that can be copied on the redo allocation latch is specified by the initialization parameter LOG_SMALL_ENTRY_MAX_SIZE.<br />
Redo Copy Latches<br />
select round(greatest(<br />
(sum(decode(ln.name,'redo copy',misses,0))<br />
/greatest(sum(decode(ln.name,'redo copy',gets,0)),1)),<br />
(sum(decode(ln.name,'redo copy',immediate_misses,0))<br />
/greatest(sum(decode(ln.name,'redo copy',immediate_gets,0))<br />
+sum(decode(ln.name,'redo copy',immediate_misses,0)),1)) )*100,2)<br />
from v$latch l,v$latchname ln<br />
where l.latch#=ln.latch#<br />
The user process first obtains the copy latch. Then it obtains the allocation latch, performs allocation, and releases the allocation latch. Next the process performs the copy under the copy latch, and releases the copy latch. The allocation latch is thus held for only a very short period of time, as the user process does not try to obtain the copy latch while holding the allocation latch.<br />
If the redo entry is too large to copy on the redo allocation latch, the user process must obtain a redo copy latch before copying the entry into the buffer. While holding a redo copy latch, the user process copies the redo entry into its allocated space in the buffer and then releases the redo copy latch.<br />
With multiple CPUs the redo log buffer can have multiple redo copy latches. These allow multiple processes to copy entries to the redo log buffer concurrently. The number of redo copy latches is determined by the parameter LOG_SIMULTANEOUS_COPIES.<br />
<br />
Optimum Very Low <br />
init.ora parameter LOG_SMALL_ENTRY_MAX_SIZE <br />
init.ora parameter LOG_SIMULTANEOUS_COPIES <br />
Table Contention<br />
There are two figures which give indications of how well table storage is working. Figures are averaged across all tables in use. This means one table may be seriously at fault or many tables may have low level problems.<br />
Chained Fetch Ratio<br />
select round((cont.value/(scn.value+rid.value))*100,2)<br />
from v$sysstat cont, v$sysstat scn, v$sysstat rid<br />
where cont.name= 'table fetch continued row' <br />
and scn.name= 'table scan rows gotten'<br />
and rid.name= 'table fetch by rowid'<br />
This is a proportion of all rows fetched which resulted in a chained row continuation. Such a continuation means that data for the row is spread across two blocks, which can occur in either of two ways:<br />
Row Migration<br />
This occurs when an update to a row cannot fit within the current block. In this case, the data for the row is migrated to a new block leaving a pointer to the new location in the original block.<br />
Row Chaining<br />
This occurs when a row cannot fit into a single data block, e.g. due to having large or many fields. In this case, the row is spread over two or more blocks.<br />
<br />
Optimum Very Low <br />
Free List Contention<br />
select round((sum(decode(w.class,'free list',count,0))/<br />
(sum(decode(name,'db block gets',value,0))<br />
+ sum(decode(name,'consistent gets',value,0))))*100,2)<br />
from v$waitstat w, v$sysstat<br />
Free list contention occurs when more than one process is attempting to insert data into a given table. The table header structure maintains one or more lists of blocks which have free space for insertion. If more processes are attempting to make insert than there are free lists some will have to wait for access to a free list.<br />
<br />
Optimum Very Low <br />
CPU Parse Overhead<br />
select round((prs.value/(prs.value+exe.value))*100,2)<br />
from v$sysstat prs, v$sysstat exe<br />
where prs.name like 'parse count (hard)' <br />
and exe.name= 'execute count'<br />
The CPU parse overhead is the proportion of database CPU time being spent in parsing SQL and PL/SQL code. High values of this figure indicate that either a large amount of once-only code is being used by the database or that the shared sql area is too small.<br />
<br />
Optimum Low <br />
init.ora parameter SORT_AREA_SIZE <br />
Latches<br />
Latches are simple, low-level serialization mechanisms to protect shared data structures in the SGA. When attempting to get a latch a process may be willing to wait, hence this graph includes two figures. See also redo log allocation latches.<br />
Willing to Wait Latch Gets<br />
select round(((sum(gets) - sum(misses)) / sum(gets))*100,2)<br />
from v$latch<br />
An attempt by a process to obtain a latch which is willing to wait will sleep and retry until it obtains the latch.<br />
<br />
Optimum High <br />
Immediate Latch Gets<br />
select round(((sum(immediate_gets) - sum(immediate_misses)) / sum(immediate_gets))*100,2)<br />
from v$latch<br />
An attempt to obtain a latch which a process is not allowed to wait for will timeout.<br />
<br />
Optimum High <br />
Rollback Segment Contention<br />
select round(sum(waits)/sum(gets)*100,2)<br />
from v$rollstat<br />
This figure is an indication of whether a process had to wait to get access to a rollback segment. To improve figures, increase the number of rollback segments available.<br />
<br />
Optimum Low <br />
<br />
Redo Log Buffer Latches <br />
When a transaction is ready to write its changes to the redo log, it first has to grab the Redo Allocation Latch, of which there is only one, to keep others from writing to the log at the same time. If someone else has that latch, it has to wait for the latch, resulting in a "miss".<br />
Once it grabs that latch, if the change is larger than log_small_entry_max_size bytes and if your server has multiple CPU's, it then tries to grab a Redo Copy Latch, of which there can be up to 2 times the number of CPU's, which would allow it to release the Redo Allocation Latch for someone else to use. If none of them are available, resulting in an "immediate miss", it will not wait for a Redo Copy Latch (thus, the "immediate"), but, instead, hangs on to the Redo Allocation Latch until the change is written.<br />
Oracle keeps statistics for these latches in v$latch, including the number of gets and misses for the Redo Allocation Latch and the number of immediate gets and immediate misses for the Redo Copy Latches, which are cumulative values since instance startup. If you've got a 100% hit ratio for either of those latch types, that's a good thing. It just means that all of your transactions were able to grab and use the latch without retrying. It's when you get below a 99% hit ratio that you need to start looking out. The following sql figures the current hit ratios for those latches:<br />
column latch_name format a20<br />
select name latch_name, gets, misses,<br />
round(decode(gets-misses,0,1,gets-misses)/<br />
decode(gets,0,1,gets),3) hit_ratio<br />
from v$latch where name = 'redo allocation';<br />
column latch_name format a20<br />
select name latch_name, immediate_gets, immediate_misses,<br />
round(decode(immediate_gets-immediate_misses,0,1,<br />
immediate_gets-immediate_misses)/<br />
decode(immediate_gets,0,1,immediate_gets),3) hit_ratio<br />
from v$latch where name = 'redo copy';<br />
<br />
If your Redo Allocation Latch hit ratio consistently falls below 99%, and if you have a multi-CPU machine, you can lower the value for log_small_entry_max_size (see below) in your init.ora file (ours is currently 800 bytes, but, maybe 100 or so bytes may be better - you'll have to try out different values over time), which says that any change smaller than that will hang onto the Redo Allocation Latch until Oracle is finished writing that change. Anything larger than that grabs a Redo Copy Latch, if currently available, and releases the Redo Allocation Latch for another transaction to use.<br />
<br />
If your Redo Copy Latch hit ratio consistently falls below 99%, and if you have a multi-CPU machine, you can raise the value of log_simultaneous_copies in your init.ora file up to twice the number of CPU's to provide more Redo Copy Latches (there is only one Redo Allocation Latch, so it is at a premium). Remember that you have to shut down your database instance and restart it to reread the new parameter values in the init.ora file ($ORACLE_HOME/dbs/initSID.ora). The following sql shows the current values for those associated parameters:<br />
<br />
column name format a30<br />
column value format a10<br />
select name,value from v$parameter where name in<br />
('log_small_entry_max_size','log_simultaneous_copies',<br />
'cpu_count');<br />
<br />
Database Buffer Cache Size <br />
The Database Buffer Cache is part of the Shared Global Area (SGA) in memory for a single database instance (SID) and holds the blocks of data and indexes that you and everyone else is currently using. It may even contain multiple copies of the same data block if, for example, more than one transaction is making changes to it but not yet committed, or, if you are looking at the original copy (select) and someone else is looking at their modified but uncommitted copy (insert, update, or delete). The parameters db_block_buffers and db_block_size in your init.ora file determine the size of the buffer cache. db_block_size, in bytes, is set at database creation, and cannot be changed (unless you recreate the database from scratch), so, the only thing that you can adjust is the number of blocks in db_block_buffers (one buffer holds one block).<br />
<br />
The Cache Hit Ratio shows how many blocks were already in memory (logical reads, which include "db block gets" for blocks you are using and "consistent gets" of original blocks from rollback segments that others are updating) versus how many blocks had to be read from disk ("physical reads"). Oracle recommends that this ratio be at least 80%, but, I like at least 90% myself. The ratio can be obtained from values in v$sysstat, which are constantly being updated and show statistics since database startup (it is only accessable from a DBA user account). You will get a more representative sample if the database has been running several hours with normal user transactions taking place. The Cache Hit Ratio is determined as follows:<br />
<br />
select (1-(pr.value/(dbg.value+cg.value)))*100<br />
from v$sysstat pr, v$sysstat dbg, v$sysstat cg<br />
where pr.name = 'physical reads'<br />
and dbg.name = 'db block gets'<br />
and cg.name = 'consistent gets';<br />
<br />
If you have a low Cache Hit Ratio, you can test to see what the effect of adding buffers would be by putting "db_block_lru_extended_statistics = 1000" in the init.ora file, doing a shutdown and startup of the database, and waiting a few hours to get a representative sample. Oracle determines how many Additional Cache Hits (ACH) would occur for each query and transaction for each of the 1000 buffer increments (or whatever other maximum value you might want to try out), and places them into the x$kcbrbh table, which is only accessable from user "sys". To measure the new Cache Hit Ratio with, for example, 100 extra buffers, determine ACH as follows:<br />
<br />
select sum(count) "ACH" from x$kcbrbh where indx < 100;
and plug that value into the Cache Hit Ratio formula as follows:
select (1-((pr.value-&ACH)/(dbg.value+cg.value)))*100
from v$sysstat pr, v$sysstat dbg, v$sysstat cg
where pr.name = 'physical reads'
and dbg.name = 'db block gets'
and cg.name = 'consistent gets';
If the ratio originally was lower than 80% and is now higher with ACH, you may want to increase db_block_buffers by that number of extra buffers, restarting your database to put the increase into effect. Be sure to try several values for the number of extra buffers to find an optimum for your work load. Also, remove db_block_lru_extended_statistics from your init.ora file before restarting your database to stop gathering statistics, which tends to slow down the transaction time. (Removing that clears the x$kcbrbh table.) Also, make sure that your server has enough memory to accomodate the increase!
If you are running really tight on memory, and the Cache Hit Ratio is running well above 80%, you might want to check the effect of lowering the number of buffers, which would release Oracle memory that could then be used by other processes, but would also potentially slow down database transactions. To test this, put "db_block_lru_statistics = true" in your init.ora file and restart your database. This gathers statistics for Additional Cache Misses (ACM) that would occur for each query and transaction for each of the buffer decrements up to the current db_block_buffers value, placing them into the x$kcbcbh table, also only accessable from user "sys". To measure the new Cache Hit Ratio with, for example, 100 fewer buffers, determine ACM as follows:
select sum(count) "ACM" from x$kcbcbh
where indx >= (select max(indx)+1-100 from x$kcbcbh);<br />
<br />
and plug that value into the Cache Hit Ratio formula as follows:<br />
<br />
select (1-((pr.value+&ACM)/(dbg.value+cg.value)))*100<br />
from v$sysstat pr, v$sysstat dbg, v$sysstat cg<br />
where pr.name = 'physical reads'<br />
and dbg.name = 'db block gets'<br />
and cg.name = 'consistent gets';<br />
<br />
If the ratio is still above 80%, you may want to decrease db_block_buffers by that number of fewer buffers, restarting your database to put the decrease into effect. Be sure to try several values for the number of fewer buffers to find an optimum for your work load. Also, remove db_block_lru_statistics from your init.ora file before restarting your database to stop gathering statistics, which tends to slow down the transaction time. (Removing that clears the x$kcbcbh table.)<br />
<br />
I have three scripts which you can use to figure your instance's optimum number of db_block_buffers. The cache_hit_ratio.sql script computes the current ratio for the database buffer cache, and can be run from any DBA account. The adding_buffers.sql script computes the resulting ratio for an increase in the buffer cache size of the given number of buffer blocks (figuring ACH itself). It must be run from user "sys", after a representative sampling time with db_block_lru_extended_statistics in place. The removing_buffers.sql script computes the resulting ratio for a decrease in the buffer cache size of the given number of buffer blocks (figuring ACM itself). It must be run from user "sys", after a representative sampling time with db_block_lru_statistics in place.<br />
<br />
Shared Pool Size <br />
<br />
The Shared Pool is also part of the Shared Global Area (SGA) in memory for a single database instance (SID) and holds the Library Cache with the most recently used SQL statements and parse trees along with PL/SQL blocks, and the Data Dictionary Cache with definitions of tables, views, and other dictionary objects. Both of those sets of cached objects can be used by one or more users, and are aged out (Least Recently Used) as other objects need the space. (You can pin large frequently-used objects in the Shared Pool for performance and other reasons, but, I won't go into that here.)<br />
<br />
There are several ratios that you can check after a representative sample time that may indicate that you need to enlarge the shared pool, which is set by the shared_pool_size parameter in your init.ora file and defaults to 3500000 (3.5 Meg). One indicator is the Library Cache Get Hit Ratio, which shows how many cursors are being shared (SQL statements (gets) which were already found and parsed (gethits) in the shared pool, with no parsing or re-parsing needed), and is determined by:<br />
<br />
select gethits,gets,gethitratio from v$librarycache<br />
where namespace = 'SQL AREA';<br />
<br />
If the gethitratio is less than 90%, you should consider increasing the shared pool size. Another indicator is the reloads per pin ratio, which shows how many parsed statements (pins) have been aged out (reloaded) of the shared pool for lack of space (ideally 0), and is determined by:<br />
<br />
select reloads,pins,reloads/pins from v$librarycache<br />
where namespace = 'SQL AREA';<br />
<br />
If the reloads/pins ratio is more than 1%, you should consider increasing the shared pool size. A third indicator, which is not as important as the first two, is the dictionary object getmisses per get ratio, which shows how many cached dictionary object definitions in the dictionary cache are encountering too many misses (aged out?), and is determined by:<br />
<br />
select sum(getmisses),sum(gets),sum(getmisses)/sum(gets)<br />
from v$rowcache;<br />
<br />
If the getmisses/gets ratio is more than 15%, you should consider increasing the shared pool size.<br />
<br />
If these ratios indicate that your shared pool is too small, you can estimate the size of the shared pool by doing the following. Set the shared_pool_size to a very large number, maybe a fourth or more of your system's available memory, depending on how many other instances and processes that you have running that are also using memory, then shutdown and startup your database and let it run for a representative time (like all day or when a large batch job is running that you want to accomodate), then, figure the memory required for packages and views, memory required for frequently used SQL statements, and memory required for users SQL statements executed, as shown below:<br />
<br />
select sum(sharable_mem) "Packages/Views" from v$db_object_cache;<br />
select sum(sharable_mem) "SQL Statements" from v$sqlarea<br />
where executions > 5;<br />
select sum(250 * users_opening) "SQL Users" from v$sqlarea;<br />
<br />
Then, add the above three numbers and multiply the results by 2.5. Use this estimated size as a guideline for the value for shared_pool_size, changing that parameter to the estimated size or back to the original size and doing another shutdown/startup to put the value into effect. The shared_pool_size.sql script can be used to figure these values for you, which uses an example of the Select From Selects tip:<br />
<br />
select sum(a.spspv) "Packages/Views", sum(a.spssql) "SQL Statements",<br />
sum(a.spsusr) "SQL Users", round((sum(a.spspv) + sum(a.spssql) +<br />
sum(a.spsusr)) * 2.5,-6) "Estimated shared_pool_size"<br />
from (select sum(sharable_mem) spspv, 0 spssql, 0 spsusr<br />
from v$db_object_cache<br />
union all<br />
select 0, sum(sharable_mem), 0 from v$sqlarea<br />
where executions > 5<br />
union all<br />
select 0, 0, sum(250 * users_opening) from v$sqlarea) a;<br />
<br />
<br />
set echo off<br />
rem<br />
rem Script: tuning.sql<br />
rem<br />
rem Purpose: Check various statistics for the currently-running database<br />
rem to see if there are any database parameters or other modifications<br />
rem you could make to tune the database for faster response.<br />
rem<br />
rem Author: David Midgett <Dave.Midgett@EKU.EDU><br />
rem Eastern Kentucky University<br />
rem<br />
rem Modifications: Stephen Rea <srea@uaex.edu><br />
rem University of Arkansas Cooperative Extension Service<br />
rem<br />
rem Updates:<br />
rem 7/16/03 - Added database SID to init.ora file names in output. Show<br />
rem current values of init.ora parameters and other values in question. <br />
rem Branch around multi-threaded server output if MTS not being used.<br />
rem Cleaned up the formatting of the output for consistency.<br />
rem<br />
set feedback off<br />
column SID new_value SID<br />
select substr(substr(global_name,1,30),1,instr(substr(global_name,1,30),'.')-1) SID<br />
from global_name;<br />
prompt<br />
rem Bypass multi-threaded server checks if there aren't any mts servers.<br />
set termout off heading off<br />
spool tmp~~~.sql<br />
select '/*' from v$parameter where name = 'mts_servers' and value = '0';<br />
spool off<br />
@tmp~~~.sql<br />
set termout on heading on<br />
prompt ================================================================================;<br />
prompt<br />
prompt . Tuning Multi-Threaded Server<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt If SERVERS_HIGHWATER exceeds the MTS_SERVERS parameter, increase the number of<br />
prompt MTS_SERVERS in the init&SID..ora file.<br />
set heading off<br />
select 'Current MTS_SERVERS number is ' || value from v$parameter where name = 'mts_servers';<br />
set heading on<br />
select * from v$mts;<br />
-- */<br />
!rm tmp~~~.sql<br />
set termout on heading on<br />
prompt ================================================================================;<br />
prompt<br />
prompt . Tuning The Library Cache<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt Library cache get/hit ratio for SQL AREA should be in high 90's. If not, there<br />
prompt is room to improve the efficiency of your application.<br />
select namespace, gets, gethits, gethitratio from v$librarycache;<br />
prompt<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt If reloads-to-pins ratio is greater than .01, increase SHARED_POOL_SIZE in<br />
prompt the init&SID..ora file.<br />
set heading off<br />
select 'Current SHARED_POOL_SIZE value is ' || value from v$parameter where name = 'shared_pool_size';<br />
set heading on<br />
select sum(pins) "Executions", sum(reloads) "LC Misses",<br />
sum(reloads)/sum(pins) "Ratio" from v$librarycache;<br />
prompt<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt Data Dictionary Cache -- If ratio is greater than .15, consider increasing<br />
prompt SHARED_POOL_SIZE in the init&SID..ora file.<br />
set heading off<br />
select 'Current SHARED_POOL_SIZE value is ' || value from v$parameter where name = 'shared_pool_size';<br />
set heading on<br />
select sum(gets) "Total Gets", sum(getmisses) "Total Get Misses",<br />
sum(getmisses)/sum(gets) "Ratio" from v$rowcache;<br />
prompt<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt Packages you might want to consider pinning into the shared pool:<br />
column owner format a12<br />
column name format a25<br />
column type format a15<br />
set feedback on<br />
select owner, name, type, loads, executions, sharable_mem<br />
from v$db_object_cache<br />
where kept = 'NO'<br />
and loads > 1 and executions > 50 and sharable_mem > 10000<br />
and type in ('PACKAGE', 'PACKAGE BODY', 'FUNCTION', 'PROCEDURE')<br />
order by loads desc;<br />
set feedback off<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt Shared Pool Reserved space -- The goal is to have zero REQUEST_MISSES and<br />
prompt REQUEST_FAILURES, so increase SHARED_POOL_RESERVED_SIZE in the init&SID..ora<br />
prompt file if either of them are greater than 0.<br />
set heading off<br />
select 'Current SHARED_POOL_RESERVED_SIZE value is ' || value from v$parameter where name = 'shared_pool_reserved_size';<br />
set heading on<br />
select request_misses, request_failures, last_failure_size<br />
from v$shared_pool_reserved;<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt . Tuning the Data Dictionary Cache<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt If the gethitratio is greater than .15 -- increase the SHARED_POOL_SIZE<br />
prompt parameter in the init&SID..ora file.<br />
set heading off<br />
select 'Current SHARED_POOL_SIZE value is ' || value from v$parameter where name = 'shared_pool_size';<br />
set heading on<br />
select sum(gets) "totl_gets", sum(getmisses) "totl_get_misses",<br />
sum(getmisses)/sum(gets) * 100 "gethitratio"<br />
from v$rowcache;<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt . Tuning The Data Buffer Cache<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt Goal is to have a Cache Hit Ratio greater than 90% -- if lower, increase value<br />
prompt for DB_BLOCK_BUFFERS in the init&SID..ora file.<br />
set heading off<br />
select 'Current DB_BLOCK_BUFFERS value is ' || value from v$parameter where name = 'db_block_buffers';<br />
set heading on<br />
column value format 999,999,999,999<br />
select name, value from v$sysstat where<br />
name in ('db block gets', 'consistent gets', 'physical reads');<br />
select 1 - (phy.value / (cur.value + con.value)) "Cache Hit Ratio"<br />
from v$sysstat cur, v$sysstat con, v$sysstat phy<br />
where cur.name = 'db block gets'<br />
and con.name = 'consistent gets'<br />
and phy.name = 'physical reads';<br />
prompt<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt If the number of free buffers inspected is high or increasing, consider<br />
prompt increasing the DB_BLOCK_BUFFERS parameter in the init&SID..ora file.<br />
set heading off<br />
select 'Current DB_BLOCK_BUFFERS value is ' || value from v$parameter where name = 'db_block_buffers';<br />
set heading on<br />
column value format 999,999,999<br />
select name, value from v$sysstat where name = 'free buffer inspected';<br />
prompt<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt A high or increasing number of waits indicates that the db writer cannot keep<br />
prompt up writing dirty buffers. Consider increasing the number of writers using the<br />
prompt DB_WRITER_PROCESSES parameter in the init&SID..ora file.<br />
set heading off<br />
select 'Current DB_WRITER_PROCESSES value is ' || value from v$parameter where name = 'db_writer_processes';<br />
set heading on<br />
select event, total_waits from v$system_event where event in<br />
('free buffer waits', 'buffer busy waits');<br />
prompt<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt If the LRU Hit percentage is less than 99%, consider adding more<br />
prompt DB_WRITER_PROCESSES and increasing the DB_BLOCK_LRU_LATCHES parameter<br />
prompt in the init&SID..ora file.<br />
set heading off<br />
select 'Current DB_WRITER_PROCESSES value is ' || v1.value || chr(10) ||<br />
'Current DB_BLOCK_LRU_LATCHES value is ' || v2.value<br />
from v$parameter v1,v$parameter v2<br />
where v1.name = 'db_writer_processes' and v2.name = 'db_block_lru_latches';<br />
set heading on<br />
select name, 100 - (sleeps/gets * 100) "LRU Hit%" from v$latch<br />
where name = 'cache buffers lru chain';<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt . Tuning The Redo Log Buffer<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt Ideally, there should never be a wait for log buffer space. Increase LOG_BUFFER<br />
prompt in the init&SID..ora file if the selection below doesn't show "no rows selected".<br />
set heading off<br />
select 'Current LOG_BUFFER size is ' || value from v$parameter where name = 'log_buffer';<br />
set heading on<br />
set feedback on<br />
select sid, event, state from v$session_wait<br />
where event = 'log buffer space';<br />
set feedback off<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt If there are any Wait events because of log switches, consider increasing the<br />
prompt size of the redo log files.<br />
set heading off<br />
select 'Current size of redo log files is ' || bytes || ' bytes' from v$log where rownum = 1;<br />
set heading on<br />
column event format a30<br />
select event, total_waits, time_waited, average_wait from v$system_event<br />
where event like 'log file switch completion%';<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt Tables with Chain count greater than 10% of the number of rows:<br />
set feedback on<br />
select owner, table_name, num_rows, chain_cnt, chain_cnt/num_rows "Percent"<br />
from dba_tables where (chain_cnt/num_rows) > .1 and num_rows > 0;<br />
set feedback off<br />
prompt ================================================================================;<br />
prompt<br />
prompt . Tuning Sorts<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt The ratio of disk sorts to memory sorts should be less than 5%. Consider<br />
prompt increasing the SORT_AREA_SIZE parameter in the init&SID..ora file. You<br />
prompt might also consider setting up separate temp tablespaces for frequent<br />
prompt users of disk sorts.<br />
set heading off<br />
select 'Current SORT_AREA_SIZE value is ' || value from v$parameter where name = 'sort_area_size';<br />
set heading on<br />
select disk.value "Disk", mem.value "Mem", (disk.value/mem.value) * 100 "Ratio"<br />
from v$sysstat mem, v$sysstat disk<br />
where mem.name = 'sorts (memory)'<br />
and disk.name = 'sorts (disk)';<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt . Tuning Rollback segments<br />
prompt<br />
prompt ================================================================================;<br />
prompt<br />
prompt If ratio of waits to gets is greater than 1%, you need more rbs segments.<br />
set heading off<br />
select 'Current number of rollback segments is ' || count(*) from dba_rollback_segs<br />
where status = 'ONLINE' and owner = 'PUBLIC';<br />
set heading on<br />
select sum(waits)*100/sum(gets) "Ratio", sum(waits) "Waits", sum(gets) "Gets"<br />
from v$rollstat;<br />
prompt<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt Rollback segment waits -- any waits indicates need for more segments.<br />
set heading off<br />
select 'Current number of rollback segments is ' || count(*) from dba_rollback_segs<br />
where status = 'ONLINE' and owner = 'PUBLIC';<br />
set heading on<br />
select * from v$waitstat where class = 'undo header';<br />
column event format a25<br />
prompt<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt Rollback segment waits for transaction slots -- any waits indicates need for<br />
prompt more segments.<br />
set heading off<br />
select 'Current number of rollback segments is ' || count(*) from dba_rollback_segs<br />
where status = 'ONLINE' and owner = 'PUBLIC';<br />
set heading on<br />
set feedback on<br />
select * from v$system_event where event = 'undo segment tx slot';<br />
set feedback off<br />
prompt --------------------------------------------------------------------------------;<br />
prompt<br />
prompt Rollback contention -- should be zero for all rbs's.<br />
column name format a10<br />
select n.name,round (100 * s.waits/s.gets) "%contention"<br />
from v$rollname n, v$rollstat s<br />
where n.usn = s.usn;<br />
prompt<br />
clear columns<br />
set feedback on echo on<br />
<br />
<br />
<br />
<br />
-- From http://www.oramag.com/code/cod46dba.html <br />
prompt****************************************************<br />
prompt Hit Ratio Section <br />
prompt****************************************************<br />
prompt <br />
prompt ========================= <br />
prompt BUFFER HIT RATIO <br />
prompt ========================= <br />
prompt (should be > 70, else increase db_block_buffers in init.ora) <br />
<br />
--select trunc((1-(sum(decode(name,'physical reads',value,0))/ <br />
-- (sum(decode(name,'db block gets',value,0))+ <br />
-- (sum(decode(name,'consistent gets',value,0))))) <br />
-- )* 100) "Buffer Hit Ratio" <br />
--from v$sysstat; <br />
<br />
column "logical_reads" format 99,999,999,999 <br />
column "phys_reads" format 999,999,999 <br />
column "phy_writes" format 999,999,999 <br />
select a.value + b.value "logical_reads", <br />
c.value "phys_reads", <br />
d.value "phy_writes", <br />
round(100 * ((a.value+b.value)-c.value) / <br />
(a.value+b.value)) <br />
"BUFFER HIT RATIO" <br />
from v$sysstat a, v$sysstat b, v$sysstat c, v$sysstat d <br />
where <br />
a.statistic# = 37 <br />
and <br />
b.statistic# = 38 <br />
and <br />
c.statistic# = 39 <br />
and <br />
d.statistic# = 40; <br />
<br />
prompt <br />
prompt <br />
prompt ========================= <br />
prompt DATA DICT HIT RATIO <br />
prompt ========================= <br />
prompt (should be higher than 90 else increase shared_pool_size in init.ora) <br />
prompt <br />
<br />
column "Data Dict. Gets" format 999,999,999 <br />
column "Data Dict. cache misses" format 999,999,999 <br />
select sum(gets) "Data Dict. Gets", <br />
sum(getmisses) "Data Dict. cache misses", <br />
trunc((1-(sum(getmisses)/sum(gets)))*100) "DATA DICT CACHE HIT <br />
RATIO" <br />
from v$rowcache; <br />
prompt <br />
prompt ========================= <br />
prompt LIBRARY CACHE MISS RATIO <br />
prompt ========================= <br />
prompt (If > .1, i.e., more than 1% of the pins resulted in reloads, then increase the shared_pool_size in init.ora) <br />
prompt <br />
column "LIBRARY CACHE MISS RATIO" format 99.9999 <br />
column "executions" format 999,999,999 <br />
column "Cache misses while executing" format 999,999,999 <br />
select sum(pins) "executions", sum(reloads) "Cache misses while executing", <br />
(((sum(reloads)/sum(pins)))) "LIBRARY CACHE MISS RATIO" <br />
from v$librarycache; <br />
<br />
prompt <br />
prompt ========================= <br />
prompt Library Cache Section <br />
prompt ========================= <br />
prompt hit ratio should be > 70, and pin ratio > 70 ... <br />
prompt <br />
<br />
column "reloads" format 999,999,999 <br />
select namespace, trunc(gethitratio * 100) "Hit ratio", <br />
trunc(pinhitratio * 100) "pin hit ratio", reloads "reloads" <br />
from v$librarycache; <br />
prompt <br />
prompt <br />
prompt ========================= <br />
prompt REDO LOG BUFFER <br />
prompt ========================= <br />
prompt <br />
set heading off <br />
column value format 999,999,999 <br />
select substr(name,1,30), <br />
value <br />
from v$sysstat where name = 'redo log space requests'; <br />
<br />
set heading on <br />
prompt <br />
prompt <br />
prompt <br />
<br />
column bytes format 999,999,999 <br />
select name, bytes from v$sgastat where name = 'free memory'; <br />
<br />
prompt <br />
prompt**************************************************** <br />
prompt SQL Summary Section <br />
prompt**************************************************** <br />
prompt <br />
column "Tot SQL run since startup" format 999,999,999 <br />
column "SQL executing now" format 999,999,999 <br />
select sum(executions) "Tot SQL run since startup", <br />
sum(users_executing) "SQL executing now" <br />
from v$sqlarea; <br />
<br />
prompt <br />
prompt <br />
prompt**************************************************** <br />
prompt Lock Section <br />
prompt**************************************************** <br />
prompt <br />
prompt ========================= <br />
prompt SYSTEM-WIDE LOCKS - all requests for locks or latches <br />
prompt ========================= <br />
prompt <br />
select substr(username,1,12) "User", <br />
substr(lock_type,1,18) "Lock Type", <br />
substr(mode_held,1,18) "Mode Held" <br />
from sys.dba_lock a, v$session b <br />
where lock_type not in ('Media Recovery','Redo Thread') <br />
and a.session_id = b.sid; <br />
prompt <br />
prompt ========================= <br />
prompt DDL LOCKS - These are usually triggers or other DDL <br />
prompt ========================= <br />
prompt <br />
select substr(username,1,12) "User", <br />
substr(owner,1,8) "Owner", <br />
substr(name,1,15) "Name", <br />
substr(a.type,1,20) "Type", <br />
substr(mode_held,1,11) "Mode held" <br />
from sys.dba_ddl_locks a, v$session b <br />
where a.session_id = b.sid; <br />
<br />
prompt <br />
prompt ========================= <br />
prompt DML LOCKS - These are table and row locks... <br />
prompt ========================= <br />
prompt <br />
select substr(username,1,12) "User", <br />
substr(owner,1,8) "Owner", <br />
substr(name,1,20) "Name", <br />
substr(mode_held,1,21) "Mode held" <br />
from sys.dba_dml_locks a, v$session b <br />
where a.session_id = b.sid; <br />
<br />
prompt <br />
prompt <br />
prompt**************************************************** <br />
prompt Latch Section <br />
prompt**************************************************** <br />
prompt if miss_ratio or immediate_miss_ratio > 1 then latch <br />
prompt contention exists, decrease LOG_SMALL_ENTRY_MAX_SIZE in init.ora <br />
prompt <br />
column "miss_ratio" format .99 <br />
column "immediate_miss_ratio" format .99 <br />
select substr(l.name,1,30) name, <br />
(misses/(gets+.001))*100 "miss_ratio", <br />
(immediate_misses/(immediate_gets+.001))*100 <br />
"immediate_miss_ratio" <br />
from v$latch l, v$latchname ln <br />
where l.latch# = ln.latch# <br />
and ( <br />
(misses/(gets+.001))*100 > .2 <br />
or <br />
(immediate_misses/(immediate_gets+.001))*100 > .2 <br />
) <br />
order by l.name; <br />
<br />
prompt <br />
prompt <br />
prompt**************************************************** <br />
prompt Rollback Segment Section <br />
prompt**************************************************** <br />
prompt if any count below is > 1% of the total number of requests for data <br />
prompt then more rollback segments are needed <br />
--column count format 999,999,999 <br />
select class, count <br />
from v$waitstat <br />
where class in ('free list','system undo header','system undo block', <br />
'undo header','undo block') <br />
group by class,count; <br />
<br />
column "Tot # of Requests for Data" format 999,999,999 <br />
select sum(value) "Tot # of Requests for Data" from v$sys stat where <br />
name in ('db block gets', 'consistent gets'); <br />
prompt <br />
prompt ========================= <br />
prompt ROLLBACK SEGMENT CONTENTION <br />
prompt ========================= <br />
prompt <br />
prompt If any ratio is > .01 then more rollback segments are needed <br />
<br />
column "Ratio" format 99.99999 <br />
select name, waits, gets, waits/gets "Ratio" <br />
from v$rollstat a, v$rollname b <br />
where a.usn = b.usn; <br />
<br />
column "total_waits" format 999,999,999 <br />
column "total_timeouts" format 999,999,999 <br />
prompt <br />
prompt <br />
set feedback on; <br />
prompt**************************************************** <br />
prompt Session Event Section <br />
prompt**************************************************** <br />
prompt if average-wait > 0 then contention exists <br />
prompt <br />
select substr(event,1,30) event, <br />
total_waits, total_timeouts, average_wait <br />
from v$session_event <br />
where average_wait > 0 ; <br />
--or total_timeouts > 0; <br />
<br />
prompt <br />
prompt <br />
prompt**************************************************** <br />
prompt Queue Section <br />
prompt**************************************************** <br />
prompt average wait for queues should be near zero ... <br />
prompt <br />
column "totalq" format 999,999,999 <br />
column "# queued" format 999,999,999 <br />
select paddr, type "Queue type", queued "# queued", wait, totalq, <br />
decode(totalq,0,0,wait/totalq) "AVG WAIT" from v$queue; <br />
<br />
set feedback on; <br />
prompt <br />
prompt <br />
--prompt**************************************************** <br />
--prompt Multi-threaded Server Section <br />
--prompt**************************************************** <br />
--prompt <br />
--prompt If the following number is > 1 <br />
--prompt then increase MTS_MAX_SERVERS parm in init.ora <br />
--prompt <br />
-- select decode( totalq, 0, 'No Requests', <br />
-- wait/totalq || ' hundredths of seconds') <br />
-- "Avg wait per request queue" <br />
-- from v$queue <br />
-- where type = 'COMMON'; <br />
<br />
--prompt <br />
--prompt If the following number increases, consider adding dispatcher processes <br />
<br />
--prompt <br />
-- select decode( sum(totalq), 0, 'No Responses', <br />
-- sum(wait)/sum(totalq) || ' hundredths of seconds') <br />
-- "Avg wait per response queue" <br />
-- from v$queue q, v$dispatcher d <br />
-- where q.type = 'DISPATCHER' <br />
-- and q.paddr = d.paddr; <br />
<br />
--set feedback off; <br />
--prompt <br />
--prompt <br />
--prompt ========================= <br />
--prompt DISPATCHER USAGE <br />
--prompt ========================= <br />
--prompt (If Time Busy > 50, then change<br />
MTS_MAX_DISPATCHERS in init.ora) <br />
--column "Time Busy" format 999,999.999 <br />
--column busy format 999,999,999 <br />
--column idle format 999,999,999 <br />
--select name, status, idle, busy, <br />
-- (busy/(busy+idle))*100 "Time Busy" <br />
--from v$dispatcher; <br />
<br />
--prompt <br />
--prompt <br />
--select count(*) "Shared Server Processes" <br />
-- from v$shared_server <br />
-- where status = 'QUIT'; <br />
<br />
--prompt <br />
--prompt <br />
--prompt high-water mark for the multi-threaded server <br />
--prompt <br />
<br />
--select * from v$mts; <br />
<br />
--prompt <br />
--prompt**************************************************** <br />
--prompt file i/o should be evenly distributed across drives. <br />
--prompt <br />
<br />
--select <br />
--substr(a.file#,1,2) "#", <br />
--substr(a.name,1,30) "Name", <br />
--a.status, <br />
--a.bytes, <br />
--b.phyrds, <br />
--b.phywrts <br />
--from v$datafile a, v$filestat b <br />
--where a.file# = b.file#; <br />
<br />
--select substr(name,1,55) system_statistic, value <br />
-- from v$sysstat <br />
-- order by name; <br />
<br />
Description: The output of this script will display all sql statements in the shared pool that are doing full table scans.<br />
Code:<br />
--run this as sys in SQL worksheet<br />
create table full_sql (sql_text varchar2(1000), executions number);<br />
create or replace procedure p_findfullsql as<br />
<br />
v_csr number;<br />
v_rc number;<br />
v_string varchar2(2000);<br />
<br />
v_count number;<br />
<br />
<br />
cursor c1 is select sql_text,executions from v$sqlarea where lower(sql_text) like '%select%';<br />
<br />
begin<br />
<br />
for x1 in c1 loop<br />
<br />
delete from plan_table ;<br />
Begin<br />
v_Csr := DBMS_SQL.OPEN_CURSOR; <br />
v_string := 'explain plan for ' ;<br />
v_string := v_string||x1.sql_text ;<br />
DBMS_SQL.PARSE(v_csr, v_string, DBMS_SQL.V7);<br />
v_rc := DBMS_SQL.EXECUTE(v_csr);<br />
DBMS_SQL.CLOSE_CURSOR(v_csr); <br />
Exception<br />
when others then <br />
null; <br />
End ;<br />
<br />
select count(*) into v_count from plan_table where options like '%FULL%' and operation like '%TABLE%' ;<br />
if v_count > 0 then<br />
insert into full_sql(sql_text,executions) values (x1.sql_text, x1.executions) ;<br />
end if;<br />
end loop ;<br />
commit;<br />
end ;<br />
/<br />
execute p_findfullsql ;<br />
select * from full_sql;<br />
drop table full_sql;<br />
Description: Find the top 20 longest running processes in unix. Useful for high CPU bound systems with large number of users. Script also identifies processes without a oracle session.<br />
Code:<br />
#!/bin/ksh<br />
#<br />
# Find Highest CPU used Oracle processes and get the Username <br />
# and SID from oracle<br />
# Only 3 character SIDNAME is displayed - Adjust the script according to your need.<br />
#<br />
date<br />
echo " Top 20 CPU Utilized Session from `hostname`"<br />
echo " ============================================"<br />
echo "O/S Oracle Session Session Serial UNIX Login Ora CPU Time"<br />
echo "ID User ID Status ID No ID MMDD:HHMISS SID Used"<br />
echo "-------- ----------- -------- ------- ------- ------- ----------- --- --------"<br />
ps -ef|grep LOCAL|cut -c1-15,42-79|sort -rn +2 | head -20 | while read LINE<br />
do<br />
SIDNAME=`echo $LINE | awk '{ print $4 }' | cut -c7-14`<br />
CPUTIME=`echo $LINE | awk '{ print $3 }'`<br />
UNIXPID=`echo $LINE | awk '{ print $2 }'`<br />
#echo $SIDNAME $CPUTIME $UNIXPID<br />
export ORACLE_SID=$SIDNAME<br />
SIDNAME=`echo $ORACLE_SID | cut -c4-6`<br />
export ORACLE_HOME=`/dba_script/bin/find_ohome.sh ${ORACLE_SID}`<br />
export SHLIB_PATH=$ORACLE_HOME/lib:/usr/lib<br />
export TMPDIR=/tmp<br />
export LD_LIBRARY_PATH=$ORACLE_HOME/lib<br />
export SQLPLUS="$ORACLE_HOME/bin/sqlplus -s / "<br />
$SQLPLUS >> $wlogfile <
set pages 0 lines 80 trims on echo off verify off pau off
column pu format a8 heading 'O/S|ID' justify left
column su format a11 heading 'Oracle|User ID' justify left
column stat format a8 heading 'Session|Status' justify left
column ssid format 999999 heading 'Session|ID' justify right
column sser format 999999 heading 'Serial|No' justify right
column spid format 999999 heading 'UNIX|ID' justify right
column ltime format a11 heading 'Login|Time' justify right
select p.username pu,
s.username su,
s.status stat,
s.sid ssid,
s.serial# sser,
lpad(p.spid,7) spid,
to_char(s.logon_time, 'MMDD:HH24MISS') ltime,
'$SIDNAME $CPUTIME'
from v\$process p,
v\$session s
where p.addr=s.paddr
and p.spid=$UNIXPID
union all
select a.username, 'Kill Me', 'NoOracle', a.pid, a.serial#,
lpad(a.spid,7) spid, 'KILL UNIXID', '$SIDNAME $CPUTIME'
from v\$process a
where a.spid = $UNIXPID
and not exists (select 1 from v\$session s
where a.addr=s.paddr);
EOF
done
echo "-------- ----------- -------- ------- ------- ------- ----------- --- --------"
date
#
# End of Script
Description: Monitor and Verify DEAD Locks: Holding and Waiting Sessions
Code:
Doc
Verify DEAD LOCK situation
Script: waiters.sql
Date: 07/1999
Revision:
#
set lines 80 echo on ver off timing on term on pages 60 feed on head on
spool DEAD_LOCK_WAITERS.LST
col " " for A25
col "Holding Session Info" for A25
col "Waiting Session Info" for A25
select --+ ORDERED
'Session ID' || CHR(10) ||
'Mode Held' || CHR(10) ||
'Lock Type' || CHR(10) ||
'Mode Requested' || CHR(10) ||
'Lock ID 1' || CHR(10) ||
'Lock ID 2' " "
-------------------------------- END of Header
,
HH.session_id || CHR(10) ||
HH.mode_held || CHR(10) ||
HH.lock_type || CHR(10) ||
HH.mode_requested || CHR(10) ||
HH.lock_id1 || CHR(10) ||
HH.lock_id2 "Holding Session Info"
------------------------------ END of Holding Session
,
Ww.session_id || CHR(10) ||
WW.mode_held || CHR(10) ||
Ww.lock_type || CHR(10) ||
Ww.mode_requested || CHR(10) ||
Ww.lock_id1 || CHR(10) ||
Ww.lock_id2 "Waiting Session Info"
------------------------------ END of Waiting Session
from
-----------------------------------------------------
(
select /*+ RULE */ *
from SYS.dba_locks
where blocking_others = 'Blocking' and
mode_held != 'None' and
mode_held != 'Null'
) HH,
-----------------------------------------------------
(
select /*+ RULE */ *
from SYS.dba_locks
where mode_requested != 'None'
) WW
-----------------------------------------------------
where WW.lock_type = HH.lock_type and
WW.lock_id1 = HH.lock_id1 and
WW.lock_id2 = HH.lock_id2
;
spool off
Description: An interesting script submitted by one of our readers to size the SGA
Code:
set serverout on
DECLARE
l_uplift CONSTANT NUMBER := 0.3;
/* i.e. 30% above calculated */
l_numusers NUMBER DEFAULT 50;
/* Change ths to a predicted number existing database */
l_avg_uga NUMBER;
l_max_uga NUMBER;
l_sum_sql_shmem NUMBER;
l_sum_obj_shmem NUMBER;
l_total_avg NUMBER;
l_total_max NUMBER;
BEGIN
dbms_output.enable(20000);
IF ( l_numusers = 0) THEN
SELECT sessions_highwater
INTO l_numusers
FROM v$license;
dbms_output.put_line('Maximum concurrent users on this database = '||TO_CHAR(l_numusers));
ELSE
dbms_output.put_line('Calculating SGA for = '||TO_CHAR(l_numusers)||' concurrent users');
END IF;
dbms_output.new_line;
SELECT
avg(value)*l_numusers
,max(value)*l_numusers
INTO l_avg_uga, l_max_uga
FROM v$sesstat s, v$statname n
WHERE s.statistic# = n.statistic#
AND n.name = 'session uga memory max';
SELECT sum(sharable_mem) INTO l_sum_sql_shmem FROM v$sqlarea;
SELECT sum(sharable_mem) INTO l_sum_obj_shmem FROM v$db_object_cache;
l_total_avg := l_avg_uga + l_sum_sql_shmem + l_sum_obj_shmem;
l_total_max := l_max_uga + l_sum_sql_shmem + l_sum_obj_shmem;
dbms_output.put_line('Recommended Shared_pool size between :' || TO_CHAR(ROUND(l_total_avg + (l_total_avg * l_uplift), 0) ) ||' and '
|| TO_CHAR(ROUND(l_total_max + (l_total_max * l_uplift), 0) ) ||' bytes');
dbms_output.put_line('Recommended Shared_pool size between :' || TO_CHAR(ROUND((l_total_avg + (l_total_avg * l_uplift)) /(1024*1024), 0) )
||' and '|| TO_CHAR(ROUND((l_total_max + (l_total_max * l_uplift )) /1024*1024) ,0) ) ||' M bytes');
end;
/
Description: This script uses the new way of calculating DB_Block Buffer Efficiency. Logical Read: consistent gets + db block gets
Hit-Ratio: (logical reads - physical reads) / (logical reads)
Our Aim: OLTP >= 95%, DSS/Batch >= 85%<br />
Solution: enlarge block buffers, tune SQL, check appropriateness of indexes<br />
Code:<br />
select name, ((consistent_gets + db_block_gets) - physical_reads) /<br />
(consistent_gets + db_block_gets) * 100 "Hit Ratio%"<br />
from v$buffer_pool_statistics<br />
where physical_reads > 0;<br />
<br />
Description: Haven't you ever thought there should be an easier way to do the EXPLAIN PLAN and TKPROF statistics than to edit your<br />
queries to add the commands (like EXPLAIN PLAN SET...), or to have to find or write a script that automates this? It should<br />
be an automatic part of SQL*Plus. Well, as of SQL*Plus 3.3 it is!! The command is called 'SET AUTOTRACE ON'!<br />
Code:<br />
The SET AUTOTRACE Command<br />
<br />
In SQL*Plus 3.3 there is a little known command (at least I didn't know about it until recently) called SET AUTOTRACE. It is<br />
documented in the newest SQL*Plus document set, but who really reads the whole document set for changes? Well I did not. It<br />
is very simple to use. Just type the command: <br />
<br />
<br />
SET AUTOTRACE ON<br />
<br />
And then run your select statement. Example: <br />
<br />
<br />
SQL> SET AUTOTRACE ON<br />
SQL> SELECT d.deptno, d.dname, e.empno, e.ename<br />
2 FROM dept d, emp e<br />
3 WHERE d.deptno = e.deptno<br />
4 /<br />
<br />
DEPTNO DNAME EMPNO ENAME <br />
---------- -------------- ---------- ---------- <br />
10 ACCOUNTING 7839 KING <br />
.<br />
. <br />
30 SALES 7900 JAMES <br />
30 SALES 7521 WARD <br />
<br />
14 rows selected.<br />
<br />
Execution Plan<br />
---------------------------------------------------------- <br />
0 SELECT STATEMENT Optimizer=CHOOSE <br />
1 0 MERGE JOIN <br />
2 1 SORT (JOIN) <br />
3 2 TABLE ACCESS (FULL) OF 'EMP' <br />
4 1 SORT (JOIN) <br />
5 4 TABLE ACCESS (FULL) OF 'DEPT' <br />
<br />
Statistics<br />
---------------------------------------------------------- <br />
0 recursive calls <br />
4 db block gets <br />
2 consistent gets <br />
0 physical reads <br />
0 redo size <br />
670 bytes sent via SQL*Net to client <br />
376 bytes received via SQL*Net from client <br />
3 SQL*Net roundtrips to/from client <br />
2 sorts (memory) <br />
0 sorts (disk) <br />
14 rows processed <br />
<br />
<br />
There are also some other options, for example there is a TRACEONLY option which supresses the SQL output. See the<br />
SQL*Plus 3.3 manual for a full description. <br />
<br />
Some setup issues:<br />
<br />
If you go off and try this on your instance, you may run into some problems. There are a few setup steps that need to be taken to<br />
make this work: <br />
<br />
1.Make sure you have access to PLAN_TABLE. If you don't, create it using utlxplan.sql (It should be in a directory like<br />
$ORACLE_HOME/rdbms73/admin/) and make sure you have access to it from the user you are using to tune the SQL. <br />
2.You also need to create the PLUSTRACE role, and grant it to the desired users. The script to create this role is in:<br />
<br />
$ORACLE_HOME/plus33/Plustrce.sql<br />
<br />
It has to be run from SYS in order to have the correct security access. Then grant the role to the desired users or ROLEs.<br />
<br />
Health Check <br />
Description: Look at the statistics of your Database<br />
Code:<br />
REM Database Health monitoring script.<br />
REM<br />
REM Segments Max extents & Current extent comparison<br />
REM<br />
set line 180<br />
set pagesize 10000<br />
set feedback off<br />
prompt<br />
col Time format a50 heading "System Time"<br />
select to_char(sysdate,'DD-MON-YYYY:HH24:MI:SS') Time from dual;<br />
prompt<br />
prompt<br />
prompt<br />
prompt**---------------Objects Reaching Max extents-----------------------**<br />
prompt<br />
col segment_name format a40 heading "Object Name"<br />
col max_extents format 9999999999 heading "Max Extents"<br />
col curr_extents format 99999999999 heading "Curent Extents"<br />
select a.segment_name,a.max_extents,b.curr_extents from dba_segments a,(select<br />
segment_name,max(extent_id) curr_extents from dba_extents group by segment_name) b where a.segment_name = b.segment_name<br />
and (a.max_extents - b.curr_extents) <= 10;
prompt
prompt**---------------User Session Information----------------------------**
prompt
col sid format 9999 heading "SID"
col serial# format 9999999 heading "SERIAL NO"
col logon_time format 9999999 heading "Login Time"
col username format a12
col osuser format a24
col program format a38
select s.sid,s.serial#,p.spid,s.username,s.osuser,s.program,to_char(s.LOGON_TIME,'DD-MON-YY:HH24:MI:SS') "Log on Time",round((s.LAST_CALL_ET/(60*60*24)),2)"Wait in Days" from v$session s,v$process p where s.paddr = p.addr and s.username is not null order by 8 desc;
prompt
prompt**---------------File Information-------------------------------------**
prompt
col file_name format a55 heading "File Name"
col SizeInMB format 99999999 heading "Total Size (MB)"
col MAXSIZE format 99999999 heading "Maximum Size (MB)"
select file_name,BYTES/(1024*1024) SizeInMB,AUTOEXTENSIBLE,MAXBYTES/(1024*1024) MAXSIZE from dba_data_files;
prompt
prompt**---------------Tablespace Information-------------------------------**
prompt
col tablespace_name format a25 heading "Tablespace Name"
col logging format a10
col status format a12
col extent_management format a30 heading "Local/Dict"
select tablespace_name,status,contents,decode(logging,LOGGING,'YES','NO') Logging,extent_management from dba_tablespaces;
SELECT Total.name "Tablespace Name",
Free_space, (total_space-Free_space) Used_space, total_space
FROM
(select tablespace_name, sum(bytes/1024/1024) Free_Space
from sys.dba_free_space
group by tablespace_name
) Free,
(select b.name, sum(bytes/1024/1024) TOTAL_SPACE
from sys.v_$datafile a, sys.v_$tablespace B
where a.ts# = b.ts#
group by b.name
) Total
WHERE Free.Tablespace_name = Total.name;
prompt
prompt**---------------Tablespace Fragmentation Status----------------**
prompt
col TABLESPACE_NAME format a25 heading "Tablespace Name"
select TABLESPACE_NAME Name,TOTAL_EXTENTS "Total Extents",EXTENTS_COALESCED,decode(PERCENT_EXTENTS_COALESCED,100,'NO','YES') "Frag" from
dba_free_space_coalesced;
prompt
prompt**---------------Latch Contention-------------------------------**
prompt
col name format a40 heading "Latch Name"
SELECT name, gets, misses,
round((gets-misses)/decode(gets,0,1,gets),3) hit_ratio,
sleeps,
round(sleeps/decode(misses,0,1,misses),3) "sleeps/misses"
from v$latch
where gets != 0
order by name;
select name,immediate_gets,immediate_misses,(immediate_gets)/(immediate_gets+immediate_misses) Hit_Ratio
from v$latch where immediate_gets != 0;
prompt
prompt**---------------Shared Pool Statistics-------------------------------**
prompt
col namespace format a22
select namespace,gets,gethits,round(gethitratio,2) gethitratio,pins,pinhits,round(pinhitratio,2)
pinhitratio,reloads,invalidations from v$librarycache;
prompt
prompt**---------------Roll back segment Statistics-------------------------------**
prompt
col segment_name format a30 heading "Segment Name"
col status format a15
select segment_name,a.status,initial_extent/(1024) "Initial KB",next_extent/(1024)"NextKB",max_extents,min_extents,optsize/102
"Opt Size KB",curext "Current Extents" from dba_rollback_segs a,v$rollstat b where a.segment_id = b.usn;
prompt
prompt**---------------Top 20 Events and System Statistics-------------------------------**
prompt
col event format a40 heading "Event Name"
col Stat format a40 heading "Stat Name"
select * from ( select name "Stat",value from v$sysstat order by value desc ) where rownum <= 20 ;
select * from ( select event,total_waits from v$system_event order by total_waits desc ) where rownum <=
20;
prompt
prompt**---------------Buffer Cache statistics-------------------------------**
prompt
select (1-(a.value/(b.value+c.value))) *100 "Buffer Cache Hit ratio" from v$sysstat a,v$sysstat b,v$sysstat c where a.name
= 'physical reads' and b.name = 'consistent gets' and c.name = 'db block gets';
col name format a20 heading "Buffer Pool Name"
select name,free_buffer_wait,write_complete_wait,buffer_busy_wait,db_block_gets,consistent_gets,physical_reads,physical_writes
from v$buffer_pool_statistics;
prompt
prompt
prompt
prompt**---------------File I/O statistics-------------------------------**
prompt
col file# format 99 heading "File No"
select file#,PHYRDS "Physical Reads",PHYWRTS "Physical Writes",READTIM "Read Time",WRITETIM "Write Time",AVGIOTIM "Avg Time" from v$filestat;
set feedback on
Description: This script will display the estimated work and time remaining for long operations (i.e. over 10 minutes).
Code:
select sid,
message || '(' || time_remaining || ')' "Long Ops"
from v$session_longops
where time_remaining > 600;<br />
Description: This is possibly one of the most useful tuning scripts available. The V$SYSTEM_EVENT dynamic performance view is the highest-level view of the "Session Wait Interface". Information in this view is cumulative since the database instance was started, and one can get a very good idea of what types of contention a database instance is (or is not) experiencing by monitoring this view.<br />
Code:<br />
/**********************************************************************<br />
* File: systemevent.sql<br />
* Type: SQL*Plus script<br />
* Description:<br />
* This is possibly one of the most useful tuning scripts<br />
* available.<br />
*<br />
* The V$SYSTEM_EVENT dynamic performance view is the highest-level<br />
* view of the "Session Wait Interface". Information in this view<br />
* is cumulative since the database instance was started, and one<br />
* can get a very good idea of what types of contention a database<br />
* instance is (or is not) experiencing by monitoring this view.<br />
*<br />
* NOTE: this script sorts output by the TIME_WAITED column in the<br />
* V$SYSTEM_EVENT view. If the Oracle initialization parameter<br />
* TIMED_STATISTICS is not set to TRUE, then the TIME_WAITED<br />
* column will not be populated.<br />
*<br />
* Please *disregard* the advice of people who insist that turning<br />
* off TIMED_STATISTICS is somehow a performance boost. Whatever<br />
* performance overhead that might be incurred is more than<br />
* compensated for by the incredible tuning information that<br />
* results...<br />
*<br />
*********************************************************************/<br />
set echo off feedback off timing off pause off verify off<br />
set pagesize 100 linesize 500 trimspool on trimout on<br />
col event format a26 truncate heading "Event Name"<br />
col total_waits format 999,990.00 heading "Total|Waits|(in 1000s)"<br />
col total_timeouts format 999,990.00 heading "Total|Timeouts|(in 1000s)"<br />
col time_waited format 999,990.00 heading "Time|Waited|(in Hours)"<br />
col pct_significant format 90.00 heading "% of|Concern"<br />
col average_wait format 990.00 heading "Avg|Wait|(Secs)"<br />
<br />
col instance new_value V_INSTANCE noprint<br />
select lower(replace(t.instance,chr(0),<br />
)) instance<br />
from sys.v_$thread t,<br />
sys.v_$parameter p<br />
where p.name = 'thread'<br />
and t.thread# = to_number(decode(p.value,'0','1',p.value));<br />
<br />
col total_time_waited new_value V_TOTAL_TIME_WAITED noprint<br />
select sum(time_waited) total_time_waited<br />
from sys.v_$system_event<br />
where event not in ('SQL*Net message from client',<br />
'rdbms ipc message',<br />
'slave wait',<br />
'pmon timer',<br />
'smon timer',<br />
'rdbms ipc reply',<br />
'SQL*Net message to client',<br />
'SQL*Net break/reset to client',<br />
'inactive session',<br />
'Null event')<br />
/<br />
<br />
select event,<br />
(total_waits / 1000) total_waits,<br />
(total_timeouts / 1000) total_timeouts,<br />
(time_waited / 360000) time_waited,<br />
decode(event,<br />
'SQL*Net message from client', 0,<br />
'rdbms ipc message', 0,<br />
'slave wait', 0,<br />
'pmon timer', 0,<br />
'smon timer', 0,<br />
'rdbms ipc reply', 0,<br />
'SQL*Net message to client', 0,<br />
'SQL*Net break/reset to client', 0,<br />
'inactive session', 0,<br />
'Null event', 0,<br />
(time_waited / &&V_TOTAL_TIME_WAITED)*100) pct_significant,<br />
(average_wait / 100) average_wait<br />
from sys.v_$system_event<br />
where (time_waited/360000) >= 0.01<br />
order by pct_significant desc, time_waited desc<br />
<br />
spool sysevent_&&V_INSTANCE<br />
/<br />
spool off<br />
Description: The Script lists the Session ID,Event Name,Time Waited on the Database. This Script is useful in identifying the resource intensive Queries. Can Be used on oracle 8i & 9i Databases.<br />
Code:<br />
select se.sid,s.username,Se.Event,se.time_waited<br />
from v$session s, v$session_event se<br />
where s.username is not null<br />
and s.username not in ('SYS','SYSTEM')<br />
and se.sid=s.sid<br />
and s.status='ACTIVE'<br />
and se.event not like '%SQL*Net%'<br />
/<br />
Description: Anything appear in the result will be a locked object .<br />
Code:<br />
select o.object_name,l.oracle_username,l.os_user_name,l.session_id<br />
,decode(l.locked_mode,2,'Row-S',3,'Row-X',4,'Share',5,'S/Row-X',6 ,'Exclusive','NULL')<br />
from user_objects o , v$locked_object l<br />
where o.object_id = l.object_id;<br />
<br />
Description: This script will display the waits on the instance since the instance has been created. Good place to look for system bottle necks<br />
Code:<br />
col event form A50 <br />
col Prev form 999 <br />
col Curr form 999 <br />
col Tot form 999 <br />
select <br />
event,<br />
sum(decode(wait_Time,0,0,1)) "Prev", <br />
sum(decode(wait_Time,0,1,0)) "Curr",<br />
count(*) "Tot" <br />
from <br />
v$session_Wait <br />
group by event <br />
order by 4<br />
Description: Display the contention in rollback segments<br />
Code:<br />
column "Ratio" format 99.99999 <br />
select name, waits, gets, waits/gets "Ratio" <br />
from v$rollstat a, v$rollname b <br />
where a.usn = b.usn;<br />
Description: Display IO by file<br />
Code:<br />
select <br />
substr(a.file#,1,2) "#", <br />
substr(a.name,1,30) "Name", <br />
a.status, <br />
a.bytes, <br />
b.phyrds, <br />
b.phywrts <br />
from v$datafile a, v$filestat b <br />
where a.file# = b.file#;<br />
Description: Try to balance the I/O by mixing active and inactive data files / tablespaces on the same drive<br />
Code:<br />
column name format a18 heading 'Tablespace' jus cen<br />
column file format a50 heading 'File Name' jus cen<br />
column pbr format 99,999,999,990 heading 'Physical|Blocks|Read' jus cen <br />
column pbw format 99,999,999,990 heading 'Physical |Blocks|Written' jus cen <br />
column pyr format 99,999,999,990 heading 'Physical |Reads' jus cen <br />
column pyw format 99,999,999,990 heading 'Physical|Writes' jus cen<br />
<br />
ttitle center 'Disk I/O Activity by file' skip 2<br />
compute sum of pyr on report<br />
compute sum of pyw on report<br />
compute sum of pbr on report<br />
compute sum of pbw on report<br />
break on report<br />
select <br />
df.tablespace_name name,<br />
df.file_name "file",<br />
f.phyrds pyr,<br />
f.phyblkrd pbr,<br />
f.phywrts pyw,<br />
f.phyblkwrt pbw<br />
from v$filestat f, dba_data_files df<br />
where<br />
f.file# = df.file_id<br />
order by df.tablespace_name<br />
<br />
Description: This script takes a snapshot of v$filestats at the current time and saves it. It then waits 10 seconds and takes another snapshot and reports on the delta.<br />
Code:<br />
col name for a50<br />
set linesize 132<br />
set pages 666<br />
-- drop temporary table<br />
drop table jh$filestats;<br />
create table jh$filestats as <br />
select file#, PHYBLKRD, PHYBLKWRT<br />
from v$filestat;<br />
prompt Waiting......<br />
exec dbms_lock.sleep(10);<br />
<br />
prompt NOTE: Only the top 10 files...<br />
select * from (<br />
select df.name, fs.phyblkrd - t.phyblkrd "Reads",<br />
fs.PHYBLKWRT - t.PHYBLKWRT "Writes",<br />
(fs.PHYBLKRD+fs.PHYBLKWRT) - (t.PHYBLKRD+t.PHYBLKWRT) "Total IO"<br />
from v$filestat fs, v$datafile df, jh$filestats t<br />
where df.file# = fs.file#<br />
and t.file# = fs.file#<br />
and (fs.PHYBLKRD+fs.PHYBLKWRT) - (t.PHYBLKRD+t.PHYBLKWRT) > 0<br />
order by "Total IO" desc )<br />
where rownum <= 10
/
Description: This script will display the table name and column name of the tables that need to be indexed.
Code:
COL table_name format A20 head 'TABLE_NAME'
COL constraint_name format A20 head 'CONSTRAINT_NAME'
COL table2 format A20 head 'TABLE_TO_BE_INDEXED'
COL column_name format A20 head 'COLUMN_TO_BE_INDEXED'
SET linesize 100
SELECT t.table_name,c.constraint_name,c.table_name table2
,acc.column_name
FROM all_constraints t, all_constraints c
, all_cons_columns acc
WHERE c.r_constraint_name = t.constraint_name
AND c.table_name =acc.table_name
AND c.constraint_name = acc.constraint_name
AND NOT EXISTS ( SELECT '1' FROM all_ind_columns aid
WHERE aid.table_name = acc.table_name
AND aid.column_name = acc.column_name)
ORDER BY c.table_name;
Description: Table to Index Cross Reference Report
Code:
ttitle left 'Date: ' format a10 cur_date -
center 'Table/Index Cross Reference for ' format a8 sql.user -
right 'Page:' format 999 sql.pno
set heading on
set pagesize 56
set linesize 100
set newpage 0
column uniqueness format a9 heading 'Unique?'
column index_name format a30 heading 'Index|Name'
column table_name format a24 heading 'Table|Name'
column column_name format a24 heading 'Column|Name'
column table_type format a8 heading 'Index|Type'
break on table_name skip 1 on table_type on index_name on uniqueness
select user_indexes.table_name, user_indexes.index_name,
uniqueness, column_name
from user_ind_columns, user_indexes
where user_ind_columns.index_name = user_indexes.index_name
and user_ind_columns.table_name = user_indexes.table_name
order by user_indexes.table_type, user_indexes.table_name,
user_indexes.index_name, column_position
;
clear breaks
clear columns
ttitle off
Description: This script lists foreign keys that are missing indexes on the foreign key columns in the child table. If the index is not in place, share lock problems may occur on the parent table.
Code:
ttitle 'Foreign Keys with Indexes Missing on Child Table'
SELECT acc.owner||'-> '||acc.constraint_name||'('||acc.column_name<br />
||'['||acc.position||'])'||' ***** Missing Index' "Missing Index"<br />
FROM all_cons_columns acc, all_constraints ac<br />
WHERE ac.constraint_name = acc.constraint_name<br />
AND ac.constraint_type = 'R'<br />
and acc.owner not in ('SYS','SYSTEM')<br />
AND (acc.owner, acc.table_name, acc.column_name, acc.position) <br />
IN<br />
(SELECT acc.owner, acc.table_name, acc.column_name, acc.position <br />
FROM all_cons_columns acc, all_constraints ac<br />
WHERE ac.constraint_name = acc.constraint_name<br />
AND ac.constraint_type = 'R'<br />
MINUS<br />
SELECT table_owner, table_name, column_name, column_position<br />
FROM all_ind_columns)<br />
ORDER BY acc.owner, acc.constraint_name, <br />
acc.column_name, acc.position;<br />
ttitle off<br />
escription: Calculate buffer cache hit ratio in the database. Make sure it is more than 80 for an oltp environment and 99 is the best value.<br />
Code:<br />
column "logical_reads" format 99,999,999,999 <br />
column "phys_reads" format 999,999,999 <br />
column "phy_writes" format 999,999,999 <br />
select a.value + b.value "logical_reads", <br />
c.value "phys_reads",<br />
round(100 * ((a.value+b.value)-c.value) / <br />
(a.value+b.value)) <br />
"BUFFER HIT RATIO" <br />
from v$sysstat a, v$sysstat b, v$sysstat c<br />
where <br />
a.statistic# = 38 <br />
and <br />
b.statistic# = 39 <br />
and <br />
c.statistic# = 40;<br />
Description: Increase Shared pool size to reach a 90% hit ratio on Dictionary Cache. Entries for dc_table_grants, d_user_grants, and dc_users should be under 5% each in the MISS RATE % column<br />
Code:<br />
select <br />
parameter, <br />
gets,<br />
Getmisses , <br />
getmisses/(gets+getmisses)*100 "miss ratio",<br />
(1-(sum(getmisses)/ (sum(gets)+sum(getmisses))))*100 "Hit ratio"<br />
from v$rowcache<br />
where gets+getmisses <>0<br />
group by parameter, gets, getmisses ;<br />
escription: Reduce the Reloads and try to increase the hit ratios to above 85%<br />
Code:<br />
ttitle center 'LIBRARY CACHE STATS' skip 2<br />
column namespace format a8 heading 'Library'<br />
column gets format 9,999,990 heading 'GETS'<br />
column gethits format 9,999,990 heading 'GETHITS'<br />
column gethitratio format 99.90 heading 'GET|HIT|RATIO'<br />
column pins format 999,999,990 heading 'PINS'<br />
column pinhits format 999,999,990 heading 'PINHITS'<br />
column pinhitratio format 99.90 heading 'PIN|HIT|RATIO'<br />
column reloads format 999,990 heading 'RELOADS' <br />
compute sum of gets on report<br />
compute sum of gethits on report<br />
compute sum of pins on report<br />
compute sum of pinhits on report <br />
compute sum of reloads on report<br />
break on report<br />
select<br />
namespace,gets,gethits,gethitratio,pins,pinhits,<br />
pinhitratio, reloads<br />
from v$librarycache<br />
where gets+gethits+pins+pinhits>0<br />
Description: This should be near 0.If the Ratio is larger than 1% then increase the SHARED_POOL_SIZE.<br />
Code:<br />
column libcache format 99.99 heading 'Percentage' jus cen<br />
select sum(pins) "Total Pins", sum(reloads) "Total Reloads",<br />
sum(reloads)/sum(pins) *100 libcache<br />
from v$librarycache<br />
Description: To run this script you must get connect<br />
as user sys<br />
Code:<br />
select name,<br />
1 - ( physical_reads / ( db_block_gets +<br />
consistent_gets)) "HIT RATIO"<br />
from sys.v$buffer_pool_statistics<br />
where db_block_gets + consistent_gets > 0<br />
Description: THIS SCRIPT CHECK THE FREE SPACE IN THE SHARED POOL AND WHEN THE SIZE ITS VERY SMALL EXECUTE AN ALTER SYSTEM FLUSH SHARED POOL<br />
Code:<br />
CREATE OR REPLACE PROCEDURE flush_it ( pct_full IN NUMBER) AS<br />
--<br />
-- Cursor definitions<br />
--<br />
CURSOR get_share IS<br />
SELECT SUM(sharable_mem) FROM<br />
sql_summary;<br />
--<br />
CURSOR get_var IS<br />
SELECT value FROM v$sga WHERE name LIKE 'Var%';<br />
--<br />
CURSOR get_time IS <br />
SELECT sysdate FROM dual;<br />
--<br />
-- Variable definitions<br />
--<br />
todays_date DATE;<br />
mem_ratio NUMBER;<br />
share_mem NUMBER;<br />
variable_mem NUMBER;<br />
cur INTEGER;<br />
sql_com VARCHAR2(60);<br />
row_proc NUMBER;<br />
--<br />
-- Procedure Body<br />
--<br />
BEGIN<br />
OPEN get_share;<br />
OPEN get_var;<br />
FETCH get_share INTO share_mem;<br />
dbms_output.put_line('share_mem: '||to_char(share_mem));<br />
FETCH get_var INTO variable_mem;<br />
dbms_output.put_line('variable_mem: '||to_char(variable_mem));<br />
mem_ratio:=share_mem/variable_mem;<br />
dbms_output.put_line(to_char(mem_ratio));<br />
IF mem_ratio>(pct_full/100) THEN<br />
cur:=dbms_sql.open_cursor;<br />
sql_com:='ALTER SYSTEM FLUSH SHARED_POOL';<br />
dbms_sql.parse(cur,sql_com,dbms_sql.v7);<br />
row_proc:=dbms_sql.execute(cur);<br />
dbms_sql.close_cursor(cur);<br />
OPEN get_time;<br />
FETCH get_time INTO todays_date;<br />
INSERT INTO dba_running_stats VALUES <br />
(<br />
'Flush of Shared Pool',1,35,todays_date,0<br />
);<br />
COMMIT;<br />
END IF;<br />
END;<br />
Description: The Hit Ratio should be higher than 90%<br />
Code:<br />
select Username,<br />
OSUSER,<br />
Consistent_Gets,<br />
Block_Gets,<br />
Physical_Reads,<br />
100*( Consistent_Gets + Block_Gets - Physical_Reads)/<br />
( Consistent_Gets + Block_Gets ) "Hit Ratio %"<br />
from V$SESSION,V$SESS_IO<br />
where V$SESSION.SID = V$SESS_IO.SID<br />
and ( Consistent_Gets + Block_Gets )>0<br />
and username is not null<br />
order by Username,"Hit Ratio %";<br />
Description: If you want to copy the data from one database to another having many tables and constraints,all you need is this script which will first disable all the constraints then delete the prior data and then finaly copy the data from any remote database and finaly it enables all the constraints on the table of that database.......all you need is to have a table having names of all the tables and the corresponding constraints<br />
Code:<br />
CREATE OR REPLACE procedure pre_tra<br />
as<br />
cursor cconstraints is select table_name, constraint_name, status from all_constraints where owner='owner name';<br />
<br />
c2 cconstraints%rowtype;<br />
mytab varchar2(200);<br />
mytab1 varchar2(200);<br />
mytab3 varchar2(200);<br />
mytab4 varchar2(200);<br />
retrycounter number;<br />
counter number;<br />
cstatus varchar2(20);<br />
enableerrorfound boolean:=true;<br />
--MAX_TRY number:=10000;<br />
begin<br />
begin<br />
open cconstraints;<br />
loop<br />
<br />
fetch cconstraints into c2;<br />
exit when cconstraints%notfound;<br />
mytab := c2.table_name;<br />
mytab1 := c2.constraint_name;<br />
mytab3 :='alter table ' || mytab || ' disable constraint ' || mytab1;<br />
execute immediate mytab3;<br />
<br />
end loop;<br />
close cconstraints;<br />
<br />
<br />
<br />
open cconstraints;<br />
loop<br />
fetch cconstraints into c2;<br />
exit when cconstraints%notfound;<br />
mytab :=c2.table_name;<br />
execute immediate 'delete ' || mytab;<br />
<br />
<br />
end loop;<br />
close cconstraints;<br />
end;<br />
<br />
<br />
open cconstraints;<br />
loop<br />
fetch cconstraints into c2;<br />
exit when cconstraints%notfound;<br />
mytab :=c2.table_name;<br />
mytab4 :='DATABASENAME';<br />
execute immediate 'insert into ' || mytab || ' (select * from '|| mytab || '@' || mytab4 || ')';<br />
DBMS_OUTPUT.PUT_LINE(mytab);<br />
commit;<br />
DBMS_OUTPUT.PUT_LINE(mytab);<br />
end loop;<br />
close cconstraints;<br />
<br />
<br />
begin<br />
DBMS_OUTPUT.PUT_LINE('Starting enable script.....');<br />
<br />
RETRYCOUNTER := 3;<br />
<br />
WHILE RETRYCOUNTER > 0<br />
loop<br />
open cconstraints;<br />
<br />
loop<br />
fetch cconstraints into c2;<br />
exit when cconstraints%notfound;<br />
<br />
DBMS_OUTPUT.PUT_LINE('retrycounter==>'||mytab);<br />
mytab := c2.table_name;<br />
mytab1 := c2.constraint_name;<br />
--cstatus:= c2.status;<br />
--if cstatus = 'DISABLED' then<br />
mytab3 :='alter table ' || mytab || ' enable constraint ' || mytab1;<br />
DBMS_OUTPUT.PUT_LINE('Q' || mytab3);<br />
DBMS_OUTPUT.PUT_LINE(retrycounter);<br />
--enableerrorfound:=false;<br />
BEGIN<br />
execute immediate mytab3;<br />
EXCEPTION<br />
WHEN OTHERS THEN<br />
enableerrorfound:=true;<br />
END;<br />
--end if;<br />
<br />
<br />
<br />
end loop;<br />
RETRYCOUNTER := RETRYCOUNTER - 1;<br />
DBMS_OUTPUT.PUT_LINE(retrycounter);<br />
close cconstraints;<br />
end loop;<br />
<br />
DBMS_OUTPUT.PUT_LINE(retrycounter);<br />
<br />
IF (enableerrorfound) THEN<br />
DBMS_OUTPUT.PUT_LINE('enableerrorfound==> TRUE');<br />
else<br />
DBMS_OUTPUT.PUT_LINE('enableerrorfound==> FALSE');<br />
end if;<br />
end;<br />
<br />
end pre_tra;<br />
/<br />
Description: List the UGA and PGA used by each session on the server<br />
Code:<br />
column name format a25<br />
column total format 999 heading 'Cnt'<br />
column bytes format 9999,999,999 heading 'Total Bytes'<br />
column avg format 99,999,999 heading 'Avg Bytes'<br />
column min format 99,999,999 heading 'Min Bytes'<br />
column max format 9999,999,999 heading 'Max Bytes'<br />
ttitle 'PGA = dedicated server processes - UGA = Client machine process'<br />
<br />
<br />
compute sum of minmem on report<br />
compute sum of maxmem on report<br />
break on report<br />
<br />
select se.sid,n.name, <br />
max(se.value) maxmem<br />
from v$sesstat se,<br />
v$statname n<br />
where n.statistic# = se.statistic#<br />
and n.name in ('session pga memory','session pga memory max',<br />
'session uga memory','session uga memory max')<br />
group by n.name,se.sid<br />
order by 3<br />
/<br />
Description: This script lists the size of stored objects<br />
Code:<br />
column num_instances heading "Num" format 999 <br />
column type heading "Object Type" format a12 <br />
column source_size heading "Source" format 99,999,999 <br />
column parsed_size heading "Parsed" format 99,999,999 <br />
column code_size heading "Code" format 99,999,999 <br />
column error_size heading "Errors" format 999,999 <br />
column size_required heading "Total" format 999,999,999 <br />
compute sum of size_required on report <br />
<br />
select count(name) num_instances <br />
,type <br />
,sum(source_size) source_size <br />
,sum(parsed_size) parsed_size <br />
,sum(code_size) code_size <br />
,sum(error_size) error_size <br />
,sum(source_size) <br />
+sum(parsed_size) <br />
+sum(code_size) <br />
+sum(error_size) size_required <br />
from dba_object_size <br />
group by type <br />
order by 2 <br />
/<br />
<br />
Description: This script will display the active user and the rollback segment being used in the database<br />
Code:<br />
column rr heading 'RB Segment' format a18 <br />
column us heading 'Username' format a15 <br />
column os heading 'OS User' format a10 <br />
column te heading 'Terminal' format a10 <br />
<br />
SELECT r.name rr,<br />
nvl(s.username,'no transaction') us,<br />
s.osuser os,<br />
s.terminal te <br />
FROM <br />
v$lock l, <br />
v$session s,<br />
v$rollname r <br />
WHERE <br />
l.sid = s.sid(+) AND <br />
trunc(l.id1/65536) = r.usn AND<br />
l.type = 'TX' AND <br />
l.lmode = 6 <br />
ORDER BY r.name <br />
/<br />
<br />
escription: This script reports how many hours it has been since the rollback segments wrapped<br />
Code:<br />
select n.name, <br />
round(24*(sysdate-to_date(i1.value||' '||i2.value,'j SSSSS')) / <br />
(s.writes/s.rssize),1) "Hours" <br />
from v$instance i1,<br />
v$instance i2,<br />
v$rollname n,<br />
v$rollstat s <br />
where <br />
i1.key = 'STARTUP TIME - JULIAN' <br />
and i2.key = 'STARTUP TIME - SECONDS' <br />
and n.usn = s.usn <br />
and s.status = 'ONLINE'<br />
Description: Gives lots of usefull easy to read info on how your RBS are performing. Needs 132 char display.<br />
Code:<br />
ol name for a7<br />
col xacts for 9990 head "Actv|Trans"<br />
col InitExt for 990.00 head "Init|Ext|(Mb)"<br />
col NextExt for 990.00 head "Next|Ext|(Mb)"<br />
col MinExt for 99 head "Min|Ext"<br />
col MaxExt for 999 head "Max|Ext"<br />
col optsize for 9990.00 head "Optimal|Size|(Mb)"<br />
col rssize for 9990.00 head "Curr|Size|(Mb)"<br />
col hwmsize for 9990.00 head "High|Water|Mark|(Mb)"<br />
col wraps for 999 head "W|R|A|P|S"<br />
col extends for 990 head "E|X|T|E|N|D|S"<br />
col shrinks for 990 head "S|H|R|I|N|K|S"<br />
col aveshrink for 990.00 head "AVG|Shrink|(Mb)"<br />
col gets head "Header|Gets"<br />
col waits for 99990 head "Header|Waits"<br />
col writes for 999,999,990 head "Total|Writes|Since|Startup|(Kb)"<br />
col wpg for 9990 head "AVG|Writes|Per|HedGet|(bytes)"<br />
set lines 132 pages 40 feed off<br />
break on report<br />
compute sum of gets on report<br />
compute sum of waits on report<br />
compute avg of aveshrink on report<br />
compute avg of wpg on report<br />
<br />
select name,<br />
XACTS,<br />
initial_extent/1048576 InitExt,<br />
next_extent/1048576 NextExt,<br />
min_extents MinExt,<br />
max_extents MaxExt,<br />
optsize/1048576 optsize,<br />
RSSIZE/1048576 rssize,<br />
HWMSIZE/1048576 hwmsize,<br />
wraps,<br />
extends,<br />
shrinks,<br />
aveshrink/1048576 aveshrink,<br />
gets,<br />
waits,<br />
writes/1024 writes,<br />
writes/gets wpg<br />
from v$rollstat,v$rollname,dba_rollback_segs<br />
where v$rollstat.usn=v$rollname.usn<br />
and dba_rollback_segs.segment_id=v$rollname.usn<br />
order by name<br />
Description: This script will show you the user's OS name, Username in the database and the SQL Text they are running<br />
Code:<br />
SELECT osuser, username, sql_text<br />
from v$session a, v$sqltext b<br />
where a.sql_address =b.address<br />
order by address, piece<br />
Description: When you connect to sqlplus, you see the the following sql prompt<br />
SQL><br />
<br />
By using this sql in the glogin.sql, you will see a prompt similar to the following:<br />
<br />
SCOTT@DB-01><br />
Code:<br />
The following code works on Oracle 8i (8.1.5, 8.1.6, 8.1.7).<br />
You have to insert the following line of code in glogin.sql which is usually found in <br />
$ORACLE_HOME/sqlplus/admin<br />
<br />
set termout off<br />
set echo off<br />
define X=NotConnected<br />
define Y=DBNAME<br />
<br />
Column Usr New_Value X<br />
Column DBName New_Value Y<br />
<br />
<br />
Select SYS_CONTEXT('USERENV','SESSION_USER' ) Usr From Dual;<br />
<br />
--- The following does not work in 8.1.5 but works in 8.1.6 or above<br />
---Select SYS_CONTEXT('USERENV','DB_NAME') DBNAME From Dual;<br />
<br />
--- If you are using 8.1.5, use this .<br />
Select Global_Name DBNAME from Global_Name;<br />
<br />
set termout on<br />
set sqlprompt '&X@&Y> '<br />
<br />
Please note that this method will work only when you make a new sql plus session because when you make a new sql plus session, then only glogin.sql is executed.<br />
Description: In N tier structure it is really hard to track<br />
user session specially in a web enviroment,<br />
All users connect to database using the same <br />
application , so all requests are going to database on behalf of one oracle user.<br />
then how can you track actions of one ENDUSER<br />
(application user).<br />
Code:<br />
In the v$session, there are serval columns you <br />
can use to distinguish ENDUSER such as Moule,Action and Client info.<br />
for example, you can use <br />
dbms_application_info.set_client_info('client info')<br />
to set client info accroding to user's ip or <br />
loginname in your application,<br />
then it is easy to find out what sql statement <br />
was execute by this client use following script<br />
<br />
select s.sid, s.username, s.program, oc.sql_text<br />
from v$session s, v$open_cursor oc<br />
where s.saddr=oc.saddr and s.sid=oc.sid<br />
and s.client_info='client_info';<br />
<br />
if you want to capture all sql in real time, then <br />
there is a tool Oracle Session Manager in <br />
http://www.wangz.net can help you do it.<br />
Description: Purpose: To read raw session trace file with option to <br />
format trace file with TKPROF and read it in SQLPLUS Session<br />
Code:<br />
#!/bin/sh<br />
#################################################################<br />
#################################################################<br />
## Copyright (c)2004 Kalson Systems ##<br />
## Author :Neaman Ahmed ##<br />
## ScriptName:trace ##<br />
## Purpose: To read raw session trace file with option to ## <br />
## format trace file with TKPROF and read it in SQLPLUS Session##<br />
## You must set TRACE_DIR and ORACLE_SID in ##<br />
## this script Put trace in < $HOME/bin > make it executable ##<br />
## and run it from sqlplus session ## <br />
## Example: SQL> ! trace ##<br />
## Critics, Comment and suggestion are welcome nomiak@yahoo.com##<br />
## Note:You Must Run this script from your sqlplus session ##<br />
#################################################################<br />
#################################################################<br />
clear<br />
n1=`ps |grep sqlplus|awk '{print $1}' `<br />
n2=`ps -ef|grep $n1|grep oracle"$ORACLE_SID"|awk '{print $2}'`<br />
TRACE_DIR=/u02/app/oracle/admin/$ORACLE_SID/udump <br />
<br />
echo " 1.Do you want to view raw trace file"<br />
echo " 2.Do you want to format trace file and view it"<br />
read choice<br />
if [ "$choice" = "1" ];<br />
then cat $TRACE_DIR/"$ORACLE_SID"_ora_$n2.trc|more<br />
else <br />
if [ "$choice" = "2" ]; <br />
then <br />
tkprof $TRACE_DIR/"$ORACLE_SID"_ora_$n2.trc $TRACE_DIR/trace_session_$n2 <br />
clear<br />
echo "Your TKPROF formated file name is $TRACE_DIR/trace_session_$n2.prf "<br />
echo " Do you want to read it now y/n "<br />
read ans<br />
if [ "$ans" = "y" ];<br />
then <br />
cat $TRACE_DIR/trace_session_$n2.prf|more<br />
else <br />
if [ "$ans" = "n" ]; <br />
then <br />
echo "Happy Hunting Perfomance Problems"<br />
fi<br />
fi<br />
fi<br />
fiKrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-36695371245998641622009-10-06T08:58:00.001-07:002009-10-06T08:58:20.742-07:00Tuning SQL Statements That Use the IN OperatorA product manager at a software development company called me recently in a panic. His team had built an application that runs against Oracle databases, and in their test environment the application offered good response time. Then a key customer installed the software on their system and discovered that mission critical queries that should return results in under a second were taking over a minute to complete.<br />
<br />
The product manager, suspecting the customer’s database was not properly tuned or configured, asked me to examine the system. Gaining access to the customer’s database was at first politically touchy, but in the end I got excellent cooperation from the customer’s DBA staff and found their database was in fine shape.<br />
<br />
I quickly switched to my application developer hat as I traced sessions on an active database and ran Tkprof to collect information about what was going on inside the customer’s Oracle server. After an hour of generating trace files and looking at the results, the problem was clear: The application was using the IN operator in its SQL to an extreme I had never seen before, and the customer’s Oracle 7.3 database didn’t take kindly to this.<br />
<br />
The software development company, which I’ll refer to as “Acme Software,” had used an Oracle 7.2 database in-house to build the application. It turned out that Oracle 7.2 handled this application’s far-out use of the IN operator somewhat better than Oracle 7.3. Also clouding the issue was a flaw in Acme’s testing strategy that made it appear as if the application ran very fast on the Oracle 7.2 database when in reality performance was still not so hot.<br />
<br />
So at the end of the day Acme had built an application whose core queries ran in five seconds against an Oracle 7.2 database, 70 seconds against an Oracle 7.3 database, and under one second against an Oracle 8.0 database. All three databases had the exact same data and ran on comparable hardware. My mission was to explain to the product manager why different releases of Oracle would give such varied results and, more importantly, help his developers get the application to run quickly on all releases of Oracle. <br />
<br />
The IN Operator<br />
<br />
The IN operator in SQL gives you the ability to allow for multiple conditions in one SQL statement. The following query, for example, will give you the order numbers of all orders that are either canceled or invalid:<br />
<br />
SELECT order_number<br />
FROM orders<br />
WHERE status IN (‘canceled’, ‘invalid’)<br />
<br />
The list of values enclosed in the parentheses is called an inlist. The above example had an inlist with two values in it. The following two queries will list the names of all employees in departments 10, 20 and 30. Both queries yield the same results—the IN operator and the = ANY operator mean the same thing to the Oracle server:<br />
<br />
SELECT ename<br />
FROM emp<br />
WHERE deptno IN (10, 20, 30)<br />
<br />
SELECT ename<br />
FROM emp<br />
WHERE deptno = ANY (10, 20, 30)<br />
<br />
In each of these queries, the IN operator has been used to select data based on multiple constant values (order status codes in the first query and department numbers in the second and third). The IN operator can also be used with subqueries. The query below retrieves names of all employees in departments where the name of the department starts with the word “Shoes”:<br />
<br />
SELECT ename<br />
FROM emp<br />
WHERE deptno IN<br />
(<br />
SELECT deptno<br />
FROM dept<br />
WHERE deptname LIKE ‘Shoes %’<br />
)<br />
<br />
IN operators used with subqueries can often be rewritten using a join between the two tables instead. For example, the previous query is identical to:<br />
<br />
SELECT emp.ename<br />
FROM dept, emp<br />
WHERE dept.deptname LIKE ‘Shoes %’<br />
AND emp.deptno = dept.deptno<br />
<br />
The query optimizer in the Oracle server is well aware of this trick, and will usually process IN operators with subqueries as joins instead wherever possible. This has been true since the Oracle V6 days. Developers still use the IN operator with subqueries, though, because sometimes it makes the code easier to understand. Also, there are times when this use of the IN operator is not the same as an ordinary join.<br />
<br />
How Oracle 7 Processes IN Operators<br />
<br />
So far we’ve looked at examples of how the IN operator can be used to handle multiple conditions in one query—either from a list of enumerated constants or a subquery that returns multiple values. But how does Oracle actually process the IN operator when executing a SQL statement? The answer can depend on which release of Oracle you are using, and whether the IN operator is being used with a subquery or enumerated values.<br />
<br />
First let’s look at the scenario where the IN operator is being used with a subquery. This is the more straightforward case, and seems to be consistent across all releases of Oracle. Fig. 1 shows the same query used in a previous example, along with its execution plan. You can see that the Oracle server performs the query as if it were a simple join between the dept and emp tables: First Oracle finds the departments with names starting with the word “Shoes” and then it finds all of the employees in these departments.<br />
<br />
SELECT ename<br />
FROM emp<br />
WHERE deptno IN<br />
(<br />
SELECT deptno<br />
FROM dept<br />
WHERE deptname LIKE ‘Shoes %’<br />
)<br />
<br />
Execution Plan<br />
-------------------------------------------------------<br />
SELECT STATEMENT<br />
NESTED LOOPS<br />
TABLE ACCESS BY ROWID (DEPT)<br />
INDEX RANGE SCAN (DEPT_NAME_IDX)<br />
TABLE ACCESS BY ROWID (EMP)<br />
INDEX RANGE SCAN (EMP_FK_DEPT_IDX)<br />
<br />
Fig. 1: A query using the IN operator with a subquery, and its execution plan.<br />
<br />
Next let’s look at how Oracle processes IN operators where constant values are enumerated in the inlist instead of a subquery. Fig. 2 shows the same query used in the very first example along with its execution plan. In this query we wish to retrieve the order numbers of all orders that have been canceled or are invalid. <br />
<br />
SELECT order_number<br />
FROM orders<br />
WHERE status IN (‘canceled’, ‘invalid’)<br />
<br />
Execution Plan<br />
-------------------------------------------------------<br />
SELECT STATEMENT<br />
CONCATENATION<br />
TABLE ACCESS BY ROWID (ORDERS)<br />
INDEX RANGE SCAN (ORDER_STATUS_IDX)<br />
TABLE ACCESS BY ROWID (ORDERS)<br />
INDEX RANGE SCAN (ORDER_STATUS_IDX)<br />
<br />
Fig. 2: A simple query using the IN operator with a list of constant values, and the resulting execution plan.<br />
<br />
If we take it as a given that the status column in the orders table is indexed, you might expect Oracle to simply scan the status index twice (once for the value “canceled” and once for the value “invalid”), and then combine the results. In a roundabout way, that is in fact exactly what Oracle is doing. First, the Oracle server has transformed the original query into the following:<br />
<br />
SELECT order_number<br />
FROM orders<br />
WHERE (status = ‘canceled’ OR status = ‘invalid’)<br />
<br />
Next, Oracle transforms the query further as follows:<br />
<br />
SELECT order_number<br />
FROM orders<br />
WHERE status = ‘canceled’<br />
UNION ALL<br />
SELECT order_number<br />
FROM orders<br />
WHERE status = ‘invalid’<br />
<br />
This second transformation should help you understand the execution plan shown in Fig. 2. Conceptually, the Oracle server is doing about what we would expect: It is looking up all canceled orders and all invalid orders and combining the results together. Oracle calls this combining action a concatenation. <br />
<br />
But suppose the query were more complex. Fig. 3 shows a more sophisticated query using the IN operator with enumerated values in the inlist, and the resulting execution plan. In this query we are retrieving more information about canceled and invalid orders than just the order number as in the previous example. Because of the way Oracle 7 transforms the IN operator into an OR clause and subsequently into a compound SQL statement using the UNION ALL set operator, you can see that the execution plan doubles in size for a query with an IN operator that has two constant values in its inlist. <br />
<br />
SELECT ORD.order_id, ORD.order_number, <br />
ORD.total, CUST.customer_name<br />
FROM orders ORD, customers CUST<br />
WHERE ORD.status IN (‘canceled’, ‘invalid’)<br />
AND CUST.customer_id = ORD.customer_id<br />
<br />
Execution Plan<br />
-------------------------------------------------------<br />
SELECT STATEMENT<br />
CONCATENATION<br />
NESTED LOOPS<br />
TABLE ACCESS BY ROWID (ORDERS)<br />
INDEX RANGE SCAN (ORDER_STATUS_IDX)<br />
TABLE ACCESS BY ROWID (CUSTOMERS)<br />
INDEX UNIQUE SCAN (CUSTOMERS_PK)<br />
NESTED LOOPS<br />
TABLE ACCESS BY ROWID (ORDERS)<br />
INDEX RANGE SCAN (ORDER_STATUS_IDX)<br />
TABLE ACCESS BY ROWID (CUSTOMERS)<br />
INDEX UNIQUE SCAN (CUSTOMERS_PK)<br />
<br />
Fig. 3: A more complex query using the IN operator with a list of constant values, and the resulting execution plan.<br />
<br />
As you can imagine, Oracle 7’s strategy for processing SQL statements with IN operators and constant values works well for small inlists, but is not very scaleable. In fact Oracle 7 will return an ORA-01795 error if the inlist for an IN operator contains more than 254 values. <br />
<br />
With this in mind, let’s go back to the core queries in Acme’s application. These queries, it turns out, hinge on a four table join and an IN operator with 200 constant values enumerated in the inlist. The execution plan for this query on their Oracle 7.2 database was over 2000 lines long!<br />
<br />
Fig. 4a shows a query similar to one of Acme’s core queries with the extreme IN operators. Fig. 4b shows the execution plan generated by an Oracle 7.2 database. Part of it anyway; you don’t need to see all fifty pages to get the idea!<br />
<br />
SELECT ORD.order_id, ORD.order_number, <br />
ORD.status, CUST.customer_name, <br />
CAT.category, SHIP.description<br />
FROM orders ORD, customers CUST, <br />
customer_categories CAT,<br />
shipping_methods SHIP<br />
WHERE CUST.customer_id = ORD.customer_id<br />
AND CAT.category_id = CUST.category_id<br />
AND SHIP.shipping_method_id = <br />
ORD.shipping_method_id<br />
AND ORD.order_id IN <br />
(<br />
385, 409, 507, 511, 601, 633, 641, 651,<br />
715, 799, 972, 984, 1038, 1074, 1122, 1135,<br />
1143, 1189, 1241, 1242, 1370, 1442, 1565, 1581,<br />
1593, 1614, 1803, 1881, 1890, 1899, 1943, 2015,<br />
2049, 2217, 2269, 2389, 2409, 2411, 2486, 2527,<br />
2576, 2696, 2704, 2759, 2768, 2846, 2924, 2927,<br />
2973, 3129, 3138, 3162, 3181, 3252, 3341, 3357,<br />
3383, 3503, 3551, 3738, 3741, 3753, 3822, 3864,<br />
3914, 4090, 4096, 4185, 4221, 4236, 4263, 4271,<br />
4278, 4305, 4396, 4465, 4485, 4570, 4578, 4607,<br />
4656, 4742, 4777, 4799, 4824, 4838, 4940, 5030,<br />
5134, 5137, 5199, 5302, 5360, 5531, 5569, 5592,<br />
5787, 5946, 5986, 6051, 6161, 6191, 6193, 6196,<br />
6261, 6392, 6465, 6597, 6611, 6725, 6782, 6793,<br />
6813, 6822, 6843, 6896, 6995, 7037, 7076, 7141,<br />
7305, 7307, 7327, 7374, 7387, 7410, 7424, 7442,<br />
7443, 7473, 7488, 7496, 7579, 7637, 7662, 7664,<br />
7748, 7830, 7891, 7902, 7947, 7997, 8008, 8092,<br />
8130, 8180, 8224, 8304, 8351, 8400, 8466, 8475,<br />
8480, 8491, 8509, 8572, 8592, 8704, 8766, 8775,<br />
8860, 8897, 8965, 9024, 9034, 9052, 9083, 9089,<br />
9102, 9107, 9230, 9269, 9358, 9412, 9423, 9429,<br />
9490, 9507, 9530, 9581, 9591, 9730, 9768, 9770,<br />
9771, 9772, 9784, 9791, 9854, 9858, 9929, 9964,<br />
9974, 9982, 9987, 9996, 9998,10003,10004,10092<br />
)<br />
<br />
Fig. 4a: A query using the IN operator with a long inlist of constant values.<br />
<br />
Execution Plan<br />
-------------------------------------------------------<br />
SELECT STATEMENT<br />
CONCATENATION<br />
NESTED LOOPS<br />
NESTED LOOPS<br />
NESTED LOOPS<br />
TABLE ACCESS BY ROWID (ORDERS)<br />
INDEX UNIQUE SCAN (ORDERS_PK)<br />
TABLE ACCESS BY ROWID (SHIPPING_METHODS)<br />
INDEX UNIQUE SCAN (SHIPPING_METHODS_PK)<br />
TABLE ACCESS BY ROWID (CUSTOMERS)<br />
INDEX UNIQUE SCAN (CUSTOMERS_PK)<br />
TABLE ACCESS BY ROWID (CUSTOMER_CATEGORIES)<br />
INDEX UNIQUE SCAN (CUSTOMER_CATEGORIES_PK)<br />
NESTED LOOPS<br />
NESTED LOOPS<br />
NESTED LOOPS<br />
TABLE ACCESS BY ROWID (ORDERS)<br />
INDEX UNIQUE SCAN (ORDERS_PK)<br />
TABLE ACCESS BY ROWID (SHIPPING_METHODS)<br />
INDEX UNIQUE SCAN (SHIPPING_METHODS_PK)<br />
TABLE ACCESS BY ROWID (CUSTOMERS)<br />
INDEX UNIQUE SCAN (CUSTOMERS_PK)<br />
TABLE ACCESS BY ROWID (CUSTOMER_CATEGORIES)<br />
INDEX UNIQUE SCAN (CUSTOMER_CATEGORIES_PK)<br />
. . .<br />
<br />
Fig. 4b: The execution plan generated by an Oracle 7.2 database for the query shown in Fig. 4a.<br />
<br />
With release 7.3, Oracle apparently altered the optimizer’s strategy for transforming SQL statements with large inlists. Instead of transforming one query into 200 queries and using UNION ALL to bring the results together, Oracle 7.3 will often ignore the relevant index and instead perform one full table scan. In this scenario, Oracle reads every row of data from the table and compares each to all values listed in the inlist of the IN operator. If the underlying table is large, this full table scan can clobber performance.<br />
<br />
To illustrate this point, Fig. 4c shows the execution plan generated by an Oracle 7.3 database for the query shown in Fig. 4a. Instead of looking up each of the 200 order_id values in the primary key index on the orders table, the Oracle 7.3 server has elected to perform a full scan of the orders table, finding all rows that have any of the 200 values in the order_id column, and then join the results to the other three tables.<br />
<br />
Execution Plan<br />
-------------------------------------------------------<br />
SELECT STATEMENT<br />
NESTED LOOPS<br />
NESTED LOOPS<br />
NESTED LOOPS<br />
TABLE ACCESS FULL (ORDERS)<br />
TABLE ACCESS BY ROWID (CUSTOMERS)<br />
INDEX UNIQUE SCAN (CUSTOMERS_PK)<br />
TABLE ACCESS BY ROWID (CUSTOMER_CATEGORIES)<br />
INDEX UNIQUE SCAN (CUSTOMER_CATEGORIES_PK)<br />
TABLE ACCESS BY ROWID (SHIPPING_METHODS)<br />
INDEX UNIQUE SCAN (SHIPPING_METHODS_PK)<br />
<br />
Fig. 4c: The execution plan generated by an Oracle 7.3 database for the query shown in Fig. 4a.<br />
<br />
Its not clear exactly where Oracle 7.3 draws the line. If the inlist is quite small (only a few constant values) then Oracle 7.3 will use the same transformation strategy as Oracle 7.2. But when the inlist is large (over perhaps 30 or 40 enumerated values) the full table scan seems to be used instead. You can force Oracle to use the transformation strategy, regardless of software release and inlist size, by using the USE_CONCAT optimizer hint.<br />
<br />
Enter Oracle 8<br />
<br />
Oracle 8 seems to handle IN operators with subqueries in much the same way that Oracle 7 and V6 did. This was already very efficient and needed no improvement. But for IN operators with many constant values in the inlist, Oracle 8 introduces the new inlist iterator data access path. This new method in the query optimizer makes handling of large inlists of constants extremely efficient and intuitive. Fig. 4d shows the execution plan generated by an Oracle 8.0 database for the query shown in Fig. 4a.<br />
<br />
Execution Plan<br />
-------------------------------------------------------<br />
SELECT STATEMENT <br />
NESTED LOOPS <br />
NESTED LOOPS <br />
NESTED LOOPS <br />
INLIST ITERATOR CONCATENATED <br />
TABLE ACCESS BY INDEX ROWID (ORDERS) <br />
INDEX RANGE SCAN (ORDERS_PK) <br />
TABLE ACCESS BY INDEX ROWID (CUSTOMERS) <br />
INDEX UNIQUE SCAN (CUSTOMERS_PK) <br />
TABLE ACCESS BY INDEX ROWID (CUSTOMER_CATEGORIES) <br />
INDEX UNIQUE SCAN (CUSTOMER_CATEGORIES_PK) <br />
TABLE ACCESS BY INDEX ROWID (SHIPPING_METHODS) <br />
INDEX UNIQUE SCAN (SHIPPING_METHODS_PK) <br />
<br />
Fig. 4d: The execution plan generated by an Oracle 8.0 database for the query shown in Fig. 4a.<br />
<br />
With the inlist iterator of Oracle 8, there is no transforming of the SQL statement into something far more complex than the original. And there is also no setting aside indexes in favor of full table scans. Instead, the Oracle 8 server just does what you would expect: It walks down the list of constant values spelled out in the inlist and looks each one up in the index, collects matching rows from the table, and moves on.<br />
<br />
There is no hint to turn on the inlist iterator data access path. The optimizer will simply use it when appropriate. If for some reason you need Oracle 8 to mimic the Oracle 7 behavior, you can use the USE_CONCAT hint to force the UNION ALL transformation.<br />
<br />
Back to the Original Puzzle<br />
<br />
Using the query shown in Fig. 4a one can now explain the puzzling results Acme Software observed. Their in-house Oracle 7.2 database transformed the core queries in their application into extremely complex statements as demonstrated in Fig. 4b. It took their Oracle server about five seconds to parse such a query and build the enormous execution plan. Actually executing and fetching the query, though, proved fast because only unique index scans and concatenations were involved. <br />
<br />
The first time Acme ran a regression test of their application, the core query took five seconds because of the long parse. Subsequent runs took under one second because the SQL statement was already in the shared pool and did not need to be parsed. So Acme had concluded that the query ran very fast in their regression test. (This was actually a flawed conclusion because in real life the criteria would be slightly different on each invocation, and so the five second parse would apply each time an end-user initiated a search.)<br />
<br />
On an Oracle 7.3 database, meanwhile, the query always took approximately 70 seconds. This was directly attributable to the full table scan of the orders table—one of the largest tables in their database—as demonstrated in Fig. 4c.<br />
<br />
And finally on the Oracle 8.0 database, the query always ran in less than one second because the new inlist iterator data access path put an end to complex transformations and full table scans.<br />
<br />
All of this served to explain why Acme’s application performed differently on different Oracle releases. But the real interest was in getting the application to run fast on any release of Oracle. One approach would be to add a USE_CONCAT optimizer hint to the query in Fig. 4a to cause Oracle releases 7.2, 7.3, and 8.0 to all process the query in the same way (as shown in Fig. 4b). This could benefit Oracle 7.3 users, but would significantly harm Oracle 8 users. And at five seconds per query, the performance would not be very satisfying.<br />
<br />
A better solution was to do away with the enormous inlists altogether. After some research, it turned out that the application in question was a PowerBuilder screen where an end-user could query order information from the database based on many different types of criteria. Instead of building one query to retrieve order information based on the end-user’s criteria, the application worked by retrieving order_ids of orders meeting the criteria and then retrieving the orders themselves using a second query along the lines of the query shown in Fig. 4a.<br />
<br />
Breaking the operation into two separate SQL statements like this perhaps made the application easier for the developer to build, but offered no other benefits. At the same time, this strategy came at a high performance cost for Oracle 7.2 and 7.3 users. Acme ultimately got the application to run faster on all releases of Oracle by merging the two SQL statements together and replacing queries like the one in Fig. 4a with dynamically generated queries such as the one shown in Fig. 5.<br />
<br />
SELECT ORD.order_id, ORD.order_number, <br />
ORD.status, CUST.customer_name, <br />
CAT.category, SHIP.description<br />
FROM orders ORD, customers CUST, <br />
customer_categories CAT,<br />
shipping_methods SHIP<br />
WHERE CUST.customer_id = ORD.customer_id<br />
AND CAT.category_id = CUST.category_id<br />
AND SHIP.shipping_method_id = <br />
ORD.shipping_method_id<br />
AND ORD.order_date BETWEEN :date_lo AND :date_hi<br />
AND ORD.status = :order_status<br />
AND ORD.total BETWEEN :total_lo AND :total_hi <br />
<br />
Fig. 5: A query to efficiently retrieve orders based on ad hoc search criteria.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-65011648891021567312009-10-06T08:56:00.001-07:002009-10-06T08:56:48.308-07:00Tuning Your SQL for Maximum PerformanceNothing can make a good application perform badly like poorly tuned SQL. <br />
Suppose that you have a database that is used by the HR and payroll systems to keep track of employee information and payroll transactions. Consider the Employees table as shown below:<br />
create table employees (<br />
employee_id varchar2(9) not null,<br />
first_name varchar2(30) not null,<br />
last_name varchar2(30) not null,<br />
hire_date date not null,<br />
department_id varchar2(10) not null,<br />
salary number(9,2)<br />
);<br />
Now, suppose you want to find out how a simple query that looks up an employee by employee ID executes. To do this, you need to explain the execution plan of the query you are executing. In SQL*Plus, you could do this as follows:<br />
set autotrace traceonly explain;<br />
select first_name, last_name, hire_date<br />
from employees<br />
where employee_id = '123456789';<br />
set autotrace off;<br />
Execution Plan<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE<br />
(Cost=18 Card=1 Bytes=29)<br />
1 TABLE ACCESS (FULL) OF 'EMPLOYEES' (TABLE)<br />
(Cost=18 Card=1 Bytes=29)<br />
This output shows the execution plan for the query.<br />
Line 0 shows important information about the entire query. First, note the portion that says Optimizer=CHOOSE. This indicates that Oracle's cost-based optimizer is choosing the mode in which it runs on the fly. There are two possible modes: on-line transaction processing (OLTP) and data warehousing (DW). An OLTP database, such as an airline reservation system, would likely see a high volume of transactions, while a data warehouse, such as a business process management system, might only be updated nightly, weekly, or monthly. Here, the default configuration of Oracle gives the optimizer the freedom to choose which mode to use based on the characteristics of this particular database.<br />
Next, you will notice that line 0 indicates Cost=18. This is the cost for the entire query, and is a relative indication of how much processing has to be done to execute it. In general, lower cost equals better performance. However, this is a relative measurement, not an absolute measurement of the computing resources needed to run the query. So, a query that has a cost of 16 would run slightly faster than this query does, and a query with a cost of 6000 would run significantly more slowly. The actual execution time and resources needed to run the query depend on many different factors, so the cost should be used to determine whether you are helping or hindering query performance when tuning a query. <br />
Next on this line, you will see Card=1. This value indicates how many rows must be processed in order to execute the query. This is Oracle's best guess, based on the data currently existing in the table.<br />
Finally on line 0, we have Bytes=29. This number indicates the amount of I/O in bytes that must occur between the client and the server to give back the results of the query.<br />
So, for purposes of query tuning, line 0 gives a good idea of the total resources needed to run the query, how many rows will be processed, and how much data must be exchanged between client and server to return the results. Now, on to the meat of the tune-up for this query.<br />
Line 1 shows TABLE ACCESS (FULL) OF 'EMPLOYEES' as step 1 of the execution plan. This means that, to run this query, every row must be read in the EMPLOYEES table until the right one is found. Data is kept in this table in the order in which it was inserted, so this might require one read, 20 reads, or 2 million reads, depending on when the employee ID 123456789 was inserted. In general, a full-table scan is the absolute worst-performing way to get data from a table.<br />
So, how can the query be fixed? By creating a primary key on the table. Create the primary key as follows:<br />
alter table employees<br />
add constraint employees_pk primary key (employee_id);<br />
Now, performing the explanation of the query again gives the following execution plan:<br />
Execution Plan<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE<br />
(Cost=1 Card=1 Bytes=29)<br />
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMPLOYEES' (TABLE)<br />
(Cost=1 Card=1 Bytes=29)<br />
2 1 INDEX (UNIQUE SCAN) OF 'EMPLOYEES_PK' (INDEX (UNIQUE)) (<br />
Cost=0 Card=1)<br />
So, what has been accomplished by adding the primary key? On line 2, notice that you have INDEX (UNIQUE SCAN) OF 'EMPLOYEES_PK'. Scanning an index is far quicker than scanning an entire table for data. In this case, a unique scan is used; this means that the Oracle optimizer is able to figure out whether the row exists in the table by doing one lookup on the index. Also, line 1 shows TABLE ACCESS (BY INDEX ROWID) OF 'EMPLOYEES'. When Oracle goes to the primary key to look up the data, it is able to find the row ID of the needed row. The row ID is like a unique street address of this row. Once this value is known, it takes only one access of the table to retrieve the desired data.<br />
Finally, looking at line 0, you can see that the cost has decreased from 18 down to 1. It is easy to see that adding the primary key has improved the query's performance.<br />
Now, on to some more complicated examples.<br />
Searching For Answers<br />
Suppose that you want to perform a search on the Employees table by first or last name. Although there now is a primary key on this table, it will not help in this case, because only the employee ID was included in the primary key. So, running a plan explanation on the following query:<br />
set autotrace traceonly explain;<br />
select first_name, last_name, hire_date<br />
from employees<br />
where last_name like 'Doe%';<br />
set autotrace off;<br />
results in the following familiar and less-than-ideal results:<br />
Execution Plan<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE<br />
(Cost=18 Card=1 Bytes=19)<br />
1 0 TABLE ACCESS (FULL) OF 'EMPLOYEES' (TABLE)<br />
(Cost=18 Card=1 Bytes=19)<br />
How can you speed up this query's performance? The solution is to create an index on the search fields as follows:<br />
create index employees_idx01 on employees (last_name, first_name);<br />
Creating this index yields the following new execution plan:<br />
Execution Plan<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE<br />
(Cost=2 Card=1 Bytes=19)<br />
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMPLOYEES' (TABLE)<br />
(Cost=2 Card=1 Bytes=19)<br />
2 1 INDEX (RANGE SCAN) OF 'EMPLOYEES_IDX01' (INDEX)<br />
(Cost=1 Card=1)<br />
<br />
The index range scan on line 2 is like your search through the phone book; you must scan through the names to find the address of the row you want. Then, once the address is found, the data is retrieved by index row ID, which is the most efficient way to locate the data in the table.<br />
Suppose you have a table of departments as shown below:<br />
create table departments (<br />
department_id varchar2(10) not null,<br />
department_name varchar2(30)<br />
);<br />
<br />
alter table departments<br />
add constraint department_pk primary key (department_id);<br />
<br />
alter table employees<br />
add constraint employees_fk1<br />
foreign key (department_id)<br />
references departments (department_id);<br />
Now, suppose you want to find all employees in the Accounting department, as shown in the following query:<br />
select e.employee_id, e.first_name, e.last_name<br />
from employees e, departments d<br />
where d.department_name = 'Accounting' and<br />
d.department_id = e.department_id;<br />
This query has the following execution plan:<br />
Execution Plan<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE<br />
(Cost=22 Card=1 Bytes=49)<br />
1 0 NESTED LOOPS<br />
2 1 NESTED LOOPS<br />
(Cost=22 Card=1 Bytes=49)<br />
3 2 TABLE ACCESS (FULL) OF 'EMPLOYEES' (TABLE)<br />
(Cost=18 Card=4 Bytes=100)<br />
<br />
4 2 INDEX (UNIQUE SCAN) OF 'DEPARTMENT_PK' (INDEX (UNIQUE) )<br />
(Cost=0 Card=1)<br />
<br />
5 1 TABLE ACCESS (BY INDEX ROWID) OF 'DEPARTMENTS' (TABLE)<br />
(Cost=1 Card=1 Bytes=24)<br />
Obviously, the poorest-performing portion of this query is the full-table scan on the Employees table. Creating an index on this table to optimize a search by department_id as follows:<br />
create index employees_idx02<br />
on employees (department_id, employee_id);<br />
yields the following execution plan:<br />
Execution Plan<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE<br />
(Cost=7 Card=1 Bytes=49)<br />
1 0 NESTED LOOPS<br />
2 1 NESTED LOOPS<br />
(Cost=7 Card=1 Bytes=49)<br />
3 2 VIEW OF 'index$_join$_001' (VIEW)<br />
(Cost=3 Card=4 Bytes=100)<br />
4 3 HASH JOIN<br />
5 4 INDEX (FAST FULL SCAN) OF 'EMPLOYEES_IDX01' (INDEX)<br />
(Cost=1 Card=4 Bytes=100)<br />
6 4 INDEX (FAST FULL SCAN) OF 'EMPLOYEES_IDX02' (INDEX)<br />
(Cost=1 Card=4 Bytes=100)<br />
<br />
7 2 INDEX (UNIQUE SCAN) OF 'DEPARTMENT_PK' (INDEX (UNIQUE) )<br />
(Cost=0 Card=1)<br />
<br />
8 1 TABLE ACCESS (BY INDEX ROWID) OF 'DEPARTMENTS' (TABLE)<br />
(Cost=1 Card=1 Bytes=24)<br />
In this situation, the department_id field is an example of a foreign key. This means that a value in a table is a key value in another table. In this case, the department_id field is unique on the departments table because it is the table's primary key. It is also an attribute of the Employees table because each employee belongs to a department. Having a foreign key constraint on the Employees table insures the data integrity of the employee data because this constraint will not allow an employee record to contain any value that does not exist in the Departments table. <br />
Having an index that contains the department_id field on the Employees table increases the efficiency of the search, as shown from the decrease in cost after the index is created. Any time that a foreign key relationship exists like the one described here, it is always best to have an index that contains that foreign key.<br />
So, if indexes are such wonderful things for query performance, why not just create an index on every column of every table? In theory, this sounds great, but the convenience of an index comes at a price. Having many indexes on a table will slow down DML operations, such as inserts, updates, and deletes. This is because each time a row in the table is inserted, changed, or removed, the associated indexes also have to be changed to reflect the table data. Also, if there are any foreign key constraints to maintain data integrity as shown in this example, Oracle must verify that any value that is about to be inserted is valid.<br />
This will slow down these operations significantly for tables with many columns that contain many indexes. The definition of "many" indexes is rather subjective. However, the best practice is to create indexes only where foreign key constraints or search functionality calls for them.<br />
Take A Hint<br />
Creating indexes can improve query performance; however, sometimes, even when appropriate indexes are created, Oracle is still not able to figure out the best way to execute a query. Consider the following example.<br />
Suppose you have a payroll transactions table and indexes as follows:<br />
create table payroll_transactions (<br />
transaction_id number(38) not null,<br />
transaction_type varchar2(10) not null,<br />
transaction_date date not null,<br />
employee_id varchar2(9) not null,<br />
amount number(9, 2) not null<br />
);<br />
<br />
alter table payroll_transactions<br />
add constraint payroll_transaction_pk primary key<br />
(transaction_id);<br />
<br />
create index payroll_transaction_idx01 on<br />
payroll_transactions (transaction_date);<br />
<br />
create index payroll_transaction_idx02 on<br />
payroll_transactions (transaction_date, transaction_type);<br />
<br />
create index payroll_transaction_idx03 on<br />
payroll_transactions (employee_id);<br />
<br />
create index payroll_transaction_idx04 on<br />
payroll_transactions (transaction_date, employee_id);<br />
Now, suppose you want to figure out how much money a particular employee has been paid via automatic deposit since his hire date:<br />
select sum(amount) from payroll_transactions where transaction_date > to_date('01/01/2008','mm/dd/yyyy') and employee_id = '123456789';<br />
This query would use the payroll_transaction_idx03 index because it is the first index on the table containing the employee ID, as shown by the following execution plan:<br />
Execution Plan<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE<br />
(Cost=2 Card=1 Bytes=20)<br />
1 0 SORT (AGGREGATE)<br />
2 1 TABLE ACCESS (BY INDEX ROWID) OF<br />
'PAYROLL_TRANSACTIONS' (TABLE)<br />
(Cost=2 Card=1 Bytes=20)<br />
<br />
3 2 INDEX (RANGE SCAN) OF 'PAYROLL_TRANSACTION_IDX03' (IND EX)<br />
(Cost=1 Card=2)<br />
However, the table also has payroll_transaction_idx04, which contains both the transaction date and the employee ID, which is a perfect fit. Luckily, there is a way to show Oracle the preferred way for the query to be executed: by giving the optimizer a hint. Executing the following query:<br />
select /*+INDEX(payroll_transactions payroll_transaction_idx04)*/<br />
sum(amount)<br />
from payroll_transactions<br />
where transaction_date > to_date('01/01/2008',<br />
'mm/dd/yyyy') and<br />
employee_id = '123456789';<br />
yields the following execution plan, which shows that the optimizer is using the desired index:<br />
Execution Plan<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE<br />
(Cost=2 Card=1 Bytes=20)<br />
1 0 SORT (AGGREGATE)<br />
2 1 TABLE ACCESS (BY INDEX ROWID) OF 'PAYROLL_TRANSACTIONS' (TABLE)<br />
(Cost=2 Card=1 Bytes=20)<br />
Although the cost for both queries is the same for the small sample data set used here, consider a payroll transaction table containing data for thousands of employees over the course of many years. The index containing the transaction date and employee ID would be the choice that would contain the most unique information. The hint forces the optimizer to use the correct index with the following:<br />
/*+INDEX(payroll_transactions payroll_transaction_idx04)*/<br />
The hint is always enclosed in /* and */, and must always begin with a plus sign. Then, the hint instructs Oracle to use an index by specifying the INDEX keyword. Finally, the desired table name and index on that table are specified by putting them in parentheses after the index hint. Failing to get the syntax of the hint exactly right will result in Oracle ignoring the hint.<br />
The same thing could be accomplished by telling Oracle to exclude the payroll_transaction_idx03 index specifically by executing the following query:<br />
<br />
select<br />
/*+NO_INDEX(payroll_transactions payroll_transaction_idx03)*/<br />
sum(amount) from payroll_transactions where transaction_date > to_date('01/01/2008', 'mm/dd/yyyy') and employee_id = '123456789';<br />
This would result in the same execution plan above where payroll_transaction_idx04 is used.<br />
A word of caution is necessary here concerning table aliases. Aliases are often used as shorthand to make complicated queries a little easier to read. If aliases were to be used, as in the following query:<br />
select /*+INDEX(payroll_transactions payroll_transaction_idx04)*/ b.first_name, b.last_name, sum(a.amount) from payroll_transactions a, employees b where a.transaction_date > to_date('01/01/2008',<br />
'mm/dd/yyyy') and a.employee_id = '123456789' and a.employee_id = b.employee_id;<br />
the hint specified would be ignored. Instead, you must use the alias within the hint in place of the table name:<br />
select /*+INDEX(a payroll_transaction_idx04)*/ sum(amount)<br />
from payroll_transactions where transaction_date > to_date('01/01/2008', 'mm/dd/yyyy') and employee_id = '123456789';<br />
Now, consider another handy hint. As discussed earlier, performing a full-table scan is usually very inefficient, and should be avoided, but this is not always the case. Consider the following query on the Departments table:<br />
select department_id from departments where department_name = 'Accounting';<br />
If the Departments table contains a small number of rows, it may be more efficient to read the entire table than it is to use an index. To force Oracle to bypass the index, you can specify a hint as follows:<br />
select /*+FULL(departments)*/ department_id from departments<br />
where department_name = 'Accounting';<br />
This may seem counterintuitive, but small tables are likely to be cached in memory, and would require no disk reads. Specifying the hint actually cuts down on I/O and speeds up the query.<br />
Another example where a full-table scan might prove to be more efficient than an index is a case where the makeup of the data is not very unique. Consider a Gender column on the Employees table with only M for male or F for female employees. To get a count of the number of male and female employees:<br />
select gender, count(*) from employees group by gender;<br />
Oracle would need to scan all rows in the table to get the counts anyway, so using any index would just add overhead to the process. If an index were used, each row lookup would require that Oracle find it in the index, requiring one I/O operation, and then another to read the data from the table when the row is found from the index. This would nearly double the amount of I/O required to execute the query.<br />
Hints should be used only in cases where a database design change or creation of a well-thought-out index does not help boost performance. Ideally, once a query is tuned using a hint, changes should be put in place so that Oracle's cost-based optimizer can run the query efficiently without the hint. Still, hints can be extremely helpful when solving performance issues, and are still necessary in cases where changes to the database are not possible.<br />
The Numbers Game<br />
So far, the cost value shown in the query execution plan has been used to gauge how well a query will perform. Now, it's time to take a look at how that cost is calculated.<br />
For each table, index, and column in the database, Oracle keeps statistics on the composition of the data.<br />
For tables, this includes:<br />
1. the number of rows in the table<br />
2. the amount of storage required to store all of the table's data<br />
3. the average size of each row in the table. <br />
The number of rows is important when determining, for example, whether it is more efficient to use an index or to scan through a table sequentially. The number of data blocks would give Oracle an idea of how much I/O would be necessary to read through the table's data.<br />
For indexes, Oracle tracks the number of index entries and the uniqueness of these entries. This is how Oracle calculates the cardinality number seen earlier in the query execution plans.<br />
For columns, Oracle tracks the number of distinct values, null values, and the density of the data. For example, consider a Gender column in a table that only contains M or F for male or female. Oracle would use the information about the relative non-uniqueness of this data when determining an execution plan.<br />
All of this information helps Oracle's optimizer to make wise decisions about query execution.<br />
Statistics are usually recomputed periodically by the database administrator based either on a period of time since the last time the statistics were gathered, or on the amount of data that has changed in the table since that time. If statistics become so out-of-date that they no longer reflect the actual makeup of the table's data, Oracle's optimizer may make bad decisions about query execution plans. Likewise, if a significant amount of data has changed over a short period of time, query performance might suddenly take a nose-dive. To address this, most DBAs use scripts to periodically re-analyze tables whose statistics are becoming old. Oracle also keeps track of the amount of data that has changed since the last table analysis, and can flag the statistics as stale if too much activity has occurred since the last analysis.<br />
So, how could stale statistics impact database performance? Suppose the Employees table has only 2 records in it when statistics are created. At that point, the Oracle query optimizer would decide that a full-table scan of Employees is the quickest way to find the desired row. Then, suppose that 100,000 rows are inserted into the table, and statistics are not re-created. The query optimizer would still be working under the assumption that the table is very small, and would recommend a full-table scan, which would obviously perform poorly now that the table contained much more data.<br />
The first way to gather statistics is to have Oracle analyze the complete table data and recompute all of the statistics for the table. The command to do this is as follows:<br />
Analyze table Employees compute statistics;<br />
During this process, Oracle exhaustively looks at all of the data and recalculates all of the statistical values for that table. Although this is likely to yield a very accurate analysis of the table, this process can take a very long time if performed on large tables. The process usually must be performed during a maintenance window because Oracle must lock the table being analyzed, which would hold up any pending transactions. This problem is now somewhat addressed by the implementation of Oracle RAC, a system of clustering databases. Oracle RAC allows the same database to be clusterd on multiple nodes, so maintenance can be done on one node while another is still available to applications. If Oracle RAC is not an option, however, there is another way to compute statistics that takes less time:<br />
Analyze table Employees estimate statistics sample 1000 rows;<br />
Or:<br />
Analyze table Employees estimate statistics sample 10 percent;<br />
This command does recompute statistics for the table based on actual data, but only on a fraction of it. In the first example, Oracle will sample 1000 rows of data; in the second, it will sample 10% of the total data in the table. Thus, even for a large table, the time to complete the analysis can be far less than the full analysis performed by totally recomputing all statistics.<br />
Later versions of Oracle have implemented special packages for the purpose of analyzing databases, schemas, and tables. Using these utility packages offers a bit more flexibility than the analyze commands above. To analyze a table and compute statistics, one can use the following command:<br />
exec DBMS_STATS.gather_table_stats('MYSCHEMA', 'Employees');<br />
To estimate statistics for a table, use the following:<br />
Exec DBMS_STATS.gather_table_stats('MYSCHEMA', 'Employees',<br />
estimate_percent = 10);<br />
It is also possible to set statistics manually. This might be necessary as a stop-gap measure to fix poorly running queries until a more thorough job of gathering statistics can be done. It might also be useful if a table often fluctuates between a small and large amount of data over a short period of time, such as a log table whose data is purged out every night.<br />
It is especially important to be sure that all tables referenced in a query have statistics available to the optimizer. Having one table with statistics and another without them is a recipe for inconsistent query performance. Oracle may attempt to estimate the statistics on-the-fly when this occurs, resulting in a performance hit. Your database administrator should have measures in place to insure that statistics are recalculated regularly on all tables. However, if it is not possible to do this for all tables in a query, you can totally bypass the cost-based optimizer by specifying the /*+RULE*/ hint. This causes Oracle to revert to a much simpler approach that does not use statistics, and relies solely on the primary keys and indexes available on the table.<br />
Conclusion<br />
In this article, you have explored how to get an execution plan for a query, maximize query performance with indexes, use hints to tune queries, and generate statistics for Oracle's cost-based optimizer to use when creating query execution plans. Hopefully, these tools will get you well on your way to writing efficient SQL for your applications.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com1tag:blogger.com,1999:blog-8692881074449232849.post-67792704828461083332009-10-06T08:54:00.001-07:002009-10-06T08:54:23.232-07:00Tuning Oracle's Buffer CacheIntroduction <br />
Oracle maintains its own buffer cache inside the system global area (SGA) for each instance. A properly sized buffer cache can usually yield a cache hit ratio over 90%, meaning that nine requests out of ten are satisfied without going to disk. <br />
If a buffer cache is too small, the cache hit ratio will be small and more physical disk I/O will result. If a buffer cache is too big, then parts of the buffer cache will be under-utilized and memory resources will be wasted. <br />
Checking The Cache Hit Ratio <br />
Oracle maintains statistics of buffer cache hits and misses. The following query will show you the overall buffer cache hit ratio for the entire instance since it was started: <br />
SELECT (P1.value + P2.value - P3.value) / (P1.value + P2.value)<br />
FROM v$sysstat P1, v$sysstat P2, v$sysstat P3<br />
WHERE P1.name = 'db block gets'<br />
AND P2.name = 'consistent gets'<br />
AND P3.name = 'physical reads'<br />
You can also see the buffer cache hit ratio for one specific session since that session started: <br />
SELECT (P1.value + P2.value - P3.value) / (P1.value + P2.value)<br />
FROM v$sesstat P1, v$statname N1, v$sesstat P2, v$statname N2,<br />
v$sesstat P3, v$statname N3<br />
WHERE N1.name = 'db block gets'<br />
AND P1.statistic# = N1.statistic#<br />
AND P1.sid = <enter SID of session here><br />
AND N2.name = 'consistent gets'<br />
AND P2.statistic# = N2.statistic#<br />
AND P2.sid = P1.sid<br />
AND N3.name = 'physical reads'<br />
AND P3.statistic# = N3.statistic#<br />
AND P3.sid = P1.sid<br />
You can also measure the buffer cache hit ratio between time X and time Y by collecting statistics at times X and Y and computing the deltas. <br />
Adjusting The Size Of The Buffer Cache <br />
The db_block_buffers parameter in the parameter file determines the size of the buffer cache for the instance. The size of the buffer cache (in bytes) is equal to the value of the db_block_buffers parameter multiplied by the data block size. <br />
You can change the size of the buffer cache by editing the db_block_buffers parameter in the parameter file and restarting the instance. <br />
Determining If The Buffer Cache Should Be Enlarged <br />
If you set the db_block_lru_extended_statistics parameter to a positive number in the parameter file for an instance and restart the instance, Oracle will populate a dynamic performance view called v$recent_bucket. This view will contain the same number of rows as the setting of the db_block_lru_extended_statistics parameter. Each row will indicate how many additional buffer cache hits there might have been if the buffer cache were that much bigger. <br />
For example, if you set db_block_lru_extended_statistics to 1000 and restart the instance, you can see how the buffer cache hit ratio would have improved if the buffer cache were one buffer bigger, two buffers bigger, and so on up to 1000 buffers bigger than its current size. Following is a query you can use, along with a sample result: <br />
SELECT 250 * TRUNC (rownum / 250) + 1 || ' to ' || <br />
250 * (TRUNC (rownum / 250) + 1) "Interval", <br />
SUM (count) "Buffer Cache Hits"<br />
FROM v$recent_bucket<br />
GROUP BY TRUNC (rownum / 250)<br />
<br />
Interval Buffer Cache Hits<br />
--------------- --------------------<br />
1 to 250 16083<br />
251 to 500 11422<br />
501 to 750 683<br />
751 to 1000 177<br />
This result set shows that enlarging the buffer cache by 250 buffers would have resulted in 16,083 more hits. If there were about 30,000 hits in the buffer cache at the time this query was performed, then it would appear that adding 500 buffers to the buffer cache might be worthwhile. Adding more than 500 buffers might lead to under-utilized buffers and therefore wasted memory. <br />
There is overhead involved in collecting extended LRU statistics. Therefore you should set the db_block_lru_extended_ statistics parameter back to zero as soon as your analysis is complete. <br />
In Oracle7, the v$recent_bucket view was named X$KCBRBH. Only the SYS user can query X$KCBRBH. Also note that in X$KCBRBH the columns are called indx and count, instead of rownum and count. <br />
Determining If The Buffer Cache Is Bigger Than Necessary <br />
If you set the db_block_lru_statistics parameter to true in the parameter file for an instance and restart the instance, Oracle will populate a dynamic performance view called v$current_bucket. This view will contain one row for each buffer in the buffer cache, and each row will indicate how many of the overall cache hits have been attributable to that particular buffer. <br />
By querying v$current_bucket with a GROUP BY clause, you can get an idea of how well the buffer cache would perform if it were smaller. Following is a query you can use, along with a sample result: <br />
SELECT 1000 * TRUNC (rownum / 1000) + 1 || ' to ' || <br />
1000 * (TRUNC (rownum / 1000) + 1) "Interval",<br />
SUM (count) "Buffer Cache Hits"<br />
FROM v$current_bucket<br />
WHERE rownum > 0 <br />
GROUP BY TRUNC (rownum / 1000)<br />
<br />
Interval Buffer Cache Hits<br />
------------ ----------------- <br />
1 to 1000 668415 <br />
1001 to 2000 281760 <br />
2001 to 3000 166940 <br />
3001 to 4000 14770 <br />
4001 to 5000 7030 <br />
5001 to 6000 959 <br />
This result set shows that the first 3000 buffers are responsible for over 98% of the hits in the buffer cache. This suggests that the buffer cache would be almost as effective if it were half the size; memory is being wasted on an oversized buffer cache. <br />
There is overhead involved in collecting LRU statistics. Therefore you should set the db_block_lru_statistics parameter back to false as soon as your analysis is complete. <br />
In Oracle7, the v$current_bucket view was named X$KCBCBH. Only the SYS user can query X$KCBCBH. Also note that in X$KCBCBH the columns are called indx and count, instead of rownum and count. <br />
Full Table Scans <br />
When Oracle performs a full table scan of a large table, the blocks are read into the buffer cache but placed at the least recently used end of the LRU list. This causes the blocks to be aged out quickly, and prevents one large full table scan from wiping out the entire buffer cache. <br />
Full table scans of large tables usually result in physical disk reads and a lower buffer cache hit ratio. You can get an idea of full table scan activity at the data file level by querying v$filestat and joining to SYS.dba_data_files. Following is a query you can use and sample results: <br />
SELECT A.file_name, B.phyrds, B.phyblkrd<br />
FROM SYS.dba_data_files A, v$filestat B<br />
WHERE B.file# = A.file_id<br />
ORDER BY A.file_id<br />
<br />
FILE_NAME PHYRDS PHYBLKRD<br />
-------------------------------- ---------- ----------<br />
/u01/oradata/PROD/system01.dbf 92832 130721<br />
/u02/oradata/PROD/temp01.dbf 1136 7825<br />
/u01/oradata/PROD/tools01.dbf 7994 8002<br />
/u01/oradata/PROD/users01.dbf 214 214<br />
/u03/oradata/PROD/rbs01.dbf 20518 20518<br />
/u04/oradata/PROD/data01.dbf 593336 9441037<br />
/u05/oradata/PROD/data02.dbf 4638037 4703454<br />
/u06/oradata/PROD/index01.dbf 1007638 1007638<br />
/u07/oradata/PROD/index02.dbf 1408270 1408270<br />
PHYRDS shows the number of reads from the data file since the instance was started. PHYBLKRD shows the actual number of data blocks read. Usually blocks are requested one at a time. However, Oracle requests blocks in batches when performing full table scans. (The db_file_multiblock_read_count parameter controls this batch size.) <br />
In the sample result set above, there appears to be quite a bit of full table scan activity in the data01.dbf data file, since 593,336 read requests have resulted in 9,441,037 actual blocks read. <br />
Spotting I/O Intensive SQL Statements <br />
The v$sqlarea dynamic performance view contains one row for each SQL statement currently in the shared SQL area of the SGA for the instance. v$sqlarea shows the first 1000 bytes of each SQL statement, along with various statistics. Following is a query you can use: <br />
SELECT executions, buffer_gets, disk_reads, <br />
first_load_time, sql_text<br />
FROM v$sqlarea<br />
ORDER BY disk_reads<br />
EXECUTIONS indicates the number of times the SQL statement has been executed since it entered the shared SQL area. BUFFER_GETS indicates the collective number of logical reads issued by all executions of the statement. DISK_READS shows the collective number of physical reads issued by all executions of the statement. (A logical read is a read that resulted in a cache hit or a physical disk read. A physical read is a read that resulted in a physical disk read.) <br />
You can review the results of this query to find SQL statements that perform lots of reads, both logical and physical. Consider how many times a SQL statement has been executed when evaluating the number of reads. <br />
Conclusion <br />
This brief document gives you the basic information you need in order to optimize the buffer cache size for your Oracle database. Also, you can zero in on SQL statements that cause a lot of I/O, and data files that experience a lot of full table scans.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-19364922498622727552009-10-06T08:52:00.001-07:002009-10-06T08:52:30.462-07:00Tuning JoinsAbstract<br />
<br />
A SQL statement involving two or more tables requires that Oracle join data from multiple sources in order to achieve the desired result. The Oracle Server has several built in methods for performing joins, and each method has its own performance and resource usage qualities. While an application developer or DBA doesn’t need to explicitly tell Oracle which join method to use or the order in which different tables should be joined, a solid understanding of how Oracle processes joins allows you to design and build systems for optimal performance. In this presentation we’ll examine the different methods Oracle uses for joining data from multiple sources, and we’ll see how this understanding can be used to design better database schemas, tune SQL statements, and tune databases as a whole for better join performance. Since most SQL statements involve joins, tuning joins leads to improved overall system performance.<br />
<br />
How Oracle Processes SQL Statements Involving Joins<br />
<br />
Oracle’s optimizer must choose an execution plan for each SQL statement submitted. The plan will indicate how data will be accessed—which indexes will be used and so forth. If the SQL statement involves a join or a sub query, then the optimizer must also choose the order in which tables will be joined together, and the method to be used for each join operation. <br />
<br />
Join Order<br />
<br />
A row source is a logical set of rows from a partially complete query. The row source may be rows in an actual table or rows in a temporary segment stored on disk or in memory. Oracle always joins two row sources at a time. Therefore, if a SQL statement joins three tables, Oracle will actually join two tables together and then join the results with the third table.<br />
<br />
The order in which row sources are joined can have a significant impact on performance. The optimizer makes a list of all physically possible join orders and then chooses what it believes will be the best option. The cost-based optimizer makes this decision by estimating the cost of each option and choosing the one with the lowest cost. The rule-based optimizer, meanwhile, scores each option based on a fixed set of rules and chooses the option with the best score.<br />
<br />
Outer joins limit the number of possible join orders because a table being outer joined must appear in the join order after the table it joins to.<br />
<br />
Let’s use the following query as an example:<br />
<br />
SELECT E1.name “Employee Name”, D.deptname “Department”,<br />
E2.name “Manager Name”<br />
FROM emp E1, dept D, emp E2<br />
WHERE E1.empno = :empno<br />
AND D.deptno = E1.deptno<br />
AND E2.empno (+) = E1.mgr_empno;<br />
<br />
Oracle could execute this SQL statement by joining emp table E1 to dept, and then joining the results to emp table E2. Alternatively, the order could be E1 - E2 - D. Note that the order D - E1 - E2 is also possible, but probably not too efficient. Also note that the order D - E2 - E1 is not physically possible because E2 is being outer joined to E1 and therefore must appear after E1 in the join order. And since D and E2 share no criteria in the WHERE clause, such a join order wouldn’t seem to make much sense anyway.<br />
<br />
Join Methods<br />
<br />
Since the days of Oracle V6, the optimizer has used three different ways to join row sources together. These are the nested loops join, the sort-merge join, and the cluster join. With Oracle 7.3 the hash join was introduced, making for a total of four join methods. Each has a unique set of features and limitations.<br />
<br />
Nested Loops Joins<br />
<br />
Suppose somebody gave you a telephone book and a list of 20 names, and asked you to write down each person’s name and telephone number on a fresh sheet of paper. You would probably go down the list of names, looking each up in the telephone book one at a time. This would be pretty easy for you, because the telephone book is alphabetized by name. Moreover, somebody looking over your shoulder could begin calling the first few numbers you write down while you are still looking up the rest. This scene describes a nested loops join.<br />
<br />
In a nested loops join, Oracle reads the first row from the first row source and then checks the second row source for matches. All matches are then placed in the result set and Oracle goes on to the next row from the first row source. This continues until all rows in the first row source have been processed. The first row source is often called the outer table or driving table, while the second row source is called the inner table. <br />
<br />
Nested loops joins are ideal when the driving row source is small and the joined columns of the inner row source are uniquely indexed or have a highly selective non-unique index. Nested loops joins have an advantage over other join methods in that they can quickly retrieve the first few rows of the result set without having to wait for the entire result set to be determined. This is ideal for query screens where an end-user can read the first few records retrieved while the rest are being fetched. Nested loops joins are also flexible in that any two row sources can always be joined by nested loops—regardless of join condition and schema definition. <br />
<br />
However, nested loops joins can be very inefficient if the inner row source does not have an index on the joined columns or if the index is not highly selective. Also, if the driving row source is quite large, other join methods may be more efficient.<br />
<br />
Sort-Merge Joins<br />
<br />
Suppose two salespeople attend a conference and each collects over 100 business cards from potential new customers. Each salesperson now has a pile of cards in random order, and they want to see how many cards are duplicated in both piles. So each salesperson alphabetizes their pile, and then they call off names one at a time. Since both piles of cards have been sorted, it becomes much easier to find the names that appear in both piles. This scene describes a sort-merge join.<br />
<br />
In a sort-merge join, Oracle sorts the first row source by its join columns, sorts the second row source by its join columns, and then “merges” the sorted row sources together. As matches are found, they are put into the result set.<br />
<br />
Sort-merge joins can be effective when lack of data selectivity or useful indexes render a nested loops join inefficient, or when both of the row sources are quite large. However, sort-merge joins can only be used for equijoins (WHERE D.deptno = E1.deptno, as opposed to WHERE T.temperature >= D.daily_avg_temperature). Also, sort-merge joins require temporary segments for sorting. This can lead to extra memory utilization and extra disk I/O in the temporary tablespace. <br />
<br />
Cluster Joins<br />
<br />
Imagine a filing cabinet full of personnel records. For each employee there is a separate folder containing a basic information sheet, a list of vacation days used, and other papers pertaining to the employee. If the boss wants to see a report that lists each employee’s name, social security number, and number of vacation days used this year, then you could work through the filing cabinet one file at a time. When you pull out one employee’s file, you immediately have access to the basic information sheet (which has the employee’s social security number on it) and the list of vacation days used. If you imagine that the basic information is stored in one database table and the vacation days used in another, then this scene could describe a cluster join.<br />
<br />
A cluster join is really just a special case of the nested loops join. If the two row sources being joined are actually tables that are part of a cluster and if the join is an equijoin between the cluster keys of the two tables, then Oracle can use a cluster join. In this case, Oracle reads each row from the first row source and finds all matches in the second row source by using the cluster index. In the personnel cabinet example, the filing cabinet is the cluster and the individual employee folders are the cluster key.<br />
<br />
Cluster joins are extremely efficient, since the joining rows in the two row sources will actually be located in the same physical data block. However, clusters carry certain caveats of their own, and you can’t have a cluster join without a cluster. Therefore, cluster joins are not very commonly used. In a later section we’ll look at the pros and cons of clusters.<br />
<br />
Hash Joins<br />
<br />
In a hash join, Oracle reads all of the join column values from the second row source, builds a hash table, and then probes the hash table for each of the join column values from the first row source. This is like a nested loops join, except that first Oracle builds a hash table to facilitate the operation.<br />
<br />
Hash joins can be effective when the lack of a useful index renders nested loops joins inefficient. The hash join might be faster than a sort-merge join in this case because only one row source needs to be sorted, and could possibly be faster than a nested loops join because probing a hash table in memory can be faster than traversing a B-tree index. As with sort-merge joins and cluster joins, though, hash joins only work on equijoins. Also, as with sort-merge joins, hash joins use memory resources and can drive up I/O in the temporary tablespace. Finally, hash joins are only available in Oracle 7.3 and later, and then only when cost-based optimization is used.<br />
<br />
Effective Schema Design<br />
<br />
The database schema plays an important role in tuning joins. Certain features in the schema, or the absence of these features, will limit the options available to the optimizer for joining tables efficiently. Also, poorly placed features in the schema can fool the optimizer into making a poor decision. <br />
<br />
Indexing Keys<br />
<br />
All primary and unique keys should be identified and appropriate constraints declared. This will cause the creation of unique indexes on the primary and unique key columns. Indexes should usually be created explicitly on foreign key columns to facilitate joins and to eliminate locking problems when Oracle enforces referential integrity constraints. <br />
<br />
Consider an orders table with primary key order_id, and an order_lines table with primary key order_line_id and foreign key order_id referencing the orders table.<br />
<br />
SELECT O.order_number, L.line_number, L.item_number, L.quantity<br />
FROM orders O, order_lines L<br />
WHERE O.order_id = :order_id<br />
AND L.order_id = O.order_id<br />
ORDER BY O.order_number, L.line_number<br />
<br />
If the database will contain many orders, then the order_id column in the order_lines table is very selective and should be indexed. This index, plus the unique index on the order_id column of the orders table will allow Oracle to execute this query extremely efficiently using a nested loops join. <br />
<br />
First Oracle will use the unique index on the orders table to create a row source with one row from the orders table. Then Oracle will join this row source with all rows from the order_lines table. The index on the order_id column of the order_lines table allows Oracle to efficiently find all order lines for the specified order. Without this index, Oracle would probably perform a sort-merge join or hash join, requiring a scan of all rows in the order_lines table.<br />
<br />
Its not a hard and fast rule, though, that all foreign key columns should be indexed. Indexes should not be placed on columns that are not selective. Such indexes could trick the optimizer into choosing a bad execution plan. Suppose that an order tracking application has a table called order_placement_methods that lists how orders can be placed and an orders table that lists active orders. The orders table has an order_placement_method_id referencing the order_placement_methods table to indicate how a specific order was placed. If orders can be placed by telephone or via the Internet, and it is expected that roughly half the orders will be placed each way, then the foreign key in the orders table should definitely not be indexed. This index would not be selective, always retrieving about half the rows in the orders table.<br />
<br />
The Star Formation<br />
<br />
Beginning in Oracle 7.3, the optimizer recognizes a schema design known as a star formation. This design is common in data warehouses, and the optimizer can handle queries against these types of schemas using a special nested loops join that is an exception to the “only two row sources can be joined at a time” rule.<br />
<br />
In a star formation, several relatively small tables called dimension tables or lookup tables have simple primary keys. A large table called the fact table has a primary key that is a concatenation of the primary keys of the dimension tables. A schema with small tables for time, region, and product and a large table for sales figures would follow a star formation if each record in the sales table indicated sales of one product in one region during one time period. <br />
<br />
The following query would be executed very efficiently if the primary key on the sales table consists of the concatenation of the primary keys of the other three tables:<br />
<br />
SELECT S.*<br />
FROM time T, regions R, products P, sales S<br />
WHERE T.end_date BETWEEN SYSDATE - 1 AND SYSDATE<br />
AND R.name = ‘West Coast’<br />
AND P.bar_code = ‘184901724’<br />
AND S.time_id = T.time_id<br />
AND S.region_id = R.region_id<br />
AND S.product_id = P.product_id<br />
<br />
Clusters<br />
<br />
Clusters can improve join performance when two or more tables are usually accessed together, as in a master-detail relationship. Queries that join the clustered tables together run faster because each master record is stored in the same data block with its related detail records. Even a single table that is frequently self-joined or queried for all records with the same value of a key can benefit from being stored in a cluster.<br />
<br />
However, updates are slower when a table is clustered, as are full table scans. Also, storage tends not to be as efficient in a cluster. Therefore, much thought should be put in before clustering tables. In practice, clusters are not frequently used.<br />
<br />
Distributed Queries<br />
<br />
Because each node of a distributed database is an autonomous unit, distributed queries can be a serious bottleneck. Whereas Oracle can efficiently join two row sources located on the same node, joins between a local row source and a remote one can be costly. In a cross-node join, the local node must connect to the remote node and then formulate standard SQL statements to extract the desired data from the remote node. The optimizer on the local node does not have access to the remote node’s optimizer and therefore must make educated guesses as to the optimal way to execute the distributed query.<br />
<br />
In many cases, distributed queries can be tuned to work efficiently. This will be discussed in the section below on tuning SQL. For distributed queries that just can’t be made efficient, or where maximum performance is critical, replication can be used. Simple snapshots or Oracle’s Advanced Replication Option can be used to copy data from a remote node so that queries do not need to join row sources across nodes.<br />
<br />
Choosing Data Types<br />
<br />
Choose data types for all primary and unique keys carefully, and remain consistent throughout the schema. For example, if the primary key on an invoice table is a column called invoice_number and you select a data type of NUMBER(9) for this column, then always use this same data type for invoice numbers throughout the entire schema. If you were to use a data type of VARCHAR2(10) for the invoice_number column of the invoice_items table, then Oracle would need to perform a data type conversion when joining the invoices and invoice_items tables. This conversion costs CPU cycles and can sometimes defeat indexes, having a disastrous effect on performance.<br />
<br />
Primary and unique keys for large tables or tables that are heavily joined should be kept small and simple. I’ve worked on multiple projects where a critical table had as its primary key a column with a data type such as VARCHAR2(100). The primary key was actually a series of strings, concatenated together in order to make a unique “key”. Such a primary key is expensive because the large key must be carried in every referencing table, and also because Oracle must compare long strings when performing a join. <br />
<br />
A NUMBER(9) or VARCHAR2(5) column requires far fewer bytes and allows faster comparisons during a join. If the true unique key for a table in your application involves many columns or long strings, then consider declaring a unique key on these columns and adding a surrogate primary key column with data type NUMBER(9) that is populated from a sequence. All referencing tables could then carry the NUMBER(9) value instead of the more complex true unique key.<br />
<br />
Tuning SQL<br />
<br />
The more a database schema is designed with an eye toward optimizing joins, the less individual SQL statements need to be tuned. But still, SQL is such a flexible language that there are usually many ways to write a query. Being able to identify the efficient ways is a good skill to have.<br />
<br />
The Usual Suspects<br />
<br />
All of the usual fundamentals of tuning SQL still apply when tuning joins, including such techniques as using selective indexes where possible, avoiding unintentional index defeat caused by NVL() and other functions, and avoiding excessive sorting action by overworking the DISTINCT and GROUP BY operators. Since this presentation already has so much ground to cover, these basic tuning concepts will not be discussed here.<br />
<br />
Early Elimination of Candidate Rows<br />
<br />
Suppose you have a list of 1000 residents of your town along with each resident’s street address, and you are asked to prepare an alphabetized list of residents who have propane delivered to their home.<br />
<br />
You could first alphabetize the list of 1000 names and then look up each street address in the list of propane delivery locations, but instead you probably would look up each street address first and then alphabetize second. Why? Either way you will need to perform 1000 street address lookups. However, these lookups will eliminate many names from the list (since not all residents have propane delivered to their home) and thus your alphabetizing job will be easier.<br />
<br />
You can apply the same concept when writing SQL that joins tables together. The Oracle optimizer is pretty smart about choosing the most efficient order in which to do things, but how a query is written can constrain the options available to the optimizer.<br />
<br />
The query below leaves the optimizer no choice but to read all of Acme’s invoice lines, when in fact only the unpaid invoices are of interest:<br />
<br />
SELECT V.vendor_num, I.invoice_num, SUM (L.amount)<br />
FROM vendors V, invoices I, invoice_lines L<br />
WHERE V.vendor_name = ‘Acme’<br />
AND L.vendor_num = V.vendor_num<br />
AND I.vendor_num = L.vendor_num<br />
AND I.invoice_num = L.invoice_num<br />
AND I.paid = ‘N’<br />
GROUP BY V.vendor_num, I.invoice_num<br />
ORDER BY I.invoice_num<br />
<br />
This query could be rewritten as follows:<br />
<br />
SELECT V.vendor_num, I.invoice_num, SUM (L.amount)<br />
FROM vendors V, invoices I, invoice_lines L<br />
WHERE V.vendor_name = ‘Acme’<br />
AND I.vendor_num = V.vendor_num<br />
AND I.paid = ‘N’<br />
AND L.vendor_num = I.vendor_num<br />
AND L.invoice_num = I.invoice_num<br />
GROUP BY V.vendor_num, I.invoice_num<br />
ORDER BY I.invoice_num<br />
<br />
In the rewritten query, the optimizer eliminates all of the paid invoices before joining to the invoice_lines table. If most of the invoices in the database have already been paid, then the rewritten query will be significantly faster. (The schema design in this example is dubious, and is used for illustrative purposes only.)<br />
<br />
Forcing a Specific Join Method<br />
<br />
When choosing an execution plan for a query involving joins, the optimizer considers all possible join methods and join orders. The optimizer does its best to evaluate the merits of each option and to choose the optimal execution plan, but sometimes there is just no substitute for human intelligence. <br />
<br />
In these situations the USE_NL, USE_MERGE, and USE_HASH hints can be used to request a specific join method, and the ORDERED hint can be used to request a specific join order. The optimizer will do its best to observe the wishes of these hints, but if you ask for something impossible (such as a sort-merge join on an anti-join) the hint will be ignored. Note that there is no hint to request a cluster join.<br />
<br />
When tuning SQL that uses joins, you may wish to run benchmark comparisons between different join methods and orders if it’s not immediately obvious which would be best. For example, if a report joins two tables that form a master-detail relationship and the proper primary key and foreign key indexes are in place, the optimizer will probably choose to use a nested loops join. But if you know that this particular report joins all of the master records to all of the detail records, you might think it faster to use a sort-merge join or hash join instead. Run a benchmark and see. <br />
<br />
Figure 1 shows an example query and its TKPROF output. Figure 2 shows the same query with a USE_MERGE hint, and Figure 3 shows it with a USE_HASH hint. You can see that in this situation the hash join cut CPU time by almost 40 percent and logical I/Os by about 98 percent.<br />
<br />
SELECT A.BUSINESS_UNIT,A.PO_NUMBER,A.VENDOR_TYPE,B.LINE_NUMBER,<br />
B.LINE_AMOUNT,B.LINE_STATUS,B.DESCRIPTION <br />
FROM<br />
PURCHASE_ORDERS A,PURCHASE_ORDER_LINES B WHERE B.BUSINESS_UNIT =<br />
A.BUSINESS_UNIT AND B.PO_NUMBER = A.PO_NUMBER ORDER BY <br />
A.BUSINESS_UNIT,A.PO_NUMBER,B.LINE_NUMBER<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.01 0.01 0 0 0 0<br />
Execute 1 0.04 0.12 0 0 1 0<br />
Fetch 73370 23.47 23.55 2071 298667 2089 73369<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 73372 23.52 23.68 2071 298667 2090 73369<br />
<br />
Misses in library cache during parse: 0<br />
Optimizer goal: CHOOSE<br />
Parsing user id: 455 (RSCHRAG)<br />
<br />
Rows Execution Plan<br />
------- ---------------------------------------------------<br />
0 SELECT STATEMENT GOAL: CHOOSE<br />
73369 SORT (ORDER BY)<br />
73369 NESTED LOOPS<br />
73726 TABLE ACCESS GOAL: ANALYZED (FULL) OF <br />
'PURCHASE_ORDER_LINES'<br />
73369 TABLE ACCESS GOAL: ANALYZED (BY ROWID) OF <br />
'PURCHASE_ORDERS'<br />
73726 INDEX GOAL: ANALYZED (UNIQUE SCAN) OF <br />
'PURCHASE_ORDERS_PK' (UNIQUE)<br />
<br />
Figure 1: Benchmark of a query using a nested loops join.<br />
<br />
SELECT /*+ USE_MERGE (B) */A.BUSINESS_UNIT,A.PO_NUMBER,<br />
A.VENDOR_TYPE, B.LINE_NUMBER,B.LINE_AMOUNT,B.LINE_STATUS,<br />
B.DESCRIPTION <br />
FROM<br />
PURCHASE_ORDERS A,PURCHASE_ORDER_LINES B WHERE B.BUSINESS_UNIT = <br />
A.BUSINESS_UNIT AND B.PO_NUMBER = A.PO_NUMBER ORDER BY <br />
A.BUSINESS_UNIT, A.PO_NUMBER,B.LINE_NUMBER<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.01 0.01 0 0 0 0<br />
Execute 1 0.02 0.15 0 0 2 0<br />
Fetch 73370 17.49 19.57 3772 4165 3798 73369<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 73372 17.52 19.73 3772 4165 3800 73369<br />
<br />
Misses in library cache during parse: 1<br />
Optimizer goal: CHOOSE<br />
Parsing user id: 455 (RSCHRAG)<br />
<br />
Rows Execution Plan<br />
------- ---------------------------------------------------<br />
0 SELECT STATEMENT GOAL: CHOOSE<br />
73369 SORT (ORDER BY)<br />
73369 MERGE JOIN<br />
886 SORT (JOIN)<br />
886 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'PURCHASE_ORDERS'<br />
73726 SORT (JOIN)<br />
73726 TABLE ACCESS GOAL: ANALYZED (FULL) OF <br />
'PURCHASE_ORDER_LINES'<br />
<br />
Figure 2: Benchmark of the query from Figure 1, but using a sort-merge join instead.<br />
<br />
SELECT /*+ USE_HASH (B) */A.BUSINESS_UNIT,A.PO_NUMBER,<br />
A.VENDOR_TYPE,B.LINE_NUMBER,B.LINE_AMOUNT,B.LINE_STATUS,<br />
B.DESCRIPTION <br />
FROM<br />
PURCHASE_ORDERS A,PURCHASE_ORDER_LINES B WHERE B.BUSINESS_UNIT = <br />
A.BUSINESS_UNIT AND B.PO_NUMBER = A.PO_NUMBER ORDER BY <br />
A.BUSINESS_UNIT,A.PO_NUMBER,B.LINE_NUMBER<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.00 0 0 0 0<br />
Execute 1 0.05 0.13 0 0 1 0<br />
Fetch 73370 14.88 14.95 2071 4165 2093 73369<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 73372 14.93 15.08 2071 4165 2094 73369<br />
<br />
Misses in library cache during parse: 0<br />
Optimizer goal: CHOOSE<br />
Parsing user id: 455 (RSCHRAG)<br />
<br />
Rows Execution Plan<br />
------- ---------------------------------------------------<br />
0 SELECT STATEMENT GOAL: CHOOSE<br />
73369 SORT (ORDER BY)<br />
137807 HASH JOIN<br />
886 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'PURCHASE_ORDERS'<br />
73726 TABLE ACCESS GOAL: ANALYZED (FULL) OF <br />
'PURCHASE_ORDER_LINES'<br />
<br />
Figure 3 Benchmark of the query from Figure 1, but using a hash join instead.<br />
<br />
Distributed Queries<br />
<br />
As we discussed earlier, distributed queries can sometimes be disastrous for performance. In particular a nested loops join between two row sources on separate nodes of a distributed database can be very slow. Figure 4 shows a simple distributed query and its execution plan. This query will be slow because for each row retrieved from the customers table, a separate query will be dispatched to the remote node in order to see if there is an open booking for the customer. This will result in many small network packets moving between the two nodes of the database, and the network latency and overhead will degrade performance.<br />
<br />
Figure 5 shows the same query rewritten in a form that causes less network traffic. One query is sent to the remote node to determine all customers with open bookings. The results are the same, but performance is greatly improved. Both versions of the query use roughly the same CPU time and logical I/Os on the local node, but the elapsed time is about 97% less in Figure 5. This is due to reduced network overhead.<br />
<br />
SELECT CUSTOMER_ID,CUSTOMER_NAME,CLASS_CODE <br />
FROM<br />
CUSTOMERS CUST WHERE EXISTS (SELECT 1 FROM BOOKINGS@BOOK BKG WHERE <br />
BKG.CUSTOMER_ID = CUST.CUSTOMER_ID AND BKG.STATUS = 'open' ) ORDER BY <br />
CUSTOMER_NAME<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.01 0 0 0 0<br />
Execute 1 0.00 0.00 0 0 0 0<br />
Fetch 156 0.41 11.85 0 476 2 155<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 158 0.41 11.86 0 476 2 155<br />
<br />
Misses in library cache during parse: 0<br />
Optimizer goal: CHOOSE<br />
Parsing user id: 455 (RSCHRAG)<br />
<br />
Rows Execution Plan<br />
------- ---------------------------------------------------<br />
0 SELECT STATEMENT GOAL: CHOOSE<br />
155 SORT (ORDER BY)<br />
467 FILTER<br />
467 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'CUSTOMERS'<br />
0 REMOTE [BOOK.WORLD]<br />
SELECT "CUSTOMER_ID","STATUS" FROM "BOOKINGS" BKG WHERE <br />
"STATUS"='open' AND "CUSTOMER_ID"=:1<br />
<br />
Figure 4: Benchmark of a query using a nested loops join between two tables on separate nodes of a distributed database.<br />
<br />
SELECT CUSTOMER_ID,CUSTOMER_NAME,CLASS_CODE <br />
FROM<br />
CUSTOMERS WHERE CUSTOMER_ID IN (SELECT CUSTOMER_ID FROM BOOKINGS@BOOK <br />
WHERE STATUS = 'open' )ORDER BY CUSTOMER_NAME<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
Parse 1 0.00 0.01 0 0 0 0<br />
Execute 1 0.00 0.00 0 0 0 0<br />
Fetch 156 0.07 0.27 0 467 0 155<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
total 158 0.07 0.28 0 467 0 155<br />
<br />
Misses in library cache during parse: 0<br />
Optimizer goal: CHOOSE<br />
Parsing user id: 455 (RSCHRAG)<br />
<br />
Rows Execution Plan<br />
------- ---------------------------------------------------<br />
0 SELECT STATEMENT GOAL: CHOOSE<br />
155 SORT (ORDER BY)<br />
155 NESTED LOOPS<br />
156 VIEW<br />
1000 SORT (UNIQUE)<br />
1000 REMOTE [BOOK.WORLD]<br />
SELECT "CUSTOMER_ID","STATUS" FROM "BOOKINGS" BOOKINGS WHERE <br />
"STATUS"='open'<br />
155 TABLE ACCESS GOAL: ANALYZED (BY ROWID) OF 'CUSTOMERS'<br />
156 INDEX GOAL: ANALYZED (UNIQUE SCAN) OF 'SYS_C002109' <br />
(UNIQUE)<br />
<br />
Figure 5: Benchmark of the query from Figure 4, after being optimized to reduce network traffic.<br />
<br />
When distributed queries cannot be avoided, use IN clauses, set operators such as UNION and MINUS, and whatever else you can to reduce the network traffic between nodes of the database. Using views will not necessarily improve performance. Views will hide the complexity from the application developer, but all of the complexity and issues are still there under the covers.<br />
<br />
With Oracle 8, you can use the DRIVING_SITE hint to control which node of the distributed database drives the distributed query. In Oracle 7, the driving site will always be the node where the query originated. <br />
<br />
Tuning the Database<br />
<br />
In addition to optimizing the schema and tuning individual SQL statements, the database as a whole can be tuned in a way to optimize join performance. As in the previous section, we’ll focus on issues specific to joins and only glance at general tuning principles.<br />
<br />
Optimizer Statistics <br />
<br />
The query optimizer evaluates all possible execution plans for a given SQL statement and attempts to choose the best option. If you are using cost-based optimization, the optimizer will rely heavily on statistics in the data dictionary. Since queries with joins tend to have many possible execution paths, its especially important to keep these statistics in order.<br />
<br />
For every database schema except SYS, you should collect statistics for all clusters, tables, and indexes on a regular basis. Estimated statistics are sufficient; its usually not worth the overhead to collect computed statistics. Analyzing clusters and tables explicitly is sufficient, because indexes are analyzed automatically when a table is analyzed. If some critical tables have badly skewed data distributions, you might want to have Oracle build histograms while collecting statistics. You can do this in Oracle 7.3 and later, and this can make a big difference.<br />
<br />
I was once called in to look at a database where the initial loads ran quickly, but performance quickly degraded. It turned out that statistics had been collected when the database was almost empty. Statistics were not updated as tables grew in size, and so the optimizer began to make bad decisions. This is why it is important to collect statistics regularly. You can use an operating system scheduling function, such as cron in Unix, to do this. Or you can let Oracle take care of this itself by creating a database job (using dbms_job) that calls dbms_utility.analyze_schema on a regular basis.<br />
<br />
Arranging Tablespaces<br />
<br />
As you probably learned in DBA 101, it is important to create separate tablespaces for tables, indexes, temporary segments, and rollback segments. Separating tables from indexes is key to balancing disk I/O and reducing contention. This will significantly impact nested loops join performance since nested loops joins make heavy use of indexes.<br />
<br />
Since sort-merge joins rely on temporary segments, it’s important to dedicate a tablespace to holding temporary segments only and to designate each user’s temporary tablespace accordingly. In a large database with many concurrent users performing lots of sorts, it can be beneficial to have several temporary tablespaces spread across many disks. In Oracle 7.3 and later, you can designate the contents of a tablespace as temporary with the CREATE TABLESPACE command, and this speeds up dynamic space allocation and release for temporary segments.<br />
<br />
Be sure to size the temporary tablespace large enough to accommodate enough concurrent sort-merge joins, and size the initial, next, and max extents properly so that temporary segments will be able to grow large enough to perform the desired sorts. Even though Oracle 7.3 and later allow unlimited extents, using very small extent sizes in the temporary tablespace will lead to unnecessary dynamic space allocation.<br />
<br />
Initialization Parameters<br />
<br />
Performance of sort-merge joins and hash joins are strongly impacted by certain initialization parameters. Join performance can be crippled if these parameters are not set properly.<br />
<br />
Sort-Merge Joins<br />
<br />
db_file_multiblock_read_count specifies how many blocks Oracle should read at a time from disk when performing a sequential read such as a full table scan. Since sort-merge joins often involve full table scans, this parameter should be set properly. Larger values will reduce overhead when scanning large tables.<br />
<br />
sort_area_size specifies how much memory can be used for sorting, and this has a strong impact on performance of all sorts. Since sort-merge joins require sorting of both row sources, sort_area_size strongly impacts sort-merge join performance. If an entire sort cannot be completed in the amount of memory specified by this parameter, then a temporary segment in the temporary tablespace will be allocated. In this case the sort will be performed in memory one part at a time, and partial results will be stored on disk in the temporary segment. <br />
<br />
If sort_area_size is set very small, then excessive disk I/O will be necessary to perform even the smallest of sorts. If it is set too high, then the operating system may run out of physical memory and resort to swapping. Sort space is private to each session, and can consume lots of memory if there are a high number of concurrent users. For example, if sort_area_size is set to 5000000, then 100 concurrent users could theoretically use up 500 Mb of memory for sorting.<br />
<br />
Hash Joins<br />
<br />
The three parameters hash_join_enabled, hash_area_size, and hash_multiblock_io_count control hash join behavior. Unlike most other initialization parameters, these ones can also be dynamically altered with the ALTER SESSION command. <br />
<br />
hash_join_enabled dictates whether or not the optimizer should consider using hash joins. If you don’t want hash joins to be used, set this parameter to false.<br />
<br />
hash_area_size specifies how much memory can be used to build a hash table for a hash join, and bears a lot in common with sort_area_size. If this parameter is set too small, then partial hash tables will need to be stored in temporary segments. If this parameter is set too big, then physical memory may be exhausted. As with sort_area_size, hash_area_size indicates how much memory can be used per session. Many concurrent sessions can consume a lot of memory.<br />
<br />
hash_multiblock_io_count specifies how many blocks should be read at a time when building the hash table, somewhat similar to db_file_multiblock_read_count for sort-merge joins. However, the interplay between hash_multiblock_io_count and hash_area_size significantly impacts hash join performance. Before changing either of these parameters, consult the formula listed in the Oracle 7.3 or 8.0 Server Reference Guide.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com1tag:blogger.com,1999:blog-8692881074449232849.post-90138158365379965532009-10-06T08:50:00.001-07:002009-10-06T08:50:38.179-07:00Script to query the database for tablespaces and estimates and space utilizationDescription: This script queries your database for tablespaces and estimates their space utilization like allocated space, used space, free space available and percentage of used space.<br />
The script can also search if %used space is more than 80, send an email to concerned, and log details to a file. <br />
Code: <br />
<br />
#Description:<br />
# This script queries your database for tablespaces and estimates their space utilization like allocated space, used space, free space available and percentage of used space.<br />
# The script can also search if %used space is more than 80, send an email to concerned, and log details to a file.<br />
<br />
#You should not have to pass any parameter. It will search all the databases available in the oratab file<br />
<br />
#Code:<br />
#!/bin/sh<br />
#<br />
#Script name: estimate_space.sh<br />
#<br />
#Comments: Script checks for disk space utilisation of Tablespces in all DBs available in the oratab file<br />
#sends email messages to concerned if any tablespace is utilised 80 percent of allocated space.<br />
#-------------------------------------------------------------------------------------------------<br />
# Revision Log<br />
#<br />
# 00/00/00 : Name of modifier - description of Modifications<br />
#-------------------------------------------------------------------------------------------------<br />
#<br />
#Variables declared<br />
CWD=/opt/oracle<br />
DIR=/var/opt/oracle<br />
export CWD DIR<br />
<br />
#Variables picked up from the environment <br />
HOST=`uname -a | awk '{print $2}'`<br />
ORACLE_HOME=`grep -v '^#' $DIR/oratab | awk -F':' '{print $2}' | tail -1`<br />
export HOST ORACLE_HOME<br />
<br />
grep -v '^#' $DIR/oratab | awk -F':' '{print $1}' >/tmp/xyz <br />
<br />
for SID in `cat /tmp/xyz`<br />
do<br />
ORACLE_SID=$SID;<br />
export ORACLE_SID<br />
<br />
#Querry database for all tablespaces and their disk space utilisation and makes a list<br />
$ORACLE_HOME/bin/sqlplus system/manager </tmp/tbs1.list
set heading off;
set echo off;
#set verify off;
set feedback off;
set pagesize 50;
SELECT
dts.tablespace_name||':'||
NVL(ddf.bytes / 1024 / 1024, 0)||':'||
ROUND(NVL(ddf.bytes - NVL(dfs.bytes, 0), 0)/1024/1024,2)||':'||
ROUND(NVL(dfs.bytes / 1024 / 1024, 0),2)||':'||
ROUND(NVL((ddf.bytes - NVL(dfs.bytes, 0)) / ddf.bytes * 100, 0), 0)||':'||
dts.status
FROM
sys.dba_tablespaces dts,
(select tablespace_name, sum(bytes) bytes
from dba_data_files group by tablespace_name) ddf,
(select tablespace_name, sum(bytes) bytes
from dba_free_space group by tablespace_name) dfs
WHERE
dts.tablespace_name = ddf.tablespace_name(+)
AND dts.tablespace_name = dfs.tablespace_name(+);
exit;
EOF
#Removes all junk lines from the above list
egrep -v "^SQL|^With|^JServer|^Connected|^Oracle8i" /tmp/tbs1.list > /tmp/tbs2.list<br />
grep -v '^(c)' /tmp/tbs2.list > $CWD/tablespace$SID.list<br />
<br />
cat $CWD/tablespace$SID.list|while read LINE<br />
do<br />
TBS=`echo $LINE | awk -F':' '{print $1}'`<br />
RATIO=`echo $LINE | awk -F':' '{print $5}'`<br />
<br />
#Checks if any tablespace having used space for more than 80%<br />
if [ "$RATIO" -ge 60 ]<br />
then<br />
<br />
#If this entry is there more than 2 times in the log, sending email will be ignored <br />
y=`cat $CWD/tablespace$SID.log | egrep -ch $TBS`<br />
if [ $y -ge 2 ]<br />
then<br />
#continue<br />
echo "Problem still exists in the above tablespace in $SID - <br />
delete the above entries after correcting Database `date '+%m/%d/%y %H:%M'`" >>$CWD/tablespace$SID.log<br />
else<br />
echo "Tablespace $TBS in $SID is filled around $RATIO percent in $HOST `date '+%m/%d/%y %H:%M'`" >>$CWD/tablespace$SID.log<br />
echo "Tablespace $TBS in $SID is filled around $RATIO percent in $HOST" | /usr/bin/mailx -s "check $CWD/tablespace$SID.list" oracle;<br />
fi<br />
fi<br />
<br />
done<br />
doneKrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-62626723973074284822009-10-06T08:47:00.001-07:002009-10-06T08:47:45.694-07:00RAID and Oracle - Performance IssuesThe various RAID configurations are all appropriate if used correctly, and no RAID configuration is 'best' for all situations. <br />
Mirroring <br />
· use where availability is essential AND write performance requirements eliminates RAID 5 <br />
· doubles cost of storage (2x2Gb disk = 2Gb storage) <br />
· costs 5-15% in performance is using SW RAID (e.g. NT) <br />
RAID 5 <br />
· use where availability is important, AND 'read' will be the majority of I/O's <br />
· costs about 20% (5 member stripe) to 33% (3 member stripe) <br />
· decreases WRITE performance by 1/4 (5 members) to 1/3 (3) <br />
· increases READ (for larger reads) by 1.5 to 3x (usually limited by xfer capacity of SCSI channel) <br />
STRIPING <br />
· a single failure will crash entire stripe set <br />
· 100% storage efficiency <br />
· READ and WRITE performance can be much higher (depending upon geom. of stripe set) <br />
NOTE: THIS ASSUMES SOFTWARE RAID ONLY - there are some VERY GOOD implementations of RAID with caching, separate RISC cpu's etc These have to be evaluated independently due to the various algor. they can use. <br />
These are also more expensive (redundant everything ...) but can really make your day when needed !KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-24351866562076702972009-10-06T08:46:00.000-07:002009-10-06T08:46:45.183-07:00Pre-Loading Oracle Text indexes into MemoryIntroduction<br />
Oracle Text indexes are stored in Oracle Database relational tables. These tables normally reside on disk. Systems with large memory configurations generally benefit greatly from Oracle caching - a second or subsequent search for a particular term is generally much faster than the first search. <br />
But what about the first search? Can we pre-load the database cache in order to make sure that most of what we need is already in memory before the first search? <br />
This paper explores some techniques which make this possible. <br />
Prerequisites<br />
This paper assumes a good understanding of the various underlying tables of an Oracle Text index. If you are not aware of the functions of the various tables ($I, $K etc), then the paper should still offer some useful techniques, but to gain full benefit you should review documents such as this. <br />
The Keep Pool<br />
Normally the database buffer cache size is set with the parameter DB_CACHE_SIZE. If our intention is to load all of our indexes into memory and keep them there, we would probably be better off putting them into the "keep pool". This is a buffer cache from which objects will never be aged out. The size of the keep pool is determined by the parameter DB_KEEP_CACHE_SIZE. <br />
A table or index may be forced to reside in the keep cache using the commands <br />
ALTER TABLE <tablename> STORAGE (BUFFER POOL KEEP);<br />
ALTER INDEX <indexname> STORAGE (BUFFER_POOL KEEP);<br />
What to cache?<br />
If we have unlimited memory available, then we would want to put all of the following into the keep pool: <br />
· $I token table <br />
· $X index on $I table <br />
· $R table <br />
· The base table itself (assuming we are selecting more than just ROWID, or use a sort on a real column). <br />
If our table uses local partition indexes, we would need to store these tables for each partition. Additionally, if our application uses functional lookups in the text table we would need the $K table, and if we're using prefix indexing we would need the $P table. <br />
However, in real life it very likely that all of these tables will exceed the amount of memory we can afford to allocate to the keep pool. We would therefore need to select only certain tables. If we are using local partition indexes, and our queries are normally sorted by the partition key, it might make sense to just cache the tables for the first few partitions. <br />
Alternatively, if our table isn't partitioned, it might be better to cache certain tables and not others. <br />
In decreasing order of importance, we could rank them as <br />
· $X <br />
· $R <br />
· equivalence ($I, $K, base table) <br />
· $P (but way below) <br />
Now usually, $R is very small, and $X is small compared to the size of the base table and $I, so it is suggested that these two ($R andf $X) always be loaded. <br />
$I, base table and $K are perhaps equally important, - though the sizes are clearly not comparable, and different usage scenarios would tend to favor different approaches. In the normal case (with little or no functional invocation), we would suggest base-table, $I, $K in that order. If the index is badly fragmented, but we're still not using functional invocation, then we'd flip the base table and $I. If there's a large amount of functional invocation being used, $K goes to the head of the list. <br />
Determining the size of an index<br />
The CTX_REPORT index package can be used to determine the size of a index, which will aid us in planning how much we can cache. <br />
For testing purposes, I built a local partition index on a table called TEST with one million rows - each row consisting of just the unique key ID, and a filename pointing to an external XML file. <br />
The output of CTX_REPORT for this table is as follows: <br />
SQL> set long 50000<br />
SQL> select ctx_report.index_size('test_idx') from dual;<br />
<br />
CTX_REPORT.INDEX_SIZE('TEST_IDX')<br />
--------------------------------------------------------------------------------<br />
===========================================================================<br />
INDEX SIZE FOR TESTUSER.TEST_IDX<br />
===========================================================================<br />
<br />
...<br />
<br />
TOTALS FOR INDEX TESTUSER.TEST_IDX<br />
---------------------------------------------------------------------------<br />
<br />
CTX_REPORT.INDEX_SIZE('TEST_IDX')<br />
--------------------------------------------------------------------------------<br />
TOTAL BLOCKS ALLOCATED: 206608<br />
TOTAL BLOCKS USED: 198235<br />
TOTAL BYTES ALLOCATED: 1,692,532,736 (1,614.13 MB)<br />
TOTAL BYTES USED: 1,623,941,120 (1,548.71 MB)<br />
So we can see that the total index size is about 1.6GB. Since the machine has "only" 2GB of memory, we can't really afford to put all of that into memory. So we're going to cache a subset. <br />
Determining what is in the buffer pool<br />
One way to find out what we need to cache is to make the normal buffer cache as large as possible, bounce the database, run a representative query, and find out what ended up in the buffer cache. <br />
SQL> connect sys as sysdba<br />
SQL> alter system set db_cache_size = 1G;<br />
SQL> shutdown<br />
SQL> startup<br />
SQL> connect testuser/testuser<br />
SQL> select /* FIRST_ROWS */ * from (<br />
select id, filename from test<br />
where contains (filename, 'district and region) > 0<br />
order by id desc )<br />
where rownum <= 20;
...
Note that this query fetches only the top 20 hits, ordered by "id", which is the partition key for my table. This means that a query for a common word (or words) will generally only have to hit one - or a few - partitions of the index.
After running this query, we can find out what's in the buffer cache by running the following SQL as a dba:
COLUMN OBJECT_NAME FORMAT A30
COLUMN OBJECT_TYPE FORMAT A15
COLUMN OWNER FORMAT A15
COLUMN NUMBER_OF_BLOCKS FORMAT 999,999,999,999
SELECT o.OBJECT_NAME, o.OBJECT_TYPE, o.OWNER, COUNT(*) NUMBER_OF_BLOCKS
FROM DBA_OBJECTS o, V$BH bh
WHERE o.DATA_OBJECT_ID = bh.OBJD
AND o.OWNER NOT IN
('SYS', 'SYSTEM', 'SYSMAN', 'XDB', 'IX', 'WMSYS', 'CTXSYS')
GROUP BY o.OBJECT_NAME, o.OWNER, o.OBJECT_TYPE
ORDER BY COUNT(*);
This will tell us which user tables are now in the cache. If you want to see system / data dictionary tables as well, you can vary the "NOT IN" list.
After running the previous CONTAINS query, this gives me:
OBJECT_NAME OBJECT_TYPE OWNER NUMBER_OF_BLOCKS
------------------------------ --------------- --------------- ----------------
DR#TEST_IDX0009$R TABLE TESTUSER 2
DR#TEST_IDX0010$R TABLE TESTUSER 2
SYS_LOB0000056864C00002$$ LOB TESTUSER 4
DR#TEST_IDX0009$X INDEX TESTUSER 12
DR#TEST_IDX0010$X INDEX TESTUSER 12
SYS_LOB0000056876C00002$$ LOB TESTUSER 17
TEST TABLE PARTITION TESTUSER 22
DR#TEST_IDX0010$I TABLE TESTUSER 459
DR#TEST_IDX0009$I TABLE TESTUSER 494
9 rows selected.
So what does that tell us?
Firstly, we can see from the last two lines that the majority of blocks are from two $I tables - those for the last two partitions of the table (since it has ten partitions). Then, moving up a bit, we see 22 blocks from the base table TEST. Note that this doesn't tell us which partitions of the table these blocks come from, but it's reasonable to assume that they come from the last two partitions as well. There are also a few blocks from the $R table.
We'll be covering the SYS_LOB... objects in a moment. For now, we're going to ignore them as tracking them down will be a useful exercise coming up.
Pre-Loading the tables
There is no simple command in Oracle to load data into the buffer cache. You must run a SQL command (normally a query, though DML may do it as well).
We need to load both the tables, and the indexes for indexed tables. We can do this by running queries with appropriate hints. Rather than actually fetching every row which would require lots of traffic between the kernel and client, we can make use of aggregate functions so that the rows are fetched only to the kernel:
SELECT /*+ FULL(ITAB) */ SUM(TOKEN_COUNT), SUM(LENGTH(TOKEN_INFO)) FROM DR#TEST_IDX0010$I ITAB;
SELECT /*+ FULL(ITAB) */ SUM(TOKEN_COUNT), SUM(LENGTH(TOKEN_INFO)) FROM DR#TEST_IDX0009$I ITAB;
SELECT /*+ INDEX(ITAB) */ SUM(LENGTH(TOKEN_TEXT)) FROM DR#TEST_IDX0010$I ITAB;
SELECT /*+ INDEX(ITAB) */ SUM(LENGTH(TOKEN_TEXT)) FROM DR#TEST_IDX0009$I ITAB;
SELECT SUM(ROW_NO) FROM DR#TEST_IDX0010$R;
SELECT SUM(ROW_NO) FROM DR#TEST_IDX0009$R;
SELECT /*+ FULL(BTAB) */ SUM(ID) FROM TEST BTAB WHERE ID >= 900000;<br />
<br />
There's a couple of points to note here. The final select from the base table is fetching specific rows that I know are in the last two partitions. You would need to vary this according to your system design. <br />
Also note that the $R loading is incomplete. This is related to those system LOB objects we saw earlier - and we'll be coming back to that later. <br />
Running these queries on my system took just over a minute. That's to load one-fifth of the partitions for a million-row table. Here's the output after rebooting the machine to avoid any effects of disk caching: <br />
SUM(TOKEN_COUNT) SUM(LENGTH(TOKEN_INFO))<br />
---------------- -----------------------<br />
6963707 37472389<br />
<br />
Elapsed: 00:00:27.31<br />
<br />
SUM(TOKEN_COUNT) SUM(LENGTH(TOKEN_INFO))<br />
---------------- -----------------------<br />
6587719 35349528<br />
<br />
Elapsed: 00:00:27.77<br />
<br />
SUM(LENGTH(TOKEN_TEXT))<br />
-----------------------<br />
8994078<br />
<br />
Elapsed: 00:00:03.74<br />
<br />
SUM(LENGTH(TOKEN_TEXT))<br />
-----------------------<br />
8855116<br />
<br />
Elapsed: 00:00:03.21<br />
<br />
SUM(ID)<br />
----------<br />
9.5001E+10<br />
<br />
Elapsed: 00:00:01.74<br />
<br />
SUM(ROW_NO)<br />
-----------<br />
231<br />
<br />
Elapsed: 00:00:00.12<br />
<br />
SUM(ROW_NO)<br />
-----------<br />
231<br />
Now we'll run our query against V$BH again to see what's been cached. Remember, this is after pre-loading the various tables rather than running a simple query. <br />
OBJECT_NAME OBJECT_TYPE OWNER NUMBER_OF_BLOCKS<br />
------------------------------ --------------- --------------- ----------------<br />
DR#TEST_IDX0009$R TABLE TESTUSER 2<br />
DR#TEST_IDX0010$R TABLE TESTUSER 2<br />
TEST TABLE PARTITION TESTUSER 612<br />
DR#TEST_IDX0009$X INDEX TESTUSER 5,905<br />
DR#TEST_IDX0010$X INDEX TESTUSER 6,018<br />
DR#TEST_IDX0009$I TABLE TESTUSER 14,607<br />
DR#TEST_IDX0010$I TABLE TESTUSER 15,090<br />
<br />
7 rows selected.<br />
Now this looks similar to our previous output, except that the number of blocks is greater for most of the tables (because we've loaded ALL blocks, rather than just the ones needed by the query), and we don't have those mysterious SYS_LOB objects (again we'll come back to those later). <br />
We're going to try to put all those tables into the keep pool, so we need to know how much space they're taking to decide how big to make our keep pool. We can sum up the NUMBER_OF_BLOCKS column above, or use: <br />
SELECT COUNT(*) TOTAL_BLOCKS, COUNT(*)*8/1024 MEGABYTES<br />
FROM DBA_OBJECTS o, V$BH bh<br />
WHERE o.DATA_OBJECT_ID = bh.OBJD<br />
AND o.OWNER NOT IN<br />
('SYS', 'SYSTEM', 'SYSMAN', 'XDB', 'IX', 'WMSYS', 'CTXSYS')<br />
ORDER BY COUNT(*);<br />
<br />
TOTAL_BLOCKS MEGABYTES<br />
------------ ----------<br />
49503 386.742188<br />
So we need just under 400MB of keep pool for these tables. We'll adjust this via the DB_KEEP_CACHE_SIZE parameter. I'm going to run out of memory if I just bump that up without reducing my large DB_CACHE_SIZE, so I'll do that at the same time. I'm using an spfile rather than init.ora, so I'll do: <br />
ALTER SYSTEM SET DB_CACHE_SIZE = 500M SCOPE=SPFILE;<br />
-- Down from 1G before<br />
ALTER SYSTEM SET DB_KEEP_CACHE_SIZE = 400M SCOPE=SPFILE;<br />
And then of course I'll bounce the database. <br />
Forcing Objects to the KEEP POOL<br />
Next we need to ensure that our tables go into the Keep Pool rather than the normal buffer cache. We do that with ALTER TABLE or ALTER INDEX statements as appropriate: <br />
alter table DR#TEST_IDX0010$I storage (buffer_pool keep);<br />
alter table DR#TEST_IDX0009$I storage (buffer_pool keep);<br />
<br />
alter index DR#TEST_IDX0010$X storage (buffer_pool keep);<br />
alter index DR#TEST_IDX0009$X storage (buffer_pool keep);<br />
<br />
alter table DR#TEST_IDX0010$R storage (buffer_pool keep);<br />
alter table DR#TEST_IDX0009$R storage (buffer_pool keep);<br />
<br />
alter table DR#TEST_IDX0010$R storage (buffer_pool keep);<br />
alter table DR#TEST_IDX0009$R storage (buffer_pool keep);<br />
<br />
alter table TEST modify partition p10 storage (buffer_pool keep);<br />
alter table TEST modify partition p9 storage (buffer_pool keep);<br />
We'll run these statements, bounce our database, then run our pre-loading queries again. It would be handy if we could use the query against V$BH to make sure that the tables ARE in fact in the keep pool, but this information is not stored in V$BH - the various buffer caches are considered together. <br />
Using TKPROF to monitor disk reads<br />
Now we're going to run our query again and see what effect it has. This time, we're going to generate a trace file and examine it too see the number of physical I/O's. In SQL*Plus: <br />
@?/rdbms/admin/utlxplan.sql -- avoid tkprof plan table error<br />
<br />
alter session set sql_trace=true;<br />
<br />
-- Use a bind variable to cut down on recursive SQL from optimizer:<br />
<br />
variable bnd varchar2(4000)<br />
<br />
exec :bnd := 'district and region'<br />
<br />
select /* FIRST_ROWS */ * from (<br />
select id, filename from test<br />
where contains (filename, :bnd) > 0<br />
order by id desc )<br />
where rownum <= 20;
Then, in our udump directory (typically $ORACLE_HOME/admin//udump), we need to find the latest tracefile, and do something like:
tkprof mysid_ora_01234.trc traceout.txt sort=exeela table=testuser.plan_table
This will generate a formatted output file "traceout.txt" which we can examine for evidence of physical I/O's. To do this we need to look at the summary for each SQL statement, which might look like this:
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 1 0.02 0.03 0 402 0 0
Fetch 3 0.00 0.11 34 65 0 20
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 5 0.02 0.15 34 467 0 20
The "disk" column is the one we're interested in. In this case, the SQL statement above this summary has fetched 34 database blocks from disk. The next column, query, is the number of block fetches, but most of these will have been from cache.
Now if we look at the end of the file, we will see:
OVERALL TOTALS FOR ALL NON-RECURSIVE STATEMENTS
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 2 0.00 0.00 0 0 0 0
Execute 3 0.03 0.06 0 402 0 1
Fetch 3 0.00 0.11 34 65 0 20
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 8 0.03 0.17 34 467 0 21
Misses in library cache during parse: 2
Misses in library cache during execute: 2
OVERALL TOTALS FOR ALL RECURSIVE STATEMENTS
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 41 0.02 0.01 0 0 0 0
Execute 66 0.08 0.08 0 543 0 7
Fetch 59 0.00 0.02 2 190 0 61
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 166 0.10 0.12 2 733 0 68
This tells us that of our 34 disk block reads, only two are during recursive statements. Checking back through the tracefile finds the culprit:
select /*+ rule */ bucket, endpoint, col#, epvalue
from
histgrm$ where obj#=:1 and intcol#=:2 and row#=:3 order by bucket
call count cpu elapsed disk query current rows
------- ------ -------- ---------- ---------- ---------- ---------- ----------
Parse 1 0.00 0.00 0 0 0 0
Execute 2 0.00 0.00 0 0 0 0
Fetch 2 0.00 0.01 2 6 0 11
------- ------ -------- ---------- ---------- ---------- ---------- ----------
total 5 0.00 0.01 2 6 0 11
That's clearly some kind of system SQL, and it's only 2 blocks anyway, so we won't worry about it (TODO: Are these the same as the two "Misses in library cache"?)
That leaves us with 32 unexplained disk reads from our main SQL statement.
To figure out where these come from, we need to dig a bit deeper.
We'll bounce the database again, re-preload the cache, but this time set an event flag before running our query:
ALTER SESSION SET EVENTS '10046 TRACE NAME CONTEXT FOREVER, LEVEL 8';
This will write a large trace file into the udump directory (typically $ORACLE_HOME/admin//udump). The trace file will list all the "wait" events, which will include the physical reads.
To extract just the reads, we could do:
grep WAIT sid_ora_1234.trc | grep read
which on my system gave me:
WAIT #15: nam='db file sequential read' ela= 53 p1=1 p2=55741 p3=1
WAIT #15: nam='db file sequential read' ela= 29 p1=1 p2=55740 p3=1
WAIT #1: nam='db file sequential read' ela= 35 p1=7 p2=168890 p3=1
WAIT #1: nam='db file scattered read' ela= 63 p1=7 p2=191027 p3=2
WAIT #1: nam='db file scattered read' ela= 53 p1=7 p2=191030 p3=3
WAIT #1: nam='db file scattered read' ela= 68 p1=7 p2=191034 p3=4
WAIT #1: nam='db file scattered read' ela= 50 p1=7 p2=191038 p3=3
WAIT #1: nam='db file sequential read' ela= 7 p1=7 p2=191067 p3=1
WAIT #1: nam='db file sequential read' ela= 6 p1=7 p2=206475 p3=1
WAIT #1: nam='db file sequential read' ela= 23 p1=7 p2=206506 p3=1
WAIT #1: nam='db file scattered read' ela= 59 p1=7 p2=206552 p3=3
WAIT #1: nam='db file scattered read' ela= 104 p1=7 p2=206555 p3=6
WAIT #1: nam='db file sequential read' ela= 16 p1=7 p2=150156 p3=1
WAIT #1: nam='db file sequential read' ela= 8 p1=7 p2=150162 p3=1
WAIT #1: nam='db file sequential read' ela= 8 p1=7 p2=169084 p3=1
WAIT #1: nam='db file sequential read' ela= 9 p1=7 p2=184356 p3=1
WAIT #1: nam='db file sequential read' ela= 9 p1=7 p2=184461 p3=1
WAIT #1: nam='db file sequential read' ela= 23 p1=7 p2=77 p3=1
WAIT #1: nam='db file sequential read' ela= 21 p1=7 p2=5434 p3=1
WAIT #1: nam='db file sequential read' ela= 19 p1=7 p2=5545 p3=1
WAIT #1: nam='db file sequential read' ela= 20 p1=7 p2=5614 p3=1
The meanings of the arguments here are as follows:
· ela: time waited in miliseconds (?)
· p1: file number (used in dba_files)
· p2: block number (used in dba_segments)
· p3: number of blocks read
We're not too concerned with the difference between scattered read and sequential read - they both represent physical I/O's.
In order to translate these file and block numbers into useful information, we need to run check DBA_EXTENTS, like this:
column owner format a30
column segment_name format a30
select owner, segment_name
from dba_extents
where &1 = file_id
and &2 between block_id and block_id + blocks - 1
/
If we want to be really clever, we might save this in a file called seg.sql (together with SET VERIFY OFF and any necessary column formatting) and then do:
cat $ORACLE_HOME/admin/$ORACLE_SID/udump/`ls -tr $ORACLE_HOME/admin/$ORACLE_SID/udump | tail -1` \
| sed -n -e 's/^WAIT.*read.*p1=\(.*\).*p2=\(.*\) .*/\1 \2/p' \
| xargs -l1 sqlplus -s "/ as sysdba" @seg.sql
This extracts all the file numbers and block numbers from the latest trace file in udump, then feeds them into seg.sql (note this works for me with the bash shell in Linux, it may not work for you!)
The start of the output might look like this:
OWNER SEGMENT_NAME
---------- ------------------------------
SYS C_OBJ#_INTCOL#
OWNER SEGMENT_NAME
---------- ------------------------------
TESTUSER SYS_IL0000056876C00002$$
OWNER SEGMENT_NAME
---------- ------------------------------
TESTUSER SYS_LOB0000056876C00002$$
We'll ignore the SYS segment, and look at the other two. Clearly they are system generated object names - and at least one of them matches the segment name we saw in our original query against V$BH (the reason we ignored them earlier was precisely so we could go through this diagnostic process). We know the segment names, but what actual database object do they represent?
select segment_name, segment_type from dba_segments
where segment_name in ('SYS_IL0000056876C00002$$', 'SYS_LOB0000056876C00002$$');
SEGMENT_NAME SEGMENT_TYPE
------------------------------ ------------------
SYS_IL0000056876C00002$$ LOBINDEX
SYS_LOB0000056876C00002$$ LOBSEGMENT
No we're getting somewhere - they're both LOB related. Now we need to find out what table these LOBS are associated with. So how about:
select table_name from dba_indexes where owner = 'TESTUSER' and index_name = 'SYS_IL0000056876C00002$$';
TABLE_NAME
------------------------------
DR#TEST_IDX0010$R
select table_name from dba_lobs where owner = 'TESTUSER' and segment_name = 'SYS_LOB0000056876C00002$$';
TABLE_NAME
------------------------------
DR#TEST_IDX0010$R
So - clearly our earlier attempts at preloading the $R table into the keep pool did not work perfectly. Why would that be?
In order to understand that we need to know a bit more about LOBs. LOBs can be either In-Line or Out-of-Line, meaning that the actual LOB is stored either in the same segment as the rest of the table blocks, or is stored its own segment. By default, lobs are stored in-line. However, if the LOB size exceeds 4K, it will automatically be stored out-of-line. You can force all blocks to be stored out-of-line using the LOB storage clause "DISABLE STORAGE IN ROW".
In the case of the $I tables, the LOB lengths are restricted by the Text code to 4K, to make sure they can always be stored in-line to minimize I/O requirements. Hence there should never be any requirement for separate LOB segments for the $I tables.
However, in the case of $R tables, things are different. The $R table consists of a BLOB (or BLOBs) which are basically a list of ROWIDs for the base table, ordered by DOCID. In a small table with less than about 220 rows, this BLOB will be less than 4K and will be stored in-line. However when it exceeds that size, it will me moved to out-of-line storage in a new segment.
To ensure that we get the $R LOB segments pre-cached in the keep pool, we need to do two things. First, we need to change our ALTER TABLE statements for the $R tables:
alter table DR#TEST_IDX0010$R storage (buffer_pool keep) modify lob (data) (storage (buffer_pool keep));
alter table DR#TEST_IDX0009$R storage (buffer_pool keep) modify lob (data) (storage (buffer_pool keep));
(Of course, we could have done this by altering the initial R_STORAGE_CLAUSE attribute when creating our text index).
Secondly, we need to force the LOB blocks to be loaded, as well as the main table blocks for the $R. This is a little more complicated. We might think we could do:
SELECT SUM(ROW_NO), SUM(LENGTH(DATA)) FROM DR#TEST_IDX0010$R; /* WRONG!! */
but that won't, in fact, work. The length of the LOB is held in the header block - it's not necessary to read the whole LOB to get its length. Instead, we must physically read the LOB - and read it at enough points along its length to make sure we've pulled in all the LOBs. This requires a PL/SQL procedure. The following procedure will load ALL the $R tables for a local partitioned index - it could be simplified for a non-partitioned index, and you MIGHT decide you didn't want to preload all the partitions. $R tables are relatively small (13MB in total for my system), so often you may as well load the lot.
set serveroutput on size 1000000
create or replace procedure loadAllDollarR (idx_name varchar2) is
v_idx_name varchar2(30) := upper(idx_name);
type c_type is ref cursor;
c2 c_type;
s varchar2(2000);
b blob;
buff varchar2(100);
siz number;
off number;
cntr number;
begin
for c1 in (select table_name t from user_tables
where table_name like 'DR_'||v_idx_name||'%$R') loop
dbms_output.put_line('loading from table '||c1.t);
s := 'select data from '||c1.t;
open c2 for s;
loop
fetch c2 into b;
exit when c2%notfound;
siz := 10;
off := 1;
cntr := 0;
if dbms_lob.getlength(b) > 0 then<br />
begin<br />
loop <br />
dbms_lob.read(b, siz, off, buff);<br />
cntr := cntr + 1;<br />
off := off + 4096;<br />
end loop;<br />
exception when no_data_found then<br />
if cntr > 0 then<br />
dbms_output.put_line('4K chunks fetched: '||cntr);<br />
end if;<br />
end;<br />
end if;<br />
end loop;<br />
end loop;<br />
end;<br />
/<br />
You would call this procedure from SQL*Plus like this: <br />
exec LoadAllDollarR('test_idx')<br />
So now we have everything we need to preload the first two partitions of our index. <br />
Finishing Up<br />
Let's run back over our our full set of commands for moving tables to the keep pool, and then loading them into memory. I'm going to do all of this each time I restart the database (the ALTER TABLES only really need to be run once, but they're quick and this will catch any situation where the index - or individual tables - have been rebuilt). <br />
-- This script preloads the last two partitions from a 10-way<br />
-- partitioned table and its text index. The table is called TEST,<br />
-- the index is called TEST_IDX. It is owned by user TESTUSER.<br />
<br />
connect testuser/testuser<br />
<br />
--------------------------------------------------------<br />
-- First make sure all tables go into the keep buffer --<br />
--------------------------------------------------------<br />
<br />
alter table DR#TEST_IDX0010$I storage (buffer_pool keep);<br />
alter table DR#TEST_IDX0009$I storage (buffer_pool keep);<br />
<br />
alter index DR#TEST_IDX0010$X storage (buffer_pool keep);<br />
alter index DR#TEST_IDX0009$X storage (buffer_pool keep);<br />
<br />
<br />
alter table DR#TEST_IDX0010$R storage (buffer_pool keep);<br />
alter table DR#TEST_IDX0009$R storage (buffer_pool keep);<br />
<br />
alter table DR#TEST_IDX0010$R storage (buffer_pool keep)<br />
modify lob (data) (storage (buffer_pool keep));<br />
alter table DR#TEST_IDX0009$R storage (buffer_pool keep)<br />
modify lob (data) (storage (buffer_pool keep));<br />
<br />
alter table TEST storage (buffer_pool keep);<br />
<br />
set timing on<br />
<br />
--------------------------------------------------------------<br />
-- Then perform the necessary queries to preload the tables --<br />
--------------------------------------------------------------<br />
<br />
SELECT /*+ FULL(ITAB) */ SUM(TOKEN_COUNT), SUM(LENGTH(TOKEN_INFO))<br />
FROM DR#TEST_IDX0010$I ITAB;<br />
SELECT /*+ FULL(ITAB) */ SUM(TOKEN_COUNT), SUM(LENGTH(TOKEN_INFO))<br />
FROM DR#TEST_IDX0009$I ITAB;<br />
<br />
SELECT /*+ INDEX(ITAB) */ SUM(LENGTH(TOKEN_TEXT))<br />
FROM DR#TEST_IDX0010$I ITAB;<br />
SELECT /*+ INDEX(ITAB) */ SUM(LENGTH(TOKEN_TEXT))<br />
FROM DR#TEST_IDX0009$I ITAB;<br />
<br />
SELECT /*+ FULL(BTAB) */ SUM(ID)<br />
FROM TEST WHERE ID >= 900000;<br />
<br />
SELECT SUM(ROW_NO) FROM DR#TEST_IDX0010$R;<br />
SELECT SUM(ROW_NO) FROM DR#TEST_IDX0009$R;<br />
<br />
exec LoadAllDollarR('test_idx')<br />
<br />
If we run this after bouncing the database and before running our test query, we should find that now there are NO reads from the $R tables at all. In fact, in my testcase I found the system reads went away as well, and my query now completed without any physical I/O at all.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-44770562180254665652009-10-06T08:45:00.000-07:002009-10-06T08:45:06.513-07:00Performance tuning - Necessary StepsPerformance tuning, step 1: Planning a tuning engagement<br />
Introduction : <br />
Tuning a database or database applications is a time-intensive, repetitive task. Over time, the database changes in many ways and must be reevaluated. In addition, changes made to improve performance will need to be evaluated. Did the change actually improve performance? If so, how much? How much effort did it take to achieve the gain (or loss)? What is "good performance" and how can we make it better? <br />
This series identifies a standard approach for performing or evaluating database tuning efforts. It is applicable to data warehouses, custom databases, custom database applications or Oracle Applications Database. The criteria used in the evaluations will depend on individual circumstances. For example, a data warehouse has different performance requirements than a customer order application. These requirements will affect the database architecture, which affects how performance can be optimized. In short, the goal is to identify how performance can be optimized, as measured by how operations are transacted by users. <br />
When users start complaining that the database is too slow, the natural desire is to tune everything. But the real answer, based on customer needs, is a combination of training, tuning and business process management. This primer identifies the basic elements of performance tuning, the tools used and a basic methodology for obtaining the information used to evaluate a database. It is designed for the non-technical user i.e., the applications user, project managers or anyone who would like understand what those techies and sysadmins do, and why they talk to themselves... <br />
It identifies the basic steps for each stage of performance tuning, explains what tools may be used and function or expected results yielded by the tool. It also provides a set of basic questions for initial performance problem assessment -- a manager's checklist to guide the non-technical manager to the business of performance tuning. <br />
What is good performance? <br />
There are a number of published articles on what constitutes good -- and bad -- performance. These articles are usually defined for a specific architecture. This is because the machine configuration -- size, speed and number of CPUs significantly impacts the system. Simply put, some systems have more horsepower than others. However, CPU is not the most important measure in dealing with performance issues. The most important measure is response time. Time is the metric the user understands, time is the measure the manager uses and time is the most effective measure of performance improvement. While there are many important ratios that measure whether a database is working effectively, the goal is for the database to effectively support user transactions. Tuning hit ratios and cache statistics is like patching potholes. Sometimes patching fixes the problem; sometimes repaving is better. That's why a more effective measure of performance tuning results is response time. Response time concentrates on the specific business process that users have identified as problematic. <br />
Planning a tuning engagement is like the initial stages of any project and uses the same steps: <br />
1. Determine what you've got.<br />
How big is it in terms of users, machines, operating systems and connections. This is the current state. <br />
2. Determine what they think they need.<br />
What does the user see as the primary problem? No matter what you actually find (good, needing improvement, dearly beloved, etc.), you will frame your results and recommendations in terms of the user's perceptions. This is the future state, or the "to-be" state. It establishes an understanding of perceived priorities. <br />
3. Determine the details of what you have.<br />
In this case, you or your tame techie will use commands and scripts to retrieve detailed information about each area under consideration. <br />
4. Establish goals.<br />
If it ain't broke, don't fix it. What is it that returns the most value for the user? Remember that tuning for tuning's sake is not effective…. <br />
5. Prepare a plan.<br />
Documenting your plan is an important management. If you are lucky, when you distribute your plan, you'll get feedback from all sides. (Recite this daily: "Feedback is my friend.") Feedback allows you to plan targeted communication as results are achieved. <br />
6. Work the plan.<br />
Your improvement plan will probably have multiple phases. At each phase completion, evaluate the accomplishments in comparison to both the original baseline and with accomplishments of previous phases. Update your communications, give feedback to the users regarding results. <br />
7. Keep going until you reach your goal or reassess your goals.<br />
As time elapses, priorities may change. For instance, a company with seasonal sales may need to have all performance enhancements completed before the sales rush or put in place some temporary "fixes" for the duration. Care must be taken when introducing temporary solutions, however. <br />
<br />
Performance tuning, step 2: Determining the current state<br />
<br />
In step 1 we identified the elements of a basic performance tuning plan. The next step is to determine the "current state" of the system. The current state is documented in order to form a baseline for comparison in later stages. We will also look at determining relative priorities with users and management, and how to define the details of the current state. <br />
The current state <br />
Determining the current state involves both users and the technical architecture of the database. This includes working with knowledgeable users, often called power users, to find out what they see as problems. Ask your power users if they have observed any specific trends, jot down what time problems occur and what they were doing. Take the time to observe the user performing their problematic tasks and record the steps. Using the same steps every time establishes a baseline for comparison. When working with the user, take care to ask questions, but not to jump to conclusions or make promises about how much you will be able to change. You are in the evaluation phase and changes may be technical, training related or both. <br />
Check the database size and basic parameters. This can be done using the enterprise manager or a set of basic SQL scripts. Do not make changes at this time. It is important to establish a baseline for comparison first. <br />
Check the database architecture. Identify network nodes used, the size and location of database files. Identify mount points. Identify network connectivity parameters, including packet sizes. Prepare a network diagram for future reference. <br />
Check how database transactions are accomplished. How many users are there? What software and hardware do they use? What transactions have been identified as problematic? Later technical personnel will check the actual code used in transactions to determine effectiveness. Is the system shared (i.e., does a single server house production and development instances)? Are other types of applications housed on the system? In one corporate IT department, top-of-the-line mainframes housed database applications as well as manufacturing production tracking applications, creating contention for resources. Changes made must be evaluated in concert with ALL operations affected. <br />
Can CPUs be added? What is the predicted effect and how does it compare with the cost to add? Scalable architectures may allow different sizes and numbers of CPUs. CPUs provided by different manufacturers may have different ratings as well. Whatever machine is run, a general rule of thumb is to know your CPU utilization target. Often this is a goal of 30 to 50 percent. What this means is that during normal, off-peak operation, measurements should indicate sufficient additional capacity for peak times, with an allowance for times of unusual load. Oracle Applications performance is not stable when CPU is over 80%. <br />
Is the system maintained internally or externally? Is there a maintenance requirement that stipulates availability or capacity to be provided? If there are service level agreements (SLAs) in place, some improvements may be made at less (or more) cost. Additionally, the more organizations involved, the more time it may take to change performance. <br />
Documentation <br />
It is important to document the baseline, distribute the information and maintain accurate records over time. Where more than one organization is involved, the need to establish priorities agreed upon -- or negotiated with all interested parties is particularly important. <br />
Typical documentation (a.k.a. "deliverables") for a performance tuning project will include:<br />
1. System or technical architecture document<br />
2. Network architecture<br />
3. Database architecture<br />
4. Capacity plan<br />
5. Change management plan<br />
6. Test plans and procedures<br />
7. Maintenance plan<br />
Plan for your deliverables early in the project. (Establish a plan, use the plan to focus, fill in the plan as you go.) Evaluate existing documentation to examine whether the data is still valid. Existing documentation may provide key information regarding previous problems, agreements that have been made or projections where problems are expected to occur. <br />
While examining the system and objectively asking questions of the users, you must consider what gives the most bang for the buck. <br />
Understanding priorities <br />
Base performance goals on the customer's system, established baseline: <br />
1. Meet with stakeholders to determine their perception of what is needed.<br />
2. Meet with primary users identified as knowledgeable personnel by the stakeholder. Observe problems. Understand the business process. List ALL customer issues.<br />
3. Get baselines.<br />
4. Based on preliminary information, agree on improvement goals with customer. Establish that applications performance improvements made by procedural changes, network architectural changes, database changes or SQL tuning. In many instances, retraining users, concurrent manager re-llocation, can affect major improvements.<br />
Make sure that the customer agrees with your strategy and present the goals in a measurable context. Establish priorities for each area to be addressed. Document the baseline, performance goals. Goals should be stated clearly and concisely and with measurable parameters. For instance: <br />
1. Sales order booking takes 45 seconds for one order line. We need it to run in five seconds for one order line.<br />
2. Custom report: Update raw material costs takes 30 minutes to complete in off-peak hours. We need regular updates throughout the day without impacting sales order pricing.<br />
3. Web pricing request takes 17 seconds to retrieve and display 4X4 cost/delivery matrix.<br />
Be careful what you promise. Vendor published baselines represent aggressively tuned, optimal systems. Any promises you make to your users should be based on improving existing performance and any existing SLAs. <br />
Evaluating the baseline will take into account information from all levels of users. A multilayer approach helps provide a 360-degree view of the system. Each type of user is asked the same questions and encouraged to give detailed examples. (When the user asserts, "Performance is rotten! I can't get my work done!" they are asked, "What are you doing when you notice it is so bad? Is it always bad, or is it sometimes bad? Does everyone have the same problem?" <br />
Obtaining details <br />
The assessment process... <br />
Is highly collaborative. It takes information from all types of users/interested parties. <br />
Focuses on business. It identifies business processes, rather than imposing a technical solution. <br />
Helps identify change impact. By identifying business processes affected, it assists in creation of important performance metrics. <br />
Leads to measurable benefits. The metrics identified will be used in subsequent comparisons. <br />
Supports future vision. The assessment and the performance tuning/analysis will often be used to support the justification for technical purchases such as more memory, additional hard drives or faster hard drives. It may also suggest changing existing configuration. On one system, a combination of disk shadowing and the selected RAID configuration created considerable degradation. What is a suitable RAID configuration for software development or manufacturing systems may not be suitable for databases. <br />
May identify new models. As you discuss the system and obtain information, you may need to assure each set of users that you have no particular agenda and that your goal is to obtain as complete information as possible. Assure them that the results of the investigation will be available, and no solution technical or otherwise can be decided until all information is available. This is where having a planned set of deliverables is critical. <br />
The goal for the assessment is to... <br />
Define the problem(s) quickly. The users responses will probably be able to be combined into categories. These categories provide the general areas where you will concentrate your efforts. <br />
Define the problem accurately. Your problems should be reported in terms of measurable behaviors. Often the performance tuning will be alleviated in stages. One set of "fixes" will improve performance, but additional improvements may be made with subsequent actions. <br />
Identify the appropriate resources. Your problem definition will include the scope of the tuning's responsibility. For instance, while your information may reveal that hardware components are undersized, the purchase and installation may not be possible in near term. Identify the times that resources are available; certain types of testing may be required to be performed in off-hours, or some personnel may be unavailable during certain periods. <br />
Solve the problem quickly. By structuring the problem statements, the analysis and the recommended solutions, you prepare the customer for your methods and the time it will take to analize. <br />
You may provide test/reporting scripts for the user to evaluate functional timing. Generally, you will want to limit any testing tools you may provide to "power" users. Let them know that these scripts may impact performance, and should not be run continuously. Power users can be your best friends. They can alert you to trends that you may not otherwise see. <br />
<br />
Performance tuning, step 3: Working the plan<br />
<br />
In step 1 we identified the elements of a basics of a performance tuning plan. Then, in step 2, we identified the current state for subsequent stages and determined the relative priorities which need to be met in terms of user requirements and management requirements. The next step in a tuning program is to assess the details of the system, in terms of specific architecture, processes and code. <br />
The overall tuning process is a systematic review based on a finite set of business processes. There are a number of published articles on what constitutes good -- and bad -- performance. These articles are usually defined for a specific architecture. This is because the machine configuration -- size, speed and number of CPUs will significantly impact the system. Simply put, some systems have more horsepower than others. However, CPU is not the most important measure in dealing with performance issues. The most important measure is response time. It is the metric the user sees, and is the most effective measure of performance improvement. While there are many important ratios that measure whether a database is working effectively, the goal is for the database to effectively support user transactions. Tuning hit ratios and cache statistics is like patching potholes. Sometimes patching fixes the problem; sometimes repaving is better. That's why a more effective measure of performance tuning results is response time. Response time concentrates on the specific business process that users have identified as problematic. Select some basic, clearly understood problems and solutions. This is often referred to as "gathering the low-hanging fruit." At this point, you evaluate the information gathered during the evaluation of the current state. <br />
Checking the business process first can have great impact on your tuning efforts. For instance, do customer support specialists indicate that their searches for open orders for a specific customer are slow? By looking at what their actual process is, you may find that the operator is not making good use of the workbench search fields, and could benefit from adding parameters to his or her search criteria. In this case, some additional one-one-one training can increase their satisfaction and the performance of their searches. <br />
Next, obtaining benchmarks -- running specific searches at non-load times, non-peak, peak times gives a measure of how results vary throughout the day. If some operators have widely disparate results at the same time, you may need to look at network configuration. For example, if Joe and Mary in operator bay A have no problems, but operations in bay B take twice as long, the network configuration should be analyzed. Note: In one instance, only one operator on a floor was slow. It was determined that someone had connected that cubicle only to a different server at the local switch as a stopgap measure and never set it back. The moral of the story is that you need to look beyond the database for performance solutions. <br />
Perform all benchmarks on a single, tuned PC or laptop client to ensure continuity. If necessary, move the client to perform benchmarks in different areas. If this is not possible, verify client configuration complies with company standards. Comparing results of disparate systems is extremely difficult. <br />
Gather benchmarks for all processes identified as problematic. When all benchmarks have been derived, discuss which processes have priority and what performance goals are appropriate. You may wish to present phased (preliminary, acceptable and target) goals, as well. This is because the final solution may combine a series of user training, network and database or application tuning actions. <br />
Documentation is used to identify the initial definition of the problem, initial conditions, goals agreed upon and the focus for the work to be done. Over time, additional documentation addresses what actions are recommended and the results of the actions. Finally, an assessment outlining the completed actions, results and recommendations for ongoing actions to maintain/further improve the increased performance is recommended. <br />
As mentioned earlier, significant improvement can be made by retraining users in specific areas of application use. Additional training performed should be documented for later use by new users. <br />
Because the solution for a specific area is often iterative, as efforts are defined and redefined, the work performed must be detailed and recorded scrupulously (and hopefully controlled in a change-management system). At each stage, results should be compared to the baseline; performance tuning is a balancing act, where action in one area may have deleterious effects on another. <br />
Also because systems change over time, repeatable scripts will allow the user to verify the degree that changes have/have not occurred... (i.e. "This time last year we had 5,000 orders placed per day, at a average rate of 10 orders per hour for each call-in sales rep.") <br />
The technical components of performance tuning<br />
Technical architecture: Server and network tuning<br />
If there is a problem with the Oracle server, such as an overloaded CPU, excessive memory swapping or a disk I/O bottleneck, no amount of tuning within the Oracle database is going to improve your performance. <br />
Application architecture: Integration of middle tier. Apache Server, Forms/Reports Server.<br />
When evaluating the performance of forms and reports, check both requirements and standards under which they were developed. Forms and reports created using standard defaults may benefit from reducing or relaxing the standards for performance reasons. In one example, approximately 20 Web forms were completely redesigned when it was determined extensive use of check constraints impacted performance. For many of the forms, data input was prevalidated; when the constraints were removed, overall performance improved over 100%. <br />
Database architecture:<br />
(a) Instance tuning: Tuning the Oracle SGA is the next step, and all of the Oracle initialization parameters must be reviewed to ensure that the database has been properly configured. This phase of Oracle tuning is directed at looking for resources shortages in the db_block_buffers, shared_pool_size and sort_area_size. Investigate important default parameters for Oracle, such as optimizer_mode.<br />
(b) Object tuning: This phase of tuning looks at the setting for Oracle tables and indexes. Settings such as PCTFREE, PCTUSED and FREELISTS can have a dramatic impact on Oracle performance. <br />
SQL tuning: This is the most time-consuming tuning operation because there can be many thousands of individual SQL statements that access the Oracle database. At a high level, we identify the most common SQL statements, tune each one by carefully reviewing the execution plan for the SQL and adjust the execution plan using Oracle hints. <br />
Networks are increasingly more important as we move to global businesses. Work with your telecom provider to evaluate link capacity and utilization. Request detailed reports. Consider resizing to reduce costs on underutilized links or upgrading those close to capacity. Evaluate network drivers. Is there a more recent, downloadable version? Vendors often update their drivers. This can make a significant improvement. <br />
An up-to-date, accurate, detailed network diagram of both the logical and physical network locating all servers and users can be used to troubleshoot and predict problem areas. Check transaction processing routes to and from the server... if they are not the same, check the routing tables on each. What, if any, network management systems are used? Can any devices be placed in diagnostic mode during evaluations? Check that software and hardware are not operated routinely in diagnostic mode, as this will generally affect performance. Determine number of packet retries and collisions, bandwith utilizaton. Note any traffic management or priority queuing set on any device. Detail any contracts or service level agreements for the LAN and WAN. Identify any protocols installed on the clients. <br />
Compare findings with benchmarks to establish size and extent of performance issues. <br />
Implementing the plan<br />
Having evaluated the current state, and performed numerous measurements, actual changes to the system must be controlled and documented. Because you will most likely continue to assess the performance measures, changes should be documented in detail. A good practice is to establish a repository of network and database assessment scripts. Similarly, record the steps of any changes. All changes should be evaluated on a test system before rolling out to production systems. Before implementing any changes, and each successive change, be sure to make a complete backup and test your ability to restore conditions to their original state. Make only one change at a time. Recovering after multiple changes is frustrating at best and may be impossible. <br />
The sequence of changes will be based on priorities discussed with the users, management and technical personnel. You will also want to run baseline assessment scripts to determine whether the changes are effective. Occasionally, a change may not have the intended effect. This is what makes performance tuning an iterative task.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-67468500498071598192009-10-06T08:39:00.000-07:002009-10-06T08:39:21.695-07:00Tuning and Optimizing Red Hat Enterprise Linux for Oracle 9i and 10g DatabasesThis article is a step by step guide for tuning and optimizing Red Hat Enterprise Linux on x86 and x86-64 platforms running Oracle 9i (32bit/64bit) and Oracle 10g (32bit/64bit) standalone and RAC databases. This guide covers Red Hat Enterprise Linux Advanced Server 3 and 4 and the older version 2.1. For instructions on installing Oracle 9i and 10g databases, see Oracle on Linux. Other Linux articles can be found at www.puschitz.com. <br />
<br />
This article covers the following topics: <br />
<br />
* Introduction<br />
* Hardware Architectures and Linux Kernels<br />
General<br />
32-bit Architecture and the hugemem Kernel<br />
64-bit Architecture<br />
* Kernel Upgrades<br />
* Kernel Boot Parameters<br />
General<br />
I/O Scheduler<br />
* Memory Usage and Page Cache<br />
Checking Memory Usage<br />
Tuning Page Cache<br />
* Swap Space<br />
General<br />
Swap Size Recommendations<br />
Checking Swap Space Size and Usage<br />
* Setting Shared Memory<br />
Setting SHMMAX Parameter<br />
Setting SHMMNI Parameter<br />
Setting SHMALL Parameter<br />
Removing Shared Memory<br />
* Setting Semaphores<br />
The SEMMSL Parameter<br />
The SEMMNI Parameter<br />
The SEMMNS Parameter<br />
The SEMOPM Parameter<br />
Setting Semaphore Parameters<br />
Example for Semaphore Settings<br />
* Setting File Handles<br />
* Adjusting Network Settings<br />
Changing Network Adapter Settings<br />
Changing Network Kernel Settings<br />
Flow Control for e1000 NICs<br />
* Setting Shell Limits for the Oracle User<br />
Limiting Maximum Number of Open File Descriptors for the Oracle User<br />
Limiting Maximum Number of Processes for the Oracle User<br />
* Enabling Asynchronous I/O Support<br />
Relinking Oracle9i R2 to Enable Asynchronous I/O Support<br />
Relinking Oracle 10g to Enable Asynchronous I/O Support<br />
Enabling Asynchronous I/O in Oracle 9i and 10g<br />
Tuning Asynchronous I/O for Oracle 9i and 10g<br />
Checking Asynchronous I/O Usage<br />
* Configuring I/O for Raw Partitions<br />
General<br />
Basics of Raw Devices<br />
Using Raw Devices for Oracle Databases<br />
Using Block Devices for Oracle 10g Release 2 in RHEL 4<br />
* Large Memory Optimization (Big Pages, Huge Pages)<br />
Big Pages in RHEL 2.1 and Huge Pages in RHEL 3/4<br />
Usage of Big Pages and Huge Pages in Oracle 9i and 10g<br />
Sizing Big Pages and Huge Pages<br />
Checking Shared Memory Before Starting Oracle Databases<br />
Configuring Big Pages in RHEL 2.1<br />
Configuring Huge Pages in RHEL 3<br />
Configuring Huge Pages in RHEL 4<br />
Huge Pages and Shared Memory Filesystem in RHEL 3/4<br />
* Growing the Oracle SGA to 2.7 GB in x86 RHEL 2.1 Without VLM<br />
General<br />
Linux Memory Layout<br />
Increasing Space for the SGA in RHEL 2.1<br />
Lowering the Mapped Base Address for Shared Libraries in RHEL 2.1<br />
Lowering the SGA Attach Address for Shared Memory Segments in Oracle 9i<br />
Allowing the Oracle User to Change the Mapped Base Address for Shared Libraries<br />
* Growing the Oracle SGA to 2.7/3.42 GB in x86 RHEL 3/4 Without VLM<br />
General<br />
Mapped Base Address for Shared Libraries in RHEL 3 and RHEL 4<br />
Oracle 10g SGA Sizes in RHEL 3 and RHEL 4<br />
Lowering the SGA Attach Address in Oracle 10g<br />
* Using Very Large Memory (VLM)<br />
General<br />
Configuring Very Large Memory (VLM)<br />
* Measuring I/O Performance on Linux for Oracle Databases<br />
General<br />
Using Orion<br />
* Appendix<br />
* References<br />
<br />
<br />
Introduction<br />
This article discusses Red Hat Enterprise Linux optimizations for x86 (32 bit) and x86-64 (64 bit) platforms running Oracle 9i R2 (32bit/64bit) and Oracle 10g R1/R2 (32bit/64bit) standalone and RAC databases. This guide covers Red Hat Enterprise Linux Advanced Server 2.1, 3, and 4. Various workarounds covered in this article are due to the 32-bit address limitations of the x86 platform. However, many steps described in this document also apply to x86-64 platforms. Sections that do not specifically say that its only applicable to 32-bit or 64-bit apply to both platforms. If you think that a section is not very clear on that, let me know. For supported system configurations and limits for Red Hat Enterprise Linux releases, see http://www.redhat.com/rhel/details/limits/. <br />
<br />
Note this document comes without warranty of any kind. But every effort has been made to provide the information as accurate as possible. I welcome emails from any readers with comments, suggestions, and corrections at webmaster_at_puschitz.com. <br />
<br />
Hardware Architectures and Linux Kernels<br />
General<br />
<br />
When it comes to large databases the hybrid x86-64 architecture platform is strongly recommended over the 32-bit x86 platform. 64-bit platforms can access more than 4GB of memory without workarounds. With 32-bit platforms there are several issues that require workaround solutions for databases that use lots of memory, for example refer to Using Very Large Memory (VLM). If you are not sure whether you are on a 32-bit or 64-bit hardware, run dmidecode or cat /proc/cpuinfo. Running uname -a can be misleading since 32-bit Linux kernels can run on x86-64 platforms. But if uname -a displays x86_64, then you are running a 64-bit Linux kernel on a x86-64 platform. <br />
<br />
<br />
32-bit Architecture and the hugemem Kernel<br />
<br />
The RHEL 3/4 smp kernel can be used on systems with up to 16 GB of RAM. The hugemem kernel is required in order to use all the memory on systems that have more than 16GB of RAM up to 64GB.However, I recommend the hugemem kernel even on systems that have 8GB of RAM or more due to the potential issue of "low memory" starvation (see next section) that can happen on database systems with 8 GB of RAM. The stability you get with the hugemem kernel on larger systems outperforms the performance overhead of address space switching. <br />
<br />
With x86 architecture the first 16MB-896MB of physical memory is known as "low memory" (ZONE_NORMAL) which is permanently mapped into kernel space. Many kernel resources must live in the low memory zone. In fact, many kernel operations can only take place in this zone. This means that the low memory area is the most performance critical zone. For example, if you run many resources intensive applications/programs and/or use large physical memory, then "low memory" can become low since more kernel structures must be allocated in this area. Low memory starvation happens when LowFree in/proc/meminfo becomes very low accompanied by a sudden spike in paging activity. To free up memory in the low memory zone, the kernel bounces buffers aggressively between low memory and high memory which becomes noticeable as paging (don't confuse it with paging to the swap partition). If the kernel is unable to free up enough memory in the low memory zone, then the kernel can hang the system. <br />
<br />
Paging activity can be monitored using the vmstat command or using the sar command (option '-B') which comes with the sysstat RPM. Since Linux tries to utilize the whole low memory zone, a lowLowFree in /proc/meminfo does not necessarily mean that the system is out of low memory. However, when the system shows increased paging activity when LowFree gets below 50MB, then thehugemem kernel should be installed. The stability you gain from using the hugemem kernel makes up for any performance impact resulting from the 4GB-4GB kernel/user memory split in this kernel (a classic 32-bit x86 system splits the available 4 GB address space into 3 GB virtual memory space for user processes and a 1 GB space for the kernel). To see some allocations in the low memory zone, refer to/proc/meminfo and slabtop(1) for more information. Note that Huge Pages would free up memory in the low memory zone since the system has less bookkeeping to do for that part of virtual memory, see Large Memory Optimization (Big Pages, Huge Pages). <br />
<br />
If you install the RHEL 3/4 hugemem kernel ensure that any proprietary drivers you are using (e.g. proprietary multipath drivers) are certified with the hugemem kernel. <br />
<br />
In RHEL 2.1, the smp kernel is capable of handling up to 4GB of RAM. The kernel-enterprise kernel should be used for systems with more than 4GB of RAM up to 16GB. <br />
<br />
<br />
64-bit Architecture<br />
<br />
This is the architecture that should be used whenever possible. If you can go with a x86-64 platform ensure that all drivers you need are supported on x86-64 (e.g. proprietary multipath drivers etc.) Furthermore, ensure that all the required applications are supported on x86-64 as well. <br />
<br />
Kernel Upgrades<br />
Make sure to install the latest kernel where all proprietary drivers, if applicable, are certified/supported. <br />
<br />
Note that proprietary drivers are often installed under /lib/modules/<kernel-version>/kernel/drivers/addon. For example, the EMC PowerPath drivers can be found in the following directory when running the 2.4.21-32.0.1.ELhugemem kernel:<br />
$ ls -al /lib/modules/2.4.21-32.0.1.ELhugemem/kernel/drivers/addon/emcpower<br />
total 732 <br />
drwxr-xr-x 2 root root 4096 Aug 20 13:50 . <br />
drwxr-xr-x 19 root root 4096 Aug 20 13:50 .. <br />
-rw-r--r-- 1 root root 14179 Aug 20 13:50 emcphr.o <br />
-rw-r--r-- 1 root root 2033 Aug 20 13:50 emcpioc.o <br />
-rw-r--r-- 1 root root 91909 Aug 20 13:50 emcpmpaa.o <br />
-rw-r--r-- 1 root root 131283 Aug 20 13:50 emcpmpap.o <br />
-rw-r--r-- 1 root root 113922 Aug 20 13:50 emcpmpc.o <br />
-rw-r--r-- 1 root root 75380 Aug 20 13:50 emcpmp.o <br />
-rw-r--r-- 1 root root 263243 Aug 20 13:50 emcp.o <br />
-rw-r--r-- 1 root root 8294 Aug 20 13:50 emcpsf.o <br />
$<br />
Therefore, when you upgrade the kernel you must ensure that all proprietary modules can be found in the right directory so that the kernel can load them. <br />
<br />
To check which kernels are installed, run the following command:<br />
$ rpm -qa | grep kernel<br />
To check which kernel is currently running, execute the following command:<br />
$ uname -r<br />
For example, to install the 2.4.21-32.0.1.ELhugemem kernel, download the kernel-hugemem RPM and execute the following command:<br />
# rpm -ivh kernel-hugemem-2.4.21-32.0.1.EL.i686.rpm<br />
Never upgrade the kernel using the RPM option '-U'. The previous kernel should always be available if the newer kernel does not boot or work properly. <br />
<br />
To make sure the right kernel is booted, check the /etc/grub.conf file if you use GRUB and change the "default" attribute if necessary. Here is an example:<br />
default=0<br />
timeout=10 <br />
splashimage=(hd0,0)/grub/splash.xpm.gz <br />
title Red Hat Enterprise Linux AS (2.4.21-32.0.1.ELhugemem)<br />
root (hd0,0) <br />
kernel /vmlinuz-2.4.21-32.0.1.ELhugemem ro root=/dev/sda2<br />
initrd /initrd-2.4.21-32.0.1.ELhugemem.img <br />
title Red Hat Enterprise Linux AS (2.4.21-32.0.1.ELsmp) <br />
root (hd0,0) <br />
kernel /vmlinuz-2.4.21-32.0.1.ELsmp ro root=/dev/sda2<br />
initrd /initrd-2.4.21-32.0.1.ELsmp.img<br />
In this example, the "default" attribute is set to "0" which means that the 2.4.21-32.0.1.ELhugemem kernel will be booted. If the "default" attribute would be set to "1", then 2.4.21-32.0.1.ELsmpwould be booted. <br />
<br />
After you installed the newer kernel reboot the system. <br />
<br />
Once you are sure that you don't need the old kernel anymore, you can remove the old kernel by running:<br />
# rpm -e <OldKernelVersion><br />
When you remove a kernel, you don't need to update /etc/grub.conf. <br />
<br />
Kernel Boot Parameters<br />
General<br />
<br />
The Linux kernel accepts boot parameters when the kernel is started. Very often it's used to provide information to the kernel about hardware parameters where the kernel would have issues/problems or to overwrite default values. <br />
<br />
For a list of kernel parameters in RHEL4, see /usr/share/doc/kernel-doc-2.6.9/Documentation/kernel-parameters.txt. This file does not exist if the kernel-doc RPM is not installed. And for a list of kernel parameters in RHEL3 and RHEL2.1, see /usr/src/linux-2.4/Documentation/kernel-parameters.txt which comes with the kernel-doc RPM. <br />
<br />
I/O Scheduler<br />
<br />
Starting with the 2.6 kernel, i.e. RHEL 4, the I/O scheduler can be changed at boot time which controls the way the kernel commits reads and writes to disks. For more information on various I/O scheduler, see Choosing an I/O Scheduler for Red Hat Enterprise Linux 4 and the 2.6 Kernel. <br />
<br />
The Completely Fair Queuing (CFQ) scheduler is the default algorithm in RHEL4 which is suitable for a wide variety of applications and provides a good compromise between throughput and latency. In comparison to the CFQ algorithm, the Deadline scheduler caps maximum latency per request and maintains a good disk throughput which is best for disk-intensive database applications. Hence, the Deadlinescheduler is recommended for database systems. Also, at the time of this writing there is a bug in the CFQ scheduler which affects heavy I/O, see Metalink Bug:5041764. Even though this bug report talks about OCFS2 testing, this bug can also happen during heavy IO access to raw/block devices and as a consequence could evict RAC nodes. <br />
<br />
To switch to the Deadline scheduler, the boot parameter elevator=deadline must be passed to the kernel that's being used. Edit the /etc/grub.conf file and add the following parameter to the kernel that's being used, in this example 2.4.21-32.0.1.ELhugemem:<br />
title Red Hat Enterprise Linux Server (2.6.18-8.el5)<br />
root (hd0,0) <br />
kernel /vmlinuz-2.6.18-8.el5 ro root=/dev/sda2 elevator=deadline<br />
initrd /initrd-2.6.18-8.el5.img<br />
This entry tells the 2.6.18-8.el5 kernel to use the Deadline scheduler. Make sure to reboot the system to activate the new scheduler. <br />
<br />
Memory Usage and Page Cache<br />
<br />
Checking Memory Usage <br />
<br />
To determine the size and usage of memory, you can enter the following command:<br />
grep MemTotal /proc/meminfo<br />
You can find a detailed description of the entries in /proc/meminfo at http://www.redhat.com/advice/tips/meminfo.html. <br />
<br />
Alternatively, you can use the free(1) command to check the memory:<br />
$ free<br />
total used free shared buffers cached<br />
Mem: 4040360 4012200 28160 0 176628 3571348<br />
-/+ buffers/cache: 264224 3776136<br />
Swap: 4200956 12184 4188772<br />
$<br />
In this example the total amount of available memory is 4040360 KB. 264224 KB are used by processes and 3776136 KB are free for other applications. Don't get confused by the first line which shows that 28160KB are free! If you look at the usage figures you can see that most of the memory use is for buffers and cache since Linux always tries to use RAM to the fullest extent to speed up disk operations. Using available memory for buffers (file system metadata) and cache (pages with actual contents of files or block devices) helps the system to run faster because disk information is already in memory which saves I/O. If space is needed by programs or applications like Oracle, then Linux will free up the buffers and cache to yield memory for the applications. So if your system runs for a while you will usually see a small number under the field "free" on the first line. <br />
<br />
<br />
Tuning Page Cache <br />
<br />
Page Cache is a disk cache which holds data of files and executable programs, i.e. pages with actual contents of files or block devices. Page Cache (disk cache) is used to reduce the number of disk reads. To control the percentage of total memory used for page cache in RHEL 3, the following kernel parameter can be changed:<br />
# cat /proc/sys/vm/pagecache<br />
1 15 30<br />
The above three values are usually good for database systems. It is not recommended to set the third value very high like 100 as it used to be with older RHEL 3 kernels. This can cause significant performance problems for database systems. If you upgrade to a newer kernel like 2.4.21-37, then these values will automatically change to "1 15 30" unless it's set to different values in /etc/sysctl.conf. For information on tuning the pagecache kernel parameter, I recommend reading the excellent article Understanding Virtual Memory. Note this kernel parameter does not exist in RHEL 4. <br />
<br />
The pagecache parameters can be changed in the proc file system without reboot:<br />
# echo "1 15 30" > /proc/sys/vm/pagecache<br />
Alternatively, you can use sysctl(8) to change it:<br />
# sysctl -w vm.pagecache="1 15 30"<br />
To make the change permanent, add the following line to the file /etc/sysctl.conf. This file is used during the boot process.<br />
# echo "vm.pagecache=1 15 30" >> /etc/sysctl.conf<br />
<br />
Swap Space<br />
General <br />
<br />
In some cases it's good for the swap partition to be used. For example, long running processes often access only a subset of the page frames they obtained. This means that the swap partition can safely be used even if memory is available because system memory could be better served for disk cache to improve overall system performance. In fact, in the 2.6 kernel, i.e. RHEL 4, you can define a threshold when processes should be swapped out in favor of I/O caching. This can be tuned with the /proc/sys/vm/swappiness kernel parameter. The default value of /proc/sys/vm/swappiness is 60 which means that applications and programs that have not done a lot lately can be swapped out. Higher values will provide more I/O cache and lower values will wait longer to swap out idle applications. <br />
<br />
Depending on the system profile you may see that swap usage slowly increases with system uptime. To display swap usage you can run the free(1) command or you can check the /proc/meminfo file. When the system uses swap space it will sometimes not decrease afterward. This saves I/O if memory is needed and pages don't have to be swapped out again when the pages are already in the swap space. However, if swap usage gets close to 80% - 100% (your threshold may be lower if you use a large swap space), then a closer look should be taken at the system, see also Checking Swap Space Size and Usage. Depending on the size of your swap space, you may want to check swap activity with vmstat or sar if swap allocation is lower than 80%. But these numbers really depend on the size of the swap space. The actual numbers of swapped pages per timeframe from vmstat or sar are the important numbers. Constant swapping should be avoided at all cost. <br />
<br />
Note, never add a permanent swap file to the system due to the performance impact of the filesystem layer. <br />
<br />
<br />
Swap Size Recommendations <br />
<br />
According to Oracle9i Installation Guide Release 2 a minimum of 512MB of RAM is required to install Oracle9i Server. <br />
According to Oracle Database Installation Guide 10g Release 2 at least 1024MB of RAM is required for 10g R2. <br />
<br />
For 10g R2, Oracle gives the following swap space requirement:<br />
RAM Swap Space<br />
--------------------------------------------<br />
1 GB - 2 GB 1.5 times the size of RAM<br />
2 GB - 8 GB Equal to the size of RAM<br />
more than 8GB 0.75 times the size of RAM<br />
<br />
<br />
Checking Swap Space Size and Usage <br />
<br />
You can check the size and current usage of swap space by running one of the following two commands:<br />
grep SwapTotal /proc/meminfo<br />
cat /proc/swaps<br />
free<br />
<br />
Swap usage may slowly increase as shown above but should stop at some point. If swap usage continues to grow steadily or is already large, then one of the following choices may need to be considered:<br />
- Add more RAM or reduce the size of the SGA<br />
- Increase the size of the swap space<br />
<br />
If you see constant swapping, then you need to either add more RAM or reduce the size of the SGA. Constant swapping should be avoided at all cost. You can check current swap activity using the following commands:<br />
$ vmstat 3 100<br />
procs memory swap io system cpu<br />
r b swpd free buff cache si so bi bo in cs us sy id wa<br />
1 0 0 972488 7148 20848 0 0 856 6 138 53 0 0 99 0<br />
0 1 0 962204 9388 20848 0 0 747 0 4389 8859 23 24 11 41<br />
0 1 0 959500 10728 20848 0 0 440 313 1496 2345 4 7 0 89<br />
0 1 0 956912 12216 20848 0 0 496 0 2294 4224 10 13 0 77<br />
1 1 0 951600 15228 20848 0 0 997 264 2241 3945 6 13 0 81<br />
0 1 0 947860 17188 20848 0 0 647 280 2386 3985 9 9 1 80<br />
0 1 0 944932 19304 20848 0 0 705 0 1501 2580 4 9 0 87<br />
The fields si and so show the amount of memory paged in from disk and paged out to disk, respectively. If the server shows continuous swap activity then more memory should be added or the SGA size should be reduced. To check the history of swap activity, you can use the sar command.<br />
For example, to check swap activity from Oct 12th:<br />
# ls -al /var/log/sa | grep "Oct 12"<br />
-rw-r--r-- 1 root root 2333308 Oct 12 23:55 sa12<br />
-rw-r--r-- 1 root root 4354749 Oct 12 23:53 sar12<br />
# sar -W -f /var/log/sa/sa12<br />
Linux 2.4.21-32.0.1.ELhugemem (rac01prd) 10/12/2005<br />
<br />
12:00:00 AM pswpin/s pswpout/s<br />
12:05:00 AM 0.00 0.00<br />
12:10:00 AM 0.00 0.00<br />
12:15:00 AM 0.00 0.00<br />
12:20:00 AM 0.00 0.00<br />
12:25:00 AM 0.00 0.00<br />
12:30:00 AM 0.00 0.00<br />
...<br />
The fields pswpin and pswpout show the total number of pages brought in and out per second, respectively. <br />
<br />
If the server shows sporadic swap activity or swap activity for a short period time at certain invervals, then you can either add more swap space or RAM. If swap usage is already very large (don't confuse it with constant swapping), then I would add more RAM. <br />
<br />
Setting Shared Memory<br />
Shared memory allows processes to access common structures and data by placing them in shared memory segments. It's the fastest form of Interprocess Communication (IPC) available since no kernel involvement occurs when data is passed between the processes. In fact, data does not need to be copied between the processes. <br />
<br />
Oracle uses shared memory segments for the Shared Global Area (SGA) which is an area of memory that is shared by Oracle processes. The size of the SGA has a significant impact to Oracle's performance since it holds database buffer cache and much more. <br />
<br />
To see all shared memory settings, execute:<br />
$ ipcs -lm<br />
<br />
Setting SHMMAX Parameter <br />
<br />
This parameter defines the maximum size in bytes of a single shared memory segment that a Linux process can allocate in its virtual address space. For example, if you use the RHEL 3 smp kernel on a 32-bit platform (x86), then the virtual address space for a user process is 3 GB. If you use the RHEL 3 hugemem kernel on a 32-bit platform (x86), then the virtual address space for a user process is almost 4GB. Hence, setting SHMMAX to 4 GB - 1 byte (4294967295 bytes) on a smp kernel on a 32-bit architecture won't increase the maximum size of a shared memory segment to 4 GB -1. Even setting SHMMAX to 4 GB - 1 byte using the hugemem kernel on a 32-bit architecture won't enable a process to get such a large shared memory segment. In fact, the upper limit for a shared memory segment for an Oracle 10g R1 SGA using the hugemem kernel is roughly 3.42 GB (~3.67 billion bytes) since virtual address space is also needed for other things like shared libraries. This means if you have three 2 GB shared memory segments on a 32-bit system, no process can attach to more than one shared memory segment at a time. Also note if you set SHMMAX to 4294967296 bytes (4*1024*1024*1024=4GB) on a 32-bit system, then SHMMAX will essentially bet set to 0 bytes since it wraps around the 4GB value. This means that SHMMAX should not exceed 4294967295 on a 32-bit system. On x86-64 platforms, SHMMAX can be much larger than 4GB since the virtual address space is not limited by 32 bits. <br />
<br />
Since the SGA is comprised of shared memory, SHMMAX can potentially limit the size of the SGA. SHMMAX should be slightly larger than the SGA size. If SHMMAX is too small, you can get error messages similar to this one:<br />
ORA-27123: unable to attach to shared memory segment<br />
It is highly recommended that the shared memory fits into the Big Pages or Huge Pages pool, see Large Memory Optimization (Big Pages, Huge Pages). <br />
<br />
To increase the default maximum SGA size on x86 RHEL 2.1 systems without VLM, refer to Growing the Oracle SGA to 2.7 GB in x86 RHEL 2.1 Without VLM. <br />
To increase the default maximum SGA size on x86 RHEL 3/4 systems without VLM, refer to Growing the Oracle SGA to 2.7/3.42 GB in x86 RHEL 3/4 Without VLM. <br />
<br />
To determine the maximum size of a shared memory segment, run:<br />
# cat /proc/sys/kernel/shmmax<br />
2147483648<br />
The default shared memory limit for SHMMAX can be changed in the proc file system without reboot:<br />
# echo 2147483648 > /proc/sys/kernel/shmmax<br />
Alternatively, you can use sysctl(8) to change it:<br />
# sysctl -w kernel.shmmax=2147483648<br />
To make a change permanent, add the following line to the file /etc/sysctl.conf (your setting may vary). This file is used during the boot process.<br />
# echo "kernel.shmmax=2147483648" >> /etc/sysctl.conf<br />
<br />
Setting SHMMNI Parameter <br />
<br />
This parameter sets the system wide maximum number of shared memory segments. <br />
<br />
Oracle recommends SHMMNI to be at least 4096 for Oracle 10g. For Oracle 9i on x86 the recommended minimum setting is lower. Since these recommendations are minimum settings, it's best to set it always to at least 4096 for 9i and 10g databases on x86 and x86-64 platforms. <br />
<br />
To determine the system wide maximum number of shared memory segments, run:<br />
# cat /proc/sys/kernel/shmmni<br />
4096<br />
The default shared memory limit for SHMMNI can be changed in the proc file system without reboot:<br />
# echo 4096 > /proc/sys/kernel/shmmni<br />
Alternatively, you can use sysctl(8) to change it:<br />
# sysctl -w kernel.shmmni=4096<br />
To make a change permanent, add the following line to the file /etc/sysctl.conf. This file is used during the boot process.<br />
# echo "kernel.shmmni=4096" >> /etc/sysctl.conf<br />
<br />
Setting SHMALL Parameter <br />
<br />
This parameter sets the total amount of shared memory pages that can be used system wide. Hence, SHMALL should always be at least ceil(shmmax/PAGE_SIZE). <br />
<br />
The default size for SHMALL in RHEL 3/4 and 2.1 is 2097152 which is also Oracle's recommended minimum setting for 9i and 10g on x86 and x86-64 platforms. In most cases this setting should be sufficient since it means that the total amount of shared memory available on the system is 2097152*4096 bytes (shmall*PAGE_SIZE) which is 8 GB. PAGE_SIZE is usually 4096 bytes unless you use Big Pages or Huge Pages which supports the configuration of larger memory pages. <br />
<br />
If you are not sure what the default PAGE_SIZE is on your Linux system, you can run the following command:<br />
$ getconf PAGE_SIZE<br />
4096<br />
To determine the system wide maximum number of shared memory pages, run:<br />
# cat /proc/sys/kernel/shmall<br />
2097152<br />
The default shared memory limit for SHMALL can be changed in the proc file system without reboot:<br />
# echo 2097152 > /proc/sys/kernel/shmall<br />
Alternatively, you can use sysctl(8) to change it:<br />
# sysctl -w kernel.shmall=2097152<br />
To make a change permanent, add the following line to the file /etc/sysctl.conf. This file is used during the boot process.<br />
# echo "kernel.shmall=2097152" >> /etc/sysctl.conf<br />
<br />
<br />
Removing Shared Memory<br />
<br />
Sometimes after an instance crash you may have to remove Oracle's shared memory segment(s) manually. <br />
<br />
To see all shared memory segments that are allocated on the system, execute:<br />
$ ipcs -m<br />
<br />
------ Shared Memory Segments --------<br />
key shmid owner perms bytes nattch status<br />
0x8f6e2129 98305 oracle 600 77694523 0<br />
0x2f629238 65536 oracle 640 2736783360 35<br />
0x00000000 32768 oracle 640 2736783360 0 dest<br />
<br />
In this example you can see that three shared memory segments have been allocated. The output also shows that shmid 32768 is an abandoned shared memory segment from a past ungraceful Oracle shutdown. Status "dest" means that this memory segment is marked to be destroyed. To find out more about this shared memory segment you can run:<br />
$ ipcs -m -i 32768<br />
Shared memory Segment shmid=32768<br />
uid=500 gid=501 cuid=500 cgid=501<br />
mode=0640 access_perms=0640<br />
bytes=2736783360 lpid=3688 cpid=3652 nattch=0<br />
att_time=Sat Oct 29 13:36:52 2005<br />
det_time=Sat Oct 29 13:36:52 2005<br />
change_time=Sat Oct 29 11:21:06 2005<br />
<br />
To remove the shared memory segment, you could copy/paste shmid and execute:<br />
$ ipcrm shm 32768<br />
<br />
Another approach to remove shared memory is to use Oracle's sysresv utility. Here are a few self explanatory examples on how to use sysresv: <br />
<br />
Checking Oracle's IPC resources:<br />
$ sysresv<br />
<br />
IPC Resources for ORACLE_SID "orcl" :<br />
Shared Memory<br />
ID KEY<br />
No shared memory segments used<br />
Semaphores:<br />
ID KEY<br />
No semaphore resources used<br />
Oracle Instance not alive for sid "orcl"<br />
$<br />
<br />
Instance is up and running:<br />
$ sysresv -i<br />
<br />
IPC Resources for ORACLE_SID "orcl" :<br />
Shared Memory:<br />
ID KEY<br />
2818058 0xdc70f4e4<br />
Semaphores:<br />
ID KEY<br />
688128 0xb11a5934<br />
Oracle Instance alive for sid "orcl"<br />
SYSRESV-005: Warning<br />
Instance maybe alive - aborting remove for sid "orcl"<br />
$<br />
<br />
Instance has crashed and resources were not released:<br />
$ sysresv -i<br />
<br />
IPC Resources for ORACLE_SID "orcl" :<br />
Shared Memory:<br />
ID KEY<br />
32768 0xdc70f4e4<br />
Semaphores:<br />
ID KEY<br />
98304 0xb11a5934<br />
Oracle Instance not alive for sid "orcl"<br />
Remove ipc resources for sid "orcl" (y/n)?y<br />
Done removing ipc resources for sid "orcl"<br />
$<br />
<br />
Setting Semaphores<br />
Semaphores can be described as counters which are used to provide synchronization between processes or between threads within a process for shared resources like shared memories. System V semaphores support semaphore sets where each one is a counting semaphore. So when an application requests semaphores, the kernel releases them in sets. The number of semaphores per set can be defined through the kernel parameter SEMMSL. <br />
<br />
To see all semaphore settings, run:<br />
ipcs -ls<br />
<br />
The SEMMSL Parameter <br />
<br />
This parameter defines the maximum number of semaphores per semaphore set. <br />
<br />
Oracle recommends SEMMSL to be at least 250 for 9i R2 and 10g R1/R2 databases except for 9i R2 on x86 platforms where the minimum value is lower. Since these recommendations are minimum settings, it's best to set it always to at least 250 for 9i and 10g databases on x86 and x86-64 platforms. <br />
<br />
NOTE:<br />
If a database gets thousands of concurrent connections where the ora.init parameter PROCESSES is very large, then SEMMSL should be larger as well. Note what Metalink Note:187405.1 and Note:184821.1 have to say regarding SEMMSL: "The SEMMSL setting should be 10 plus the largest PROCESSES parameter of any Oracle database on the system". Even though these notes talk about 9i databases this SEMMSL rule also applies to 10g databases. I've seen low SEMMSL settings to be an issue for 10g RAC databases where Oracle recommended to increase SEMMSL and to calculate it according to the rule mentioned in these notes. An example for setting semaphores for higher PROCESSES settings can be found at Example for Semaphore Settings. <br />
<br />
The SEMMNI Parameter <br />
<br />
This parameter defines the maximum number of semaphore sets for the entire Linux system. <br />
<br />
Oracle recommends SEMMNI to be at least 128 for 9i R2 and 10g R1/R2 databases except for 9i R2 on x86 platforms where the minimum value is lower. Since these recommendations are minimum settings, it's best to set it always to at least 128 for 9i and 10g databases on x86 and x86-64 platforms. <br />
<br />
The SEMMNS Parameter <br />
<br />
This parameter defines the total number of semaphores (not semaphore sets) for the entire Linux system. A semaphore set can have more than one semaphore, and as the semget(2) man page explains, values greater than SEMMSL * SEMMNI makes it irrelevant. The maximum number of semaphores that can be allocated on a Linux system will be the lesser of: SEMMNS or (SEMMSL * SEMMNI). <br />
<br />
Oracle recommends SEMMNS to be at least 32000 for 9i R2 and 10g R1/R2 databases except for 9i R2 on x86 platforms where the minimum value is lower. Setting SEMMNS to 32000 ensures that SEMMSL * SEMMNI (250*128=32000) semaphores can be be used. Therefore it's recommended to set SEMMNS to at least 32000 for 9i and 10g databases on x86 and x86-64 platforms. <br />
<br />
The SEMOPM Parameter <br />
<br />
This parameter defines the maximum number of semaphore operations that can be performed per semop(2) system call (semaphore call). The semop(2) function provides the ability to do operations for multiple semaphores with one semop(2) system call. Since a semaphore set can have the maximum number of SEMMSL semaphores per semaphore set, it is often recommended to set SEMOPM equal to SEMMSL. <br />
<br />
Oracle recommends to set SEMOPM to a minimum value of 100 for 9i R2 and 10g R1/R2 databases on x86 and x86-64 platforms. <br />
<br />
Setting Semaphore Parameters <br />
<br />
To determine the values of the four described semaphore parameters, run:<br />
# cat /proc/sys/kernel/sem<br />
250 32000 32 128<br />
These values represent SEMMSL, SEMMNS, SEMOPM, and SEMMNI. <br />
<br />
Alternatively, you can run:<br />
# ipcs -ls<br />
All four described semaphore parameters can be changed in the proc file system without reboot:<br />
# echo 250 32000 100 128 > /proc/sys/kernel/sem<br />
Alternatively, you can use sysctl(8) to change it:<br />
sysctl -w kernel.sem="250 32000 100 128"<br />
To make the change permanent, add or change the following line in the file /etc/sysctl.conf. This file is used during the boot process.<br />
echo "kernel.sem=250 32000 100 128" >> /etc/sysctl.conf<br />
<br />
Example for Semaphore Settings <br />
<br />
On systems where the ora.init parameter PROCESSES is very large, the semaphore settings need to be adjusted accordingly. <br />
<br />
As shown at The SEMMSL Parameter the SEMMSL setting should be 10 plus the largest PROCESSES parameter of any Oracle database on the system. So if you have one database instance running on a system where PROCESSES is set to 5000, then SEMMSL should be set to 5010. <br />
<br />
As shown at The SEMMNS Parameter the maximum number of semaphores that can be allocated on a Linux system will be the lesser of: SEMMNS or (SEMMSL * SEMMNI). Since SEMMNI can stay at 128, we need to increase SEMMNS to 641280 (5010*128). <br />
<br />
As shown at The SEMOPM Parameter a semaphore set can have the maximum number of SEMMSL semaphores per semaphore set and it is recommended to set SEMOPM equal to SEMMSL. Since SEMMSL is set to 5010 the SEMOPM parameter should be set to 5010 as well. <br />
<br />
Hence, if the ora.init parameter PROCESSES is set to 5000, then the semaphore settings should be as follows: <br />
sysctl -w kernel.sem="5010 641280 5010 128"<br />
<br />
Setting File Handles<br />
The maximum number of file handles specifies the maximum number of open files on a Linux system. <br />
<br />
Oracle recommends that the file handles for the entire system is set to at least 65536 for 9i R2 and 10g R1/2 for x86 and x86-64 platforms. <br />
<br />
To determine the maximum number of file handles for the entire system, run:<br />
cat /proc/sys/fs/file-max<br />
To determine the current usage of file handles, run:<br />
$ cat /proc/sys/fs/file-nr<br />
1154 133 8192<br />
The file-nr file displays three parameters:<br />
- Total allocated file handles<br />
- Currently number of used file handles (2.4 kernel); Currently number of unused file handles (2.6 kernel)<br />
- Maximum file handles that can be allocated (see also /proc/sys/fs/file-max)<br />
<br />
The kernel dynamically allocates file handles whenever a file handle is requested by an application but the kernel does not free these file handles when they are released by the application. The kernel recycles these file handles instead. This means that over time the total number of allocated file handles will increase even though the number of currently used file handles may be low. <br />
<br />
The maximum number of file handles can be changed in the proc file system without reboot:<br />
# echo 65536 > /proc/sys/fs/file-max<br />
Alternatively, you can use sysctl(8) to change it:<br />
# sysctl -w fs.file-max=65536<br />
To make the change permanent, add or change the following line in the file /etc/sysctl.conf. This file is used during the boot process.<br />
# echo "fs.file-max=65536" >> /etc/sysctl.conf<br />
<br />
Adjusting Network Settings<br />
<br />
Changing Network Adapter Settings <br />
<br />
To check the speed and settings of network adapters, use the ethtool command which works now for most NICs. For example, to check the adapter settings of eth0 run:<br />
# ethtool eth0<br />
To force a speed change to 1000 full duplex, run:<br />
# ethtool -s eth0 speed 1000 duplex full autoneg off<br />
To make a speed change permanent for eth0, set or add the ETHTOOL_OPT environment variable in /etc/sysconfig/network-scripts/ifcfg-eth0:<br />
ETHTOOL_OPTS="speed 1000 duplex full autoneg off"<br />
This environment variable is sourced in by the network scripts each time the network service is started. <br />
<br />
<br />
Changing Network Kernel Settings <br />
<br />
Oracle now uses UDP as the default protocol on Linux for interprocess communication, such as cache fusion buffer transfers between the instances. But starting with Oracle 10g network settings should be adjusted for standalone databases as well. <br />
<br />
Oracle recommends the default and maximum send buffer size (SO_SNDBUF socket option) and receive buffer size (SO_RCVBUF socket option) to be set to 256 KB. The receive buffers are used by TCP and UDP to hold the received data for the application until it's read. This buffer cannot overflow because the sending party is not allowed to send data beyond the buffer size window. This means that datagrams will be discarded if they don't fit in the receive buffer. This could cause the sender to overwhelm the receiver <br />
<br />
The default and maximum window size can be changed in the proc file system without reboot:<br />
# sysctl -w net.core.rmem_default=262144 # Default setting in bytes of the socket receive buffer<br />
# sysctl -w net.core.wmem_default=262144 # Default setting in bytes of the socket send buffer<br />
# sysctl -w net.core.rmem_max=262144 # Maximum socket receive buffer size which may be set by using the SO_RCVBUF socket option<br />
# sysctl -w net.core.wmem_max=262144 # Maximum socket send buffer size which may be set by using the SO_SNDBUF socket option<br />
To make the change permanent, add the following lines to the /etc/sysctl.conf file, which is used during the boot process:<br />
net.core.rmem_default=262144<br />
net.core.wmem_default=262144<br />
net.core.rmem_max=262144<br />
net.core.wmem_max=262144<br />
To improve failover performance in a RAC cluster, consider changing the following IP kernel parameters as well:<br />
net.ipv4.tcp_keepalive_time<br />
net.ipv4.tcp_keepalive_intvl<br />
net.ipv4.tcp_retries2<br />
net.ipv4.tcp_syn_retries<br />
Changing these settings may be highly dependent on your system, network, and other applications. For suggestions, see Metalink Note:249213.1 and Note:265194.1. <br />
<br />
<br />
On RHEL systems the default range of IP port numbers that are allowed for TCP and UDP traffic on the server is too low for 9i and 10g systems. Oracle recommends the following port range:<br />
# sysctl -w net.ipv4.ip_local_port_range="1024 65000"<br />
To make the change permanent, add the following line to the /etc/sysctl.conf file, which is used during the boot process:<br />
net.ipv4.ip_local_port_range=1024 65000<br />
The first number is the first local port allowed for TCP and UDP traffic, and the second number is the last port number. <br />
<br />
<br />
Flow Control for e1000 NICs <br />
<br />
The e1000 NICs don't have flow control enabled in the 2.6 kernel, i.e RHEL 4. If you have heavy traffic, then the RAC interconnects may lose blocks, see Metalink Bug:5058952. For more information on flow control, see Wikipedia Flow control. <br />
<br />
To enable Receive flow control for e1000 NICs, add the following line to the /etc/modprobe.conf file:<br />
options e1000 FlowControl=1<br />
The e1000 module needs to be reloaded for the change to take effect. Once the module is loaded with flow control, you should see e1000 flow control module messages in /var/log/messages. <br />
<br />
Setting Shell Limits for the Oracle User<br />
Most shells like Bash provide control over various resources like the maximum allowable number of open file descriptors or the maximum number of processes available to a user. <br />
<br />
To see all shell limits, run:<br />
ulimit -a<br />
For more information on ulimit for the Bash shell, see man bash and search for ulimit. <br />
<br />
NOTE:<br />
On some Linux systems setting "hard" and "soft" limits in the following examples might not work properly when you login as oracle via SSH. It might work if you log in as root and su to oracle. If you have this problem try to set UsePrivilegeSeparation to "no" in /etc/ssh/sshd_config and restart the SSH daemon by executing service sshd restart. The privilege separation does not work properly with PAM on some Linux systems. Make sure to talk to the Unix and/or security teams before disabling the SSH security feature "Privilege Separation". <br />
<br />
<br />
Limiting Maximum Number of Open File Descriptors for the Oracle User <br />
<br />
After /proc/sys/fs/file-max has been changed, see Setting File Handles, there is still a per user limit of maximum open file descriptors:<br />
$ su - oracle<br />
$ ulimit -n<br />
1024<br />
$<br />
To change this limit, edit the /etc/security/limits.conf file as root and make the following changes or add the following lines, respectively:<br />
oracle soft nofile 4096<br />
oracle hard nofile 63536<br />
The "soft limit" in the first line defines the number of file handles or open files that the Oracle user will have after login. If the Oracle user gets error messages about running out of file handles, then the Oracle user can increase the number of file handles like in this example up to 63536 ("hard limit") by executing the following command:<br />
<br />
ulimit -n 63536<br />
You can set the "soft" and "hard" limits higher if necessary. <br />
<br />
NOTE:<br />
I do not recommend to set the "hard" limit for nofile for the oracle user equal to /proc/sys/fs/file-max. If you do that and the user uses up all the file handles, then the entire system will run out of file handles. This could mean that you won't be able to initiate new logins any more since the system won't be able to open any PAM modules that are required for the login process. That's why I set the hard limit to 63536 and not 65536. <br />
<br />
That these limits work you also need to ensure that pam_limits is configured in the /etc/pam.d/system-auth file, or in /etc/pam.d/sshd for ssh, /etc/pam.d/su for su, or /etc/pam.d/login for local logins and telnet if you don't want to enable it for all login methods. Here are the two session entries I have in my /etc/pam.d/system-auth file:<br />
session required /lib/security/$ISA/pam_limits.so<br />
session required /lib/security/$ISA/pam_unix.so<br />
Now login to the oracle user account since the changes will become effective for new login sessions only. Note the ulimit options are different for other shells.<br />
$ su - oracle<br />
$ ulimit -n<br />
4096<br />
$<br />
The default limit for oracle is now 4096 and the oracle user can increase the number of file handles up to 63536:<br />
$ su - oracle<br />
$ ulimit -n<br />
4096<br />
$ ulimit -n 63536<br />
$ ulimit -n<br />
63536<br />
$<br />
To make this change permanent, you could add "ulimit -n 63536" (for bash) to the ~oracle/.bash_profile file which is the user startup file for the bash shell on Red Hat Linux (to verify your shell execute echo $SHELL). To do this you could simply copy/paste the following commands for oracle's bash shell:<br />
su - oracle<br />
cat >> ~oracle/.bash_profile << EOF
ulimit -n 63536
EOF
To make the above changes permanent, you could also set the soft limit equal to the hard limit in /etc/security/limits.conf which I prefer:
oracle soft nofile 63536
oracle hard nofile 63536
Limiting Maximum Number of Processes for the Oracle User
After reading the procedure at Limiting Maximum Number of Open File Descriptors for the Oracle User you should now have an understanding of "soft" and "hard" limits and how to change shell limits.
To see the current limit of the maximum number of processes for the oracle user, run:
$ su - oracle
$ ulimit -u
Note the ulimit options are different for other shells.
To change the "soft" and "hard" limits for the maximum number of processes for the oracle user, add the following lines to the /etc/security/limits.conf file:
oracle soft nproc 2047
oracle hard nproc 16384
To make this change permanent, you could add "ulimit -u 16384" (for bash) to the ~oracle/.bash_profile file which is the user startup file for the bash shell on Red Hat Linux (to verify your shell execute echo $SHELL). To do this you could simply copy/paste the following commands for oracle's bash shell:
su - oracle
cat >> ~oracle/.bash_profile << EOF
ulimit -u 16384
EOF
To make the above changes permanent, you could also set the soft limit equal to the hard limit in /etc/security/limits.conf which I prefer:
oracle soft nproc 16384
oracle hard nproc 16384
Enabling Asynchronous I/O Support
Asynchronous I/O permits Oracle to continue processing after issuing I/Os requests which leads to higher I/O performance. RHEL also allows Oracle to issue multiple simultaneous I/O requests with a single system call. This reduces context switch overhead and allows the kernel to optimize disk activity.
To enable asynchronous I/O in Oracle Database, it is necessary to relink Oracle 9i and 10g Release 1. Note that 10g Release 2 is shipped with asynchronous I/O support enabled and does not need to be relinked. But you may have to apply a patch, see below.
Relinking Oracle9i R2 to Enable Asynchronous I/O Support
Note for Oracle 9iR2 on RHEL 3/4 the 9.2.0.4 patchset or higher needs to be installed together with another patch for async I/O, see Metalink Note:279069.1.
To relink Oracle9i R2 for async I/O, execute the following commands:
# shutdown Oracle
SQL> shutdown<br />
<br />
su - oracle<br />
$ cd $ORACLE_HOME/rdbms/lib<br />
$ make -f ins_rdbms.mk async_on<br />
$ make -f ins_rdbms.mk ioracle<br />
<br />
# The last step creates a new "oracle" executable "$ORACLE_HOME/bin/oracle".<br />
# It backs up the old oracle executable to $ORACLE_HOME/bin/oracleO,<br />
# it sets the correct privileges for the new Oracle executable "oracle",<br />
# and moves the new executable "oracle" into the $ORACLE_HOME/bin directory.<br />
<br />
If asynchronous I/O needs to be disabled, execute the following commands:<br />
# shutdown Oracle<br />
SQL> shutdown<br />
<br />
su - oracle<br />
$ cd $ORACLE_HOME/rdbms/lib<br />
$ make -f ins_rdbms.mk async_off<br />
$ make -f ins_rdbms.mk ioracle<br />
<br />
Relinking Oracle 10g to Enable Asynchronous I/O Support <br />
<br />
Ensure that for 10g Release 1 and 2 the libaio and libaio-devel RPMs are installed on the system:<br />
# rpm -q libaio libaio-devel<br />
libaio-0.3.96-5<br />
libaio-devel-0.3.96-5<br />
If you relink Oracle for async I/O without installing the libaio RPM, then you will get an error message similar to this one:<br />
SQL> connect / as sysdba<br />
oracleorcl: error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory<br />
ERROR:<br />
ORA-12547: TNS:lost contact<br />
The libaio RPMs provide a Linux-native asynch I/O API which is a kernel-accelerated asynch I/O for the POSIX async I/O facility. <br />
<br />
Note that 10g Release 2 is shipped with asynchronous I/O support enabled. This means that 10g Release 2 does not need to be relinked. However, there's a bug in Oracle 10.1.0.2 that causes async I/O not to be installed correctly which can result in poor DB performance, see Bug:3438751 and Note:270213.1. <br />
<br />
To relink Oracle 10g R1 for async I/O, execute the following commands:<br />
# shutdown Oracle<br />
SQL> shutdown<br />
<br />
su - oracle<br />
$ cd $ORACLE_HOME/rdbms/lib<br />
$ make PL_ORALIBS=-laio -f ins_rdbms.mk async_on<br />
<br />
If asynchronous I/O needs to be disabled, run the following commands:<br />
# shutdown Oracle<br />
SQL> shutdown<br />
<br />
su - oracle<br />
$ cd $ORACLE_HOME/rdbms/lib<br />
$ make -f ins_rdbms.mk async_off<br />
<br />
Enabling Asynchronous I/O in Oracle 9i and 10g <br />
<br />
To enable async I/O in Oracle, the disk_asynch_io parameter needs to be set to true:<br />
disk_asynch_io=true<br />
Note this parameter is set to true by default in Oracle 9i and 10g:<br />
SQL> show parameter disk_asynch_io;<br />
<br />
NAME TYPE VALUE<br />
------------------------------------ ----------- ------------------------------<br />
disk_asynch_io boolean TRUE<br />
SQL><br />
<br />
If you use filesystems instead of raw devices, block devices (available in 10gR2) or ASM for datafiles, then you need to ensure that the datafiles reside on filesystems that support asynchronous I/O (e.g., OCFS/OCFS2, ext2, ext3). To do async I/O on filesystems the filesystemio_options parameter needs to be set to "asynch" in addition to disk_asynch_io=true:<br />
filesystemio_options=asynch<br />
This parameter is platform-specific. By default, this parameter is set to none for Linux and thus needs to be changed:<br />
SQL> show parameter filesystemio_options;<br />
<br />
NAME TYPE VALUE<br />
------------------------------------ ----------- ------------------------------<br />
filesystemio_options string none<br />
SQL><br />
The filesystemio_options can have the following values with Oracle9iR2:<br />
asynch: This value enables asynchronous I/O on file system files.<br />
directio: This value enables direct I/O on file system files.<br />
setall: This value enables both asynchronous and direct I/O on file system files.<br />
none: This value disables both asynchronous and direct I/O on file system files.<br />
<br />
If you also want to enable Direct I/O Support which is available in RHEL 3/4, set filesystemio_options to "setall". <br />
<br />
Ensure that the datafiles reside on filesystems that support asynchronous I/O (e.g., OCFS, ext2, ext3). <br />
<br />
<br />
Tuning Asynchronous I/O for Oracle 9i and 10g <br />
<br />
For RHEL 3 it is recommended to set aio-max-size to 1048576 since Oracle uses I/Os of up to 1MB. It controls the maximum I/O size for asynchronous I/Os. Note this tuning parameter is not applicable to 2.6 kernel, i.e RHEL 4. <br />
<br />
To determine the maximum I/O size in bytes, execute:<br />
$ cat /proc/sys/fs/aio-max-size<br />
131072<br />
To change the maximum number of bytes without reboot:<br />
# echo 1048576 > /proc/sys/fs/aio-max-size<br />
Alternatively, you can use sysctl(8) to change it:<br />
# sysctl -w fs.aio-max-size=1048576<br />
To make the change permanent, add the following line to the /etc/sysctl.conf file. This file is used during the boot process:<br />
$ echo "fs.aio-max-size=1048576" >> /etc/sysctl.conf<br />
<br />
Checking Asynchronous I/O Usage <br />
<br />
To verify whether $ORACLE_HOME/bin/oracle was linked with async I/O, you can use the Linux commands ldd and nm. <br />
<br />
In the following example, $ORACLE_HOME/bin/oracle was relinked with async I/O:<br />
$ ldd $ORACLE_HOME/bin/oracle | grep libaio<br />
libaio.so.1 => /usr/lib/libaio.so.1 (0x0093d000)<br />
$ nm $ORACLE_HOME/bin/oracle | grep io_getevent<br />
w io_getevents@@LIBAIO_0.1<br />
$<br />
In the following example, $ORACLE_HOME/bin/oracle has NOT been relinked with async I/O:<br />
$ ldd $ORACLE_HOME/bin/oracle | grep libaio<br />
$ nm $ORACLE_HOME/bin/oracle | grep io_getevent<br />
w io_getevents<br />
$<br />
<br />
If $ORACLE_HOME/bin/oracle is relinked with async I/O it does not necessarily mean that Oracle is really using it. You also have to ensure that Oracle is configured to use async I/O calls, see Enabling Asynchronous I/O in Oracle 9i and 10g. <br />
<br />
To verify whether Oracle is making async I/O calls, you can take a look at the /proc/slabinfo file assuming there are no other applications performing async I/O calls on the system. This file shows kernel slab cache information in real time. <br />
<br />
On a RHEL 3 system where Oracle does NOT make async I/O calls, the output looks like this:<br />
$ egrep "kioctx|kiocb" /proc/slabinfo<br />
kioctx 0 0 128 0 0 1 : 1008 252<br />
kiocb 0 0 128 0 0 1 : 1008 252<br />
$<br />
Once Oracle makes async I/O calls, the output on a RHEL 3 system will look like this:<br />
$ egrep "kioctx|kiocb" /proc/slabinfo<br />
kioctx 690 690 128 23 23 1 : 1008 252<br />
kiocb 58446 65160 128 1971 2172 1 : 1008 252<br />
$<br />
The numbers in red (number of active objects) show whether Oracle makes async I/O calls. The output will look a little bit different in RHEL 4. However, the numbers in red will show same behavior in RHEL 3 and RHEL 4. The first column displays the cache names kioctx and kiocb. The second column shows the number of active objects currently in use. And the third column shows how many objects are available in total, used and unused. <br />
<br />
To see kernel slab cache information in real time, you can also use the slabtop command:<br />
$ slabtop<br />
Active / Total Objects (% used) : 293568 / 567030 (51.8%)<br />
Active / Total Slabs (% used) : 36283 / 36283 (100.0%)<br />
Active / Total Caches (% used) : 88 / 125 (70.4%)<br />
Active / Total Size (% used) : 81285.56K / 132176.36K (61.5%)<br />
Minimum / Average / Maximum Object : 0.01K / 0.23K / 128.00K<br />
<br />
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME<br />
178684 78396 43% 0.12K 5764 31 23056K size-128<br />
127632 36292 28% 0.16K 5318 24 21272K dentry_cache<br />
102815 74009 71% 0.69K 20563 5 82252K ext3_inode_cache<br />
71775 32434 45% 0.05K 957 75 3828K buffer_head<br />
19460 15050 77% 0.27K 1390 14 5560K radix_tree_node<br />
13090 13015 99% 0.03K 110 119 440K avtab_node<br />
12495 11956 95% 0.03K 105 119 420K size-32<br />
...<br />
Slab caches are a special memory pool in the kernel for adding and removing objects (e.g. data structures or data buffers) of the same size. Its a cache for commonly used objects where the kernel doesn't have to re-allocate and initialize the object each time it's being reused, and free the object each time it's being destroyed. The slab allocater scheme basically prevents memory fragmentation and it prevents the kernel from spending too much time allocating, initializing, and freeing the same objects. <br />
<br />
Configuring I/O for Raw Partitions<br />
<br />
General <br />
<br />
Raw devices allow Oracle to bypass the OS cache. A raw device can be assigned or bound to block devices such as whole disks or disk partitions. When a raw device is bound to a disk or partition, any reads or writes to the raw device will cause the disk subsystem to perform raw I/Os with the disk. A raw I/O through the /dev/raw interface bypasses the kernel's block buffer cache which is normally utilized for block device reads/writes. By bypassing the cache the physical device is accessed directly which allows applications such as Oracle databases to have more control over the I/O. In fact, Oracle does it's own data caching and raw devices allow Oracle to ensure that data gets written to the disk immediately without OS caching. <br />
<br />
Since Automatic Storage Management (ASM) is the recommended option for large amounts of storage in RAC environments, the focus of this article and section is on the usage of raw devices and block devices for ASM. ASM offers many advantages over conventional filesystems. The ASM filesystem is not buffered and supports async I/O. It allows you to group sets of physical disks to logical entities as diskgroups. You can add or remove disks without downtime. In fact, you could move a whole database from one SAN storage to another SAN without downtime. Also, ASM spreads I/O over all the available disks automatically to avoid hot spots. ASM does also it's own striping and offers mirroring. ASM can be setup using the ASM library driver or raw devices. Starting with 10g R2, neither is necessarily required, see next note. <br />
<br />
NOTE:<br />
<br />
Since raw I/O is now being deprecated by the Linux community and RHEL 4, Oracle 10g R2 no longer requires raw devices for the database. Oracle 10g R2 automatically opens all block devices such as SCSI disks using the O_DIRECT flag, thus bypasses the OS cache. But for older Oracle Database and RHEL versions raw devices are still a recommended option for ASM and datafiles. For more information on using block devices, see Using Block Devices for Oracle 10g Release 2 in RHEL 4. Unfortunately, Oracle Clusterware R2 OUI still requires raw devices or a Cluster File System. <br />
<br />
CAUTION:<br />
<br />
The name of the devices are assigned by Linux and is determined by the scan order of the bus. Therefore, the device names are not guaranteed to persist across reboots. For example, SCSI device /dev/sdbcan change to /dev/sda if the scan order of the controllers is not configured. To force the scan order of the controllers, aliases can be set in /etc/modprobe.conf. For example:<br />
alias scsi_hostadapter1 aic7xxx<br />
alias scsi_hostadapter2 lpfc<br />
These settings will guarantee that the Adaptec adapter for local storage is used first and then the Emulex adapter(s) for SAN storage. Fortunately, RHEL 4 has already addressed this issue by delaying the loading of lpfc (Emulex) and various qla (QLogic) drivers until after all other SCSI devices have been loaded. This means that the alias settings in this example would not be required in RHEL 4. For more information, see Red Hat Enterprise Linux AS 4 Release Notes. <br />
<br />
Be also careful when adding/removing devices which can change device names on the system. Starting Oracle with incorrect device names or raw devices can cause damages to the database. For stable device naming in Linux 2.4 and 2.6, see Optimizing Linux I/O. <br />
<br />
<br />
Basics of Raw Devices <br />
<br />
To bind the first raw device /dev/raw/raw1 to the /dev/sdz SCSI disk or LUN you can execute the following command:<br />
# raw /dev/raw/raw1 /dev/sdz<br />
Now when you run the dd command on /dev/raw/raw1, it will write directly to /dev/sdz bypassing the OS block buffer cache:<br />
(Warning: the following command will overwrite data on /dev/sdz)<br />
# dd if=/dev/zero of=/dev/sdz count=1<br />
To permanently bind /dev/raw/raw1 to /dev/sdz, add an entry to the /etc/sysconfig/rawdevices file:<br />
/dev/raw/raw1 /dev/sdz<br />
Now when you run /etc/init.d/rawdevices it will read the /etc/sysconfig/rawdevices file and execute the raw command for each entry:<br />
/etc/init.d/rawdevices start<br />
To have /etc/init.d/rawdevices run each time the system boot, it can be activated by executing the following command:<br />
chkconfig rawdevices on<br />
<br />
Note for each block device you need to use another raw device. To bind the third raw device to the second partition of /dev/sdz, the entry in /etc/sysconfig/rawdevices would look like this:<br />
/dev/raw/raw3 /dev/sdz2<br />
Or to bind the 100th raw device to /dev/sdz, the entry in /etc/sysconfig/rawdevices would look like this:<br />
/dev/raw/raw100 /dev/sdz<br />
<br />
Using Raw Devices for Oracle Databases <br />
<br />
Many guides and documentations show instructions on using the devices in /dev/raw/ for configuring raw devices for datafiles. I do not recommend to use the raw devices in /dev/raw/ for the following reason: When you configure raw devices for Oracle datafiles, you also have to change ownership and permissions of the devices in /dev/raw/ to allow Oracle to read and write to these raw devices. But all device names in /dev/raw/ are owned by the dev RPM. So when the Linux systems administrator upgrades the dev RPM, which may happen as part of an OS update, then all device names in /dev/raw/will automatically be recreated. This means that ownership and permissions must be set each time the dev RPM gets upgraded. Therefore I recommend to create all raw devices for Oracle datafiles in an Oracle data directory such as /u02. <br />
<br />
For example, to create a new raw device for the system datafile system01.dbf in /u02/orcl/, execute the following command:<br />
# mknod /u02/orcl/system01.dbf c 162 1<br />
This command creates a new raw device called /u02/orcl/system01.dbf with minor number 1, which is equivalent to the first raw device /dev/raw/raw1. The major number 162 designates the device as a raw device. A major number always identifies the driver associated with the device. <br />
<br />
To grant oracle:dba read and write permissions, execute:<br />
# chown oracle.dba /u02/orcl/system01.dbf<br />
# chown 660 /u02/orcl/system01.dbf<br />
To bind this new raw device to the first partition of /dev/sdb, add the following line to the /etc/sysconfig/rawdevices file:<br />
/u02/orcl/system01.dbf /dev/sdb1<br />
To activate the raw device, execute:<br />
/etc/init.d/rawdevices start<br />
<br />
Here is an example for creating raw devices for ASM:<br />
# mknod /u02/oradata/asmdisks/disk01 c 162 1<br />
# mknod /u02/oradata/asmdisks/disk02 c 162 2<br />
# mknod /u02/oradata/asmdisks/disk03 c 162 3<br />
# mknod /u02/oradata/asmdisks/disk03 c 162 4<br />
<br />
# chown oracle.dba /u02/oradata/asmdisks/disk01<br />
# chown oracle.dba /u02/oradata/asmdisks/disk02<br />
# chown oracle.dba /u02/oradata/asmdisks/disk03<br />
# chown oracle.dba /u02/oradata/asmdisks/disk04<br />
<br />
# chmod 660 /u02/oradata/asmdisks/disk01<br />
# chmod 660 /u02/oradata/asmdisks/disk02<br />
# chmod 660 /u02/oradata/asmdisks/disk03<br />
# chmod 660 /u02/oradata/asmdisks/disk04<br />
And the /etc/sysconfig/rawdevices file would look something like this if you use EMC PowerPath:<br />
/u02/oradata/asmdisks/disk01 /dev/emcpowera<br />
/u02/oradata/asmdisks/disk02 /dev/emcpowerb<br />
/u02/oradata/asmdisks/disk03 /dev/emcpowerc<br />
/u02/oradata/asmdisks/disk04 /dev/emcpowerd<br />
<br />
In this example, 4 raw devices have been created using minor numbers 1 through 4. This means that the devices /dev/raw/raw1../dev/raw/raw4 should not be used by any application on the system. But this should not be an issue since all raw devices should be configured in one place, which is the /etc/sysconfig/rawdevices file. Note that you could also partition the LUNs or disks and configure a raw device for each disk partition. <br />
<br />
<br />
Using Block Devices for Oracle 10g Release 2 in RHEL 4 <br />
<br />
For Oracle 10g Release 2 in RHEL 4 it is not recommended to use raw devices but to use block devices instead. Raw I/O is still available in RHEL 4, but it is now a deprecated interface. In fact, raw I/O has been deprecated by the Linux community. It has been replaced by the O_DIRECT flag, which can be used for opening block devices to bypass the OS cache. Unfortunately, Oracle Clusterware R2 OUI has not been updated and still requires raw devices or a Cluster File System. There is also another bug, see bug number 5021707 at http://www.oracle.com/technology/tech/linux/validated-configurations/html/vc_dell6850-rhel4-cx500-1_1.html. <br />
<br />
By default, reading and writing to block devices are buffered I/Os. Oracle 10g R2 now automatically opens all block devices such as SCSI disks using the O_DIRECT flag, thus bypassing the OS cache. For example, when you create disk groups for ASM and you want to use the SCSI block devices /dev/sdb and /dev/sdc, you can simply set the Disk Discovery Path to "/dev/sdb, /dev/sdc" to create the ASM disk group. There is no need to create raw devices and to point the Disk Discovery Path to it. <br />
<br />
Using the ASM example from Using Raw Devices for Oracle Databases, the Oracle data directory could be setup the following way:<br />
$ ln -s /dev/emcpowera /u02/oradata/asmdisks/disk01<br />
$ ln -s /dev/emcpowerb /u02/oradata/asmdisks/disk02<br />
$ ln -s /dev/emcpowerc /u02/oradata/asmdisks/disk03<br />
$ ln -s /dev/emcpowerd /u02/oradata/asmdisks/disk04<br />
And the following command needs to be executed after each reboot:<br />
# chown oracle.dba /u02/oradata/asmdisks/*<br />
You need to ensure that the ownership of block devices is changed to oracle:dba or oracle:oinstall. Otherwise Oracle can't access the block devices and ASM disk discovery won't list them. You also need to ensure that the ownership of block devices is set after each reboot since Linux changes the ownership of block devices back to "brw-rw---- 1 root disk" at boot time. <br />
<br />
Large Memory Optimization (Big Pages, Huge Pages)<br />
Big Pages in RHEL2.1 and Huge Pages in RHEL 3/4 are very useful for large Oracle SGA sizes and in general for systems with large amount of physical memory. It optimizes the use of Translation Lookaside Buffers (TLB), locks these larger pages in RAM, and the system has less bookkeeping work to do for that part of virtual memory due to larger page sizes. This is a useful feature that should be used on x86 and x86-64 platforms. The default page size in Linux for x86 is 4KB. <br />
<br />
Physical memory is partitioned into pages which are the basic unit of memory management. When a Linux process accesses a virtual address, the CPU must translate it into a physical address. Therefore, for each Linux process the kernel maintains a page table which is used by the CPU to translate virtual addresses into physical addresses. But before the CPU can do the translation it has to perform several physical memory reads to retrieve page table information. To speed up this translation process for future references to the same virtual address, the CPU saves information for recently accessed virtual addresses in its Translation Lookaside Buffers (TLB) which is a small but very fast cache in the CPU. The use of this cache makes virtual memory access very fast. Since TLB misses are expensive, TLB hits can be improved by mapping large contiguous physical memory regions by a small number of pages. So fewer TLB entries are required to cover larger virtual address ranges. A reduced page table size also means a reduction in memory management overhead. To use larger page sizes for shared memory, Big Pages (RHEL 2.1) or Huge Pages (RHEL 3/4) must be enabled which also locks these pages in physical memory. <br />
<br />
<br />
Big Pages in RHEL 2.1 and Huge Pages in RHEL 3/4<br />
<br />
In RHEL 2.1 large memory pages can be configured using the Big Pages (bigpages) feature. In RHEL 3/4 Red Hat replaced Big Pages with a feature called Huge Pages (hugetlb) which behaves a little bit different. The Huge Pages feature in RHEL 3/4 allows you to dynamically allocate large memory pages without a reboot. Allocating and changing Big Pages in RHEL 2.1 always required a reboot. However, if memory gets too fragmented in RHEL 3/4 allocation of physically contiguous memory pages can fail and a reboot may become necessary. <br />
<br />
The advantages of Big Pages and Huge Pages are:<br />
· Increased performance by through increased TLB hits<br />
· Pages are locked in memory and are never swapped out which guarantees that shared memory like SGA remains in RAM<br />
· Contiguous pages are preallocated and cannot be used for anything else but for System V shared memory (e.g. SGA)<br />
· Less bookkeeping work for the kernel for that part of virtual memory due to larger page sizes<br />
<br />
Usage of Big Pages and Huge Pages in Oracle 9i and 10g<br />
<br />
Big pages are supported implicitly in RHEL 2.1. But Huge Pages in RHEL 3/4 need to be requested explicitly by the application by using the SHM_HUGETLB flag when invoking the shmget() system call. This ensures that shared memory segments are allocated out of the Huge Pages pool. This is done automatically in Oracle 10g and 9i R2 (9.2.0.6) but earlier Oracle 9i R2 versions require a patch, see Metalink Note:262004.1. <br />
<br />
<br />
Sizing Big Pages and Huge Pages<br />
<br />
With the Big Pages and Huge Pages feature you specify how many physically contiguous large memory pages should be allocated and pinned in RAM for shared memory like Oracle SGA. For example, if you have three Oracle instances running on a single system with 2 GB SGA each, then at least 6 GB of large pages should be allocated. This will ensure that all three SGAs use large pages and remain in main physical memory. Furthermore, if you use ASM on the same system, then I recommend to add an additional 200MB. I've seen ASM instances creating between 70 MB and 150 MB shared memory segments. And there might be other non-Oracle processes that allocate shared memory segments as well. <br />
<br />
It is, however, not recommended to allocate too many Big or Huge Pages. These preallocated pages can only be used for shared memory. This means that unused Big or Huge Pages won't be available for other use than for shared memory allocations even if the system runs out of memory and starts swapping. Also take note that Huge Pages are not used for the ramfs shared memory filesystem, see Huge Pages and Shared Memory Filesystem in RHEL 3/4, but Big Pages can be used for the shm filesystem in RHEL 2.1. <br />
<br />
<br />
Checking Shared Memory Before Starting Oracle Databases<br />
<br />
It is very important to always check the shared memory segments before starting an instance. If an abandoned shared memory segment from e.g. an instance crash is not removed, it will remain allocated in the Big Pages or Huge Pages pool. This could mean that new allocated shared memory segments for the new instance SGA won't fit into the Big Pages or Huge Pages pool. For more information on removing shared memory, see Removing Shared Memory. <br />
<br />
<br />
Configuring Big Pages in RHEL 2.1<br />
<br />
Before configuring Big Pages, ensure to have read Sizing Big Pages and Huge Pages. <br />
<br />
Note that Big Pages in x86 RHEL 2.1 can only be allocated and pinned above (approx) 860MB of physical RAM which is known as Highmem or high memory region in x86. Thus, Big Pages cannot be larger than Highmem. The total amount of memory in the high region can be obtained by reading the memory statistic HighTotal from the /proc/meminfo file:<br />
$ grep "HighTotal" /proc/meminfo<br />
HighTotal: 9043840 kB <br />
$<br />
<br />
The Big Pages feature can be enabled with the following command:<br />
# echo "1" > /proc/sys/kernel/shm-use-bigpages<br />
Alternatively, you can use sysctl(8) to change it:<br />
# sysctl -w kernel.shm-use-bigpages=1<br />
To make the change permanent, add the following line to the file /etc/sysctl.conf. This file is used during the boot process.<br />
echo "kernel.shm-use-bigpages=1" >> /etc/sysctl.conf<br />
Setting kernel.shm-use-bigpages to 2 enables the Big Pages feature for the shmfs shared memory filesystem. Setting kernel.shm-use-bigpages to 0 disables the Big Pages feature. <br />
<br />
<br />
In RHEL 2.1 the size of the Big Pages pool is configured by adding a parameter to the kernel boot command. For example, if you use GRUB and you want to set the Big Pages pool to 1000 MB, edit the/etc/grub.conf file and add the "bigpages" parameter as follows:<br />
default=0<br />
timeout=10<br />
title Red Hat Linux Advanced Server (2.4.9-e.40enterprise)<br />
root (hd0,0)<br />
kernel /vmlinuz-2.4.9-e.40enterprise ro root=/dev/sda2 bigpages=1000MB<br />
initrd /initrd-2.4.9-e.40enterprise.img<br />
title Red Hat Linux Advanced Server (2.4.9-e.40smp)<br />
root (hd0,0)<br />
kernel /vmlinuz-2.4.9-e.40smp ro root=/dev/sda2<br />
initrd /initrd-2.4.9-e.40smp.img<br />
After this change the system must be rebooted:<br />
# shutdown -r now<br />
After a system reboot the 1000 MB Big Pages pool should show up under BigPagesFree in /proc/meminfo.<br />
grep BigPagesFree /proc/meminfo<br />
<br />
Note that if HighTotal in /proc/meminfo is 0 KB, then BigPagesFree will always be 0 KB as well since Big Pages can only be allocated and pinned above (approx) 860MB of physical RAM. <br />
<br />
<br />
Configuring Huge Pages in RHEL 3<br />
<br />
Before configuring Huge Pages, ensure to have read Sizing Big Pages and Huge Pages. <br />
<br />
In RHEL 3 the desired size of the Huge Pages pool is specified in megabytes. The size of the pool should be configured by the incremental size of the Huge Page size. To obtain the size of Huge Pages, execute the following command:<br />
$ grep Hugepagesize /proc/meminfo<br />
Hugepagesize: 2048 kB<br />
$<br />
The number of Huge Pages can be configured and activated by setting hugetlb_pool in the proc filesystem. For example, to allocate a 1GB Huge Page pool, execute:<br />
# echo 1024 > /proc/sys/vm/hugetlb_pool<br />
Alternatively, you can use sysctl(8) to change it:<br />
# sysctl -w vm.hugetlb_pool=1024<br />
To make the change permanent, add the following line to the file /etc/sysctl.conf. This file is used during the boot process. The Huge Pages pool is usually guaranteed if requested at boot time:<br />
# echo "vm.hugetlb_pool=1024" >> /etc/sysctl.conf<br />
<br />
If you allocate a large number of Huge Pages, the execution of the above commands can take a while. To verify whether the kernel was able to allocate the requested number of Huge Pages, execute:<br />
$ grep HugePages_Total /proc/meminfo<br />
HugePages_Total: 512 <br />
$<br />
The output shows that 512 Huge Pages have been allocated. Since the size of Huge Pages on my system is 2048 KB, a Huge Page pool of 1GB has been allocated and pinned in physical memory. <br />
<br />
If HugePages_Total is lower than what was requested with hugetlb_pool, then the system does either not have enough memory or there are not enough physically contiguous free pages. In the latter case the system needs to be rebooted which should give you a better chance of getting the memory. <br />
<br />
To get the number of free Huge Pages on the system, execute:<br />
$ grep HugePages_Free /proc/meminfo<br />
Free system memory will automatically be decreased by the size of the Huge Pages pool allocation regardless whether the pool is being used by an application like Oracle DB or not:<br />
$ grep MemFree /proc/meminfo<br />
After an Oracle DB startup you can verify the usage of Huge Pages by checking whether the number of free Huge Pages has decreased:<br />
$ grep HugePages_Free /proc/meminfo<br />
<br />
To free the Huge Pages pool, you can execute:<br />
# echo 0 > /proc/sys/vm/hugetlb_pool<br />
This command usually takes a while to finish. <br />
<br />
<br />
Configuring Huge Pages in RHEL 4<br />
<br />
Before configuring Huge Pages, ensure to have read Sizing Big Pages and Huge Pages. <br />
<br />
In RHEL 4 the size of the Huge Pages pool is specified by the desired number of Huge Pages. To calculate the number of Huge Pages you first need to know the Huge Page size. To obtain the size of Huge Pages, execute the following command:<br />
$ grep Hugepagesize /proc/meminfo<br />
Hugepagesize: 2048 kB<br />
$<br />
The output shows that the size of a Huge Page on my system is 2MB. This means if I want to allocate a 1GB Huge Pages pool, then I have to allocate 512 Huge Pages. The number of Huge Pages can be configured and activated by setting nr_hugepages in the proc filesystem. For example, to allocate 512 Huge Pages, execute:<br />
# echo 512 > /proc/sys/vm/nr_hugepages<br />
Alternatively, you can use sysctl(8) to change it:<br />
# sysctl -w vm.nr_hugepages=512<br />
To make the change permanent, add the following line to the file /etc/sysctl.conf. This file is used during the boot process. The Huge Pages pool is usually guaranteed if requested at boot time:<br />
# echo "vm.nr_hugepages=512" >> /etc/sysctl.conf<br />
<br />
If you allocate a large number of Huge Pages, the execution of the above commands can take a while. To verify whether the kernel was able to allocate the requested number of Huge Pages, run:<br />
$ grep HugePages_Total /proc/meminfo<br />
HugePages_Total: 512<br />
$<br />
The output shows that 512 Huge Pages have been allocated. Since the size of Huge Pages is 2048 KB, a Huge Page pool of 1GB has been allocated and pinned in physical memory. <br />
<br />
If HugePages_Total is lower than what was requested with nr_hugepages, then the system does either not have enough memory or there are not enough physically contiguous free pages. In the latter case the system needs to be rebooted which should give you a better chance of getting the memory. <br />
<br />
To get the number of free Huge Pages on the system, execute:<br />
$ grep HugePages_Free /proc/meminfo<br />
Free system memory will automatically be decreased by the size of the Huge Pages pool allocation regardless whether the pool is being used by an application like Oracle DB or not:<br />
$ grep MemFree /proc/meminfo<br />
<br />
NOTE: In order that an Oracle database can use Huge Pages in RHEL 4, you also need to increase the ulimit parameter "memlock" for the oracle user in /etc/security/limits.conf if "max locked memory" is not unlimited or too small, see ulimit -a or ulimit -l. For example:<br />
oracle soft memlock 1048576<br />
oracle hard memlock 1048576<br />
The memlock parameter specifies how much memory the oracle user can lock into its address space. Note that Huge Pages are locked in physical memory. The memlock setting is specified in KB and must match the memory size of the number of Huge Pages that Oracle should be able to allocate. So if the Oracle database should be able to use 512 Huge Pages, then memlock must be set to at least 512 * Hugepagesize, which is on my system 1048576 KB (512*1024*2). If memlock is too small, then no single Huge Page will be allocated when the Oracle database starts. For more information on setting shell limits, see Setting Shell Limits for the Oracle User. <br />
<br />
Now login as the oracle user again and verify the new memlock setting by executing ulimit -l before starting the database. <br />
<br />
After an Oracle DB startup you can verify the usage of Huge Pages by checking whether the number of free Huge Pages has decreased:<br />
$ grep HugePages_Free /proc/meminfo<br />
<br />
To free the Huge Pages pool, you can execute:<br />
# echo 0 > /proc/sys/vm/nr_hugepages<br />
This command usually takes a while to finish. <br />
<br />
<br />
Huge Pages and Shared Memory Filesystem in RHEL 3/4<br />
<br />
In the following example I will show that the Huge Pages pool is not being used by the ramfs shared memory filesystems. The ramfs shared memory filesystems can be used for Configuring Very Large Memory (VLM). <br />
<br />
The ipcs command shows only System V shared memory segments. It does not display shared memory of a shared memory filesystems. The following command shows System V shared memory segments on a node running a database with an SGA of 2.6 GB:<br />
# ipcs -m<br />
<br />
------ Shared Memory Segments -------- <br />
key shmid owner perms bytes nattch status <br />
0x98ab8248 1081344 oracle 600 77594624 0 <br />
0xe2e331e4 1245185 oracle 600 2736783360 0 <br />
The first shared memory segment of 74 MB was created by the ASM instance. The second shared memory segment of 2.6 GB was created by the database instance. <br />
<br />
On this database system the size of the database buffer cache is 2 GB:<br />
db_block_buffers = 262144 <br />
db_block_size = 8192<br />
The following command shows that Oracle allocated a shared memory file of 2GB (262144*8192=2147483648) for the buffer cache on the ramfs shared memory filesystem:<br />
# mount | grep ramfs<br />
ramfs on /dev/shm type ramfs (rw) <br />
# ls -al /dev/shm<br />
total 204 <br />
drwxr-xr-x 1 oracle dba 0 Oct 30 16:00 . <br />
drwxr-xr-x 22 root root 204800 Oct 30 16:00 .. <br />
-rw-r----- 1 oracle dba 2147483648 Nov 1 16:46 ora_orcl1_1277954 <br />
<br />
The next command shows how many Huge Pages are currently being used on this system:<br />
$ grep Huge /proc/meminfo<br />
HugePages_Total: 1536 <br />
HugePages_Free: 194 <br />
Hugepagesize: 2048 kB <br />
$<br />
The output shows that 1342 (1536-194) Huge Pages are being used. This translates into 2814377984 (1342*2048*1024) bytes being allocated in the Huge Pages pool. This number matches the size of both shared memory segments (2736783360+77594624=2814377984) displayed by the ipcs command above. <br />
<br />
This shows that the Huge Pages pool is not being used for the ramfs shared memory filesystem. Hence, you do not need to increase the Huge Pages pool if you use the ramfs shared memory filesystem. <br />
<br />
Growing the Oracle SGA to 2.7 GB in x86 RHEL 2.1 Without VLM<br />
General <br />
<br />
Due to 32-bit virtual address limitations workarounds have been implemented in Linux to increase the maximum size for shared memories. The workaround is to lower the Mapped Base Address (mapped_base) for shared libraries and the SGA Attach Address for shared memory segments. Lowering the Mapped Base Address and the SGA Attach Address allows SGA sizes up to 2.7 GB. By default, the shared memory segment size can only be increased to roughly 1.7 GB in RHEL 2.1. <br />
<br />
To better understand the process of lowering the Mapped Base Address for shared libraries and the SGA Attach Address for shared memory segments, a basic understanding of the Linux memory layout is necessary. <br />
<br />
<br />
Linux Memory Layout <br />
<br />
The 4 GB address space in 32-bit x86 Linux is usually split into different sections for every process on the system:<br />
0GB-1GB User space - Used for text/code and brk/sbrk allocations (malloc uses brk for small chunks)<br />
1GB-3GB User space - Used for shared libraries, shared memory, and stack; shared memory and malloc use mmap (malloc uses mmap for large chunks)<br />
3GB-4GB Kernel Space - Used for the kernel itself<br />
In older Linux systems the split between brk(2) and mmap(2) was changed by setting the kernel parameter TASK_UNMAPPED_BASE and by recompiling the kernel. However, on all RHEL systems this parameter can be changed dynamically as will be shown later. <br />
The mmaps grow bottom up from 1GB and the stack grows top down from around 3GB. <br />
The split between userspace and kernelspace is set by the kernel parameter PAGE_OFFSET which is usually 0xc0000000 (3GB). <br />
<br />
By default, in RHEL 2.1 the address space between 0x40000000 (1 GB) and 0xc0000000 (3 GB) is available for mapping shared libraries and shared memory segments. The default mapped base for loading shared libraries is 0x40000000 (1 GB) and the SGA attach address for shared memory segments is above the shared libraries. In Oracle 9i on RHEL 2.1 the default SGA attach address for shared memory is 0x50000000 (1.25 GB) where the SGA is mapped. This leaves 0.25 GB space for loading shared libraries between 0x40000000 (1 GB) and 0x50000000 (1.25 GB). <br />
<br />
The address mappings of processes can be checked by viewing the proc file /proc/<pid>/maps where pid stands for the process ID. Here is an example of a default address mapping of an Oracle 9i process in RHEL 2.1:<br />
08048000-0ab11000 r-xp 00000000 08:09 273078 /ora/product/9.2.0/bin/oracle<br />
0ab11000-0ab99000 rw-p 02ac8000 08:09 273078 /ora/product/9.2.0/bin/oracle<br />
0ab99000-0ad39000 rwxp 00000000 00:00 0<br />
40000000-40016000 r-xp 00000000 08:01 16 /lib/ld-2.2.4.so<br />
40016000-40017000 rw-p 00015000 08:01 16 /lib/ld-2.2.4.so<br />
40017000-40018000 rw-p 00000000 00:00 0<br />
40018000-40019000 r-xp 00000000 08:09 17935 /ora/product/9.2.0/lib/libodmd9.so<br />
40019000-4001a000 rw-p 00000000 08:09 17935 /ora/product/9.2.0/lib/libodmd9.so<br />
4001a000-4001c000 r-xp 00000000 08:09 16066 /ora/product/9.2.0/lib/libskgxp9.so<br />
...<br />
42606000-42607000 rw-p 00009000 08:01 50 /lib/libnss_files-2.2.4.so<br />
50000000-50400000 rw-s 00000000 00:04 163842 /SYSV00000000 (deleted)<br />
51000000-53000000 rw-s 00000000 00:04 196611 /SYSV00000000 (deleted)<br />
53000000-55000000 rw-s 00000000 00:04 229380 /SYSV00000000 (deleted)<br />
...<br />
bfffb000-c0000000 rwxp ffffc000 00:00 0<br />
<br />
As this address mapping shows, shared libraries start at 0x40000000 (1 GB) and System V shared memory, in this case SGA, starts at 0x50000000 (1.25 GB). Here is a summary of all the entries: <br />
<br />
The text (code) section is mapped at 0x08048000:<br />
08048000-0ab11000 r-xp 00000000 08:09 273078 /ora/product/9.2.0/bin/oracle<br />
The data section is mapped at 0x0ab11000:<br />
0ab11000-0ab99000 rw-p 02ac8000 08:09 273078 /ora/product/9.2.0/bin/oracle<br />
The uninitialized data segment .bss is allocated at 0x0ab99000:<br />
0ab99000-0ad39000 rwxp 00000000 00:00 0<br />
The base address for shared libraries is 0x40000000:<br />
40000000-40016000 r-xp 00000000 08:01 16 /lib/ld-2.2.4.so<br />
The base address for System V shared memory, in this case SGA, is 0x50000000:<br />
50000000-50400000 rw-s 00000000 00:04 163842 /SYSV00000000 (deleted)<br />
The stack is allocated at 0xbfffb000:<br />
bfffb000-c0000000 rwxp ffffc000 00:00 0<br />
<br />
<br />
Increasing Space for the SGA in RHEL 2.1 <br />
<br />
To increase the maximum default size of shared memory for the SGA from 1.7 GB to 2.7GB, the Mapped Base Address (mapped_base) for shared libraries must be lowered from 0x40000000 (1 GB) to 0x10000000 (0.25 GB) and the SGA Attach Address for shared memory segments must be lowered from 0x50000000 (1.25 GB) to 0x15000000 (336 MB). Lowering the SGA attach address increases the available space for shared memory almost 1 GB. If shared memory starts at 0x15000000 (336 MB), then the space between 0x15000000 (336 MB) and 0xc0000000 (3GB) minus stack size becomes available for the SGA. Note the mapped base for shared libraries should not be above the SGA attach address, i.e. between 0x15000000 (336 MB) and 0xc0000000 (3GB). <br />
<br />
To increase the space for shared memory in RHEL 2.1, the mapped base for shared libraries for the Oracle processes must be changed by root. And the oracle user must relink Oracle to relocate or lower the SGA attach address for shared memory segments. <br />
<br />
<br />
Lowering the Mapped Base Address for Shared Libraries in RHEL 2.1 <br />
<br />
The default mapped base address for shared libraries in RHEL 2.1 is 0x40000000 (1 GB). To lower the mapped base for a Linux process, the file /proc/<pid>/mapped_base must be changed where<pid> stands for the process ID. This means that his is not a system wide parameter. In order to change the mapped base for Oracle processes, the address mapping of the parent shell terminal session that spawns Oracle processes (instance) must be changed for the child processes to inherit the new mapping. <br />
<br />
Login as oracle and run the following command to obtain the process ID of the shell where sqlplus will later be executed:<br />
$ echo $$<br />
Login as root in another shell terminal session and change the mapped_base for this process ID to 0x10000000 (decimal 268435456):<br />
# echo 268435456 > /proc/<pid>/mapped_base<br />
Now when Oracle processes are started with sqlplus in this shell, they will inherit the new mapping. But before Oracle can be started, the SGA Attach Address for shared memory must be lowered as well. <br />
<br />
<br />
Lowering the SGA Attach Address for Shared Memory Segments in Oracle 9i <br />
<br />
The default SGA attach address for shared memory segments in Oracle 9i on RHEL 2.1 is 0x50000000 (1.25 GB). To lower the SGA attach address for shared memory, the Oracle utility genksms must be used before the relinking: <br />
<br />
Login as oracle and execute the following commands:<br />
# shutdown Oracle<br />
SQL> shutdown<br />
<br />
cd $ORACLE_HOME/rdbms/lib<br />
<br />
# Make a backup of the ksms.s file if it exists<br />
[[ ! -f ksms.s_orig ]] && cp ksms.s ksms.s_orig<br />
<br />
# Modify the SGA attach address in the ksms.s file before relinking Oracle<br />
genksms -s 0x15000000 > ksms.s<br />
<br />
Rebuild the Oracle executable by entering the following commands:<br />
# Create a new ksms object file<br />
make -f ins_rdbms.mk ksms.o<br />
<br />
# Create a new "oracle" executable ($ORACLE_HOME/bin/oracle):<br />
make -f ins_rdbms.mk ioracle<br />
<br />
# The last step creates a new Oracle binary in $ORACLE_HOME/bin <br />
# that loads the SGA at the address specified by sgabeg in ksms.s:<br />
# .set sgabeg,0X15000000<br />
<br />
Now when Oracle is started in the shell terminal session for which the mapped_base for shared libraries was changed at Lowering the Mapped Base Address for Shared Libraries in RHEL 2.1, the SGA attach address for Oracle's shared memory segments and hence SGA can be displayed with the following commands:<br />
# Get pid of e.g. the Oracle checkpoint process<br />
$ /sbin/pidof ora_dbw0_$ORACLE_SID<br />
13519<br />
$ grep '.so' /proc/13519/maps |head -1<br />
10000000-10016000 r-xp 00000000 03:02 750738 /lib/ld-2.2.4.so<br />
$ grep 'SYS' /proc/13519/maps |head -1<br />
15000000-24000000 rw-s 00000000 00:04 262150 /SYSV3ecee0b0 (deleted)<br />
$<br />
The SGA size can now be increased to approximately 2.7 GB. If you create the SGA larger than 2.65 GB, then I would test the database very thoroughly to ensure no memory allocation problems arise. <br />
<br />
<br />
Allowing the Oracle User to Change the Mapped Base Address for Shared Libraries <br />
<br />
As shown at Lowering the Mapped Base Address for Shared Libraries in RHEL 2.1 only root can change the mapped_base for shared libraries. Using sudo we can give the "oracle" user the privilege to change the mapped base for shared libraries for the shell terminal session without providing full root access to the system. <br />
<br />
Here is the procedure: <br />
<br />
Create a script called "/usr/local/bin/ChangeMappedBase" which changes the mapped_base for shared libraries for for its own shell:<br />
# cat /usr/local/bin/ChangeMappedBase<br />
#/bin/sh<br />
echo 268435456 > /proc/$PPID/mapped_base<br />
Make the script executable:<br />
# chown root.root /usr/local/bin/ChangeMappedBase<br />
# chmod 755 /usr/local/bin/ChangeMappedBase<br />
Allow the oracle user to execute /usr/local/bin/ChangeMappedBase via sudo without password:<br />
# echo "oracle ALL=NOPASSWD: /usr/local/bin/ChangeMappedBase" >> /etc/sudoers<br />
Now the Oracle user can run /usr/local/bin/ChangeMappedBase to change the mapped_base for its own shell:<br />
$ su - oracle<br />
$ cat /proc/$$/mapped_base; echo<br />
1073741824<br />
$ sudo /usr/local/bin/ChangeMappedBase<br />
$ cat /proc/$$/mapped_base; echo<br />
268435456<br />
$<br />
To change the mapping for shared libraries automatically during Oracle logins, execute:<br />
# echo "sudo /usr/local/bin/ChangeMappedBase" >> ~/.bash_profile<br />
Now login as oracle:<br />
$ ssh oracle@localhost<br />
oracle@localhost's password:<br />
Last login: Sun Jan 7 13:59:22 2003 from localhost<br />
$ cat /proc/$$/mapped_base; echo<br />
268435456<br />
$<br />
<br />
Note: <br />
<br />
If the mapped base address for shared libraries for the Oracle processes was changed, then every Linux shell that spawns Oracle processes (e.g. listener, sqlplus, etc.) must have the same mapped base address as well. For example, if you execute sqlplus to connect to the local database, then you will get the following error message if the mapped_base for this shell is not the same as for the running Oracle processes:<br />
SQL> connect scott/tiger<br />
ERROR:<br />
ORA-01034: ORACLE not available<br />
ORA-27102: out of memory<br />
Linux Error: 12: Cannot allocate memory<br />
Additional information: 1<br />
Additional information: 491524<br />
<br />
SQL><br />
<br />
Growing the Oracle SGA to 2.7/3.42 GB in x86 RHEL 3/4 Without VLM<br />
General <br />
<br />
Due to 32-bit virtual address limitations workarounds have been implemented in Linux to increase the maximum size for shared memories. A workaround is to lower the Mapped Base Address for shared libraries and the SGA Attach Address for shared memory segments. This enables Oracle to attain an SGA larger than 1.7 GB. To get a better understanding of address mappings in Linux and what Mapped Base Address is, see Linux Memory Layout. <br />
<br />
The following example shows how to increase the size of the SGA without a shared memory filesystem. A shared memory filesystem must be used on x86 to increase SGA beyond 3.42 GB, see Configuring Very Large Memory (VLM). <br />
<br />
<br />
Mapped Base Address for Shared Libraries in RHEL 3 and RHEL 4 <br />
<br />
In RHEL 3/4 the mapped base for shared libraries does not need to be lowered since this operation is now done automatically. <br />
<br />
To verify the mapped base (mapped_base) for shared libraries execute "cat /proc/self/maps" in a shell. The directory "self" in the proc filesytem always points to the current running process which in this example is the cat process: <br />
# cat /etc/redhat-release<br />
Red Hat Enterprise Linux AS release 3 (Taroon Update 6)<br />
# cat /proc/self/maps<br />
00a23000-00a38000 r-xp 00000000 08:09 14930 /lib/ld-2.3.2.so<br />
00a38000-00a39000 rw-p 00015000 08:09 14930 /lib/ld-2.3.2.so<br />
00b33000-00c66000 r-xp 00000000 08:09 69576 /lib/tls/libc-2.3.2.so<br />
00c66000-00c69000 rw-p 00132000 08:09 69576 /lib/tls/libc-2.3.2.so<br />
00c69000-00c6c000 rw-p 00000000 00:00 0<br />
00ee5000-00ee6000 r-xp 00000000 08:09 32532 /etc/libcwait.so<br />
00ee6000-00ee7000 rw-p 00000000 08:09 32532 /etc/libcwait.so<br />
08048000-0804c000 r-xp 00000000 08:09 49318 /bin/cat<br />
0804c000-0804d000 rw-p 00003000 08:09 49318 /bin/cat<br />
099db000-099fc000 rw-p 00000000 00:00 0<br />
b73e7000-b75e7000 r--p 00000000 08:02 313698 /usr/lib/locale/locale-archive<br />
b75e7000-b75e8000 rw-p 00000000 00:00 0<br />
bfff8000-c0000000 rw-p ffffc000 00:00 0<br />
#<br />
# cat /etc/redhat-release<br />
Red Hat Enterprise Linux AS release 4 (Nahant Update 2)<br />
# cat /proc/self/maps<br />
00b68000-00b7d000 r-xp 00000000 03:45 1873128 /lib/ld-2.3.4.so<br />
00b7d000-00b7e000 r--p 00015000 03:45 1873128 /lib/ld-2.3.4.so<br />
00b7e000-00b7f000 rw-p 00016000 03:45 1873128 /lib/ld-2.3.4.so<br />
00b81000-00ca5000 r-xp 00000000 03:45 1938273 /lib/tls/libc-2.3.4.so<br />
00ca5000-00ca6000 r--p 00124000 03:45 1938273 /lib/tls/libc-2.3.4.so<br />
00ca6000-00ca9000 rw-p 00125000 03:45 1938273 /lib/tls/libc-2.3.4.so<br />
00ca9000-00cab000 rw-p 00ca9000 00:00 0<br />
08048000-0804c000 r-xp 00000000 03:45 1531117 /bin/cat<br />
0804c000-0804d000 rw-p 00003000 03:45 1531117 /bin/cat<br />
08fa0000-08fc1000 rw-p 08fa0000 00:00 0<br />
b7df9000-b7ff9000 r--p 00000000 03:45 68493 /usr/lib/locale/locale-archive<br />
b7ff9000-b7ffa000 rw-p b7ff9000 00:00 0<br />
bffa6000-c0000000 rw-p bffa6000 00:00 0<br />
ffffe000-fffff000 ---p 00000000 00:00 0<br />
#<br />
The outputs show that the mapped base is already very low in RHEL 3 and RHEL 4. In the above example shared libraries start at 0x00a23000 (decimal 10629120) in RHEL 3 and 0xb68000 (decimal 11960320) in RHEL 4. This is much lower than 0x40000000 (decimal 1073741824) in RHEL 2.1:<br />
# cat /etc/redhat-release<br />
Red Hat Linux Advanced Server release 2.1AS (Pensacola)<br />
# cat /proc/self/maps<br />
08048000-0804c000 r-xp 00000000 08:08 44885 /bin/cat<br />
0804c000-0804d000 rw-p 00003000 08:08 44885 /bin/cat<br />
0804d000-0804f000 rwxp 00000000 00:00 0<br />
40000000-40016000 r-xp 00000000 08:08 44751 /lib/ld-2.2.4.so<br />
40016000-40017000 rw-p 00015000 08:08 44751 /lib/ld-2.2.4.so<br />
40017000-40018000 rw-p 00000000 00:00 0<br />
40022000-40155000 r-xp 00000000 08:08 47419 /lib/i686/libc-2.2.4.so<br />
40155000-4015a000 rw-p 00132000 08:08 47419 /lib/i686/libc-2.2.4.so<br />
4015a000-4015f000 rw-p 00000000 00:00 0<br />
bffea000-bffee000 rwxp ffffd000 00:00 0<br />
#<br />
The above mappings show that the Mapped Base Address does not have to be lowered in RHEL 3/4 to gain more SGA space. <br />
<br />
<br />
Oracle 10g SGA Sizes in RHEL 3 and RHEL 4 <br />
<br />
The following table shows how large the Oracle 10g SGA can be configured in RHEL 3/4 without using a shared memory filesystem. Shared memory filesystems for the SGA are covered at Configuring Very Large Memory (VLM). <br />
<br />
RHEL 3/4 Kernel 10g DB Version Default Supported SGA<br />
without VLM Max Supported SGA<br />
without VLM Comments <br />
smp kernel (x86) 10g Release 1 Up to 1.7 GB Up to 2.7 GB 10g R1 must be relinked to increase the SGA size to approx 2.7 GB <br />
hugemem kernel (x86) 10g Release 1 Up to 2.7 GB Up to 3.42 GB 10g R1 must be relinked to increase the SGA size to approx 3.42 GB <br />
smp kernel (x86) 10g Release 2 Up to ~2.2 GB (*) Up to ~2.2 GB (*) No relink of 10g R2 is necessary but the SGA Attach Address is a little bit higher than in R1 <br />
hugemem kernel (x86) 10g Release 2 Up to ~3.3 GB (*) Up to ~3.3 GB (*) No relink of 10g R2 is necessary but the SGA Attach Address is a little bit higher than in R1 <br />
<br />
In Oracle 10g R2 the SGA size can be increased to approximately 2.7 GB using the smp kernel and to approximately 3.42 GB using the hugemem kernel. The SGA attach address does not have to be changed for that. To accommodate the same SGA sizes in Oracle 10g R1, the SGA Attach Address must be lowered. <br />
<br />
(*) In my test scenarios I was not able to startup a 10g R2 database if sga_target was larger than 2350000000 bytes on a smp kernel, and if sga_target was larger than 3550000000 bytes on a hugemem kernel. <br />
<br />
NOTE: Lowering the SGA attach address in Oracle restricts the remaining 32-bit address space for Oracle processes. This means that less address space will be available for e.g. PGA memory. If the application uses a lot of PGA memory, then PGA allocations could fail even if there is sufficient free physical memory. Therefore, in certain cases it may be prudent not to change the SGA Attach Address to increase the SGA size but to use Very Large Memory (VLM) instead. Also, if the SGA size is larger but less than 4GB to fit in memory address space, then the Very Large Memory (VLM) solution should be considered first before switching to the hugemem kernel on a small system, unless the system has lots of physical memory. The hugemem kernel is not recommended on systems with less than 8GB of RAM due to some overhead issues in the kernel, see also 32-bit Architecture. If larger SGA sizes are needed than listed in the above table, then Very Large Memory (VLM) must obviously be used on x86 platforms. <br />
<br />
<br />
Lowering the SGA Attach Address in Oracle 10g <br />
<br />
Starting with Oracle 10g R2 the SGA attach address does not have to be lowered for creating larger SGAs. However, Oracle 10g R1 must be relinked for larger SGAs. <br />
<br />
The following commands were executed on a 10g R1 database system:<br />
# ps -ef | grep "[o]ra_ckpt"<br />
oracle 3035 1 0 23:21 ? 00:00:00 ora_ckpt_orcl<br />
# cat /proc/3035/maps | grep SYSV<br />
50000000-aa200000 rw-s 00000000 00:04 262144 /SYSV8b1d1510 (deleted)<br />
#<br />
The following commands were executed on a 10g R2 database system:<br />
# ps -ef | grep "[o]ra_ckpt"<br />
oracle 4998 1 0 22:29 ? 00:00:00 ora_ckpt_orcl<br />
# cat /proc/4998/maps | grep SYSV<br />
20000000-f4200000 rw-s 00000000 00:04 4390912 /SYSV950d1f70 (deleted)<br />
#<br />
The output shows that the SGA attach address in 10g R2 is already lowered to 0x20000000 vs. 0x50000000 in 10g R1. This means that Oracle 10g R2 does not have to be relinked for creating larger SGAs. For 10g R1 the SGA attach address must be lowered from 0x50000000 to e.g. 0xe000000. You could also set it a little bit higher like 0x20000000 as its done by default in 10g Release 2. <br />
<br />
The following example shows how to lower the SGA attach address to 0xe000000 in 10g R1 (see also Metalink Note:329378.1):<br />
su - oracle<br />
cd $ORACLE_HOME/rdbms/lib<br />
[[ ! -f ksms.s_orig ]] && cp ksms.s ksms.s_orig<br />
genksms -s 0Xe000000 > ksms.s<br />
make -f ins_rdbms.mk ksms.o<br />
make -f ins_rdbms.mk ioracle<br />
For a detailed description of these commands, see Lowering the SGA Attach Address for Shared Memory Segments in Oracle 9i. <br />
<br />
You can verify the new lowered SGA attach address by running the following command:<br />
$ objdump -t $ORACLE_HOME/bin/oracle |grep sgabeg<br />
0e000000 l *ABS* 00000000 sgabeg<br />
$<br />
Now when 10g R1 is restarted the SGA attach address should be at 0xe000000:<br />
# ps -ef | grep "[o]ra_ckpt"<br />
oracle 4998 1 0 22:29 ? 00:00:00 ora_ckpt_orcl<br />
# cat /proc/4998/maps | grep SYSV<br />
0e000000-c1200000 rw-s 00000000 00:04 0 /SYSV8b1d1510 (deleted)<br />
#<br />
Now you should be able to create larger SGAs. <br />
<br />
NOTE: If you increase the size of the SGA, essentially using more process address space for the SGA, then less address space will be available for PGA memory. This means that if your application uses a lot of PGA memory, PGA allocations could fail even if you have sufficient RAM. In this case, you need to set the SGA attach address to a higher value which will lower the SGA size. <br />
<br />
Using Very Large Memory (VLM)<br />
General <br />
<br />
This chapter does not apply to 64-bit systems. <br />
<br />
With hugemem kernels on 32-bit systems, the SGA size can be increased but not significantly as shown at Oracle 10g SGA Sizes in RHEL 3 and RHEL 4 (note that the hugemem kernel is always recommended on systems with large amounts of RAM, see 32-bit Architecture and the hugemem Kernel). This chapter shows how the SGA can be significantly increased using VLM on 32-bit systems. <br />
<br />
Starting with Oracle9i Release 2 the SGA can theoretically be increased to about 62 GB (depending on block size) on a 32-bit system with 64 GB RAM. A processor feature called Page Address Extension (PAE) provides the capability of physically addressing 64 GB of RAM. However, it does not enable a process or program to address more than 4GB directly or have a virtual address space larger than 4GB. Hence, a process cannot attach to shared memory directly if it has a size of 4GB or more. To address this issue, a shared memory filesystem (memory-based filesystem) can be created which can be as large as the maximum allowable virtual memory supported by the kernel. With a shared memory filesystem processes can dynamically attach to regions of the filesystem allowing applications like Oracle to have virtually a much larger shared memory on 32-bit systems. This is not an issue on 64-bit systems. <br />
<br />
For Oracle to use a shared memory filesystem, a feature called Very Large Memory (VLM) must be enabled. VLM moves the database buffer cache part of the SGA from the System V shared memory to the shared memory filesystem. It is still considered one large SGA but it consists now of two different OS shared memory entities. It is noteworthy to say that VLM uses 512MB of the non-buffer cache SGA to manage VLM. This memory area is needed for mapping the indirect data buffers (shared memory filesystem buffers) into the process address space since a process cannot attach to more than 4GB directly on a 32-bit system. For example, if the non-buffer cache SGA is 2.5 GB, then you will only have 2 GB of non-buffer cache SGA for shared pool, large pool, and redo log buffer since 512MB is used for managing VLM. If the buffer cache is less than 512 MB, then the init.ora parameter VLM_WINDOW_SIZE must be changed to reflect the size of the database buffer cache. However, it is not recommended to use VLM if db_block_buffers is not greater than 512MB. <br />
<br />
In RHEL 3 and RHEL 4 there are two different memory filesystems that can be used for VLM:<br />
- shmfs/tmpfs: This memory filesystem is pageable/swappable. And to my knowledge it cannot be backed by Huge Pages because Huge Pages are not swappable.<br />
- ramfs: This memory filesystems is not pageable/swappable and not backed by Huge Pages, see also Huge Pages and Shared Memory Filesystem in RHEL 3/4. <br />
<br />
Note the shmfs filesystem is available in RHEL 3 but not in RHEL 4:<br />
$ cat /etc/redhat-release<br />
Red Hat Enterprise Linux AS release 3 (Taroon Update 6) <br />
$ egrep "shm|tmpfs|ramfs" /proc/filesystems<br />
nodev tmpfs <br />
nodev shm <br />
nodev ramfs <br />
$<br />
<br />
$ cat /etc/redhat-release<br />
Red Hat Enterprise Linux AS release 4 (Nahant Update 2)<br />
$ egrep "shm|tmpfs|ramfs" /proc/filesystems<br />
nodev tmpfs<br />
nodev ramfs<br />
$<br />
This means that if you try to mount a shmfs filesystem in RHEL 4, you will get the following error message:<br />
mount: fs type shm not supported by kernel<br />
The difference between shmfs and tmpfs is you don't need to specify the size of the filesystem if you mount a tmpfs filesystem. <br />
<br />
<br />
Configuring Very Large Memory (VLM) <br />
<br />
The following example shows how to use the RAM disk ramfs to allocate 8 GB of shared memory for the 10g database buffer cache on a 32-bit RHEL 3/4 systems (hugemem kernel). If this setup is performed on a server that does not have enough RAM, then Linux will appear to hang and the kernel will automatically start killing processes due to memory shortage (ramfs is not swappable). Furthermore, ramfs is not backed by Huge Pages and therefore the Huge Pages pool should not be increased for database buffers, see Huge Pages and Shared Memory Filesystem in RHEL 3/4. In fact, if there are too many Huge Pages allocated, then there may not be enough memory for ramfs. <br />
<br />
Since ramfs is not swappable, it is by default only usable by root. If you put too much on a ramfs filesystem, you can easily hang the system. To mount the ramfs filesystem and to make it usable for the Oracle account, execute:<br />
# umount /dev/shm <br />
# mount -t ramfs ramfs /dev/shm <br />
# chown oracle:dba /dev/shm<br />
When Oracle starts it will create a file in the /dev/shm directory that corresponds to the extended buffer cache. Ensure to add the above lines to /etc/rc.local. If ointall is the primary group of the Oracle account, use chown oracle:oinstall /dev/shm instead. For security reasons you do not want to give anyone write access to the shared memory filesystem. Having write access to the ramfs filesystem allows you to allocate and pin a large chunk of memory in RAM. In fact, you can kill a machine by allocating too much memory in the ramfs filesystem. <br />
<br />
To enable VLM, set the Oracle parameter use_indirect_data_buffers to true:<br />
use_indirect_data_buffers=true<br />
<br />
For 10g R1 and R2 databases it's important to convert DB_CACHE_SIZE and DB_xK_CACHE_SIZE parameters to DB_BLOCK_BUFFERS, and to remove SGA_TARGET if set. Otherwise you will get errors like these:<br />
ORA-00385: cannot enable Very Large Memory with new buffer cache parameters<br />
<br />
Here is an example for configuring a 8 GB buffer cache for a 10g R2 database with RHEL 3/4 hugemem kernels:<br />
use_indirect_data_buffers=true <br />
db_block_size=8192 <br />
db_block_buffers=1048576 <br />
shared_pool_size=2831155200<br />
Note that shmmax needs to be increased for shared_pool_size to fit into the System V shared memory. In fact, it should be slightly larger than the SGA size. Since shared_pool_size is less than 3 GB in this example, shmmax doesn't need to be larger than 3GB. The 8 GB indirect buffer cache will be in the RAM disk and hence it doesn't have to be accounted for in shmmax. On a 32-bit system the shmmaxkernel paramter cannot be larger than 4GB, see also Setting SHMMAX Parameter. <br />
<br />
In order to allow oracle processes to lock more memory into its address space for the VLM window size, the ulimit parameter memlock must be changed for oracle. <br />
Ensure to set memlock in /etc/security/limits.conf to 3145728:<br />
oracle soft memlock 3145728<br />
oracle hard memlock 3145728<br />
Login as Oracle again and check max locked memory limit:<br />
$ ulimit -l<br />
3145728<br />
If it's not working after a ssh login, then you may have to set the SSH parameter UsePrivilegeSeparation, see Setting Shell Limits for the Oracle User. <br />
<br />
If memlock is not set or too small, you will get error messages similar to this one:<br />
ORA-27103: internal error<br />
Linux Error: 11: Resource temporarily unavailable<br />
<br />
Now try to start the database. Note the database startup can take a while. Also, the sqlplus banner or show sga may not accurately reflect the actual SGA size in older Oracle versions. <br />
<br />
The 8GB file for the database buffer cache can be seen in the ramfs shared memory filesystem:<br />
$ ls -al /dev/shm<br />
total 120 <br />
drwxr-xr-x 1 oracle dba 0 Nov 20 16:29 . <br />
drwxr-xr-x 22 root root 118784 Nov 20 16:25 .. <br />
-rw-r----- 1 oracle dba 8589934592 Nov 20 16:30 ora_orcl_458754 <br />
$<br />
<br />
If the shared pool size is configured too large, you will get error messages similar to this one:<br />
ORA-27103: internal error<br />
Linux Error: 12: Cannot allocate memory<br />
<br />
Measuring I/O Performance on Linux for Oracle Databases<br />
General <br />
<br />
Oracle provides now a tool called Orion which simulates Oracle workloads without having to install Oracle or create a database. It uses Oracle's I/O software stack to perform various test scenarios to predict performance of Oracle databases. Orion can also simulate ASM striping. So it's a great tool for testing and optimizing Linux for Oracle databases. But note that at the time of this writing Orion is available on x86 Linux only and it's still in beta and not supported by Oracle. For more information on Orion, see Oracle ORION. <br />
<br />
<br />
Using Orion <br />
<br />
WARNING: Running write tests with the Orion tool will wipe out all data on the disks where tests are performed. <br />
<br />
In the following example I will use Orion to measure the performance of small random reads at different loads and then (separately) large random reads at different loads. <br />
<br />
<br />
Before running any tests, verify the speed of the Host Bus Adapters (HBA) if you use SAN attached storage. <br />
<br />
For Emulex Fibre Channel adapters in RHEL 3, execute:<br />
# grep speed /proc/scsi/lpfc/*<br />
For QLogic Fibre Channel adapters in RHEL 3, execute:<br />
# grep "data rate" /proc/scsi/qla*/*<br />
<br />
Go to Oracle ORION downloads to download the Orion tool. The downloadable file is in a compressed format that contains a single binary that simulates the workloads. To uncompress the file and make it executable, run:<br />
# gunzip orion10.2_linux.gz<br />
# chmod 755 orion10.2_linux<br />
Make sure the libaio RPM is installed on the system to avoid the following error:<br />
# ./orion10.2_linux<br />
./orion10.2_linux: error while loading shared libraries: libaio.so.1: cannot open shared object file: No such file or directory<br />
#<br />
Next create a file that lists the raw volumes or files that should be tested by Orion. For example, if the name of the test run is "test1", then the file name should be test1.lun: <br />
# cat test1.lun<br />
/dev/raw/raw1<br />
#<br />
Now to run a "simple" test to measure the performance of small random reads at different loads and then (separately) large random reads at different loads, execute:<br />
# ./orion10.2_linux -run simple -testname test1 -num_disks 1<br />
The option "-run simple" specifies to run a "simple" test which measures the performance of small random reads at different loads and then (separately) large random reads at different loads. <br />
The option "-testname test" specifies the name of the test run. This means that test.lun must contain a list of raw volumes or files to be tested. And the results of the test will be recorded in files that start with suffix "test". <br />
The option "-num_disks 1" specifies that I have only one raw volume or file listed in the test.lun file. <br />
<br />
<br />
A test run creates several output files. The summary file contains information like MBPS, IOPS, latency, etc. Here is the list of files of my "test1" test run:<br />
# ls test1*<br />
test1_iops.csv test1_lat.csv test1.lun test1_mbps.csv test1_summary.txt test1_trace.txt<br />
For more information on Orion and the output files, refer to Oracle ORION. <br />
<br />
Appendix<br />
Here is a list of various Linux monitoring tools and statistics files: <br />
<br />
Overall Tools: <br />
top, vmstat, sar, ps, pstree, ipcs <br />
<br />
CPU: <br />
top, mpstat, tload, /proc/cpuinfo, x86info <br />
<br />
Memory: <br />
free, /proc/meminfo, slabtop, /proc/slabinfo, ipcs <br />
<br />
I/O: <br />
iostat, vmstat, sar <br />
<br />
sar examples: <br />
<br />
To display CPU utilization:<br />
sar 3 100<br />
To display paging activity:<br />
sar -B 3 100<br />
To display swapping activity:<br />
sar -W 3 100<br />
To display block I/O activity:<br />
sar -b 3 100<br />
To display block I/O activity for each block device:<br />
sar -d 3 100<br />
To display network activity:<br />
sar -n DEV 3 100KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-54944871527307937242009-10-06T08:35:00.000-07:002009-10-06T08:35:54.414-07:00Oracle Tuning Parameters§ Performance<br />
§ Parallel Query Option<br />
§ Analysis Tools<br />
§ General<br />
§ Recovery Manager<br />
§ Multithreaded Server<br />
§ Distributed Option<br />
§ Parallel Server Parameters<br />
§ Security<br />
§ Trusted Oracle7<br />
§ National Language Support<br />
<br />
This lists the Oracle tuning parameters, grouping them into general areas of use and then sorting them alphabetically within the group. The section headings are the syntax for the parameters. The syntax contains information in the following format:<br />
PARAMETER [option1, option2, option3, etc..] <DEFAULT VALUE><br />
A value that is italicized indicates that the value should be replaced with one of your own. A value that is italicized and in CAPS indicates a choice of this keyword. The / character indicates an OR condition. A value enclosed in brackets (<>) indicates the default value for that parameter.<br />
These parameters are divided into sections based on whether the parameter affects performance, enables system analysis, is a general parameter, and so on. There might be some overlap, so if a parameter is not in the section you expected, keep looking.<br />
Performance<br />
These parameters change the performance characteristics of the system.<br />
ALWAYS_ANTI_JOIN [NESTED_LOOPS/MERGE/ HASH] <NESTED_LOOPS> This parameter sets the type of anti-join that the Oracle server uses. This specifies the algorithm chosen for the anti-join.<br />
B_TREE_BITMAP_PLANS [TRUE/FALSE] <FALSE><br />
When set to TRUE, the optimizer considers a bitmap access path even though a table might have only a regular B*-tree index.<br />
BITMAP_MERGE_AREA_SIZE [System Dependent] <1MB><br />
This parameter specifies the amount of memory used to merge bitmaps retrieved from a range scan of the index. Larger values typically improve performance.<br />
CLOSE_CACHED_OPEN_CURSORS [TRUE/FALSE] <FALSE><br />
This parameter specifies whether cursors opened and cached in memory are automatically closed at each commit. If you frequently use cursors, this should be set to FALSE.<br />
CPU_COUNT [0-unlimited] <Automatic><br />
This parameter specifies the number of CPUs used by Oracle. This parameter is set automatically and should not be changed.<br />
CREATE_BITMAP_AREA_SIZE [OS Dependent] <8MB><br />
This parameter specifies the amount of memory to be used for bitmap creation. A larger value might provide greater bitmap-creation performance. If the cardinality is small, this number can be small.<br />
CURSOR_SPACE_FOR_TIME [TRUE/FALSE] <FALSE><br />
CURSOR_SPACE_FOR_TIME causes the system to use more space for cursors, thus increasing performance. This parameter affects both the shared SQL areas and the user's private SQL area. This parameter speeds performance but uses more memory.<br />
If CURSOR_SPACE_FOR_TIME is TRUE, the shared SQL areas remain pinned in the shared pool as long as an open cursor references them. This parameter should be used only if you have a sufficiently large shared pool to simultaneously hold all the processes' cursors.<br />
The user's private SQL area is also retained during cursor execution, thus saving time and I/Os at the expense of memory. DB_BLOCK_BUFFERS [4..65535] <32 buffers> This parameter controls the number of database block buffers in the SGA. DB_BLOCK_BUFFERS is probably the most significant instance tuning parameter because the majority of I/Os in the system are generated by database blocks. Increasing DB_BLOCK_BUFFERS increases performance at the expense of memory. You can calculate the amount of memory that will be consumed with the following formula:<br />
Buffer size = DB_BLOCK_BUFFERS * DB_BLOCK_SIZE<br />
A larger number of database block buffers in the system creates a higher cache-hit rate, thus reducing the amount of utilized I/O and CPU and improving performance.<br />
DB_BLOCK_CHECKPOINT_BATCH [0..derived] <8><br />
This parameter specifies the number of blocks that the DBWR writes in one batch when performing a checkpoint. Setting this value too high causes the system to flood the I/O devices during the checkpoint, severely degrades performance, and increases response times--maybe to unacceptable levels.<br />
You should set DB_BLOCK_CHECKPOINT_BATCH to a level that allows a checkpoint to finish before the next checkpoint occurs. Setting DB_BLOCK_CHECKPOINT_BATCH to 0 causes the default value of 8 to be used.<br />
DB_BLOCK_SIZE [1024..8192 (OS dependent)] <OS dependent><br />
This parameter specifies in bytes the size of the Oracle database blocks. The typical values are 2048 and 4096. If you set the block size relative to the size of the rows in a database, you can reduce I/O. In some types of applications in which large amounts of sequential accesses are performed, a larger database block size can be beneficial. This value is useful only at database-creation time.<br />
DB_FILE_MULTIBLOCK_READ_COUNT [number (OS dependent)] <OS dependent><br />
DB_FILE_MULTIBLOCK_READ_COUNT specifies the maximum number of blocks read in one I/O during a sequential scan. The default is a function of DB_BLOCK_BUFFERS and PROCESSES. Reasonable values are 4, 16, or 32. The maximum allowed values are OS dependent.<br />
This parameter can be especially useful if you perform a large number of table scans, such as in a DSS system.<br />
DB_FILE_SIMULTANEOUS_WRITES [1..24] <4><br />
This parameter specifies the number of simultaneous writes for each database file when written by the DBWR. For disk arrays that handle large numbers of requests in the hardware simultaneously, it is advantageous to set DB_FILE_SIMULTANEOUS_WRITES to its maximum.<br />
DISCRETE_TRANSACTIONS_ENABLED [TRUE/FALSE] <FALSE><br />
This parameter implements a simpler, faster rollback mechanism that, under certain conditions, can improve performance. You can obtain greater efficiency in this mode, but the qualification criteria for what kind of transactions can take advantage of discrete transactions are quite strict.<br />
DISK_ASYNCH_IO [TRUE/FALSE] <TRUE><br />
This parameter specifies that I/O to datafiles, control files, and log files are asynchronous. This should be left enabled and not altered.<br />
DML_LOCKS [20..unlimited,0] <4 * TRANSACTIONS><br />
This parameter specifies the maximum number of DML locks. A DML lock is used for each table-modification transaction. DML locks are used in the DROP TABLE, CREATE INDEX, and LOCK TABLE IN EXCLUSIVE MODE statements. If the value is set to 0, enqueues (Oracle locking mechanisms) are disabled, which improves performance slightly.<br />
DBWR_IO_SLAVES [0..OS Dependent] <0><br />
This parameter specifies the number of I/O slaves used by the DBWR process.<br />
HASH_AREA_SIZE [0..OS Dependent] <2*SORT_AREA_SIZE><br />
This parameter specifies the maximum amount of memory to be used for hash joins.<br />
HASH_MULTIBLOCK_IO_COUNT [OS Dependent] <1><br />
This parameter specifies how many sequential blocks a hash join reads and writes in one I/O.<br />
LARGE_POOL_MIN_ALLOC [16K-64KB] <16KB><br />
This parameter specifies the minimum allocation size from the large pool. LARGE_POOL_SIZE<br />
[300K or LARGE_POOL_MIN_ALLOC, whichever is larger] <0><br />
This parameter specifies the size of the large pool allocation heap.<br />
LGWR_IO_SLAVES [0..OS Dependent] <0><br />
This parameter specifies the number of I/O slaves used by the LGWR process.<br />
LOG_ARCHIVE_BUFFER_SIZE [1..OS Dependent] <OS dependent><br />
When running in ARCHIVELOG mode, this parameter specifies the size of each archival buffer in redo log blocks. This parameter can be used in conjunction with the LOG_ARCHIVE_BUFFERS parameter to make the archiving speed faster or slower to affect overall system performance.<br />
LOG_ARCHIVE_BUFFERS [1..OS Dependent] <OS dependent><br />
When running in ARCHIVELOG mode, this parameter specifies the number of buffers to allocate to archiving. This parameter is used with the LOG_ARCHIVE_BUFFER_SIZE parameter to control the speed of archiving.<br />
LOG_BUFFER [OS Dependent] <OS dependent><br />
LOG_BUFFER specifies the number of bytes allocated to the redo log buffer. Larger values reduce I/Os to the redo log by writing fewer blocks of a larger size. This might help performance, particularly in a heavily used system.<br />
LOG_CHECKPOINT_INTERVAL [2..unlimited] <OS dependent><br />
This parameter specifies the number of redo log file blocks to be filled to cause a checkpoint to occur. Remember that a checkpoint always happens when a log switch occurs. This parameter can be used to cause checkpoints to occur more frequently. Sometimes, frequent checkpoints have less effect on the system than one large checkpoint when the log switch occurs.<br />
LOG_CHECKPOINT_TIMEOUT [0..unlimited] <OS dependent><br />
This parameter specifies the maximum amount of time that can pass before another checkpoint must occur. This parameter can also be used to increase the frequency of the checkpoint process, thus changing the overall system effect.<br />
LOG_SIMULTANEOUS_COPIES [0..unlimited] <CPU_COUNT><br />
LOG_SIMULTANEOUS_COPIES specifies the number of redo buffer copy latches simultaneously available to write log entries. You can have up to two redo copy latches per CPU. This helps the LGWR process keep up with the extra load generated by multiple CPUs.<br />
If this parameter is 0, redo copy latches are turned off and all log entries are copied on the redo allocation latch.<br />
LOG_SMALL_ENTRY_MAX_SIZE [number (OS dependent)] <OS dependent><br />
This parameter specifies the size in bytes of the largest copy to the log buffers that can occur under the redo allocation latch without obtaining the redo buffer copy latch. If LOG_SIMULTANEOUS_COPIES is zero, this parameter is ignored.<br />
OPTIMIZER_MODE [RULE/COST/FIRST_ROWS/ALL_ROWS] COST<br />
When set to RULE, this parameter causes rule-based optimization to be used, unless hints are supplied in the query. When set to COST, this parameter causes a cost-based approach for the SQL statement, providing that there are any statistics in the data dictionary. When set to FIRST_ROWS, the optimizer chooses execution plans that minimize response time. When set to ALL_ROWS, the optimizer chooses execution plans that minimize total execution time.<br />
OPTIMIZER_PERCENT_PARALLEL [0..100] <0><br />
This parameter specifies the amount of parallelism the optimizer uses in its cost functions.<br />
OPTIMIZER_SEARCH_LIMIT <5><br />
This parameter specifies the search limit for the optimizer.<br />
PRE_PAGE_SGA [TRUE/FALSE] <FALSE><br />
When set to TRUE, this parameter specifies that at instance startup all pages of the SGA are touched, causing them to be allocated in memory. This increases startup time but reduces page faults during runtime. This is useful if you have a large number of processes starting at once. This parameter can increase the system performance in that case by avoiding memory-allocation overhead.<br />
ROLLBACK_SEGMENTS [Any rollback segment names] <NULL><br />
ROLLBACK_SEGMENTS specifies one or more rollback-segment names to be allocated to this instance. If ROLLBACK_SEGMENTS is not specified, the public rollback segments are used. If you want to move your rollback segments to a different disk device, you must specify it here. The parameter is specified as follows:<br />
ROLLBACK_SEGMENTS = (roll1, roll2, roll3)<br />
If you use the Oracle Parallel Server option, you must name different rollback segments for each instance.<br />
ROW_CACHE_CURSORS [10..3300] <10><br />
This parameter specifies the number of cached recursive cursors used by the row cache manager for selecting rows from the data dictionary. The default is usually sufficient unless you have particularly high access to the data dictionary.<br />
ROW_LOCKING [ALWAYS/INTENT] <ALWAYS><br />
The value ALWAYS specifies that only row locks are acquired when a table is updated. If you set this value to INTENT, row locks are acquired on a SELECT FOR UPDATE, but when the update occurs, a table lock is acquired. SEQUENCE_CACHE_ENTRIES [10..32000] <10> This parameter specifies the number of sequences that can be cached in the SGA. By caching the sequences, an immediate response is achieved for sequences. Set a large value for SEQUENCE_CACHE_ENTRIES if you have a high concurrency of processes requesting sequences.<br />
SEQUENCE_CACHE_HASH_BUCKETS [1..32000 (prime number)] <7><br />
This parameter specifies the number of buckets to speed up access to sequences in the cache. The cache is arranged as a hash table.<br />
SERIAL_REUSE [DISABLE/SELECT/DML/PLSQL/ALL/NULL] <NULL><br />
This parameter specifies which type of SQL cursors should make use of serial-reusable memory.<br />
SERIALIZABLE [TRUE/FALSE] <FALSE><br />
If this value is set to TRUE, queries obtain table-level read locks, which prohibits other transactions from modifying that table until the transaction has committed or rolled back the transaction. This mode provides repeatable reads and ensures that within the transactions multiple queries to the same data achieve the same result.<br />
With SERIALIZABLE set to TRUE, degree-three consistency is provided. You pay a performance penalty when you run in this mode. Running in this mode is usually not necessary.<br />
SESSION_CACHED_CURSORS [0..OS dependent] <0><br />
This parameter specifies the number of session cursors to cache. If parse calls of the same SQL statement are repeated, this can cause the session cursor for that statement to be moved into the session cursor cache. Subsequent calls need not reopen the cursor.<br />
SESSION_MAX_OPEN_FILES [1..MAX_OPEN_FILES] <10><br />
This parameter specifies the maximum number of BFILEs that can be opened by any given session. The BFILE stores unstructured binary data in OS files outside the database.<br />
SHARED_POOL_RESERVED_MIN_ALLOC [5000..SHARED_POOL_RESERVE_SIZE] <5000><br />
Memory allocations larger than this value cannot allocate space from the reserved list.<br />
SHARED_POOL_RESERVE_SIZE [SHARED_POOL_RESERVE_MIN_ALLOC.. (SHARED_POOL_SIZE/2)] <5% of SHARED_POOL_SIZE><br />
This parameter specifies the shared pool space that is reserved for large contiguous requests for shared-pool memory.<br />
SHARED_POOL_SIZE [300KB..OS dependent] <3.5MB><br />
This parameter specifies the size of the shared pool in bytes. The shared pool contains the data dictionary cache (row cache) and the library cache as well as session information. Increasing the size of the shared pool should help performance, but at the cost of memory.<br />
SMALL_TABLE_THRESHOLD [0..OS dependent] <4><br />
This parameter specifies the number of buffers available in the SGA for table scans. A small table might be read entirely into cache if it fits in SMALL_TABLE_THRESHOLD number of buffers. When scanning a table larger than this, these buffers are reused immediately. This provides a mechanism to prohibit a single-table scan from taking over the buffer cache.<br />
SORT_AREA_RETAINED_SIZE [0..SORT_AREA_SIZE] <SORT_AREA_SIZE><br />
SORT_AREA_RETAINED_SIZE defines the maximum amount of session memory in bytes that can be used for an in-memory sort. The memory is released when the last row is fetched from the sort area.<br />
If the sort does not fit in SORT_AREA_RETAINED_SIZE bytes, a temporary segment is allocated and the sort is performed in this temporary table. This is called an external (disk) sort. This value is important if sort performance is critical. SORT_AREA_SIZE [number of bytes] <OS dependent> This value specifies the maximum amount of PGA memory to use for an external sort. This memory is released when the sorted rows are written to disk. Increasing this value increases the performance of large sorts.<br />
Remember that each user process has its own PGA. You can calculate the potential memory usage if all the users are doing a large sort with the following formula:<br />
Potential memory usage = SORT_AREA_SIZE * (number of users doing a large sort)<br />
If very large indexes are being created, you might want to increase the value of this parameter. SORT_SPACEMAP_SIZE [bytes] <OS dependent> This parameter specifies the size in bytes of the sort spacemap in the context area. If you have very large indexes, increase the value of this parameter. Optimal performance is achieved when this parameter has the following value:<br />
SORT_SPACEMAP_SIZE = (total-sort-bytes / sort-area-size) + 64<br />
In this formula, total-sort-bytes has the following value:<br />
total-sort-bytes = record-count * ( sum-of-average-column-sizes + ( 2 * number-of-columns ) )<br />
number-of-columns includes the SELECT list for ORDER BY, GROUP BY, and the key list for the CREATE INDEX. You should also add 10 or 20 extra bytes for overhead.<br />
SORT_WRITE_BUFFER_SIZE [32KB/64KB] <32768><br />
This parameter specifies the size of the sort I/O buffer when SORT_DIRECT_WRITES is set to TRUE.<br />
SORT_WRITE_BUFFERS [2..8] <1><br />
This parameter specifies the number of sort buffers when SORT_DIRECT_WRITES is set to TRUE.<br />
SPIN_COUNT [1..1,000,000] <1><br />
This parameter specifies the number of times to spin on a latch before sleeping.<br />
STAR_TRANSFORMATION_ENABLED [TRUE/FALSE] <FALSE><br />
This parameter specifies whether a cost-based query transformation will be applied to star queries.<br />
USE_ISM [TRUE/FALSE] <TRUE><br />
This parameter specifies that the shared page table is enabled.<br />
Parallel Query Option<br />
The following parameters affect the operation of the Parallel Query option, which has been available in Oracle since version 7.1. The Parallel Query option can dramatically affect the performance of certain operations.<br />
PARALLEL_DEFAULT_MAX_SCANS [0..unlimited] <OS dependent><br />
This value specifies the maximum number of query servers to be used by default for a query. This valued is used only if there are no values specified in a PARALLEL hint or in the PARALLEL definition clause. This limits the number of query servers used by default when the value of PARALLEL_DEFAULT_SCANSIZE is used by the query coordinator.<br />
PARALLEL_DEFAULT_SCANSIZE [0..OS Dependent ] <OS dependent><br />
This parameter is used to determine the number of query servers to be used for a particular table. The size of the table divided by PARALLEL_DEFAULT_SCANSIZE determines the number of query servers, up to PARALLEL_DEFAULT_MAX_SCANS.<br />
PARALLEL_MAX_SERVERS [0..100] <OS dependent><br />
This parameter specifies the maximum number of query servers or parallel recovery processes available for this instance.<br />
PARALLEL_MIN_MESSAGE_POOL [0..(SHARED_POOLSIZE*.9)] <equation><br />
This parameter specifies the minimum permanent amount of memory that will be allocated from the shared pool for messages in parallel execution.<br />
PARALLEL_MIN_PERCENT [0..100] <0><br />
This parameter specifies the minimum percent of threads required for parallel query.<br />
PARALLEL_MIN_SERVERS [0..PARALLEL_MAX_SERVERS] <0><br />
This parameter determines the minimum number of query servers for an instance. It is also the number of query servers started at instance startup.<br />
PARALLEL_SERVER_IDLE_TIME [0..unlimited] <OS dependent><br />
This parameter specifies the number of minutes before Oracle terminates an idle query server process.<br />
RECOVERY_PARALLELISM [0..PARALLEL_MAX_SERVERS] <OS dependent><br />
This parameter specifies the number of processes to be used for instance or media recovery. A large value can greatly reduce instance recovery time. A value of 0 or 1 indicates that parallel recovery will not be performed and that recovery will be serial.<br />
Analysis Tools<br />
These parameters turn on special features in Oracle for detailed analysis and debugging.<br />
DB_BLOCK_CHECKSUM [TRUE/FALSE] <FALSE><br />
Setting this parameter to TRUE causes the DBWR and direct loader to calculate a checksum for every block they write to disk. This checksum is written into the header of each block.<br />
DB_LOG_CHECKSUM [TRUE/FALSE] <FALSE><br />
Setting this parameter to TRUE causes the LGWR to calculate a checksum for every block it writes to disk. The checksum is written into the header of the redo block.<br />
DB_BLOCK_LRU_EXTENDED_STATISTICS [0..unlimited] <0><br />
This parameter enables statistics in the X$KCBRBH table to be gathered. These statistics estimate the increased number of database block buffer cache hits for each additional buffer. Any value over zero specifies the number of buffers to estimate the cache hits for. If you are interested in estimating the cache hits for an additional 100 buffers, set this parameter to 100.<br />
This parameter affects performance and should be turned off during normal operation.<br />
DB_BLOCK_LRU_LATCHES [1.. number of CPUs] <CPU_COUNT/2><br />
This parameter specifies the upper bound of the number of LRU latch sets. This is the number of LRU latch sets that you want. Oracle decides whether to use this number or a smaller one.<br />
DB_BLOCK_LRU_STATISTICS [TRUE/FALSE] <FALSE><br />
This parameter specifies whether statistics are gathered for database block buffer cache hit estimates as specified in DB_BLOCK_LRU_EXTENDED_STATISTICS. Set this parameter to TRUE when you want to gather these statistics.<br />
DB_BLOCK_MAX_DIRTY_TARGET [100..all buffers or 0] <all buffers><br />
This parameter specifies the number of buffers that can be dirty. If the number of dirty buffers exceeds this, the DBWR writes out buffers to reduce the number of dirty buffers.<br />
EVENT <NULL><br />
The EVENT parameter modifies the scope of ALTER SESSION SET EVENTS commands so that they pertain to the entire instance rather than just the session. This is an Oracle internal parameter and should be changed only at the direction of Oracle support.<br />
FIXED_DATE [date string] <NULL><br />
FIXED_DATE allows you to set as a constant the Oracle function SYSDATE in the format YYYY-MM-DD-HH24:MI:SS. Use this parameter for debug only. This parameter allows you to test your application's functionality with certain dates, such as the turn of the century.<br />
ORACLE_TRACE_COLLECTION_NAME [valid name] <NULL><br />
This parameter specifies the Oracle Trace collection name.<br />
ORACLE_TRACE_COLLECTION_PATH [valid path] <NULL><br />
This parameter specifies the directory where Oracle Trace collection definition and datafiles are located.<br />
ORACLE_TRACE_COLLECTION_SIZE [0..4294967295] <5242880><br />
The maximum size in bytes of the Oracle Trace collection file.<br />
ORACLE_TRACE_ENABLE [TRUE/FALSE] <FALSE><br />
Enables Oracle Trace collections for the server.<br />
ORACLE_TRACE_FACILITY_NAME [valid name] <OS Specific><br />
This parameter specifies the name of the Oracle Trace product definition file.<br />
ORACLE_TRACE_FACILITY_PATH [valid directory name] <OS Specific><br />
This parameter specifies the directory where the Oracle Trace facility definition files are located.<br />
SQL_TRACE [TRUE/FALSE] <FALSE><br />
This parameter specifies whether the SQL*Trace facility is enabled. The SQL*Trace facility can provide valuable information but at the price of some overhead. Use SQL*Trace only when you are tracking down a specific problem.<br />
SORT_READ_FAC [integer] <OS Dependent><br />
SORT_READ_FAC defines a unitless ratio that describes the amount of time to read a single database block divided by the block transfer rate.<br />
TIMED_OS_STATISTICS [OFF/CALL/LOGOFF] <OFF><br />
This parameter allows the system administrator to gather OS statistics when calls are pushed or popped or when a user logs off.<br />
TIMED_STATISTICS [TRUE/FALSE] <FALSE><br />
When TIMED_STATISTICS is set to TRUE, the time-related statistics in the dynamic performance tables are enabled. This information can be quite useful, but there is considerable overhead involved. Only enable TIMED_STATISTICS when you are analyzing the system.<br />
General<br />
These parameters are of a general nature; they typically set limits and do not significantly affect performance--except that they might take up space in the SGA. AQ_TM_PROCESS [0/1] <0> This parameter specifies whether a time manager is created. If AQ_TM_PROCESS is set to 1, a time-manager process is created to monitor the messages.<br />
ARCH_IO_SLAVES [0-15] <0><br />
The number of I/O slaves to be used by the ARCH process. This should be adjusted if archiving is running into an I/O bottleneck.<br />
BACKGROUND_CORE_DUMP [FULL/PARTIAL] <FULL><br />
This parameter specifies whether the SGA is dumped as part of the generated core file.<br />
BACKGROUND_DUMP_DEST [pathname] <OS dependent><br />
This parameter specifies the destination directory where the debugging trace files for the background processes are written. The background processes log all startup and shutdown messages and errors to these files, as well as any other error logs. A log of all CREATE, ALTER, or DROP statements is also stored here.<br />
BLANK_TRIMMING [TRUE/FALSE] <FALSE><br />
If the value of BLANK_TRIMMING is TRUE, this allows a data assignment of a string variable to a column value that is smaller (assuming that the truncated characters are blank).<br />
CHECKPOINT_PROCESS [TRUE/FALSE] <FALSE><br />
This parameter determines whether the CKPT background process is enabled. During a checkpoint, the headers of all the datafiles must be updated. This task is usually performed by the LGWR process. Writing the blocks to disk is the job of the DBWR process. If you notice that the LGWR is slowing down during checkpoints, it might be necessary to enable CKPT to eliminate the extra work that LGWR is doing.<br />
CLEANUP_ROLLBACK_ENTRIES [number] <20><br />
This parameter specifies the number of undo records processed at a time when a rollback occurs. This breaks up the rollback and limits a large rollback from locking out smaller rollbacks.<br />
CLOSE_CACHED_OPEN_CURSORS [TRUE/FALSE] <FALSE><br />
This parameter specifies whether cursors that have been opened and cached by PL/SQL are automatically closed at COMMIT. A value of FALSE allows these cursors to remain open for further use. If cursors are rarely reused, you can save space in the SGA by setting this value to TRUE. If cursors are reused, you can improve performance by leaving this parameter at the default value of FALSE.<br />
COMPATIBLE [variable] <release dependent><br />
Setting this variable guarantees that the DBMS will remain compatible with the specified release. Some features might have to be limited for the compatibility to be maintained.<br />
COMPATIBLE_NO_RECOVERY [variable] <release dependent><br />
This parameter works like the COMPATIBLE parameter except that the earlier version (specified as the parameter) might not work on the current database if recovery is necessary.<br />
CONTROL_FILE_RECORD_KEEP_TIME [0-365] <7><br />
This parameter specifies the minimum age (in days) that a record in the control file must be kept before it can be reused.<br />
CONTROL_FILES [1..8 filenames] <OS dependent><br />
This parameter specifies the path names of one to eight control files. It is recommended that there always be more than one control file and that they exist on different physical devices.<br />
CORE_DUMP_DEST [directory name] <ORACLE_HOME/DBS/><br />
This parameter specifies the directory where core files are dumped.<br />
DB_DOMAIN [extension components of a global db name] <WORLD><br />
This parameter specifies the extension components of the global database name consisting of valid identifiers separated by periods (for example, texas.us.widgets.com). This allows multiple divisions to each have an ACCOUNTING database that is uniquely identified by the addition of the domain.<br />
DBLINK_ENCRYPT_LOGIN [TRUE/FALSE] <FALSE><br />
When you connect to another server, Oracle encrypts the password. If the value of DBLINK_ENCRYPT_LOGIN is FALSE and the connection fails, Oracle tries to connect again with a nonencrypted password. If DBLINK_ENCRYPT_LOGIN is TRUE and the connection fails, Oracle does not attempt to reconnect.<br />
DB_FILES [min: MAXDATAFILES, max OS dependent] <OS dependent><br />
This parameter specifies the maximum number of database files that can be open. This value can be reduced if you want to reclaim space in the SGA. No performance degradation is incurred by leaving this value high, just additional memory usage in the SGA.<br />
DB_FILE_DIRECT_IO_COUNT [OS Dependent] <64><br />
This parameter specifies the number of blocks to be used for I/O operations done by backup, restore, or direct path read/write functions.<br />
DB_NAME [valid name] <NULL><br />
This parameter provides a string of up to eight characters in length that specifies the name of the database. The following characters are valid:<br />
· Alphabetic characters<br />
<br />
· Numbers<br />
<br />
· Underscore (_)<br />
<br />
· Pound sign (#)<br />
<br />
· Dollar sign ($)<br />
No other characters can be used. Double quotation marks are removed and cannot be part of the name. The characters used in the DB_NAME parameter are case insensitive, so SALES, Sales, and sales are equal.<br />
ENQUEUE_RESOURCES [10..65535] <derived><br />
This parameter specifies the number of resources that can be locked by the lock manager. The default value is derived from PROCESSES and is usually sufficient. The value is derived from this formula:<br />
PROCESSES <= 3; default values = 20
PROCESSES 4-10; default value = ((PROCESSES - 3) * 5) + 20
PROCESSES > 10; default value = ((PROCESSES - 10) * 2) + 55<br />
If you use a large number of tables, you might have to increase this value. This value should never exceed DML_LOCKS + DDL_LOCKS + 20 (overhead).<br />
GLOBAL_NAMES [TRUE/FALSE] <FALSE><br />
This parameter determines whether a database link is required to have the same name as the database to which it connects. Oracle recommends setting this parameter to TRUE to ensure the use of consistent naming conventions for databases and links.<br />
IFILE [parameter filename] <NULL><br />
This parameter embeds another parameter file into the current parameter file. This can be very useful to separate specific changes from the general changes that you often make. The parameter also allows you to separate different types of parameters such as parallel options.<br />
INIT_SQL_FILES [SQL filename] <NULL><br />
This parameter lists the names of SQL files that should be run immediately after database creation. This parameter can be used to automatically create the data dictionary.<br />
JOB_QUEUE_INTERVAL [1..3600] <60><br />
This parameter specifies, in seconds, the interval between wake-ups of the SNP background process. The processes run jobs that have been queued.<br />
JOB_QUEUE_KEEP_CONNECTIONS [1..10] <0><br />
This parameter specifies the number of SNP background processes per instance.<br />
JOB_QUEUE_PROCESSES [TRUE/FALSE] <FALSE><br />
This parameter specifies whether remote connections should be shut down after remote jobs have finished executing.<br />
LICENSE_MAX_SESSIONS [0..number of session licenses] <0><br />
LICENSE_MAX_USERS sets the maximum number of concurrent user sessions allowed. When this limit is reached, only users with RESTRICTED SESSION privilege can connect to the server. A zero value indicates that this constraint is not enforced. Either LICENSE_MAX_USERS or LICENSE_MAX_SESSIONS should be set, not both.<br />
LICENSE_MAX_USERS [0..number of user licenses] <0><br />
LICENSE_MAX_USERS sets the maximum number of concurrent users that can simultaneously access the database. When this limit is reached, no more user sessions can be created. A zero value indicates that this constraint is not enforced. Either LICENSE_MAX_USERS or LICENSE_MAX_SESSIONS should be set, not both.<br />
LICENSE_SESSIONS_WARNING [0..LICENSE_MAX_SESSIONS] <0><br />
Sets a warning limit so that the administrator can be aware that the LICENSE_MAX_SESSIONS limit might soon be reached. After LICENSE_SESSIONS_WARNING number of users have connected, a message is written to the alert log for each additional user connecting.<br />
LOCAL_LISTENER [string] <Listener Identifier><br />
This parameter identifies local Net8 listeners.<br />
LOG_ARCHIVE_DEST [valid path or device name] <OS dependent><br />
When running in ARCHIVELOG mode, this text value specifies the default location and root of the file or tape device to use when archiving redo log files. Archiving to tape is not supported under all operating systems.<br />
LOG_ARCHIVE_DUPLEX_DEST [valid path] <NULL><br />
This parameter specifies a second archive destination for duplexed archiving.<br />
LOG_ARCHIVE_FORMAT [valid filename] <OS dependent><br />
This parameter uses a text string and variables to specify the default filename format of the archive log files. This string is appended to the LOG_ARCHIVE_DEST parameter name. The following variables can be used in the string:<br />
· %s--Log sequence number.<br />
<br />
· %t--Thread number. Using uppercase letters (%S, %T) causes the value to be fixed length, padded to the left with zeros. A good value is similar to the following:<br />
LOG_ARCHIVE_FORMAT = `log%S_%T.arc'<br />
<br />
LOG_ARCHIVE_MIN_SUCCEED_DEST [1..2] <1><br />
This parameter specifies the minimum number of archive log destinations that must succeed.<br />
LOG_ARCHIVE_START [TRUE/FALSE] <FALSE><br />
When running in ARCHIVELOG mode, LOG_ARCHIVE_START specifies whether archiving should be started up automatically at instance startup. A setting of TRUE indicates that archiving is automatic; FALSEindicates that archiving is manual.<br />
LOG_BLOCK_CHECKSUM [TRUE/FALSE] <FALSE><br />
Setting this parameter to TRUE causes each log block to be given a checksum. This checksum is written into the header of each block.<br />
LOG_CHECKPOINTS_TO_ALERT [TRUE/FALSE] <FALSE><br />
This parameter specifies whether you want to log the checkpoints to the alert log. This can be useful in verifying the frequency of checkpoints.<br />
LOG_FILES [2..255] <255><br />
This parameter specifies the maximum number of redo log files that can be opened at instance startup. Reducing this value can save some space in the SGA. If this value is set higher than the value ofMAXLOGFILES used at database creation, it does not override MAXLOGFILES.<br />
MAX_DUMP_FILE_SIZE [0..unlimited] <500 blocks><br />
This parameter specifies the maximum size in OS blocks of any trace file written. Set this if you are worried that trace files might consume too much space. MAX_ENABLED_ROLES [0..48] <20> This parameter specifies the maximum number of database roles (including subroles) that a user can enable.<br />
MAX_ROLLBACK_SEGMENTS [1..65536] <30><br />
This parameter specifies the maximum number of rollback segments that can be online for one instance.<br />
OBJECT_CACHE_MAX_SIZE_PERCENT [0%..OS Dependent] <10%><br />
This parameter specifies the percentage of the optimal cache size beyond which the Session object cache size can grow.<br />
OBJECT_CACHE_OPTIMAL_PERCENT [10KB..OS Dependent] <100KB><br />
This parameter specifies the optimal size of the Session object cache.<br />
OPEN_CURSORS [1..OS limit] <50><br />
This parameter specifies the maximum number of open cursors that a single user process can have open at once.<br />
OPEN_LINKS [0..255] <4><br />
This parameter specifies the maximum number of concurrent open connections to remote database processes per user process. This value should exceed the maximum number of remote systems accessed within any single SQL statement.<br />
PARTITION_VIEW_ENABLED [TRUE/FALSE] <FALSE><br />
If set to TRUE, the optimizer skips unnecessary table accesses in a partition view.<br />
PLSQL_V2_COMPATIBILITY [TRUE/FALSE] <FALSE><br />
This parameter sets the compatibility level for PL/SQL.<br />
PROCESSES [6 to OS dependent] <50><br />
This parameter specifies the maximum number of OS user processes that connect to the Oracle instance. This number must take into account the background processes and the login process that started the instance. Be sure to add an extra six processes for the background processes.<br />
REMOTE_DEPENDENCIES_MODE [TIMESTAMP/SIGNATURE] <TIMESTAMP><br />
This parameter specifies how dependencies on remote stored procedures are to be handled by the database.<br />
REMOTE_LOGIN_PASSWORDFILE [NONE/SHARED/EXCLUSIVE] <NONE><br />
This parameter specifies whether Oracle checks for a password file. A value of NONE indicates that users are authenticated through the operating system. A value of EXCLUSIVE indicates that the password file can be used only by one database and can contain names other than SYS and INTERNAL. Setting this parameter to SHARED allows more than one database to use this password file, but only SYS andINTERNAL are recognized by this password file.<br />
REPLICATION_DEPENDENCY_TRACKING [TRUE/FALSE] <TRUE><br />
This parameter specifies that dependency tracking for read/write operations to the database is turned on.<br />
RESOURCE_LIMIT [TRUE/FALSE] <FALSE><br />
A value of FALSE disables the enforcement of resource limits such as sessions, CPU time, and so on. This disables the enforcement of those limits regardless of how they are set.<br />
SESSIONS [number] <1.1 * PROCESSES><br />
This parameter specifies the total number of user and system sessions. Because recursive sessions might occur, this number should be set slightly higher than PROCESSES. DDL_LOCKS is derived from this parameter.<br />
SHADOW_CORE_DUMP [FULL/PARTIAL] <FULL><br />
This parameter specifies whether the SGA is included in core dumps.<br />
SNAPSHOT_REFRESH_INTERVAL [1..3600] <60><br />
This parameter specifies the number of seconds between wake-ups for the instance's snapshot refresh process.<br />
SNAPSHOT_REFRESH_KEEP_CONNECTION [TRUE/FALSE] <FALSE><br />
This parameter specifies whether the snapshot refresh process should keep remote connections after the refresh. If set to FALSE, the remote database connections are closed after the refreshes occur.<br />
SNAPSHOT_REFRESH_PROCESS [0..10] <0><br />
This parameter specifies the number of snapshot refresh processes per instance. You must set this value to 1 or higher for automatic refreshes. One snapshot refresh process is usually sufficient.<br />
SINGLE_PROCESS [TRUE/FALSE] FALSE<br />
If SINGLE_PROCESS is set to TRUE, the database instance is brought up in a single-user mode. A value of FALSE indicates that the database is brought up in a multiprocess mode.<br />
TEMPORARY_TABLE_LOCKS [0..OS dependent] <SESSIONS><br />
TEMPORARY_TABLE_LOCKS specifies the number of temporary tables that can be created in the temporary segment space. A temporary table lock is required whenever a sort occurs that cannot be held in memory (that is, the sort exceeds SORT_AREA_RETAINED_SIZE). If your application contains a large number of ORDER BY clauses or if you perform a large number of index sorts, you might want to increase this number.<br />
TRANSACTIONS [number] <1.1 * PROCESSES><br />
This parameter specifies the maximum number of concurrent transactions in the instance. The default value is greater than PROCESSES to provide for recursive transactions. A larger value increases the size of the SGA. If you increase the number of transactions allowed in the system, you might also want to increase the number of rollback segments available.<br />
TRANSACTIONS_PER_ROLLBACK_SEGMENT [1..OS dependent] <30><br />
This value specifies the maximum number of concurrent transactions allowed per rollback segment. You can calculate the minimum number of rollback segments enabled at startup with this formula:<br />
Rollback Segments = TRANSACTIONS / TRANSACTIONS_PER_ROLLBACK_SEGMENT<br />
Performance can be improved if there is less contention on rollback segments. In a heavily used system, you might want to reduce TRANSACTIONS_PER_ROLLBACK_SEGMENT to decrease this contention.<br />
USER_DUMP_DEST [valid path name] <OS dependent><br />
USER_DUMP_DEST specifies the path to where the debugging trace files are written.<br />
UTL_FILE_DIR [valid directory]<br />
This parameter specifies directories that are permitted for PL/SQL file I/O.<br />
Recovery Manager<br />
These parameters are used in conjunction with the Recovery Manager.<br />
BACKUP_DISK_IO_SLAVES [0..15] <0><br />
This parameter defines the number of I/O slaves used by the Recovery Manager to back up, copy, or restore.<br />
BACKUP_TAPE_IO_SLAVES [TRUE/FALSE] <FALSE><br />
This parameter specifies whether I/O slaves are used by the Recovery Manager for tape operations.<br />
DB_FILE_NAME_CONVERT [string]<br />
This parameter converts the filename of a new datafile on the primary database to a filename on the standby database.<br />
LOG_FILE_NAME_CONVERT [string]<br />
This parameter converts the filename of a new log file on the primary database to a filename on the standby database.<br />
TAPE_ASYNCH_IO [TRUE/FALSE] <TRUE><br />
This parameter specifies that I/O to sequential devices are asynchronous. This should be left enabled and not altered.<br />
Multithreaded Server<br />
These parameters are used if you are using the multithreaded server process.<br />
MTS_DISPATCHERS ["protocol, number"] <NULL><br />
This parameter specifies the configuration of the dispatcher process(es) created at startup time. The value of this parameter is a quoted string of two values separated by a comma. The values are the network protocol and the number of dispatchers. Each protocol requires a separate specification. This parameter can be specified multiple times. Here is an example of two dispatcher definitions:<br />
MTS_DISPATCHERS = "tcp, 2"<br />
MTS_DISPATCHERS = "ipx, 1"<br />
MTS_LISTENER_ADDRESS [configuration] <NULL><br />
This parameter specifies the configuration of the listener process addresses. There must be a listener process address for each protocol used in the system. Addresses are specified as the SQL*Net description of the connection address.<br />
Because each connection is required to have its own address, this parameter might be specified several times. Here is an example:<br />
MTS_LISTENER_ADDRESS = "(ADDRESS=(PROTOCOL=tcp)(HOST=hostname)(PORT=7002))"<br />
MTS_LISTENER_ADDRESS = "(ADDRESS=(PROTOCOL=ipx)()())"<br />
MTS_MAX_DISPATCHERS [OS dependent] <5><br />
This parameter specifies the maximum number of dispatcher processes allowed to run simultaneously.<br />
MTS_MAX_SERVERS [OS dependent] <20><br />
This parameter specifies the maximum number of shared server processes allowed to run simultaneously.<br />
MTS_MULTIPLE_LISTENERS [TRUE/FALSE] <FALSE><br />
This parameter is obsolete.<br />
MTS_RATE_LOG_SIZE [DEFAULTS/EVENT_LOOPS/MESSAGES/SERVER_BUFFERS/CLIENT_BUFFERS/TOTAL_BUFFERS/IN_CONNECTS/OUT_CONNECTS/RECONNECTS] <10><br />
This parameter specifies the sample size used to calculate dispatcher-rate statistics.<br />
MTS_RATE_SCALE [DEFAULTS/EVENT_LOOPS/MESSAGES/SERVER_BUFFERS/CLIENT_BUFFERS/TOTAL_BUFFERS/IN_CONNECTS/OUT_CONNECTS/RECONNECTS] <misc><br />
This parameter specifies the scale at which dispatcher-rate statistics are reported.<br />
MTS_SERVERS [OS dependent] <0><br />
This parameter specifies the number of server processes created at instance startup.<br />
MTS_SERVICE [name] <DB_NAME><br />
This parameter specifies the name of the service to be associated with the dispatcher. Using this name in the CONNECT string allows users to connect using the dispatcher. The name should be unique. Do not specify this name in quotes. It is usually a good idea to make this name the same as the instance name. Because the dispatcher is tried first, if it is not available, the CONNECT string can still connect the user into the database through a normal database connection.<br />
Distributed Option<br />
These parameters are meaningful only when you use the distributed option.<br />
COMMIT_POINT_STRENGTH [0..255] <OS dependent><br />
This value is used to determine the commit point site when executing a distributed transaction. The site with the highest value for COMMIT_POINT_STRENGTH is the commit point site. The site with the largest amount of critical data should be the commit point site.<br />
DISTRIBUTED_LOCK_TIMEOUT [1..unlimited] <60 seconds><br />
DISTRIBUTED_LOCK_TIMEOUT specifies, in seconds, how long distributed transactions should wait for locked resources.<br />
DISTRIBUTED_RECOVERY_CONNECTION_HOLD_TIME [1..1800] <200 seconds><br />
DISTRIBUTED_RECOVERY_CONNECTION_HOLD_TIME specifies, in seconds, how long to hold a remote connection open after a distributed transaction fails. A larger value holds the connection longer but also continues to use local resources even though the connection might have been severed. Any value larger than 1,800 seconds interferes with the reconnection and recovery background processes and will never drop a failed connection.<br />
DISTRIBUTED_TRANSACTIONS [0..TRANSACTIONS] <OS dependent><br />
DISTRIBUTED_TRANSACTIONS specifies the maximum number of distributed transactions that the database can process concurrently. This value cannot exceed the value of TRANSACTIONS. If you are having problems with distributed transactions because network failures are causing many in-doubt transactions, you might want to limit the number of distributed transactions.<br />
If DISTRIBUTED_TRANSACTIONS is set to 0, no distributed transactions are allowed and the RECO process does not start at instance startup.<br />
MAX_TRANSACTION_BRANCHES [1..32] <8><br />
This parameter controls the number of branches in a distributed transaction.<br />
REMOTE_OS_AUTHENT [TRUE/FALSE] <FALSE><br />
If this parameter is set to TRUE, it allows authentication to remote systems with the value of OS_AUTHENT_PREFIX.<br />
REMOTE_OS_ROLES [TRUE/FALSE] <FALSE><br />
If this parameter is set to TRUE, it allows remote clients to have their roles managed by the OS. If REMOTE_OS_ROLES is FALSE, roles are managed and identified by the database for the remote system.<br />
Parallel Server Parameters<br />
These parameters are used only in conjunction with the Oracle Parallel Server option.<br />
ALLOW_PARTIAL_SN_RESULTS [TRUE/FALSE] <FALSE><br />
This parameter allows partial results to be returned on queries to global performance tables even if a slave could not be allocated.<br />
CACHE_SIZE_THRESHOLD [number] <0.1 * DB_BLOCK_BUFFERS><br />
This parameter specifies the maximum size of a cached partition table split among the caches of multiple instances. If the partition is larger than this value, the table is not split among the caches.<br />
DELAYED_LOGGING_BLOCK_CLEANOUTS [TRUE/FALSE] <TRUE><br />
This parameter enables the delayed block cleanout feature. This can reduce OPS pinging.<br />
FREEZE_DB_FOR_FAST_INSTANCE_RECOVERY [TRUE/FALSE]<br />
This parameter specifies that the entire database freeze in order to speed recovery.<br />
GC_DEFER_TIME [integer] <0><br />
This parameter specifies the time the server waits (in hundredths of a second) before responding to a forced-write request for hot blocks.<br />
GC_DB_LOCKS [0..unlimited] <0><br />
This parameter specifies the number of PCM locks allocated. The value of GC_DB_LOCKS should be at least one greater than the sum of the locks specified with the parameter GC_FILES_TO_LOCKS.<br />
GC_FILES_TO_LOCKS [file_number=locks:filename=locks] <NULL><br />
This parameter supplies a list of filenames, each specifying how many locks should be allocated for that file. Optionally, the number of blocks and the value EACH can be added to further specify the allocation of the locks.<br />
GC_LCK_PROCS [0..10] <1><br />
This parameter specifies the number of lock processes (LCK0 to LCK9) to create for the instance. The default value of 1 is usually sufficient unless an unusually high number of locks are occurring.<br />
GC_RELEASABLE_LOCKS [0..DB_BLOCK_BUFFERS] <DB_BLOCK_BUFFERS><br />
This parameter allocates space for fine-grain locking.<br />
GC_ROLLBACK_LOCKS [number] <20><br />
This parameter specifies the number of distributed locks available for each rollback segment. The default value is usually sufficient.<br />
GC_ROLLBACK_SEGMENTS [number] <20><br />
GC_ROLLBACK_SEGMENTS specifies the maximum number of rollback segments systemwide. This includes all instances in the parallel server system, including the SYSTEM rollback segment.<br />
GC_SAVE_ROLLBACK_LOCKS [number] <20><br />
This parameter specifies the number of distributed locks reserved for deferred rollback segments. These deferred rollback segments contain rollback entries for segments taken offline.<br />
GC_SEGMENTS [number] <10><br />
This parameter specifies the maximum number of segments that might have space-management activities simultaneously performed by different instances.<br />
GC_TABLESPACES [number] <5><br />
This parameter specifies the maximum number of tablespaces that can be simultaneously brought online or offline.<br />
INSTANCE_GROUPS [string]<br />
This parameter assigns the current instance to this instance group.<br />
INSTANCE_NUMBER [1..OS dependent] <Lowest Available Number><br />
This parameter specifies a unique number that maps the instance to a group of free space lists.<br />
LM_LOCKS [512..Limited by Instance Size] <12000><br />
This parameter specifies the number of locks that are configured for the lock manager.<br />
LM_PROCS [36..PROCESSES+instances+safety factor] <64+instances><br />
This parameter represents the number of the PROCESSES parameter plus the number of instances.<br />
LM_RESS [256..Limited by Instance Size] <6000><br />
This parameter controls the number of resources that can be locked by each lock-manager process.<br />
LOCK_NAME_SPACE [string]<br />
This parameter specifies the name space that the distributed lock manager (DLM) uses to generate lock names.<br />
MAX_COMMIT_PROPAGATION_DELAY [0..90000] <90000><br />
This parameter specifies the maximum amount of time that can pass before the SCN (System Change Number) is changed by the DBWR. This value helps in certain conditions where the SCN might not be refreshed often enough because of a high load from multiple instances.<br />
OPEN_LINKS_PER_INSTANCE [0..UB4MAXVAL] <4><br />
This parameter specifies the maximum number of migratable open connections.<br />
OPS_ADMIN_GROUP [group name] <all instances><br />
This parameter allows instances to be grouped for monitoring and administration.<br />
PARALLEL_DEFAULT_MAX_INSTANCES [0..instances] <OS dependent><br />
This parameter specifies the default number of instances to spit a table among for parallel query processing. This value is used if the INSTANCES DEFAULT is specified in the table/cluster definition.<br />
PARALLEL_INSTANCE_GROUP [string] <group><br />
This parameter specifies the parallel instance group to be used for spawning parallel query slaves.<br />
PARALLEL_SERVER [TRUE/FALSE] <FALSE><br />
Setting this to TRUE enables the Parallel Server option.<br />
PARALLEL_TRANSACTION_RESOURCE_TIMEOUT [0..OS Dependent] <300><br />
This parameter specifies the maximum amount of time (seconds) that can pass before a session executing a parallel operation will time-out while waiting on a resource held by another session. THREAD [0..max threads] <0> This parameter specifies the number of the redo thread to be used by this instance. Any number can be used, but the value must be unique within the cluster.<br />
Security<br />
These parameters help set up system security; manipulate them to obtain the best mix of efficiency and security.<br />
AUDIT_FILE_DEST [dir_name] <$ORACLE_HOME/RDBMS/AUDIT><br />
This parameter specifies the directory where audit files are stored.<br />
AUDIT_TRAIL [NONE,DB,OS]<br />
The AUDIT_TRAIL parameter enables auditing to the table SYS$AUD$. Auditing causes a record of database and user activity to be logged. Because auditing causes overhead, it limits performance. The amount of overhead and the effect on performance is determined by what and how much is audited. Once AUDIT_TRAIL is enabled, auditing is turned on by the Oracle command AUDIT.<br />
O7_DICTIONARY_ACCESSIBILITY [TRUE/FALSE] <TRUE><br />
If set to TRUE (default), access to the SYS schema is allowed. This is Oracle7 behavior.<br />
OS_AUTHENT_PREFIX [] <OPS$><br />
This is the value concatenated to the beginning of the user's OS login account to give a default Oracle account name. The default value of OPS$ is OS dependent and is provided for backward compatibility with previous Oracle versions. Typically, you use the default or set the value to "" (NULL) to eliminate prefixes altogether.<br />
OS_ROLES [TRUE/FALSE] <FALSE><br />
Setting this parameter to TRUE allows the OS to have control over the username's roles. If set to FALSE, the username's roles are controlled by the database.<br />
SQL92_SECURITY [TRUE/FALSE] <FALSE><br />
This parameter specifies whether the table-level SELECT privileges are needed to execute an update or delete that reference's table-column values.<br />
TRANSACTION_AUDITING [TRUE/FALSE] <TRUE><br />
This parameter specifies that additional transaction information is included in a special redo record.<br />
Trusted Oracle7<br />
The following parameters apply to the Trusted Oracle7 option.<br />
AUTO_MOUNTING [TRUE/FALSE] <TRUE><br />
When set to TRUE, this parameter specifies that a secondary database is mounted by the primary database whenever a user connected to the primary database requests data from the secondary database.<br />
DB_MOUNT_MODE [NORMAL/READ_COMPATIBLE] <NORMAL><br />
This parameter specifies the access mode to which the database is mounted at instance startup. A value of NORMAL starts the database in normal read-write mode; READ_COMPATIBLE starts the database in read-write mode with the added feature of supporting concurrent mounting by one or more read-secure instances.<br />
LABEL_CACHE_SIZE [number> 50] <50><br />
This parameter specifies the cache size for dynamic comparison of labels. This number should be greater than the label-category combinations in the OS and should never be less than 50.<br />
MLS_LABEL_FORMAT [valid label format] <sen><br />
This parameter specifies the format used to display labels. The default value sen specifies sensitive.<br />
OPEN_MOUNTS [0..255] <5><br />
This parameter specifies the maximum number of databases that an instance can simultaneously mount in OS MAC mode. This value should be large enough to handle all the primary and secondary databases you might mount.<br />
National Language Support<br />
The following parameters are used in the configuration of National Language Support features.<br />
NLS_CURRENCY [character string] <derived from NLS_TERRITORY><br />
This parameter specifies the string to use as the local currency symbol for the L number format element.<br />
NLS_DATE_FORMAT [format mask] <derived from NLS_TERRITORY><br />
This parameter defines the default date format to use with the TO_CHAR and TO_DATE functions. The value of this parameter is any valid date format mask. Here is an example:<br />
NLS_DATE_FORMAT = `DD/MM/YYYY'<br />
NLS_DATE_LANGUAGE [NLS_LANGUAGE value] <value for NLS_LANGUAGE><br />
This parameter determines the language to use for the day and month names and date abbreviations (AM, PM, AD, BC).<br />
NLS_ISO_CURRENCY [valid NLS_TERRITORY value] <derived from NLS_TERRITORY><br />
This parameter defines the string to use as the international currency symbol for the C number format element.<br />
NLS_LANGUAGE [NLS_LANGUAGE value] <OS dependent><br />
This parameter defines the default language of the database. This specifies the language to use for messages, the language of day and month names, symbols to be used for AD, BC, A.M. and P.M., and the default sorting mechanisms.<br />
NLS_NUMERIC_CHARACTERS [two characters] <derived from NLS_TERRITORY><br />
This parameter defines the characters to be used as the group separator and decimal. The group separator is used to separate the integer groups (that is, hundreds, thousands, millions, and so on). The decimal separator is used to distinguish between the integer and decimal portion of the number. Any two characters can be used but they must be different. The parameter is specified by two characters within single quotes. To set the group separator to , (comma) and the decimal separator to . (period), use the following statement:<br />
NLS_NUMERIC_CHARACTERS = `,.'<br />
NLS_SORT [BINARY or named linguistic sort] <derived from NLS_LANGUAGE><br />
If this parameter is set to BINARY, the collating sequence for ORDER_BY is based on the numeric values of the characters. A linguistic sort decides the order based on the defined linguistic sort. A binary sort is much more efficient and uses much less overhead.<br />
NLS_TERRITORY [territory name] <OS dependent><br />
This parameter specifies the name of the territory whose conventions are used for day and week numbering. The parameter also provides defaults for other NLS parameters.<br />
Oracle Corporation has invested millions of dollars in making the cost-based SQL optimizer (CBO) one of the most sophisticated tools ever created. The job of the CBO is to always choose the most optimal execution plan for any SQL statement.<br />
<br />
However, there are some things that the CBO cannot detect, which is where the DBA comes in. The types of SQL statements, the speed of the disks and the load on the CPUs, all affect the "best" execution plan for a SQL statement. For example, the best execution plan at 4:00 A.M. when 16 CPUs are idle may be quite different from the same query at 3:00 P.M. when the system is 90 percent utilized.<br />
<br />
Despite the name "Oracle", the CBO is not psychic, and Oracle can never know, a priori, the exact load on the Oracle system. Hence the Oracle professional must adjust the CBO behavior periodically. Most Oracle professionals make these behavior adjustments using the instance-wide CBO behavior parameters such asoptimizer_index_cost_adj and optimizer_index_caching.<br />
<br />
However, Oracle does not recommend changing the default values for many of these CBO settings because the changes can affect the execution plans for thousands of SQL statements.<br />
<br />
Here are some of the major adjustable parameters that influence the behavior of the CBO:<br />
· optimizer_index_cost_adj: This parameter alters the costing algorithm for access paths involving indexes. The smaller the value, the cheaper the cost of index access.<br />
· optimizer_index_caching: This is the parameter that tells Oracle how much of your index is likely to be in the RAM data buffer cache. The setting foroptimizer_index_caching affects the CBO's decision to use an index for a table join (nested loops), or to favor a full-table scan.<br />
· optimizer_max_permutations: This controls the maximum number of table join permutations allowed before the CBO is forced to pick a table join order. For a six-way table join, Oracle must evaluate 6-factorial, or 720, possible join orders for the tables.<br />
· db_file_multiblock_read_count: When set to a high value, the CBO recognizes that scattered (multi-block) reads may be less expensive than sequential reads. This makes the CBO friendlier to full-table scans.<br />
· parallel_automatic_tuning: When set "on", full-table scans are parallelized. Because parallel full-table scans are very fast, the CBO will give a higher cost to index access, and be friendlier to full-table scans.<br />
· hash_area_size (if not using pga_aggregate_target): The setting forhash_area_size parameter governs the propensity of the CBO to favor hash joins over nested loop and sort merge table joins.<br />
· sort_area_size (if not using pga_aggregate_target): The sort_area_sizeinfluences the CBO when deciding whether to perform an index access or a sort of the result set. The higher the value for sort_area_size, the more likely that a sort will be performed in RAM, and the more likely that the CBO will favor a sort over pre-sorted index retrieval.<br />
<br />
The parameter optimizer_index_cost_adj controls the CBO's propensity to favor index scans over full-table scans. As we will see, in a dynamic system, the "ideal" value foroptimizer_index_cost_adj may change radically in just a few minutes, as the type of SQL and load on the database changes.<br />
<br />
Using optimizer_index_cost_adj<br />
The optimizer_index_cost_adj is the most important parameter of all, and the default setting of 100 is incorrect for most Oracle systems. However, for OLTP systems, resetting this parameter to a smaller value (between 10 and 30) may result in huge performance gains.<br />
<br />
Is it possible to query the Oracle environment and intelligently determine the optimal setting for optimizer_index_cost_adj? Let's examine the issue.<br />
<br />
The optimizer_index_cost_adj parameters default to a value of 100, and can range in value from 1 to 10,000. A value of 100 means that equal weight is given to index vs. multiblock reads. In other words, optimizer_index_cost_adj can be thought of as a "how much do I like full-table scans?" parameter.<br />
<br />
With a value of 100, the CBO likes full-table scans and index scans equally, and a number lower than 100 tells the CBO that index scans are faster than full-table scans. However, even with a super-low setting (optimizer_index_cost_adj=1), the CBO will still choose full-table scans for no-brainers, like tiny tables that reside on two blocks.<br />
<br />
If you are having slow performance because the CBO first_rows optimizer mode is favoring too many full-table scans, you can reset the optimizer_index_cost_adjparameter to immediately tune all of the SQL in your database to favor index scans over full-table scans. This is a "silver bullet" that can improve the performance of an entire database in cases where the database is OTLP and you have verified that the full-table scan costing is too low.<br />
<br />
The parameter is an initialization parameter that can be enabled at the session level by using the alter session set optimizer_index_cost_adj = nn syntax. This parameter lets you tune the optimizer behavior for access path selection to be more or less index-friendly, and it is very useful when you feel that the default behavior for the CBO favors full-table scans over index scans.<br />
<br />
When the CBO has minimal or too small a statistics sample, the CBO sometimes falsely determines that the cost of full-table scan is less than the cost of an index access. The optimizer_index_cost_adj parameter is a great approach to whole-system SQL tuning, but you will need to evaluate the overall effect by slowly resetting the value down from 100 and observing the percentage of full-table scans.<br />
<br />
You can also slowly bump down the value of optimizer_index_cost_adj when you bounce the database and then either use the access.sql scripts or reexamine SQL from the STATSPACK stats$sql_summary table to see the net effect of index scans on the whole database.<br />
<br />
As you can see, the suggested starting value for optimizer_index_cost_adj may be too high because 98 percent of the data waits are on index (sequential) block access. How we can "weight" this starting value for optimizer_index_cost_adj to reflect the reality that this system has only two percent waits on full-table scan reads (a typical OLTP system with few full-table scans). As a practical matter, we never want an automated value for optimizer_index_cost_adj to be less than one or more than 100.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-80549255531097502952009-10-06T08:32:00.000-07:002009-10-06T08:32:01.185-07:00How the SIZE Parameters Affect Oracle Business Intelligence Performance§SORT_MEMORY_SIZE<br />
The size specified by SORT_MEMORY_SIZE sets the upper limit on how large the sorting buffer can be in the Oracle Business Intelligence Server. When this limit is exceeded, data is sorted in allotments of the size set by SORT_MEMORY_SIZE and the sorted sets are merged together. For example, suppose SORT_MEMORY_SIZE is set to 4 MB and the size of the data to be sorted is 32 MB. The server performs the sort once per each 4 MB of data, for a total of eight sort operations, and then merge the results into a single result set. This technique allows the Oracle Business Intelligence Server to sort data of indefinite size.<br />
The merge process itself is generally not costly in terms of resources, but it does include a read and write of each result set in a temporary file. To reduce the time this takes, increase the SORT_MEMORY_SIZE. This parameter can be tuned over time by taking into consideration the data size of the query and the number of concurrent users.<br />
§ SORT_BUFFER_INCREMENT_SIZE<br />
This parameter defines the increment by which SORT_MEMORY_SIZE should be reached. For example, suppose SORT_MEMORY_SIZE is set to 4 MB and the data to be sorted is just one megabyte. As data is fed into the sort routine, the size of the sort buffer increases only by the increment size, rather than the full size allowed by SORT_MEMORY_SIZE. This mechanism allows the Oracle Business Intelligence Server to sort smaller result sets efficiently without wasting memory.<br />
§ VIRTUAL_TABLE_PAGE_SIZE<br />
Several operations—sort, join, union and database fetch—can require memory resources beyond those available to the Oracle Business Intelligence Server. To manage this condition, the server uses a virtual table management mechanism that provides a buffering scheme for processing these operations. When the amount of data exceeds the VIRTUAL_TABLE_PAGE_SIZE, the remaining data is buffered in a temporary file and placed in the virtual table as processing continues. This mechanism supports dynamic memory sizes and ensures that any row can be obtained dynamically for processing queries.<br />
When VIRTUAL_TABLE_PAGE_SIZE is increased, I/O operations are reduced. Complex queries may use 20 to 30 virtual tables, while simple queries may not even require virtual tables. The default size of 128 KB is a reasonable size when one considers that the size for virtual paging in Windows NT is 64 KB. This parameter can be tuned depending on the number of concurrent users and the average query complexity. In general, setting the size higher than 256 KB does not yield a corresponding increase in throughput due to the 64 KB size limit of Windows NT system buffers, as each I/O still goes through the system buffers.<br />
USE_LONG_MONTH_NAMES<br />
Specifies whether month names are returned as full names, such as JANUARY and FEBRUARY, or as three-letter abbreviations, such as JAN and FEB. Valid values are YES and NO. Specify YES to have month names returned as full names or NO to have months names returned as three-letter abbreviations. The default value is NO.<br />
Example: USE_LONG_MONTH_NAMES = NO ;<br />
USE_LONG_DAY_NAMES<br />
Specifies whether day names are returned as full names, such as MONDAY and TUESDAY, or as three-letter abbreviations, such as MON and TUE. Valid values are YES and NO. Specify YES to have day names returned as full names or NO to have day names returned as three-letter abbreviations. The default value is NO.<br />
Example: USE_LONG_DAY_NAMES = NO ;<br />
UPPERCASE_USERNAME_FOR_INITBLOCK<br />
Specifies whether the users are authenticated with case sensitivity. The default value is NO (or false internally). When it is set to YES, then all user names are changed to uppercase for authentication purposes in the SiebelAnalytics.rpd file. Otherwise, case is maintained.<br />
Example: UPPERCASE_USERNAME_FOR_INITBLOCK = NO ;<br />
AGGREGATE_PREFIX<br />
Specifies the Domain Server Name for Aggregate Persistence. The prefix must be between 1 and 8 characters long and should not have any special characters ('_' is allowed).<br />
Example: AGGREGATE_PREFIX = "SA_" ;KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-54224776034987119042009-10-06T08:30:00.000-07:002009-10-06T08:30:34.647-07:00Using the New Optimizer Statistics·The default value for the OPTIMIZER_MODE initialization parameter is ALL_ROWS.<br />
·Automatic Statistics Collection<br />
·Changes in the DBMS_STATS Package<br />
·Dynamic Sampling<br />
<br />
Oracle determines at compile time whether a query would benefit from dynamic sampling.<br />
Depending on the value of the OPTIMIZER_DYNAMIC_SAMPLING initialization parameter, a certain number of blocks are read by the dynamic sampling query to estimate statistics.<br />
OPTIMIZER_DYNAMIC_SAMPLING takes values from zero (OFF) to 10 (default is 2).<br />
<br />
·Table Monitoring<br />
If you use either the GATHER AUTO or STALE settings when you use the DBMS_STATS package, you don’t need to explicitly enable table monitoring in Oracle Database 10g; the MONITORING and NO MONITORING keywords are deprecated.<br />
Oracle uses the DBA_TAB_MODIFICATIONS view to determine which objects have stale statistics.<br />
Setting the STATISTICS_LEVEL to BASIC turns off the default table monitoring feature.<br />
<br />
·Collection for Dictionary Objects<br />
You can gather fixed object statistics by using the GATHER_DATABASE_STATS procedure and setting the GATHER_FIXED argument to TRUE (the default is FALSE).<br />
You can also use the new procedure:<br />
DBMS_STATS.GATHER_FIXED_OBJECTS_STATS('ALL')<br />
You must have the SYSDBA or ANALYZE ANY DICTIONARY system privilege to analyze any dictionary objects or fixed objects.<br />
<br />
To collect statistics for the real dictionary tables:<br />
·Use the DBMS_STATS.GATHER_DATABASE_STATS procedure, by setting the GATHER_SYS argument to TRUE. Alternatively, you can use the GATHER_SCHEMA_STATS ('SYS') option.<br />
<br />
·Use the DBMS_STATS.GATHER_DICTIONARY_STATS procedure.<br />
<br />
Using the SQL Tuning Advisor <br />
Providing SQL Statements to the SQL Tuning Advisor<br />
·Create a new set of statements as an input for the SQL Tuning Advisor.<br />
·The ADDM may often recommend high-load statements.<br />
·Choose a SQL statement that’s stored in the AWR.<br />
·Choose a SQL statement from the database cursor cache.<br />
<br />
How the SQL Tuning Advisor Works<br />
<br />
The optimizer will work in the new tuning mode wherein it conducts an in-depth analysis to come up with a set of recommendations, the rationale for them and the expected benefit if you follow the recommendations.<br />
When working in tuning mode, the optimizer is referred to as the Automatic Tuning Optimizer (ATO).<br />
<br />
The ATO performs the following tuning tasks:<br />
·Statistics analysis<br />
·SQL profiling<br />
·Access path analysis<br />
·SQL structure analysis<br />
Statistics Analysis<br />
ATO recommends collecting new statistics for specific objects, if required.<br />
SQL Profiling<br />
The ATO’s goal at this stage is to verify that its own estimates of factors like column selectivity and cardinality of database objects are valid.<br />
·Dynamic data sampling<br />
Using a sample of the data, the ATO can check if its own estimates for the statement in question are significantly off the mark.<br />
·Partial execution<br />
The ATO may partially execute a SQL statement, so it can check if whether a plan derived purely from inspection of the estimated statistics is actually the best plan.<br />
·Past execution history statistics<br />
The ATO may also use any existing history of the SQL statement’s execution to determine appropriate settings for parameters like OPTIMIZER_MODE.<br />
The output of this phase is a SQL Profile of the concerned SQL statement. If you create that SQL profile, it will be used later by the optimizer when it executes the same SQL statement in the normal mode. A SQL profile is simply a set of auxiliary or supplementary information about a SQL statement.<br />
Access Path Analysis<br />
The ATO analyzes the potential impact of using improved access methods, such as additional or different indexes.<br />
SQL Structure Analysis<br />
The ATO may also make recommendations to modify the structure, both the syntax and semantics, in your SQL statements.<br />
SQL Tuning Advisor Recommendations<br />
The SQL Tuning Advisor can recommend that you do the following:<br />
·Create indexes to speed up access paths<br />
·Accept a SQL profile, so you can generate a better execution plan<br />
·Gather optimizer statistics for objects with no or stale statistics<br />
·Rewrite queries based on the advisor’s advice<br />
Using the SQL Tuning Advisor<br />
Using the DBMS_SQLTUNE Package<br />
<br />
The DBMS_SQLTUNE package is the main Oracle Database 10g interface to tune SQL statements.<br />
Following are the required steps:<br />
1.Create a task. You can use the CREATE_TUNING_TASK procedure to create a task to tune either a single statement or several statements.<br />
execute :v_task := DBMS_SQLTUNE.CREATE_TUNING_TASK(sql_text=>'select count(*) from hr.employees,hr.dept')<br />
2.Execute the task. You start the tuning process by running the EXECUTE_TUNING_TASK procedure.<br />
SET LONG 1000<br />
SET LONGCHUNKSIZE 1000<br />
SET LINESIZE 100<br />
SELECT DBMS_SQLTUNE.REPORT_TUNING_TASK( :v_task) FROM DUAL;<br />
3.Get the tuning report. By using the REPORT_TUNING_TASK procedure.<br />
4.Use DROP_TUNING_TASK to drop a task, removing all results associated with the task.<br />
Managing SQL Profiles<br />
Use the DBMS_SQLTUNE.ACCEPT_SQL_PROFILE procedure to create a SQL profile based on the recommendations of the ATO.<br />
Managing SQL Tuning Categories<br />
·Any created SQL Profile will be assigned to a category defined by the parameter SQLTUNE_CATEGORY.<br />
·By default, SQLTUNE_CATEGORY has the value of DEFAULT.<br />
·You can change the SQL tuning category for all users with the following command:<br />
ALTER SYSTEM SET SQLTUNE_CATEGORY = PROD<br />
·To change a session’s tuning category, use the following command:<br />
ALTER SESSION SET SQLTUNE_CATEGORY = DEV<br />
You may also use the DBMS_SQLTUNE.ALTER_SQL_PROFILE procedure to change the SQL tuning category.<br />
Using the Database Control to Run the SQL Tuning Advisor<br />
Under the Performance tab, click the Advisor Central link and then click the SQL Tuning Advisor link.<br />
There are several possible sources for the tuning advisor’s SQL Tuning Set (STS) input:<br />
· high-load SQL statements identified by the ADDM<br />
· statements in the cursor cache<br />
· statements from the AWR<br />
· a custom workload<br />
· another new STS.<br />
<br />
Using the SQL Access Advisor <br />
The SQL Access Advisor primarily provides advice regarding the creation of indexes, materialized views, and materialized view logs, in order to improve query performance.<br />
Providing Input for the SQL Access Advisor<br />
There are four main sources of input for the advisor: SQL cache, user-defined workload, hypothetical workload, and STS from the AWR.<br />
Modes of Operation<br />
You can operate the SQL Access Advisor in two modes:<br />
Limited (partial)<br />
In this mode, the advisor will concern itself with only problematic or high cost SQL statements ignoring statements with a cost below a certain threshold.<br />
Comprehensive (full)<br />
In this mode, the advisor will perform a complete and exhaustive analysis of all SQL statements in a representative set of SQL statements, after considering the impact on the entire workload.<br />
You can also use workload filters to specify which kinds of SQL statements the SQL Access Advisor should select for analysis.<br />
Managing the SQL Access Advisor<br />
Using the DBMS_ADVISOR Package<br />
1. Create and manage a task, by using a SQL workload object and a SQL Access task.<br />
2. Specify task parameters, including workload and access parameters.<br />
3. Using the workload object, gather the workload.<br />
4. Using the SQL workload object and the SQL Access task, analyze the data.<br />
You can also use the QUICK_TUNE procedure to quickly analyze a single SQL statement:<br />
VARIABLE task_name VARCHAR2(255);<br />
VARIABLE sql_stmt VARCHAR2(4000);<br />
sql_stmt := 'SELECT COUNT(*) FROM customers WHERE cust_region=''TX''';<br />
task_name := 'MY_QUICKTUNE_TASK';<br />
DBMS_ADVISOR.QUICK_TUNE(DBMS_ADVISOR.SQLACCESS_ADVISOR, task_name, sql_stmt);<br />
Using the Database Control to Run the SQL Access Advisor<br />
Under the Performance tab, click the Advisor Central link and then click the SQL Access Advisor link.<br />
Note: Oracle creates the new indexes in the schema and tablespaces of the table on which they are created. If a user issues a query that leads to a recommendation to create a materialized view, Oracle creates the materialized view in that user’s schema and tablespace.<br />
<br />
Performance Pages in the Database Control <br />
The Database Home Page<br />
Three major tuning areas the OEM Database Control will show you: CPU and wait classes, top SQL statements, and top sessions in the instance.<br />
The Database Performance Page<br />
This page shows the three main items:<br />
Host<br />
The Host part of the page shows two important graphs:<br />
· Average Run Queue: This shows how hard the CPU is running.<br />
· Paging Rate: This shows the rate at which the host server is writing memory pages to the swap area on disk.<br />
Sessions waiting and working<br />
The sessions graph shows which active sessions are on the CPU and which are waiting for resources like locks, disk I/O, and so on.<br />
Instance throughput<br />
If your instance throughput is decreasing, along with an increasing amount of contention within the database, you should start looking into tuning your database.<br />
<br />
Indexing Enhancements <br />
Skipping Unusable Indexes<br />
In Oracle Database 10g, the SKIP_UNUSABLE_INDEXES parameter is a dynamic initialization parameter and its default value is TRUE. This setting disables error reporting of indexes and index partitions marked as UNUSABLE.<br />
Note: This setting does not disable error reporting for unusable indexes that are unique because allowing insert and update operations on the table might violate the corresponding constraint.<br />
Note: The database still records an alert message in the alert.log file whenever an index is marked as unusable.<br />
Using Hash-Partitioned Global Indexes<br />
· In Oracle 10g, you can create hash-partitioned global indexes. (Previous releases support only range-partitioned global indexes.)<br />
· You can hash-partition indexes on tables, partitioned tables, and index-organized tables.<br />
· This feature provides higher throughput for applications with large numbers of concurrent insertions.<br />
· If you have queries with range predicates, for example, hash partitioned indexes perform better than range-partitioned indexes.<br />
· You can’t perform the following operations on hash-partitioned global indexes: ALTER INDEX REBUILD, ALTER TABLE SPLIT INDEX PARTITION, ALTER TABLE MERGE INDEX PARTITITON, and ALTER INDEX MODIFY PARTITION.<br />
CREATE INDEX sales_hash<br />
on sales_items (sales_id) GLOBAL<br />
PARTITION BY HASH (sales_id) (<br />
partition p1 tablespace tbs_1,<br />
partition p2 tablespace tbs_2,<br />
partition p3 tablespace tbs_3)<br />
CREATE INDEX sales_hash<br />
on sales_items (sales_id) GLOBAL<br />
PARTITION BY HASH (sales_id)<br />
partitions 4<br />
store in (tbs_1,tbs_2,tbs_3,tbs_4)<br />
· To add a new index partition<br />
ALTER INDEX sales_hash ADD PARTITION p4<br />
TABLESPACE tbs_4 [PARALLEL]<br />
Notice the following for the previous command:<br />
o The newly added partition is populated with index entries rehashed from an existing partition of the index as determined by the hash mapping function.<br />
o If a partition name is not specified, a system-generated name of form SYS_P### is assigned to the index partition.<br />
o If a tablespace name is not specified, the partition is placed in a tablespace specified in the index-level STORE IN list, or user, or system default tablespace, in that order.<br />
· To reverse adding a partition, or in other words to reduce by one the number of index partitions, you coalesce one of the index partitions then you destroy it. Coalescing a partition distributes index entries of an index partition into one of the index partitions determined by the hash function.<br />
ALTER INDEX sales_hash COALESCE PARTITION PARALLEL<br />
Using the New UPDATE INDEXES Clause<br />
Using the new UPDATE INDEXES clause during a partitioned table DDL command will help you do two things:<br />
· specify storage attributes for the corresponding local index segments. This was not available in previous versions.<br />
· have Oracle automatically rebuild them.<br />
ALTER TABLE MY_PARTS<br />
MOVE PARTITION my_part1 TABLESPACE new_tbsp<br />
UPDATE INDEXES<br />
(my_parts_idx<br />
(PARTITION my_part1 TABLESPACE my_tbsp))<br />
Bitmap Index Storage Enhancements<br />
Oracle Database 10g provides enhancements for handling DML operations involving bitmap indexes. These improvements eliminate the slowdown of bitmap index performance, which occurs under certain DML situations. Bitmap indexes now perform better and are less likely to be fragmented when subjected to large volumes of single-row DML operations.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-36783776098734927642009-10-06T08:27:00.000-07:002009-10-06T08:27:18.988-07:00Automatic Statistics Gathering in Oracle 10gCPU and I/O Performance Statistics<br />
In Oracle 10G, the optimizer takes CPU and I/O costs into consideration during query optimization. If the optimizer understands what the performance characteristics are for I/O and CPU, it will be more effective at query optimization. Current system statistics allow the optimizer to make more intelligent decisions during the optimization process, which ultimately results in higher quality access paths being generated.<br />
Unlike object statistics, administrators must continue to gather system statistics manually in Oracle 10G. This is done by executing the DBMS_STATS.GATHER_SYSTEM_STATS procedure. The key to gathering good system statistics is to run the procedure when a normal workload is being executed. Running it when the workload is light, or too heavy, can result in poor access paths being generated.<br />
System Statistics<br />
System statistics describe the system's hardware characteristics, such as I/O and CPU performance and utilization, to the query optimizer. When choosing an execution plan, the optimizer estimates the I/O and CPU resources required for each query. System statistics enable the query optimizer to more accurately estimate I/O and CPU costs, enabling the query optimizer to choose a better execution plan.<br />
When Oracle gathers system statistics, it analyzes system activity in a specified period of time. The statistics are collected using the DBMS_STATS.GATHER_SYSTEM_STATS procedure. Oracle Corporation highly recommends that you gather system statistics.<br />
<br />
Parameter Name Description Initialization Options for Gathering or Setting Statistics <br />
cpuspeed CPU speed is the average number of CPU cycles per second. At system startup Set gathering_mode = NOWORKLOAD, INTERVAL, or START|STOP, or set statistics manually. <br />
ioseektim I/O seek time equals seek time + latency time + OS overhead time. At system startup Set gathering_mode = NOWORKLOAD or set statistics manually. <br />
iotfrspeed I/O transfer speed is the rate at which an Oracle database can read data in the single read request. At system startup Set gathering_mode = NOWORKLOAD or set statistics manually. <br />
maxthr Maximum I/O throughput is the maximum throughput that the I/O subsystem can deliver. None Set gathering_mode = NOWORKLOAD, INTERVAL, or START|STOP, or set statistics manually. <br />
slavethr Slave I/O throughput is the average parallel slave I/O throughput. None Set gathering_mode = INTERVAL or START|STOP, or set statistics manually. <br />
sreadtim Single block read time is the average time to read a single block randomly. None Set gathering_mode = INTERVAL or START|STOP, or set statistics manually. <br />
mreadtim Multiblock read is the average time to read a multiblock sequentially. None Set gathering_mode = INTERVAL or START|STOP, or set statistics manually. <br />
mbrc Multiblock count is the average multiblock read count sequentially. None Set gathering_mode = INTERVAL or START|STOP, or set statistics manually. <br />
Gathering Object Statistics<br />
Oracle provides the DBMS_STATS package to allow administrators to gather, modify, view, export, import, and delete statistics. The DBMS_STATS package gathers statistics on columns, tables, indexes and partitions and stores the results in Oracle's data dictionary. The optimizer reads the object statistics stored in the data dictionary during query optimization.<br />
After DBMS_STATS is executed, current statements in the shared pool that access the newly analyzed tables are invalidated. This forces the statement to be re-parsed which will allow the optimizer to generate access paths based on the new statistics.<br />
There are many different options available in DBMS_STATS. Administrators Or Developers are able to select which objects to analyze, how much of each object to analyze (large tables may take a long time to analyze), select the number of parallel processes that will perform the analyze, etc.<br />
In releases prior to Oracle10G, we were required to schedule DBMS_STATS jobs on a regular basis to ensure that valid statistics were available to the optimizer. In releases prior to Oracle9i, it was necessary to "guess" how much of the data changed to determine if statistics collection was necessary.<br />
In Oracle9i, the GATHER AUTO option of the DBMS_STATS procedure could be used to help determine if statistics generation was required. <br />
If more than 10% of the rows changed in the table since the last analyze was performed, the DBMS_STATS procedure (with the GATHER_AUTO option activated) analyzed the table.<br />
By default, Oracle10G automates these tasks by evaluating the statistics for all of the tables in the database and running analyze when required. Oracle10G's default maintenance window is nightly from 10 PM to 6 AM and all day on weekends. During these time periods, statistics are automatically collected using the GATHER_STATS_JOB procedure. The maintenance window time-periods can be adjusted to tailor them to each individual application's business processing requirements.<br />
In Oracle Database 10g, Automatic Optimizer Statistics Collection reduces the likelihood of poorly performing SQL statements due to stale or invalid statistics and enhances SQL execution performance by providing optimal input to the query optimizer.<br />
To ensure that 10G is automatically gathering statistics for data objects, we'll need to verify that:<br />
§ The statistics job is running by executing the following SQL statement:<br />
<br />
SELECT * FROM DBA_SCHEDULER_JOBS WHERE JOB_NAME = 'GATHER_STATS_JOB';<br />
§ The modification monitoring feature that identifies stale statistics is enabled. This feature is enabled when the STATISTICS_LEVEL parameter is set to TYPICAL (default) or ALL.<br />
Gathering Statistics Manually in Oracle10G<br />
If we have a warehouse system that contains an extremely volatile table. Hundreds of thousands of rows are added and removed around the clock. The table is loaded and literally hundreds of SQL statements are run to allow our marketing personnel to make critical business decisions.<br />
The level of activity is based on the particular business process taking place. At one particular time, the table can contain hundreds of thousands of rows and at other times it can contain dozens. If we run DBMS_STATS at the same time each day, in most cases we would think we would get a predictable set of statistics generated. Not so in this table's case, sometimes we get hundreds of rows and sometimes we get hundreds of thousands.<br />
If we are unlucky and generate statistics on the table when it has hundreds of rows, access paths suffer when the table grows to hundreds of thousands. To prevent poor access paths from being generated (a big problem in a data warehouse), we run the DBMS_STATS utility immediately after the table is loaded. The optimizer has current statistics to use as input and generates optimal access paths for our critical business queries.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-3004729500738872072009-10-06T08:26:00.000-07:002009-10-06T08:26:03.399-07:00The Automatic Workload Repository1.0 Introduction<br />
Here’s a game to play on a wet Wednesday afternoon. Take a 9i database. Write a script which will schedule the taking of a Statspack snapshot every hour -indeed, let’s just simply run Oracle’s own script (spauto.sql) to do precisely that. Whilst we’re at it, maybe we should rename the schema into which the snapshots are written -after all, PERFSTAT is a bit of weird name, isn’t it? In fact, why don’t we just let SYS own the relevant tables: why create a whole new user anyway? Of course, we’ll have to think up a brand new name for this ‘automated Statspack’, otherwise nobody will think it’s anything very much new. Hmmm... well, it would be good if we could get the word ‘repository’ in the new name somewhere, because it’s a word that is used everywhere else in the database for no good reason; that always sounds imposing and impressive; and whose meaning is sufficiently obscure that paying customers won’t quite know what they’re getting. Also: the thing is scheduled by way of our script, so it runs without manual intervention... sounds like we could maybe squeeze the word ‘automatic’ in there in that case. People pay good money for automation, after all. I know: let’s call it the ‘Automatic Workload Repository’!<br />
OK, OK! I’m being cynical, and I’m over-stating the case a little. But one of 10g’s new features is indeed called the Automatic Workload Repository; and, as a name, that does indeed sound pretty sexy and impressive. But when you boil it down to its basics, it is in fact very much what you would have achieved in 9i merely by scheduling Statpack snapshots every hour. Sure: the quality of the snapshot statistics collected in 10g has improved enormously: for a start, they are more detailed than their 9i cousins ever were. It is also true that the collection of the snapshots in 10g is done by a new background process called MMON, rather than by a server process as with Statspack in 9i. And I will concede that you control and manage the collection of statistics by executing ‘dbms_workload_repository.create_snapshot’ instead of logging on as PERFSTAT and executing ‘statspack.snap’ (though I think you might agree that the old command had a distinct advantage in brevity and ease of typing!) But despite all those acknowledges differences, Automatic Workload Repository (AWR) remains, when all is said and done, scheduled/automated Statspack on Steroids.<br />
I ought to just mention at this point, however, that AWR doesn’t physically enhance or improve on Statspack: it is truly a replacement for it, meaning that the unadorned Statspack is still there to be run (exactly as you would have done in 9i) if you want to. The strong recommendation from Oracle, however, is that you don’t bother: AWR does what Statspack used to do, for sure; but it does it better and with less fuss.<br />
So, in this article, I’ll try and explain how AWR works, what it does when it works, and how you can use its work to help improve the performance of your database. But I’d like you to start out thinking of AWR not as some new, revolutionary feature or capability, but merely as a better and different way of achieving something you were probably doing perfectly well in 9i. You’ll find, I think, that whilst the name might have gotten a lot sexier, the principles of taking snapshots of the dynamic performance views to assess database performance hasn’t.<br />
2.0 A short history of performance tuning...<br />
Oracle uses in-memory structures to record the current state and condition of the database. You can view the contents of these memory structures by querying various dynamic performance views -for example, V$SYSSTAT and V$WAITSTAT, to name but two. The contents of such views are constantly updated, because the system keeps the underlying memory structures up-to-date, in more or less real time.<br />
Unfortunately, a lot of the memory structures (and hence the views) merely accumulate statistics. That’s a problem, because -for example- knowing merely that your database has experienced 3 million buffer busy waits is meaningless. Three million waits ...since when? If your instance has been running uninterruptedly for three years, then that many waits is probably trivial. But if your instance has been running for three days, then you’ve got serious problems! A cumulative total of events is no good to anyone, in other words: you need to know a count of events over a defined period of time.<br />
The other problem with merely looking at the V$ views to assess database performance is that they are based only on memory structures -which are inevitably wiped when the instance is shut down, and re-created from scratch next time you start the instance up. So if you want to know how the database performed three months ago when you last performed that quarterly archiving and loading bulk process... well, tough luck: the information’s not there.<br />
To resolve the problem of cumulative and timeless (and hence hopeless) statistics, Oracle gave us -way back at least as far as version 7- the utlbstat.sql and utlestat.sql scripts. These let you capture the V$ statistics at one time with utlbstat, capture them again at another time with utlestat, and then compare the two sets of numbers (something utlestat did intrinsically just after it captured the second set of statistics). The number of buffer busy waits at one time might have been captured as 3,000,000 and at the second time as 3,000,105 -and suddenly you know that the database has suffered 105 buffer busy waits in the time period spanning the two captures: a much more meaningful piece of information.<br />
These ancient scripts did not, however, resolve the second problem I mentioned: of a lack of history of statistics, and hence an inability to do trend analysis. That’s because each run of utlestat.sql wiped the entire set of statistics, so each run of utlbstat started with a clean slate. Thus, in 8i, Oracle felt compelled to introduce Statspack. Statspack also did the capturing of V$ statistics at different times, thus allowing you to do comparisons over a defined period of time. But the results of the captures (the Statspack ‘snapshots’) were not simply discarded after each run, but were physically stored in ordinary, normal database tables -owned by a specially-created user called PERFSTAT. To take the actual statistics snapshots, though, you had to log onto the database as PERFSTAT and execute a special package and procedure: statspack.snap. If you wanted automatic collection of statistics every hour, for example, then you either had to schedule your own job to do the deed -or you could simply run the spauto.sql script supplied by Oracle, because that did indeed schedule snapshots to be taken every hour. If you logged on to take a manual snapshot, then it was your server process which did the actual deed. If you scheduled a job to do it, then a Job Queue process did it. When you wanted to pick which set of stored statistics snapshots you wanted to compare to assess database performance, you ran another Oracle-supplied script called spreport.sql. This output a text file which you could then pore over for hours, trying to puzzle out what it all meant... or (if you were sensible!) you could upload it to www.oraperf.com, where some scripted cleverness would analyze it into something meaningful for you in moments, and all for free.<br />
And in this way did the DBAs of the early years of the third millenium tune their databases. Here endeth the lesson...<br />
3.0 What’s new in AWR<br />
All of the above is claimed to be ancient history in 10g, but as I was hinting in my introduction to this article, it is still very much reality. You still need to grab V$ statistics at different times to convert the cumulative numbers into something that is actually meaningful. And you still need to store those ‘grabs’ in a set of tables somewhere if you want to do historical trend-analysis of performance over time.<br />
It is true, however, that we no longer have to create a new user to own the tables into which these sorts of statistics snapshots are written: SYS is now the owner of the relevant tables. And they get created automatically for us inside the new SYSAUX tablespace, with names that all begin WRx$, where x can be ‘H’ or ‘M’:<br />
SQL> select table_name from dba_tables<br />
2 where owner=''SYS'' and tablespace_name=''SYSAUX''<br />
3 and table_name like ''WR%''<br />
4 and table_name not like ''WRI%''<br />
5 order by table_name;<br />
<br />
TABLE_NAME<br />
------------------------------<br />
WRH$_ACTIVE_SESSION_HISTORY_BL<br />
WRH$_BG_EVENT_SUMMARY<br />
WRH$_BUFFER_POOL_STATISTICS<br />
WRH$_CLASS_CACHE_TRANSFER_BL<br />
WRH$_CR_BLOCK_SERVER<br />
WRH$_CURRENT_BLOCK_SERVER<br />
<br />
[and so on for 67 rows!]<br />
We also don’t have to perform our own snapshots any more (though, as I’ll go on to show, you can still do so if you wish), and we don’t have to create or run any scripts to schedule the automatic taking of snapshots. Instead, we get a whole new background process called Manageability Monitor (MMON) to take automatic snapshots for us, provided only that an initialisation parameter is set to an appropriate value (and I’ll explain which parameter and what value in just a moment). <br />
By default, MMON will take statistics snapshots for us every 60 minutes -though I note that the Oracle Press book Oracle 10g New Features by Robert Freeman incorrectly asserts the collection period to be every 30 minutes by default -see, for example, page 54. I guess this is one of those times when using a beta product as the basis of assessing a new product becomes a bit awkward -and to be fair to Mr. Freeman, the 30 minutes figure is liberally sprinkled around the Internet, too, as a quick look through Google will confirm and as this site, picked entirely at random, shows as at February 2005. Trouble is, you can prove the "60 minutes’ default theory" just by issuing this query (this comes from a freshly-created template 10g database):<br />
select snap_interval, retention<br />
from dba_hist_wr_control;<br />
<br />
SNAP_INTERVAL RETENTION<br />
------------------- -------------------<br />
+00000 01:00:00.0 +00007 00:00:00.0<br />
Here, the ‘snap interval’ is shown to be, indeed, 1 hour. But this report doesn’t just prove the ‘30 minute default’ statements to be wrong. It gives a clue to the existence of a whole bunch of new DBA_HIST_xxx data dictionary views which you can use to monitor the entire statistics collection mechanism. And it also shows that the statistics MMON captures are automatically purged -by default, as my report’s RETENTION period shows, after 7 days. Statspack snapshots, by contrast, just piled up forever, unless you remembered to clear them out -so that’s another definitely new feature.<br />
Some statistics that are important for performance tuning are session-specific ones, rather than instance-wide ones. The trouble is that to really understand what sessions are up to, and the performance issues they might be encountering, you need to sample these sorts of statistics an awful lot more frequently than once an hour. In fact, you should probably be thinking of grabbing these sorts of statistics every second or so -which is precisely what now happens in 10g. Every second, the system looks at V$SESSION to see which sessions are actually busy doing something (idle sessions are ignored). For those sessions, the system then sees if they are currently waiting on anything in the database (effectively looking at V$WAITSTAT). If sessions are indeed waiting, then that information is stored in a new memory buffer called the Active Session History (ASH) -whose contents you can view at any time by querying the new V$ACTIVE_SESSION_HISTORY view. The ASH buffer has its contents sampled (1 row in 10) and flushed to disk at each MMON statistics snapshot (that is, every 60 minutes by default). If there are too many sessions doing too many waits, however, the buffer will fill before the next MMON snapshot, so another new background process called Manageability Monitor Light (MMNL) flushes a sample of the ASH records direct to the AWR tables as needed. MMON will still flush its sample of the ASH statistics to the AWR when its next snapshot interval falls due.<br />
The statistics MMON captures at each snapshot interval are greatly improved over the kind of thing Statspack managed to collect. For a start, there are more base statistics -operating system ones, for example, along with better SQL and optimiser ones. Brand new to 10g, there are new time-based statistics that record how long activities have run for. <br />
But base statistics are simply the raw count of things happening. Metrics are statistics derived from the base statistics, and usually indicate the rate at which things are happening. For example, a base statistic might tell me the database has performed 1,000,000 physical reads since instance startup. A metric based on that statistic might be that the database has performed 5000 physical reads in the last hour, meaning an average of 1.4 physical reads per second. The base statistics are only of use, as I mentioned earlier, if you’ve got something from an earlier period to compare it with -say, 995,000. Then you can see that the difference yields you the ‘5000 since last measured’ metric. <br />
One of the main jobs that MMON performs, aside from the periodic capturing of statistics snapshots, is to calculate and re-calculate numerous system metrics almost continuously (for most of them, about once every minute). The computed metrics are visible in another bunch of new views -such as V$SYSMETRIC- as well as being stored in the AWR proper when the usual snapshot interval comes round. Having the metrics recomputed continuously in this way means you can assess the effect of running a particular job practically instantaneously: you just run the job, then query the new V$ views. Compare that to 9i, where you would have had to arrange for the calculation of the appropriate metrics by manually taking a Statspack snapshot just before and just after you ran the job, and then by running a new Statspack report to compare the two snapshots. <br />
Finally, I should mention that AWR is fully RAC-aware, in a way that Statspack never was. Certainly, you could get Statspack to capture statistics for every instance in the cluster. But you had to do that on each instance in turn: there was no way to ask for all instances in the cluster to simultaneously snapshot their individual statistics. What’s more, within the PERFSTAT tables, there was no way to distinguish which instance a particular statistics snapshot had come from, so picking the right snapshots to compare was more grief than it should have been. With AWR, by contrast, MMON in one instance co-ordinates with MMON in all the others, so that they all (virtually) simultaneously take a snapshot of their own instance -and then each of those separate snapshots is written into the AWR tables with the same Snapshot ID, though with different instance numbers. That makes picking the right snapshots to compare extremely easy; it also makes understanding what one instance was doing at precisely the same time as another a lot simpler, too.<br />
3.1 Summary<br />
Putting all of that together into one brief summary, therefore, we can say of AWR:<br />
· AWR is simply the physical and process infrastructure to collect instance and database statistics needed for proper performance tuning. <br />
· Statistics collection is automated (once per hour, by default)<br />
· Statistics purging is automated (statistics expire after a week, by default)<br />
· MMON is a new background process that does the collecting and purging<br />
· MMNL may assist in the collection of session statistics<br />
· Session statistics are collected every second for active sessions, but only in memory<br />
· One tenth of session statistics are written to the Workload Repository at the usual MMON collection<br />
· MMON collects more and better-quality statistics<br />
· Every minute, MMON re-computes metrics based on the statistics<br />
· AWR is fully RAC-enabled<br />
So it really is fair to say that AWR brings new features and new qualities to the job of statistics collection that are a distinct improvement on the sort of mechanisms we had available to us in earlier Oracle versions. Whilst it is therefore possible to run traditional Statspack in a 10g database, it is rather pointless to do so, because MMON and the AWR achieve the same goal far more effectively.{mospagebreak title=Impact of AWR}<br />
4.0 Impact of AWR<br />
You might reasonably pause at this point and ask: AWR sounds very convenient and clever, but it also sounds as though there’s an awful lot of automatic shenanigans going on in the background to make it all work... are you sure it won’t clobber my database’s performance to death? To which the semi-official answer is ‘Stop Worrying! It’s all referencing in-memory structures, so it’s fast and has been optimised’. The slightly more realistic answer is that yes, of course this puts a load on your database, and that load might be unacceptable.<br />
The Active Session History memory buffer, for example, consumes a fixed 2MB of RAM per CPU, (though with an absolute ceiling of no more than 5% of your total shared pool size). If you simply migrated from vanilla 9i to equally vanilla 10g, therefore, without also remembering to increase your shared pool size by that sort of figure to compensate, you’d be operating your database with smaller library and dictionary caches than you probably expected to have at your disposal -and that can definitely cause performance problems. Similarly, the AWR tables will probably end up consuming about 300MB of disk space, if you stick with the default retention period of 7 days, and whilst disk space is always said to be cheap, can you afford that much space even so? And never mind the megabytes: disk space has a habit of turning into physical I/O, especially when MMON starts writing to it every hour... can you afford that much additional I/O?<br />
The point is, these new structures and snapshot-taking behaviours definitely do impose a load on the database, and that load might be unacceptable. On the other hand, what you can do with the information being collected by this process by way of tuning and enhancing your database’s performance might well be thought to outweigh such considerations. And there’s no doubt that Oracle has designed the whole process so that the impact on the database’s normal operations is minimised. But ‘minimised’ is not the same thing as ‘minimal’! It really boils down to careful testing and evaluation of how it behaves in your own environment, of course, and whether the benefits do outweigh the costs. My gut feel is that for most people, most of the time, they will... but it’s early days to know that for sure.<br />
5.0 Managing AWR<br />
Given that AWR is, as its very name tells us, ‘Automatic’, it’s perhaps not surprising that there’s not actually a lot of management to do to get it working! You’ll certainly need to know how to switch it on and off, and you might want to look at moderating the load it places on on the database when it’s on. But after that, it’s all down to MMON just getting on with it. It is, however, possible that you’ll want to take manual statistics snapshots without waiting for MMON’s next wakeup call. And finally, you may need to mark particular snapshots as being typical of some ‘baseline’ level of database performance. So I’ll now examine how you go about doing each of those four things in turn.<br />
5.1 Switching MMON on and off<br />
MMON is switched on by setting the initialisation parameter STATISTICS_LEVEL. That parameter takes three possible values: ALL, TYPICAL and BASIC. If BASIC is set, then MMON is off. If either of the other values are set, then MMON is on and the Workload Repository will be populated. The default value for STATISTICS_LEVEL is TYPICAL, therefore MMON is actually on by default.<br />
Be careful about setting the parameter to ALL. Very detailed statistics related to the operating system and SQL statements are collected at this setting, and the impact of doing so on the database is quite severe. Oracle’s official line seems to be that you should never set the parameter to ALL unless Oracle support tells you to, or unless you are doing some extremely rare tuning of specific SQL statements. And even then, you wouldn’t leave it set like that for long!<br />
Bear in mind that STATISTICS_LEVEL is a global switch that affects an awful lot more in the database than just MMON. For example, switching it off (by setting it to BASIC) will stop 10g doing automatic, dynamic SGA memory re-allocations, and will similarly switch off the various advisories that exist to inform you of the best possible manual settings for things like the shared pool size, or the right size for the PGA aggregate target. If you want all of those bits of functionality, but don’t want MMON automatically taking statistics snapshots, then you’ll have to go about disabling MMON in another way entirely (see section 5.2 below). Oracle Corporation, however, strongly suggests that you don’t even think about switching off MMON: the essence of a self-tuning, self-managing, self-automated database is that you let it do its automated stuff, and stop trying to get choosey about it. That’s a new paradigm for most Oracle DBAs to swallow, and some are going to find it harder to feel comfortable about than others. But that’s 10g all over!<br />
5.2 Configuring MMON<br />
There’s not a lot to configure regarding MMON, of course: it will just do its thing without your guidance or intervention, taking a new snapshot, as we’ve already seen, every 60 minutes and purging ones older than 7 days. The only thing you might want to do is alter those timings, and you do that by using the new DBMS_WORKLOAD_REPOSITORY package as follows: <br />
exec dbms_workload_repository.modify_snapshot_settings (43200, 20)<br />
The two parameters in the brackets there are, respectively, the AWR record retention period and the MMON snapshot interval time, both measured in minutes. In this example, therefore, I’m asking for repository records to be retained for 43,200 minutes -which people good at maths will immediately recognise as being equivalent to 30 days- and for MMON to take fresh snapshots every 20 minutes.<br />
As a rough rule of thumb, allow 300MB for the repository tables if they are asked to retain snapshot data for the default period of 7 days -so a change in that retention peiod to 30 days implies my repository tables will end up occupying about 1.2GB of physical disk space. The retention period can be a minimum of 1440 (that is, one day) and a maximum of a somewhat ambitious 52,560,000 -a mere 100 years!! Incidentally, at 300MB per week, the AWR will consume about 1.5TB of data after 100 years. Just in case you were needing to budget a hardware purchase, you understand...<br />
The snapshot interval in its turn can be as short as 10 minutes, and as ridiculously silly as 100 years. The shorter the interval, the finer-grained your view of database performance will be: mini-performance spikes will be detectable in the collected snapshots. But equally, the background work the database has to do will shoot up, and that will probably start imposing a measurable and unacceptable performance burden on your database. Conversely, the longer the interval, the less work MMON is doing, and the less the impact on your database... but the more blurred your view of database performance will be. If you smudge all the little ups and downs of performance that take place during a normal day, at intervals of a few minutes at a time, into (say) a single 10 hour-broad average, then you’ll really just have wasted your time, and the statistical reports will likely be of no use to anyone. <br />
Although the technical minimum for the snapshot interval is, as I mentioned, 10 minutes, you are allowed to set it to 0. This setting has a special meaning: it disables MMON completely, and no automatic snapshots at all are taken. If you therefore wish to switch off MMON, but don’t want to set STATISTICS_LEVEL to BASIC because of all the other things that would disable at the same time, this is how you would go about doing it.<br />
5.3 Taking Manual Snapshots<br />
If you want to take your own manual statistics snapshot, instead of waiting for MMON to get around to it when the next snapshot interval elapses, then you can do so by using the CREATE_SNAPSHOT procedure of the DBMS_WORKLOAD_REPOSITORY package like this:<br />
SQL> exec dbms_workload_repository.create_snapshot()<br />
<br />
PL/SQL procedure successfully completed.<br />
I’ve specified no arguments here at all, which is why there’s a pair of empty brackets at the end of that command, but actually you can specify there something called the flush_level as being either TYPICAL or ALL. You can therefore probably guess that this is a way of specifying the statistics level for each snapshot, regardless of what statistics level may have been configured for the instance as a whole by setting the STATISTICS_LEVEL initialisation paramater. If you don’t specify a flush-level, then you get the TYPICAL one, which you might have guessed would be the default, since that is equally the default value for the instance-wide STATISTICS_LEVEL parameter.<br />
You can see the existence of the new snapshot by looking in the new data dictionary view DBA_HIST_SNAPSHOT, though it won’t yield an awful lot of particularly useful information:<br />
SQL> select snap_id, to_char(begin_interval_time,''HH:MI:SS'') as "START",<br />
2 to_char(end_interval_time,''HH:MI:SS'') as "FINISH",<br />
3 snap_level from dba_hist_snapshot<br />
4 order by snap_id;<br />
<br />
SNAP_ID START FINISH SNAP_LEVEL<br />
---------- -------- -------- ----------<br />
1 04:40:10 06:00:46 1<br />
2 06:00:46 07:00:12 1<br />
3 12:00:06 12:11:44 1<br />
4 04:27:28 04:39:15 1<br />
5 05:54:28 06:05:59 1<br />
6 12:09:09 12:20:31 1<br />
7 12:20:31 01:00:49 1<br />
8 01:00:49 02:00:26 1<br />
9 02:00:26 03:00:54 1<br />
10 03:00:54 04:00:29 1<br />
11 09:01:12 09:12:27 1<br />
12 09:12:27 10:00:45 1<br />
13 10:45:48 10:45:53 1<br />
14 10:47:22 10:47:29 2<br />
15 11:04:10 11:35:09 1<br />
16 11:35:09 11:35:10 1<br />
17 11:35:10 11:35:17 2<br />
The main thing of note here, I think, is the SNAP_LEVEL column: you get a ‘1’ if the collection was a typical one, and a ‘2’ if it was done at the ALL level. My data is not particularly good here: it appears as though my ALL snapshots took just 2 seconds or so longer to complete than my TYPICAL ones. But that’s true only for the sad and puny test database I’m using right now. In a real production database, you would expect the differential to be much larger than that.<br />
Incidentally, if you set the snapshot interval to zero as I described above, so that MMON is not performing automatic snapshotting,then you’ll actually not be able to take manual snapshots either. You’ll get this sort of error message if you try:<br />
SQL> exec dbms_workload_repository.create_snapshot()<br />
BEGIN dbms_workload_repository.create_snapshot(); END;<br />
<br />
*<br />
ERROR at line 1:<br />
ORA-13516: SWRF Operation failed: INTERVAL Setting is ZERO<br />
ORA-06512: at "SYS.DBMS_WORKLOAD_REPOSITORY", line 8<br />
ORA-06512: at "SYS.DBMS_WORKLOAD_REPOSITORY", line 31<br />
ORA-06512: at line 1<br />
This tells you that although it is you and your session issuing the command, it cannot be your server process that actually performs the statistics collection. In fact, it tells you that even manual snapshots are perfomed by MMON: all your server process does in these manual situations is tell MMON to get on with it. And, of course, if MMON has been switched off, then there’s nothing and no-one around to respond to the instruction... and hence the error you see here.<br />
5.4 Taking a Baseline<br />
Assuming you let MMON do its thing and capture snapshots automatically, there are going to be pairs of snapshots which can be considered representative of the start and end of, say, ‘normal running’ or ‘monthly bulk load’ or ‘end-of-year reports running’. You can mark such pairs of snapshots and declare them to be either end of a named baseline. You do so like this:<br />
SQL> exec dbms_workload_repository.create_baseline(13,14,''Nightly Batch Load'')<br />
<br />
PL/SQL procedure successfully completed.<br />
Here, I’m marking snapshots 13 and 14 as having been taken at the beginning and end of a nightly batch load (and yes, if you look back to section 5.3, you’ll realise that my nightly batch load took all of about one minute to process, so it can’t have been much of a load! True enough, but then I’m only interested in getting the theory right here. Realism can wait!)<br />
The point of creating a baseline in this way is that the baseline’s contributing snapshots are now exempt from the normal purging activity of MMON. They do not, in other words, ‘age out’ of the repository and in fact will be kept permanently -or, at least, until you decide to get rid of them yourself. Until then, you have a permanent record of how a nightly batch load behaved one night in January 2005 (in my case). If batch loads by July or August start running incredibly slowly, I can compare them to the way things were behaving at the start of the year, and thus (hopefully!) identify what has gone wrong in between the two periods.<br />
Remembering that if you create a baseline, it and its constituent snapshots will be forever immune to the automatic purging activity of MMON, the only way to get rid of an unwanted baseline is to delete it yourself, and you can do that by using the DROP_BASELINE procedure of the DBMS_WORKLOAD_REPOSITORY package, like so:<br />
SQL> exec dbms_workload_repository.drop_baseline(''Nightly Batch Load'',TRUE)<br />
PL/SQL procedure successfully completed.<br />
The DROP_BASELINE procedure takes as its arguments the ‘friendly’ name of the snapshot to be cleared out, and an optional second parameter which says whether the drop should ‘cascade’ down to clearing out the two snapshots which together define the baseline. The default for that argument is FALSE, so by default only the baseline ‘wrapper’ around the snapshots is itself deleted. In my example, though, I’ve wiped the participating snapshots as well.{mospagebreak title=Using AWR}<br />
6.0 Using AWR<br />
With Statspack in 9i, you ran spreport.sql to output the results of comparing one snapshot with another. With AWR, you run awrrpt.sql -either that, or you use the web-based Enterprise Manager to achieve the same sort of thing. I’ll stick with running the manual report for now, and deal with Enteprise Manager in a later paper. The awrrpt.sql script is found in the place traditionally used for this sort of thing: $ORACLE_HOME/rdbms/admin. <br />
Take a moment to have a look at the contents of this script, and you’ll find at the beginning of it this little give-away:<br />
Rem NOTES<br />
Rem Run as select_catalog privileges. <br />
Rem This report is based on the Statspack report.<br />
Rem<br />
Rem If you want to use this script in an non-interactive fashion,<br />
Rem see the ''customer-customizable report settings'' section in<br />
Rem awrrpti.sql<br />
First, note that the script must be run by a user to whom the SELECT_CATALOG role has been granted. Second, marvel at the grammatical incompetence that allows anyone to tell you to ‘run as privilges’! Third, wonder at the inability of whoever wrote this to understand the difference between a privilege and a role. And fourth, and the bit I find most interesting, pay attention to the line that confesses ‘This report is based on the Statspack report’!! I know there are some people on the planet who tend to roll their eyes whenever I say ‘Oh, new feature X is really just old feature A on steroids’, but here we have the horse’s mouth, as it were, confirming my diagnosis!<br />
Anyway, I digress! Here’s how you run that report:<br />
SQL> show user<br />
USER is "SYS"<br />
SQL> @?\\rdbms\\admin\\awrrpt<br />
<br />
Current Instance<br />
~~~~~~~~~~~~~~~~<br />
<br />
DB Id DB Name Inst Num Instance<br />
----------- ------------ -------- ------------<br />
973646425 DBNEW 1 dbnew<br />
<br />
Specify the Report Type<br />
~~~~~~~~~~~~~~~~~~~~~~~<br />
Would you like an HTML report, or a plain text report?<br />
Enter ''html'' for an HTML report, or ''text'' for plain text<br />
Defaults to ''html''<br />
Enter value for report_type: html<br />
<br />
Type Specified: html<br />
I’ll break in at this point to draw your attention to the rather nice fact that the ‘new Statspack’ report can be output in html instead of just plain text.<br />
Instances in this Workload Repository schema<br />
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br />
DB Id Inst Num DB Name Instance Host<br />
------------ -------- ------------ ------------ ------------<br />
* 973646425 1 DBNEW dbnew KOALA<br />
Using 973646425 for databas e Id<br />
Using 1 for instance number<br />
<br />
Specify the number of days of snapshots to choose from<br />
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br />
Entering the number of days (n) will result in the most recent<br />
(n) days of snapshots being listed. Pressing without<br />
specifying a number lists all completed snapshots.<br />
Listing the last 3 days of Completed Snapshots<br />
<br />
Snap<br />
Instance DB Name Snap Id Snap Started Level<br />
------------ ------------ --------- ------------------ -----<br />
11 11 Jan 2005 09:12 1<br />
12 11 Jan 2005 10:00 1<br />
15 11 Jan 2005 11:35 1<br />
16 11 Jan 2005 11:35 1<br />
17 11 Jan 2005 11:35 2<br />
18 11 Jan 2005 12:0 0 1<br />
<br />
Specify the Begin and End Snapshot Ids<br />
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~<br />
<br />
Enter value for begin_snap: 11<br />
Begin Snapshot Id specified: 11<br />
<br />
Enter value for end_snap: 18<br />
End Snapshot Id specified: 18<br />
So, just as in the old Statspack days, you get shown which snapshots actually exist, and then asked to choose which one should be compared to which other one. You can pick any two you like, provided the first one is from a time before the second (hopefully for obvious reasons!)<br />
Specify the Report Name<br />
~~~~~~~~~~~~~~~~~~~~~~~<br />
The default report file name is awrrpt_1_11_18.html. To use this name,<br />
press to continue, otherwise enter an alternative.<br />
awr_rpt1.html<br />
<br />
[snip a huge amount of screen-scrolling activity]<br />
<br />
End of Report<br />
<br />
Report written to awr_rpt1.html<br />
And at the end, as it says, your report is complete -written, incidentally, into the same directory you were in when you first launched SQL*Plus. The start of my report looks like this:<br />
...and if you’re at all familiar with the start of a Statspack report, you’ll recognise the general instance statistics this is showing you. And, just like an old Statspack report, you can scroll on to discover what your top 5 wait events were, what your most expensive SQL statements were, and so on. At which point, I hope you are an experienced Statspack analyst and have vast experience in immediately drawing the right tuning conclusions from the various statistical oddities the report will have buried within its details... because you’re going to need to be! For whilst the report just produced might look a bit like an old Statspack report, it isn’t one... and hence useful websites like Oraperf, which is able to analyse your Statspack reports for free in just a few minutes, don’t (as yet, anyway) allow you to upload their 10g equivalents. So if you’re not a Statspack Report analysis expert, it’s highly questionable why you bothered to produce an ‘awrrpt’ at all!<br />
Which would be a bit of a depressing note to conclude on, if it was the whole story. But, of course, it isn’t at all -because in 10g there are a lot of automated, in-built clients of the Workload Repository, which draw upon the statistical information it contains to suggest all manner of performance tuning tweaks you might want to apply to your database by hand, and sometimes to carry out performance tuning tweaks totally automatically. For example, the Automatic Database Diagnostic Monitor (ADDM) uses the AWR statistics to detect and resolve database configuration and performance problems. Or, again: there’s a new set of server-generated alerts which similarly draw upon the AWR data to predict when, for example, space problems are going to arise on the database. And, as one final example, the new SQL Tuning Advisor can also use the AWR to work out how SQL statements could be better written.<br />
In short, there are many AWR clients, and you as a DBA don’t actually have to be one of them in order to get benefit from having a Workload Repository. Gurus and techie types will certainly want the control that perusing your own AWR reports will give you: they will be able to use it to make informed, correct diagnoses and to implement appropriate fixes. But the vast majority of DBAs, who lack naval-gazing skills of that sort and intensity, will simply be able to let other database components use the AWR to fix things up automatically and can concentrate on being more productive in other areas.<br />
I’ll be discussing some of these AWR clients in other articles in this series.<br />
7.0 Conclusion<br />
I have, I’m afraid, been deliberately provocative in this article in the way I have cast AWR as nothing much more than a modestly-improved Statspack capability. My main reason for being so, however, is that it is all-too easy to get thoroughly confused -indeed, intimidated- when told that 10g comes with AWR, ADDM, ASM, and who knows what other acronyms and abbreviations besides. So many new things! So much to learn! So much to re-learn! At which point, it can become rather tempting just to give up and avoid all the new-ness altogether!<br />
Yet, if you could put a lot of these new features into some sort of context; if you could see them as being a development of, and an enhancement to, features with which you are already familiar, then the reason for that sense of bewilderment or intimidation disappears. You can see through to those aspects which are genuinely new very quickly, because you understand the foundation they’ve been built on. You won’t, for example, be thinking there’s a huge intellectual mountain to climb before you can even grasp what the new features are actually for.<br />
We had much the same thing in 9i with Real Application Clusters: “Thou shalt not think of this brand-new, never-before seen product as an enhanced version of Oracle Parallel Server”, said Oracle when it was first releaed. But in fact, that’s what it was! Truthfully, the technical difference between OPS and RAC was really rather slight (block transfer via the network rather than the hard disk being the main one -and even that had been introduced in basic form in 8i) -though it is fair to say that slight difference makes a huge difference in the way DBAs, Application Designers and Developers approach the topics of clustered database application design and administration. And whilst it was definitely important to get users to make that huge leap in approach, I’m not convinced that marketing a product as something it isn’t is actually the most helpful way to go about it.<br />
So, in a similar way, with AWR. Yes, of course the statistics are better, and the use to which they can be put is dramatically new in 10g compared with ye olde Statspack. But does it help people grasp the essential point of AWR better and more quickly if it is explained to them in Statspack-like terms? I think so. I certainly hope so, anyway, because otherwise this article has been a waste of time!<br />
So to conclude: yes, you will almost certainly let MMON automatically populate your Workload Repository. But it is unlikely you will be much interested in what’s inside the Repository yourself. Instead, you’ll probably be happy to let the internal, automated AWR clients like ADDM and the SQL Tuning Advisor make use of it as they see fit. AWR, in short, will probably be a vital but practically invisible 10g new feature as far as most real, human DBAs are concerned.KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-68944636038732761002009-10-06T08:25:00.000-07:002009-10-06T08:25:15.833-07:00Scheduling Jobs with DBMS_SCHEDULER1.0 Introduction<br />
We’ve had the ability to schedule work in the database for a long time... think DBMS_JOB, for example (which, I hasten to add, is still there in 10g, so old code doesn’t break). New in Oracle 10g, however, is DBMS_SCHEDULER, a more powerful, robust and flexible work scheduling engine... and a lot trickier to get sorted in your head, if you’re anything like me! Honestly: if Oracle had deliberately set out to make the topic of scheduling something to happen every ten minutes as obscure, complicated and scary as possible, they couldn’t have done a better job! That’s only my opinion, of course. But then I’m writing the article!<br />
To be fair, the complications are related to Scheduler’s power and subtlety: the comparison that springs to mind is between, say, designing a website in Microsoft Frontpage and doing it in Macromedia DreamWeaver. I know which is easier, but I also know which can achieve the better results long-term.<br />
So I’ve chosen to break the discussion of Scheduler into two parts. Part 1 sets out to treat Scheduler as the Frontpage of job scheduling programs: nothing too complicated; just the basic stuff needed to achieve the scheduling of a job. Part II then has a look at the hard stuff: creating schedules, scheduling windows, window priorities and so on. With any luck, Part II won’t look anything like as scary by the time you get to it, because you’ll be entirely at home with the basics!<br />
2.0 Privileges and Preparations<br />
There are two primary interfaces to the Job Scheduler in 10g. If you like doing things at the command line, you’ll love DBMS_SCHEDULER. If you are more a GUI kind of person, then you’ll need to get familiar with the Enterprise Manager Database Control web management utility. I’ll be writing about Enterprise Manager in another article soon, so this one is going to concentrate on doing things with the keyboard -though I’ll throw in an EM screenshot or two here as well.<br />
Before we begin, therefore, I will need to make sure that I have a database user account set up that can make use of the Scheduler -and that means it the rights to create jobs. I’ll make sure I start with a clean slate, and an account with only the minimum privileges needed for things to work effectively, like so:<br />
SQL> drop user scott cascade;<br />
User dropped.<br />
<br />
SQL> @?/rdbms/admin/utlsampl<br />
<br />
C:\WINNT\system32>sqlplus / as sysdba<br />
<br />
SQL*Plus: Release 10.1.0.2.0 - Production on Tue Feb 15 05:33:54 2005<br />
Copyright (c) 1982, 2004, Oracle. All rights reserved.<br />
<br />
SQL> grant create job, create procedure to scott;<br />
Grant succeeded.<br />
The create job system privilege gives a user execute rights on the dbms_scheduler package, and hence the ability to create and schedule jobs in his own schema. If I’d wanted Scott to be able to create jobs on behalf of, and in the schema of, some other user then I could have granted him the create any job system privilege. The grant of the create procedure system privilege is there simply so that in the rest of this article I can show Scott creating assorted stored procedures, the automatic execution of which can then be scheduled. It is not, in other words, a privilege intrinsically needed to be able to schedule work on the database.<br />
With the above privileges granted, I can now log on as Scott, and create the following procedure:<br />
SQL> connect scott/tiger<br />
Connected.<br />
<br />
SQL> create or replace procedure new_emp<br />
2 is<br />
3 begin<br />
4 insert into emp (empno,ename, job, mgr, hiredate, sal, comm, deptno)<br />
5 values ((select max(empno)+1 from emp),<br />
6 'HJR','CLERK',7698,sysdate,1000,null,30);<br />
7 commit;<br />
8 end;<br />
9 /<br />
<br />
Procedure created.<br />
It’s not a particularly brilliant piece of code: it simply inserts a new employee called ‘HJR’ (feel free to change that to your own initials!) into the ever-familiar EMP table. Most of the column information for the new record is hard-coded to specific values which are going to be of no interest to us. But you might notice that the HIREDATE column is set to the current system time. Inspecting that column, in other words, will show us precisely what clock time it was when the new record was entered.<br />
Apart from the grant of those privileges and the creation of this little procedure, that’s all the preparation you’ll need. So enjoy yourself as we go for a tour around the new 10g Job Scheduler.{mospagebreak title=Creating a Job}<br />
3.0 Creating a Job<br />
At its absolute simplest, creating a scheduled job in 10g requires that you supply just three pieces of information:<br />
· A name for the job<br />
· A job type (which I’ll describe in just a moment) and<br />
· An action for the job to perform (ditto)<br />
<br />
<br />
Commonsense additionally suggests you might like to tell Oracle when the job should start and how frequently it should be repeated thereafter.<br />
Job names can be pretty much anything you want, provided it doesn’t match anything else in the database. That is, a job can’t be called the same name as a table or a constraint, for example. <br />
There are just three Job types: PLSQL_BLOCK, STORED_PROCEDURE and EXECUTABLE. You use PLSQL_BLOCK when you want to type in, directly to DBMS_SCHEDULER, the PL/SQL code you want to run. In other words, it lets you type in an anonymous PL/SQL block. STORED_PROCEDURE is what you specify when there is an existing stored procedure, package or function you wish the scheduler to run for you. And EXECUTABLE is what you’d supply, as the name might suggest, when you want the scheduler to run some O/S program or script -for example, a shell script or a Windows Scripting Host script.<br />
The Job action is simply the name of the stored procedure to run if you’d said the job type was STORED_PROCEDURE, or the name of the O/S script or executable you want to run if you’d said the job type was EXECUTABLE. If you’d asked to schedule a PLSQL_BLOCK, then the job action is actually the PL/SQL code itself.<br />
Apart from that lot, as I mentioned earlier, you will probably want to supply a start time (and that uses the TIMESTAMP WITH TIME ZONE data type) and a repeat interval. The repeat interval can be tricky, as I’ll show you in just a moment, but if it makes you feel more comfortable then you can also use traditional date/time mathematics to specify one: so, for example, an interval of SYSDATE+(1/24) means a job will be re-run every 1 hour.<br />
One of the other new features in Oracle 10g, however, is a slew of new calendaring expressions. They can take a bit of getting used to, as with anything unfamiliar, but they are actually easier to use than the old SYSDATE+Fraction technique! These expressions usually take two components: a frequency and then an interval. For example, a job might need to be run with a frequency of HOURLY and an INTERVAL of 1 -and that would be the same as scheduling an interval of SYSDATE+(1/24). As another example, you might want the job run ‘FREQ=MINUTELY; INTERVAL=25’. That is the actual syntax you use, and in this case specifies a repeat interval of ‘every 25 minutes’: the ‘minutely’ means ‘using minutes’ not ‘I am very, very small’! <br />
It can get much more complicated than that, of course. How about this: <br />
‘FREQ=WEEKLY;BYWEEKNO=6,12,18,24,30,36,42,48;BYDAY WED,FRI;BYHOUR=21;BYMINUTE=15’<br />
Scary, huh?! It simply means: ‘At 9:15PM on Wednesday and Friday of every sixth week, starting with week number 6’. Try doing all of that using SYSDATE+something mathematics! Now you know why such expressions were invented for 10g: they make subtle and complex job schedules a (relative) piece of cake! The complete list of frequencies you can specify are: yearly, monthly, weekly, daily, hourly, minutely and secondly. Likewise, the list of interval types include BYMONTH, BYMONTHDAY, BYYEARDAY and BYSECOND (not including the ones I’ve already shown you in the command above). Some of those are distinctly silly, if you ask me: BYSECOND implies performing a job at an exact number of seconds within the minute... but you’ll recall that Oracle makes no guarantees at all about whether a job will ever get performed precisely on schedule, so trying to be that precise is just wishful thinking!<br />
Incidentally, if you don’t want to specify a repeat interval at all, then that’s fair enough: it simply means you are asking for a job to be executed once and once only, and there’s nothing wrong with that at all.<br />
Anyway: with those five pieces of information to hand, I’m ready to create my first scheduler job, and the command to do so would look something like this:<br />
SQL> exec dbms_scheduler.create_job -<br />
> (job_name=>'HIRE', -<br />
> job_type=>'STORED_PROCEDURE',-<br />
> job_action=>'NEW_EMP', -<br />
> start_date=>SYSTIMESTAMP, -<br />
> repeat_interval=>'FREQ=MINUTELY; INTERVAL=5', -<br />
> end_date=>SYSTIMESTAMP+31, -<br />
> enabled=>TRUE)<br />
<br />
PL/SQL procedure successfully completed.<br />
I’ve broken the command down into separate lines, using the ‘-’ continuation character between them, so that it’s hopefully not quite the mess it can look when you type it all on one line! Taking the syntax apart bit by bit, we see:<br />
Job Name: <br />
I’ve just invented one here that happens to be called HIRE. It could have been almost anything, though, so long as it didn’t duplicate the name of any other object in the database<br />
Job Type: <br />
I’ve told the scheduler to run a stored procedure.<br />
Job Action: <br />
And here I tell it which stored procedure to run: in my case, the NEW_EMP procedure I created at the start of this article. <br />
Start Date: <br />
I cheated on this one. I could have typed in a full-blown TIMESTAMP WITH TIME ZONE string to indicate a future start time, but instead I simply used the in-built SYSTIMESTAMP function, which means the job is to start “right now”. In fact, the repeat interval will have to elapse before the very first run of the job, but that’s close enough.<br />
Repeat Interval: <br />
Hopefully you recognise this now! I’m asking for the job to be re-executed every five minutes. I realise it makes no sense to employ a new member of staff every five minutes, but it will serve as an example!<br />
End Date: <br />
You don’t actually need to specify an end date: if you miss it out, the job just keeps repeating for ever. On the grounds that I don’t want my EMP table accidentally to acquire a bazillion new records, however, I’ve insisted the job stop running 31 days from now. Again, I’ve used the SYSTIMESTAMP function to ensure the value is returned to DBMS_SCHEDULER with the right data type.<br />
Enabled: <br />
Rather bizarrely, to my way of thinking at least, the default behaviour of the Scheduler is to create jobs in a disabled state. You would have to manually enable the job (execute dbms_scheduler.enable(‘job-name’) if you wanted the job to be actually carried out. Since I definitely want my little PL/SQL procedure to be run every five minutes, I’ve taken the opportunity here to insist that the job be run right from the word go.<br />
At this point, I suggest you go and make yourself a cup of tea and relax for a while. If I’ve got it right, the HIRE job should run itself every 5 minutes, so by the time the tea has brewed, and the biscuits nibbled, the EMP table should have a fair few more employees in it than it did to start with...<br />
Right... now you’ve stopped filling your face with cream cakes and calories, let’s look at what’s happened to the EMP table...<br />
SQL> select empno, ename, job, mgr,<br />
2 to_char(hiredate,'HH-MI-SS')<br />
3 from emp where ename='HJR';<br />
<br />
EMPNO ENAME JOB MGR TO_CHAR(<br />
---------- ---------- --------- ---------- --------<br />
7935 HJR CLERK 7698 08-20-31<br />
7936 HJR CLERK 7698 08-25-31<br />
7937 HJR CLERK 7698 08-30-31<br />
7938 HJR CLERK 7698 08-35-31<br />
7939 HJR CLERK 7698 08-40-31<br />
7940 HJR CLERK 7698 08-45-31<br />
7941 HJR CLERK 7698 08-50-31<br />
7942 HJR CLERK 7698 08-55-31<br />
7943 HJR CLERK 7698 09-00-31<br />
7944 HJR CLERK 7698 09-05-31<br />
<br />
10 rows selected.<br />
Unless I’m very much mistaken, that looks suspiciously like a new employee has been hired every five minutes! So the job creation and scheduling process has worked -and it wasn’t particularly difficult at all, was it?<br />
I should perhaps mention that these results, whilst genuine, are quite remarkable in one sense: it is extraordinary how the scheduler has activated the job every five minutes to the second. That’s because I’m running on a test rig, and there’s nothing much else happening on my database, but in a production setting, it’s most unlikely you’d get this level of spot-on precision. In fact, Oracle makes no guarantees about the execution of scheduled jobs at all: the relevant processes will get around to performing the required work when they are free to do so. If they’re tied up, busy performing the other 4000 jobs that have been scheduled, then you’ll just have to wait your turn.{mospagebreak title=Checking it has worked}<br />
4.0 Checking it has worked<br />
The example I’m using in this article, of course, provides its own check that the job has actually been performed successfully and on time -but that’s not likely to be true for most of your jobs in the real world. You need some way of validating when a job ran, and whether it ran successfully to completion or hit some sort of snag. Enterprise Manager’s Database Control will provide a nice GUI-ish way of doing that, of course, but if you’re stuck in good old SQL*Plus, then the new view USER_SCHEDULER_JOB_RUN_DETAILS is your friend, even if its name definitely isn’t! There are equivalent DBA_ and ALL_ views as well, naturally. Here’s a snippet from mine:<br />
SQL> column log_date format a45<br />
SQL> column job_name format a10<br />
<br />
SQL> select log_date, job_name, cpu_used, status<br />
2 from user_scheduler_job_run_details<br />
3 where job_name='HIRE';<br />
<br />
LOG_DATE JOB_NAME CPU_USED STATUS<br />
--------------------------------------------- ---------- ---------- -------------<br />
15-DEC-04 08.20.31.296000 AM +11:00 HIRE 0 SUCCEEDED<br />
15-DEC-04 08.25.31.328000 AM +11:00 HIRE 0 SUCCEEDED<br />
15-DEC-04 08.30.31.406000 AM +11:00 HIRE 0 SUCCEEDED<br />
15-DEC-04 08.35.31.375000 AM +11:00 HIRE 0 SUCCEEDED<br />
15-DEC-04 08.40.31.421000 AM +11:00 HIRE 0 SUCCEEDED<br />
15-DEC-04 08.45.31.359000 AM +11:00 HIRE 0 SUCCEEDED<br />
15-DEC-04 08.50.31.406000 AM +11:00 HIRE 0 SUCCEEDED<br />
15-DEC-04 08.55.31.375000 AM +11:00 HIRE 0 SUCCEEDED<br />
15-DEC-04 09.00.31.421000 AM +11:00 HIRE 0 SUCCEEDED<br />
I suppose the critical column in this report is the last one: I can see at a glance my job ran to a successful completion every time it was executed. The alternative STATUS, of course, would be FAILURE. I’ve shown the CPU_USED column here not because it is actually of much use to me in this trivial test environment, but because it’s nice to know that you can readily assess the impact which jobs have on the resources left available for the rest of the database. In a production setting, you’d have more realistic results here and it would be important to keep an eye on it: too much CPU spent on executing jobs means not enough CPU left for other users, after all. There are also other columns in the view, which I haven’t shown here, that can tell you, for example, the difference between the time a job was supposed to start and when it actually started. There’s also a RUN_DURATION column, so you don’t have to do any clock arithmetic to work out how long it takes for a job to finish.<br />
5.0 How does it work?<br />
When you create a job, it is entered as a new row in a database-wide scheduled jobs table. You can view the contents of that table by selecting from the new USER_SCHEDULER_JOBS view (and there is of course an equivalent DBA_ view, too):<br />
SQL> column job_name format a10<br />
SQL> column job_creator format a10<br />
SQL> column job_action format a10<br />
SQL> column start_date format a40<br />
SQL> column repeat_interval format a30<br />
<br />
SQL> select job_name, job_creator, job_type, job_action,start_date<br />
2 from user_scheduler_jobs;<br />
<br />
JOB_NAME JOB_CREATO JOB_TYPE JOB_ACTION START_DATE ---------- ---------- ---------------- ---------- ----------------------------------------<br />
HIRE SCOTT STORED_PROCEDURE NEW_EMP 15-FEB-05 09.33.07.500000 AM +11:00<br />
As you can see, the table pretty much stores word-for-word what you originally submitted when creating the job. Bear in mind that there is an older view available in a 10g database: USER_JOBS:<br />
SQL> select * from user_jobs;<br />
no rows selected<br />
...and as this short query tells you, the new 10g Scheduler has absolutely nothing to do with that older view. That’s there for people who continue to use DBMS_JOB as their scheduling mechanism, but the point is that they are two completely different mechanisms, and they have nothing to do with each other. So the views relating to one will not show any details concerning the other. And if I haven’t already made it clear, DBMS_JOB is there in 10g for backwards compatibility only: you are strongly urged to make the move to Scheduler and, to be quite frank, you’d be mad not to in any case. Scheduler, after all, is heaps more flexible and capable than tired old DBMS_JOB.<br />
As for how jobs get performed... Well, you’ll need to have a look at the various background processes to work that out:<br />
SQL> connect / as sysdba<br />
Connected.<br />
<br />
SQL> select distinct program from v$process;<br />
<br />
PROGRAM<br />
--------------------------------------------<br />
ORACLE.EXE (CJQ0)<br />
ORACLE.EXE (CKPT)<br />
ORACLE.EXE (DBW0)<br />
ORACLE.EXE (J000)<br />
ORACLE.EXE (J001)<br />
ORACLE.EXE (LGWR)<br />
ORACLE.EXE (MMAN)<br />
ORACLE.EXE (MMNL)<br />
ORACLE.EXE (MMON)<br />
ORACLE.EXE (PMON)<br />
ORACLE.EXE (QMNC)<br />
ORACLE.EXE (RECO)<br />
ORACLE.EXE (SHAD)<br />
ORACLE.EXE (SMON)<br />
ORACLE.EXE (q000)<br />
PSEUDO<br />
<br />
16 rows selected.<br />
This is an example from a 10g database running on Windows, of course, and I’ve therefore had to use the V$PROCESS view (as SYS) to see the assorted background processes. If you were running on Linux or Unix, you could simply grep for your ORACLE_SID:<br />
[oracle@garnet oracle]$ echo $ORACLE_SID<br />
lx10<br />
[oracle@garnet oracle]$ ps -ef | grep lx10<br />
oracle 2125 1 0 09:26 ? 00:00:00 ora_pmon_lx10<br />
oracle 2127 1 0 09:26 ? 00:00:00 ora_mman_lx10<br />
oracle 2129 1 0 09:26 ? 00:00:00 ora_dbw0_lx10<br />
oracle 2131 1 0 09:26 ? 00:00:00 ora_lgwr_lx10<br />
oracle 2133 1 0 09:26 ? 00:00:00 ora_ckpt_lx10<br />
oracle 2135 1 0 09:26 ? 00:00:00 ora_smon_lx10<br />
oracle 2137 1 0 09:26 ? 00:00:00 ora_reco_lx10<br />
oracle 2139 1 0 09:26 ? 00:00:00 ora_cjq0_lx10<br />
oracle 2145 1 0 09:26 ? 00:00:00 ora_qmnc_lx10<br />
oracle 2152 1 0 09:26 ? 00:00:00 ora_mmon_lx10<br />
oracle 2154 1 0 09:26 ? 00:00:00 ora_mmnl_lx10<br />
oracle 2191 1 0 09:32 ? 00:00:00 ora_q000_lx10<br />
oracle 2201 2083 0 09:33 pts/0 00:00:00 grep lx10<br />
In either case, you can see a process there with a name of, or including, CJQ0. That is the Co-ordinator of the Job Queue process -and as you can tell from the zero in his name, there can be more than one of them if the database is really busy. It is this process which scans the database-wide scheduled jobs table, and spawns slave processes to perform any jobs found there that are due to be performed.<br />
There is nothing to configure about the Co-ordinator: you don’t get to specify how many of them there are, nor how often or frequently it/they should check the scheduled jobs table. It is all entirely automatic. Similarly, the job queue slaves are not configurable. They certainly don’t depend on the setting of the JOB_QUEUE_PROCESSES initialisation parameter, for example.{mospagebreak title=Managing Jobs}<br />
6.0 Managing Jobs<br />
Once a job has been created, you’ll maybe want to suspend it, stop it, resume it and/or delete it. At least, I hope you will, because that’s what I’ll cover next!<br />
6.1 Suspending/Resuming a Job<br />
I’ve already touched on the fact that Jobs are created, by default, in a disabled state. I then went on to create my job in a fully-enabled state from the start, but that gives you the clue as to how you can suspend and resume a job: it simply comes down to knowing how to enable or disable a job after it has been created, and that involves using just two new procedures within the DBMS_SCHEDULER package: ENABLE and (oddly enough!) DISABLE:<br />
SQL> show user<br />
USER is "SCOTT"<br />
<br />
SQL> exec dbms_scheduler.enable('HIRE')<br />
PL/SQL procedure successfully completed.<br />
<br />
SQL> exec dbms_scheduler.disable('HIRE')<br />
PL/SQL procedure successfully completed.<br />
The procedures simply take one argument: the name of the job to be enabled or disabled. Nothing too complex there, then! <br />
One minor complication could arise if a job is currently running at the very moment you seek to disable it. If that is true, then the disable command I’ve shown here will fail with an error. However, you can force the issue by adding one extra parameter to the command:<br />
SQL> exec dbms_scheduler.disable('HIRE',TRUE)<br />
PL/SQL procedure successfully completed.<br />
That second parameter is the force one, and is either true or false (and by default, it’s false). If you set it to true, then the disabling of the job will definitely occur without error -although the currently-running instance of the job will still be allowed to run to completion.<br />
Once a job has been suspended in this way, it will stay that way until you change its status to enabled once more. That means no further runs of that job will take place until you give the go-ahead.<br />
6.2 Stopping a Job<br />
If you don’t want to semi-permanently prevent a job from running (by disabling it), you might nevertheless want to interrupt one, specific running instance of a job. That requires you to use the STOP_JOB procedure of the DBMS_SCHEDULER package (and yes, in passing, I find it incredibly irritating that some of the procedures are one-word ones -such as ENABLE or DISABLE- and some are two-word ones -such as STOP_JOB. There’s a reason for that, as we’ll see in Part II):<br />
SQL> exec dbms_scheduler.stop_job('HIRE')<br />
BEGIN dbms_scheduler.stop_job('HIRE'); END;<br />
<br />
*<br />
ERROR at line 1:<br />
ORA-27366: job "SCOTT.HIRE" is not running<br />
ORA-06512: at "SYS.DBMS_ISCHED", line 162<br />
ORA-06512: at "SYS.DBMS_SCHEDULER", line 398<br />
ORA-06512: at line 1<br />
As you can see -quite reasonably- you get an error if you try stopping a job that isn’t actually running at that moment. Had the job been running, however, then the job slaves would have gracefully halted their work. That could take some time, and in any case the job processes might not be able to halt the execution of the job -it rather depends on the nature of the job itself, of course- and in that case the command will return yet another (though different) error.<br />
If you’ve tried gracefully stopping a job, but received that ‘no can do’ error, then you’ll want to be a bit more forceful about things, and that requires that you supply a new argument to the procedure:<br />
SQL> exec dbms_scheduler.stop_job('HIRE',TRUE)<br />
BEGIN dbms_scheduler.stop_job('HIRE',TRUE); END;<br />
<br />
*<br />
ERROR at line 1:<br />
ORA-27486: insufficient privileges<br />
ORA-06512: at "SYS.DBMS_ISCHED", line 162<br />
ORA-06512: at "SYS.DBMS_SCHEDULER", line 398<br />
ORA-06512: at line 1 <br />
Well, this is how you’d do it: once again, there’s a FORCE argument to be supplied, and once again it takes a value of TRUE or FALSE, with FALSE being the ‘graceful stop’ default. As you can see, though, my attempt to specify that FORCE should be used to stop the job fails miserably with an ‘insufficient privileges’ error message. That’s because merely being able to create a job, as Scott can, doesn’t give you the right to create merry havoc on the database by aborting a job in mid-stream. For that, you’ll need the MANAGE SCHEDULER system privilege, which I’ll discuss in more detail in Part II.<br />
6.3 Deleting a Job<br />
If you want to completely obliterate a job from the system, rather than just interrupting or suspending it, then you’ll need the to use the DROP_JOB procedure:<br />
SQL> exec dbms_scheduler.drop_job('HIRE')<br />
PL/SQL procedure successfully completed.<br />
As you might have expected by now, it takes the name of the job as its sole argument. The entry in the scheduled jobs table (and hence in the USER_SCHEDULER_JOBS view) is removed completely, and it is as though the job had never been created. And, as you also might have expected by now, there is an optional FORCE parameter which can be TRUE or FALSE (the default is FALSE). The issue is whether you should be allowed to delete/drop a job when that job is currently running. If you don’t force the matter, then a running job cannot be dropped and the command will generate an error; specify TRUE, however, and Oracle will stop the job for you and then proceed with the job deletion.<br />
7.0 Conclusion<br />
That’s about all I want to say about Scheduler in this article. Hopefully, you can see it’s not a monster: in fact, it’s quite easy to use for simple scheduling, provided you can work out those calendaring expressions for the repeat intervals (and they’re not too difficult after a bit of practice). Even in this simplest of descriptions, though, I’ve hinted at capabilities that the old DBMS_JOB package never had -such as the ability to run an executable that is stored outside the database. We used to have to use cron or at to run those sorts of things... and then often wished we hadn’t, because the O/S scheduler would run the executable when the database happened to be down! With Scheduler, the executables can still be run, but it’s the database that’s doing the running. So if the database is down, no worries: the executable simply won’t be executed.<br />
Scheduler is a nice touch, therefore. It’s not one of those all-singing, all-dancing bits of functionality you wonder how we ever managed without. But it’s welcome nonetheless. Certainly recommended (and certainly recommended that you stop using DBMS_JOB).KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com1tag:blogger.com,1999:blog-8692881074449232849.post-39637784358264015212009-10-06T08:24:00.001-07:002009-10-06T08:24:15.528-07:00Oracle DBA QuestionsHow do I find out who is currently using the space in the temporary tablespace ?<br />
SELECT SE.USERNAME,<br />
SE.SID,<br />
SU.EXTENTS,<br />
SU.BLOCKS * TO_NUMBER(RTRIM(P.VALUE)) AS SPACE,<br />
TABLESPACE,<br />
SEGTYPE<br />
FROM V$SORT_USAGE SU, V$PARAMETER P, V$SESSION SE<br />
WHERE P.NAME = 'db_block_size' AND SU.SESSION_ADDR = SE.SADDR<br />
ORDER BY SE.USERNAME, SE.SID<br />
<br />
I'm trying to make a tablespace read only, but it seems to hang forever. What's wrong ?<br />
If you do a very large update in some tablespace – large enough to ensure that at least some of the updated blocks are flushed from the buffer - commit, and then make that tablespace read only, you will find that some of the blocks in the read only tablespace have not been cleaned out and can no longer be cleaned out. This means that whenever you read any of those blocks from disc Oracle will run through some of its clean-out code – specifically making a call to check the control table from the undo segment header identified in the ITL entry that your transaction used in the data block header (Call ktuwh05: ktugct for those who care about these things).<br />
The overhead is not huge – but there’s no point in paying it. So consider forcing a scan of all objects in a tablespace before switching it to read only mode to make Oracle clean out any blocks that need it. Note – however, that there are at least a couple of reasons why this might be a pointless use of resources. For example, any blocks that were in memory and not cleaned out when you switched the tablespace to read only mode will be cleaned before being written. Moreover, if the contents of the tablespace have been created through ‘create as select’ they will have been created as clean. If there are any other reasons, I can’t think of them right now.<br />
There is a beneficial side-effect to this restriction too. Oracle does not clean out all changed blocks at commit time, and some blocks will end up written to disc in a dirty state. These blocks are normally cleaned out the next time that they are read, but this puts them into a new, dirty, state that requires them to be written again, it can also be an expensive process because it can require Oracle to generate a historic image of the rollback segment header blocks. If a dirty block is found in a read only tablespace though, Oracle knows that it must have been committed before the tablespace was made readonly, and therefore never needs to worry about cleaning the block out and re-writing it - it has a good, cheap, approximation to the commit SCN from the SCN at which the tablespace went read only.<br />
How can I perform bitwise operations in Oracle ? <br />
Ever since Oracle 7 and probably even previous, Oracle has contained a BITAND function that returns the logical AND of two integers. Somewhere around 8i they finally decided to stick it in the documentation. You can safely rely on this function being retained in future versions of Oracle, because it is used extensively throughout the data dictionary. For example, a column is deemed to be "hidden" if bitand(property, 32) = 0 on the SYS.COL$ table. <br />
Note that BITAND returns a binary integer, so to use it in SQL you will need to convert it to a numeric.<br />
SQL> select bitand(49,54) from dual;<br />
ERROR:<br />
ORA-00932: inconsistent datatypes<br />
<br />
<br />
SQL> select bitand(49,54)+0 from dual;<br />
<br />
BITAND(49,54)+0<br />
---------------<br />
48<br />
Using this function, its trivial to build the other logical operations<br />
function bitor(p_dec1 number, p_dec2 number) return number is<br />
begin<br />
return p_dec1-bitand(p_dec1,p_dec2)+p_dec2;<br />
end;<br />
<br />
function bitxor(p_dec1 number, p_dec2 number) return number is<br />
begin<br />
return bitor(p_dec1,p_dec2)-bitand(p_dec1,p_dec2);<br />
-- or you could use: return p_dec1-2*bitand(p_dec1,p_dec2)+p_dec2;<br />
end;<br />
Note that BITAND only supports positive integers, so if you want to include support for negatives you will need to wrap some of your own code around the above to handle them. <br />
Because many people were unaware of the BITAND builtin, they often sought solace in the UTL_RAW package which looks promising when you describe it. <br />
SQL> desc UTL_RAW<br />
FUNCTION BIT_AND RETURNS RAW<br />
Argument Name Type In/Out Default?<br />
------------------------------ ----------------------- ------ --------<br />
R1 RAW IN<br />
R2 RAW IN<br />
<br />
FUNCTION BIT_COMPLEMENT RETURNS RAW<br />
Argument Name Type In/Out Default?<br />
------------------------------ ----------------------- ------ --------<br />
R RAW IN<br />
<br />
FUNCTION BIT_OR RETURNS RAW<br />
Argument Name Type In/Out Default?<br />
------------------------------ ----------------------- ------ --------<br />
R1 RAW IN<br />
R2 RAW IN<br />
<br />
FUNCTION BIT_XOR RETURNS RAW<br />
Argument Name Type In/Out Default?<br />
------------------------------ ----------------------- ------ --------<br />
R1 RAW IN<br />
R2 RAW IN<br />
But you need to be careful here, as can be best shown with an example. First we'll use the standard BITAND to see what the real result should be:<br />
SQL> exec dbms_output.put_line(bitand(147,63));<br />
19<br />
<br />
PL/SQL procedure successfully completed.<br />
Now we'll try UTL_RAW to accomplish the same thing<br />
SQL> declare<br />
2 x number := 147;<br />
3 y number := 63;<br />
4 begin<br />
5 dbms_output.put_line(utl_raw.bit_and(x,y));<br />
6 end;<br />
7 /<br />
<br />
ERROR at line 1:<br />
ORA-06550: line 5, column 24:<br />
PLS-00306: wrong number or types of arguments in call to 'BIT_AND'<br />
ORA-06550: line 5, column 3:<br />
PL/SQL: Statement ignored<br />
This is because the UTL_RAW package (as the name suggests) only works with RAW data. "No problem" we think, we'll just do some datatype conversions...<br />
SQL> declare<br />
2 x number := 147;<br />
3 y number := 63;<br />
4 x_raw raw(4) := utl_raw.cast_to_raw(x);<br />
5 y_raw raw(4) := utl_raw.cast_to_raw(y);<br />
6 begin<br />
7 dbms_output.put_line(utl_raw.bit_and(x_raw,y_raw));<br />
8 end;<br />
9 /<br />
303037<br />
<br />
PL/SQL procedure successfully completed.<br />
So the PL/SQL worked, but the result is not what we were expecting. Then you may click onto the fact that since the result is raw, an additional conversion is needed:<br />
SQL> declare<br />
2 x number := 147;<br />
3 y number := 63;<br />
4 x_raw raw(4) := utl_raw.cast_to_raw(x);<br />
5 y_raw raw(4) := utl_raw.cast_to_raw(y);<br />
6 begin<br />
7 dbms_output.put_line(utl_raw.cast_to_varchar2(utl_raw.bit_and(x_raw,y_raw)));<br />
8 end;<br />
9 /<br />
007<br />
<br />
PL/SQL procedure successfully completed.<br />
Still no joy! The reason for this behaviour is that we not taking into account all of the datatype conversions that are taking place. What is really happening is: <br />
The number 147 is converted to varchar2 '147', which as a raw is "31,34,37" (that is, ascii 49, ascii 52, ascii 55). Similarly, 63 becomes raw "36,33" (ascii 54, ascii 51). It is these values that get passed to the BIT_AND function, which gives<br />
49 52 55<br />
AND 54 51 -<br />
==================<br />
48 48 55<br />
which when converted back to varchar2 yields 007. <br />
It is possible to use UTL_RAW, but you just need to take care with all of the conversions. A package to do this is: <br />
create or replace<br />
package bitops2 is<br />
<br />
function bitand(p_dec1 number, p_dec2 number) return varchar2 ;<br />
function bitor(p_dec1 number, p_dec2 number) return varchar2 ;<br />
function bitxor(p_dec1 number, p_dec2 number) return varchar2 ;<br />
<br />
end;<br />
/<br />
create or replace<br />
package body bitops2 is<br />
<br />
function raw_ascii(p_dec number) return raw is<br />
v_result varchar2(1999);<br />
v_tmp1 number := p_dec;<br />
begin<br />
loop<br />
v_result := chr(mod(v_tmp1,256)) || v_result ;<br />
v_tmp1 := trunc(v_tmp1/256);<br />
exit when v_tmp1 = 0;<br />
end loop;<br />
return utl_raw.cast_to_raw(v_result);<br />
end;<br />
<br />
function ascii_raw(p_raw varchar2) return number is<br />
v_result number := 0;<br />
begin<br />
for i in 1 .. length(p_raw) loop<br />
v_result := v_result * 256 + ascii(substr(p_raw,i,1));<br />
end loop;<br />
return v_result;<br />
end;<br />
<br />
function bitand(p_dec1 number, p_dec2 number) return varchar2 is<br />
begin<br />
return<br />
ascii_raw(<br />
utl_raw.cast_to_varchar2(<br />
utl_raw.bit_and(<br />
raw_ascii(p_dec1),<br />
raw_ascii(p_dec2)<br />
)<br />
)<br />
);<br />
end;<br />
<br />
function bitor(p_dec1 number, p_dec2 number) return varchar2 is<br />
begin<br />
return<br />
ascii_raw(<br />
utl_raw.cast_to_varchar2(<br />
utl_raw.bit_or(<br />
raw_ascii(p_dec1),<br />
raw_ascii(p_dec2)<br />
)<br />
)<br />
);<br />
end;<br />
<br />
function bitxor(p_dec1 number, p_dec2 number) return varchar2 is<br />
begin<br />
return<br />
ascii_raw(<br />
utl_raw.cast_to_varchar2(<br />
utl_raw.bit_xor(<br />
raw_ascii(p_dec1),<br />
raw_ascii(p_dec2)<br />
)<br />
)<br />
);<br />
end;<br />
<br />
end;<br />
/<br />
If your bitwise requirements extend larger than the maximum size of the number datatype, or if you would rather deal with bit strings directly, the following package will assist:<br />
create or replace<br />
package bitops is<br />
<br />
function bitand(p_bit1 varchar2, p_bit2 varchar2) return varchar2 ;<br />
function bitor(p_bit1 varchar2, p_bit2 varchar2) return varchar2 ;<br />
function bitxor(p_bit1 varchar2, p_bit2 varchar2) return varchar2 ;<br />
<br />
end;<br />
/<br />
<br />
create or replace<br />
package body bitops is<br />
<br />
function bitand(p_bit1 varchar2, p_bit2 varchar2) return varchar2 is<br />
v_result varchar2(1999);<br />
v_tmp1 varchar2(1999) := p_bit1;<br />
v_tmp2 varchar2(1999) := p_bit2;<br />
v_len1 number := length(p_bit1);<br />
v_len2 number := length(p_bit2);<br />
v_len number := greatest(v_len1,v_len2);<br />
begin<br />
v_tmp1 := lpad(v_tmp1,v_len,'0');<br />
v_tmp2 := lpad(v_tmp2,v_len,'0');<br />
for i in 1 .. v_len loop<br />
v_result := v_result || least(substr(v_tmp1,i,1),substr(v_tmp2,i,1));<br />
end loop;<br />
return v_result;<br />
end;<br />
<br />
function bitor(p_bit1 varchar2, p_bit2 varchar2) return varchar2 is<br />
v_result varchar2(1999);<br />
v_tmp1 varchar2(1999) := p_bit1;<br />
v_tmp2 varchar2(1999) := p_bit2;<br />
v_len1 number := length(p_bit1);<br />
v_len2 number := length(p_bit2);<br />
v_len number := greatest(v_len1,v_len2);<br />
begin<br />
v_tmp1 := lpad(v_tmp1,v_len,'0');<br />
v_tmp2 := lpad(v_tmp2,v_len,'0');<br />
for i in 1 .. v_len loop<br />
v_result := v_result || greatest(substr(v_tmp1,i,1),substr(v_tmp2,i,1));<br />
end loop;<br />
return v_result;<br />
end;<br />
<br />
function bitxor(p_bit1 varchar2, p_bit2 varchar2) return varchar2 is<br />
v_result varchar2(1999);<br />
v_tmp1 varchar2(1999) := p_bit1;<br />
v_tmp2 varchar2(1999) := p_bit2;<br />
v_len1 number := length(p_bit1);<br />
v_len2 number := length(p_bit2);<br />
v_len number := greatest(v_len1,v_len2);<br />
begin<br />
v_tmp1 := lpad(v_tmp1,v_len,'0');<br />
v_tmp2 := lpad(v_tmp2,v_len,'0');<br />
for i in 1 .. v_len loop<br />
v_result := v_result || to_char(abs(to_number(substr(v_tmp1,i,1))-to_number(substr(v_tmp2,i,1))));<br />
end loop;<br />
return v_result;<br />
end;<br />
<br />
end;<br />
/<br />
<br />
SQL> declare<br />
2 x varchar2(20) := '10101010101010101010';<br />
3 y varchar2(20) := '11110000111100001111';<br />
4 begin<br />
5 dbms_output.put_line(bitops.bitand(x,y));<br />
6 end;<br />
7 /<br />
10100000101000001010<br />
<br />
PL/SQL procedure successfully completed.<br />
<br />
Addenda <br />
Bart Pots has extended the package to handle bit strings of differing length and provided numeric input and output. The code is below: <br />
create or replace<br />
package bitops2 is<br />
<br />
function raw_ascii(p_dec number) return raw;<br />
function ascii_raw(p_raw varchar2) return number;<br />
<br />
function bitand(p_dec1 number, p_dec2 number) return number;<br />
function bitor(p_dec1 number, p_dec2 number) return number;<br />
function bitxor(p_dec1 number, p_dec2 number) return number;<br />
<br />
end;<br />
/<br />
create or replace<br />
package body bitops2 is<br />
<br />
function raw_ascii(p_dec number) return raw is<br />
<br />
v_result varchar2(1999);<br />
v_tmp1 number := p_dec;<br />
<br />
begin<br />
<br />
loop<br />
<br />
v_result := chr(mod(v_tmp1,256)) || v_result ;<br />
v_tmp1 := trunc(v_tmp1/256);<br />
<br />
exit when v_tmp1 = 0;<br />
<br />
end loop;<br />
<br />
return utl_raw.cast_to_raw(v_result);<br />
end;<br />
<br />
function ascii_raw(p_raw varchar2) return number is<br />
<br />
v_result number := 0;<br />
<br />
begin<br />
<br />
for i in 1 .. length(p_raw) loop<br />
<br />
v_result := v_result * 256 + ascii(substr(p_raw,i,1));<br />
<br />
end loop;<br />
<br />
return v_result;<br />
<br />
end;<br />
function bitand(p_dec1 number, p_dec2 number) return number is<br />
<br />
v_tmp_raw1 raw(1999);<br />
v_tmp_raw2 raw(1999);<br />
<br />
nr_diff_length number(4,0);<br />
<br />
nr_pos number(4,0);<br />
<br />
v_tmp_raw_zero raw(1999) := raw_ascii(0);<br />
<br />
begin<br />
<br />
v_tmp_raw1 := raw_ascii(p_dec1);<br />
v_tmp_raw2 := raw_ascii(p_dec2);<br />
<br />
nr_diff_length := greatest( utl_raw.length(v_tmp_raw1),<br />
utl_raw.length(v_tmp_raw2));<br />
<br />
for nr_pos in utl_raw.length(v_tmp_raw1) + 1.. nr_diff_length loop<br />
<br />
v_tmp_raw1 := utl_raw.concat( v_tmp_raw_zero,v_tmp_raw1 );<br />
<br />
end loop;<br />
<br />
for nr_pos in utl_raw.length(v_tmp_raw2) + 1 .. nr_diff_length loop<br />
<br />
v_tmp_raw2 := utl_raw.concat( v_tmp_raw_zero,v_tmp_raw2 );<br />
<br />
end loop;<br />
<br />
return ascii_raw( utl_raw.cast_to_varchar2( utl_raw.bit_and( v_tmp_raw1, v_tmp_raw2)));<br />
<br />
end;<br />
<br />
function bitor(p_dec1 number, p_dec2 number) return number is<br />
<br />
v_tmp_raw1 raw(1999);<br />
v_tmp_raw2 raw(1999);<br />
<br />
nr_diff_length number(4,0);<br />
<br />
nr_pos number(4,0);<br />
<br />
v_tmp_raw_zero raw(1999) := raw_ascii(0);<br />
<br />
begin<br />
<br />
v_tmp_raw1 := raw_ascii(p_dec1);<br />
v_tmp_raw2 := raw_ascii(p_dec2);<br />
<br />
nr_diff_length := greatest( utl_raw.length(v_tmp_raw1),<br />
utl_raw.length(v_tmp_raw2));<br />
<br />
for nr_pos in utl_raw.length(v_tmp_raw1) + 1.. nr_diff_length loop<br />
<br />
v_tmp_raw1 := utl_raw.concat( v_tmp_raw_zero,v_tmp_raw1 );<br />
<br />
end loop;<br />
<br />
for nr_pos in utl_raw.length(v_tmp_raw2) + 1 .. nr_diff_length loop<br />
<br />
v_tmp_raw2 := utl_raw.concat( v_tmp_raw_zero,v_tmp_raw2 );<br />
<br />
end loop;<br />
<br />
return ascii_raw( utl_raw.cast_to_varchar2( utl_raw.bit_or( v_tmp_raw1, v_tmp_raw2)));<br />
<br />
end;<br />
<br />
function bitxor(p_dec1 number, p_dec2 number) return number is<br />
<br />
v_tmp_raw1 raw(1999);<br />
v_tmp_raw2 raw(1999);<br />
<br />
nr_diff_length number(4,0);<br />
<br />
nr_pos number(4,0);<br />
<br />
v_tmp_raw_zero raw(1999) := raw_ascii(0);<br />
<br />
begin<br />
<br />
v_tmp_raw1 := raw_ascii(p_dec1);<br />
v_tmp_raw2 := raw_ascii(p_dec2);<br />
<br />
nr_diff_length := greatest( utl_raw.length(v_tmp_raw1),<br />
utl_raw.length(v_tmp_raw2));<br />
<br />
for nr_pos in utl_raw.length(v_tmp_raw1) + 1.. nr_diff_length loop<br />
<br />
v_tmp_raw1 := utl_raw.concat( v_tmp_raw_zero,v_tmp_raw1 );<br />
<br />
end loop;<br />
<br />
for nr_pos in utl_raw.length(v_tmp_raw2) + 1 .. nr_diff_length loop<br />
<br />
v_tmp_raw2 := utl_raw.concat( v_tmp_raw_zero,v_tmp_raw2 );<br />
<br />
end loop;<br />
<br />
return ascii_raw( utl_raw.cast_to_varchar2( utl_raw.bit_xor( v_tmp_raw1, v_tmp_raw2)));<br />
<br />
end;<br />
end;<br />
<br />
How can I tune a tablescan?<br />
This particular example is typical of the generic case. We have a query like the following on a 40GB table, but the index is ‘not selective’:<br />
select<br />
{list of columns}<br />
from<br />
very_large_table t<br />
where<br />
colX = 'ABC' <br />
order by<br />
colY<br />
;<br />
The first problem with this question is the one of ambiguity – when the poster says “cost” do they really mean the figure reported in the COST column of the execution plan table, or do they mean the resource consumption (e.g. number of physical disk reads) when the query actually runs. <br />
In the former case, it is not necessarily safe to assume that there will be a direct correlation between the COST, the resources used, and the run-time. In the latter, there may not be a direct correlation between resource usage (e.g. number of physical disk I/Os) and the run-time when the query is transferred from development to production, because there may be more competition for physical resources on production, so response time may drop.<br />
So how can you tune a full table scan for very large tables? In a very real sense, you can’t. Once your are committed to a full tablescan, that’s it – you have to scan every block (and every row) in the table typically using multiblock reads. The block scans are likely to generate a lot of I/O; the row examinations will consume a lot of CPU. <br />
Of course, there are a few steps you can take to minimize the resource requirement – but their side effects need to be considered carefully. <br />
Can you make the scan times faster? <br />
In principle, yes, a little. Increasing the size of the parameter db_file_mulitblock_read_count so that the read requests are fewer and larger may help. In practice it is not always that simple. Apart from the side-effects on the optimizer’s calculation, you may find that odd interference effects from the various hardware levels actually make it hard to find an optimum setting for the parameter. Things like disk track size, disk buffer size, network buffer size, O/S stripe size and so on can result in some very odd “impedance” effects. You may have to find a best fit for your system by a tedious process of trial and error.<br />
In practice, you may find that the hardware “sees” your big tablescan starting, and adopts a read-ahead strategy that makes your tuning attempt redundant. (This is actually a problem with mixed-load SANs, they tend to get over-enthusiastic about tablescans at the cost of random I/Os – so on average their rate of throughput can look good while the end-users (and v$session_event) are complaining about slow disk response times)<br />
Would parallelism help ?<br />
In principle, yes; but only for the query itself. For a tablescan of degree N the table would effectively be broken up into N parts, allowing N separate processes to scan and extract the required data before forwarding the minimum possible data volume on to another set of processes to handle the sorting. So long as the number of slaves was not sufficient to overload the I/O subsystem, and provided the slaves didn’t end up colliding on the same disks all the time, then the speed of the scan would improve by a factor of roughly N.<br />
On the other hand, by making this query run as rapidly as possible – which means the highest degree of parallelism you can get away with – you are likely to put a lot of stress on the I/O subsystem – to you really want to do that, as it will probably slow everything else down.<br />
Can you do something to cache the table? <br />
At 40 GB, probably not (but who knows, maybe you have 128GB of memory to play with and that might be enough to let you put this table into a keep pool) and you’re still going to be using a lot of CPU looking at all those rows whatever you do about caching.<br />
Is it really true that the index has very poor selectivity?<br />
If so, why are you running a query that (apparently) is going to fetch a huge number of rows? Do you really need to fetch all those rows, or are you really after just the first few? <br />
If the volume is actually relatively low, then perhaps a “bad” index is still better than doing a tablescan – it may protect your I/O subsystem for other users. <br />
If the volume of data is high but you only want the first few rows, how about (in the example above) a “function-based index” of the form decode(colX,’ABC’,colY,null) – so holds entries for just the rows you are interested in, with the index entries in the right order for your order by clause. If you do this, you could re-write your query to force a range scan through this index, stopping after a few rows, rather than acquiring all the rows and sorting them before discarding most of them.<br />
Would partitioning (on the colX column in this case) work for you?<br />
A list partition strategy, or possibly even a hash partition strategy, could break the table into a reasonable number of much smaller chunks. You would still have to do a ‘full table scan’ but it would be a scan of just one partition, which could be much smaller, so use less resources, run more quickly, and cause much less damage. But can you partition on this column – or would it interfere with all the other functions of the system; and if it does could it still be worth the penalty? <br />
So, although you can’t really “tune” a large tablescan, there are some strategies you can investigate to see if you can find ways of reducing or restructuring the amount of work that you need to do. Whether any of them is appropriate for your system only you can choose.<br />
Why is dbms_stats so much slower than Analyze?<br />
The person who last posed this question didn’t mention a version number – but the problem appears in many versions of Oracle as you make the change from one technology to the other, mainly because the activity carried out by the dbms_stats does not, by default, match the activity carried out by the analyze command. <br />
Not only does dbms_stats differ from the analyze command in its behaviour, virtually every version of Oracle has introduced a few extra parameters in the calls to dbms_stats and even changed some of the default values from previous versions, so that calls to dbms_stats that used to complete in a timely fashion in one version of Oracle suddenly take much longer after an upgrade – it’s not just the switch from analyze to dbms_stats that causes problems.<br />
In the worst case, when left to its own devices, dbms_stats in 10g will work out for itself the best sample size to use on the tables and indexes, which columns to create histograms for, the number of buckets in the histograms, and the sample size to use for those histograms. The volume of work may be much larger than anything you would choose to do yourself.<br />
The first guideline for using dbms_stats is: read the manual – or the script $ORACLE_HOME/rdbms/admin/dbmsstat.sql to see what it does, and how it changes from release to release. This gives you some chance of making it do what you used to do with the analyze command. The second guideline is to try a test run with lots of tracing (e.g. events 10046, 10033, and calls to v$session_event and v$sesstat) set so that you make sure that you can see how much work the tools do, and where they are losing time.<br />
Partitioned tables have always been a particular problem for statistics – and Oracle is constantly fiddling with the code to try and reduce the impact of collecting reasonable statistics. If you have large partitioned tables, and don’t have to collect global statistics, bear in mind that you probably know more about the data than Oracle does. The best strategy for partitioned tables (probably) is to write your own code that limits Oracle to collecting statistics on the most recently added partition and then derives reasonable table level statistics programmatically by combining the latest statistics with the existing table stats. Look very carefully at the procedure with names like get_column_stats, set_column_stats. <br />
Can Bind Variable Peeking cause problems without a histogram on Predicate columns ?<br />
It is a well known fact that the reason for the majority of problems with the bind variable peek feature is an histogram on the column referenced in the access predicate. The histogram is certainly the main but not the only cause for a non appropriate execution plan. Another less known situation where a different value of bind variables can lead to a change of execution plan is an access predicate on a (sub)partitioned table. This is a particularly important scenario in case of range partitioned fact tables organized as rolling windows. These tables contain two types of partitions, those filled up and those pre allocated to future loads. As the optimiser statistics of both types are very different, the risk of getting the wrong execution plan in case of peeking “in the wrong partition” is relatively high.<br />
How to generate an Execution Plan with Bind Variable Peeking?<br />
Bind variable peeking introduced in Oracle Release 9 can be extremely helpful in some situations where the additional information leads to a better execution plan. On the other side this feature makes it much more difficult to see the 'real' execution plan using explain plan or autotrace for statements with bind variables as those tools don't perform the peek. The trivial answer to the question above is to substitute the bind variable with a literal value but there are some subtle issues with variable data type that could lead to a different plan.<br />
Why do I keep seeing tables with garbage names like BIN${something}<br />
In 10g, Oracle introduced an option for “undropping” a table, which we can demonstrate with the following script in 10g Release 2:<br />
create table t1(n1 number);<br />
create index i1 on t1(n1);<br />
select table_name from user_tables;<br />
select index_name from user_indexes;<br />
select object_name, object_type from user_objects;<br />
drop table t1;<br />
select table_name from user_tables;<br />
select index_name from user_indexes;<br />
select object_name, object_type from user_objects;<br />
flashback table t1 to before drop;<br />
select table_name from user_tables;<br />
select index_name from user_indexes;<br />
select object_name, object_type from user_objects;<br />
In my test on 10.2.0.1, the queries after the create table and index showed the following (expectd) results:<br />
TABLE_NAME<br />
--------------------------------<br />
T1<br />
INDEX_NAME<br />
--------------------------------<br />
I1<br />
OBJECT_NAME OBJECT_TYPE<br />
---------------------------------------- -------------------<br />
I1 INDEX<br />
T1 TABLE<br />
The queries following the drop showed the following:<br />
No rows for user_tables – earlier versions may show a strangely named table<br />
No rows for user_indexes - earlier versions may show a strangely named table<br />
OBJECT_NAME OBJECT_TYPE<br />
---------------------------------------- -------------------<br />
BIN$HULdSlmnRZmbCXAl/pkA9w==$0 TABLE<br />
BIN$pyWpLnQwTbOUB9rQrbwgPA==$0 INDEX<br />
The table and its indexes have not been eliminated from the database, or from the data dictionary. They have been renamed, and hidden from the table and index data dictionary views (at least, they are hidden in later versions of 10g, you could still see them in some of the earlier versions). You can get them back if you want to, you can clear them out explicitly if you want to, and (most importantly) although they count against your space quota, they will vanish spontaneously – i.e. be “properly dropped” if you account needs more space in the relevant tablespace and it isn’t available.<br />
After the “undrop” command, the queries showed the following:<br />
TABLE_NAME<br />
--------------------------------<br />
T1<br />
INDEX_NAME<br />
--------------------------------<br />
BIN$pyWpLnQwTbOUB9rQrbwgPA==$0<br />
OBJECT_NAME OBJECT_TYPE<br />
---------------------------------------- -------------------<br />
T1 TABLE<br />
BIN$pyWpLnQwTbOUB9rQrbwgPA==$0 INDEX<br />
Note how the table name has re-appeared with the correct name, but Oracle has failed to restore the index name properly – although it has made it visible. If you do “undrop” objects, make sure you check the names of dependent objects (including such things as constraints). You may need to rename them. I assume that Oracle has not renamed the secondary objects because there is a risk that you may have created other objects with conflicting names: manual resolution is the only sensible approach. Bear in mind it is also possible for Oracle to get rid of a dropped index when there is pressure for space, so when a table is “undropped”, some of its indexes could actually be missing. Because there are non-standard naming characters in the index name, you will have to quote the name when renaming it, e.g.<br />
alter index " BIN$pyWpLnQwTbOUB9rQrbwgPA==$0" rename to i1;<br />
To get rid of the objects manually, you can use one of three commands (if you have the relevant privilege)<br />
purge user_recyclebin;<br />
purge dba_recyclebin;<br />
purge recyclebin;<br />
Alternatively, if you want to avoid the recycle bin completely, you can change your drop table call:<br />
drop table t1 purge;<br />
Note – when you drop just an index, it does not go into the recycle bin, it really is dropped.<br />
How to pin a table in memory.<br />
Sometimes you wonder if you could pin or keep a table or index in memory. Many application would benefit, if some key tables could be accessed very quick and faster then average. Fast access means, that the table has to be cached in memory, to avoid wait time on disk reads and other waits associated with buffer pool management. <br />
PL/SQL objects can be pinned in SGA, but technically there is no alter table t_name pin. But the trick can be accomplished by clever use of the new buffer pools; default, keep and recycle (version 8 and up). Default pool is not new actually, it's been around and used for at while.<br />
Having a situation, where you want to pin a table in memory i.e. keep the table in the buffer pool. That would accomplish, that the table would not be aged out of the buffer cache i.e. replaced (memory space reused) by other objects through the Least Recently Used (LRU) algoritmn and table access would always be fast. <br />
Prior to version 8, this could not be accomplished. The only alternative, was to alter table t_name cache. The effect was, that full table scans would not go to the least recently used end of the lru list, but to the most recently used end, hereby surviving longer, but seldom forever in the buffer cache. <br />
After version 8, you can control if an object is loaded in default, keep or recycle pool. And that changes things. <br />
Using buffer_pool_keep for pinning a table <br />
First you have to create the keep (and/or recycle) buffer pool by editing the initSID.ora file. (NOTE: there is no difference in the behavior of the three buffer pools, the naming of the pools is merely for memo technically reasons and intended use. <br />
Just moving a table to the keep pool, does not guarantee that the table is always kept, it can be aged out of the keep pool by the LRU algoritmn.) <br />
Part of initSID.ora file using arbitrary numbers (italics is new lines) <br />
db_block_buffers = 65536 <br />
db_block_lru_latches = 4 <br />
buffer_pool_keep = 16384 -- version 8.0.x <br />
buffer_pool_keep = (buffers:16384, lru_latches:1) -- version 8.1.x <br />
buffer_pool_recycle = ......... <br />
After restarting the instance, you can isolate the table(s) in the keep pool by changing table storage <br />
alter table t_name storage ( buffer_pool keep); <br />
Now you can pre page the table and load all table rows into memory by a full table scan as select * from t_name. If the number of blocks the table occupies is less than the number of blocks in the keep pool. (tested on 8.1.7.1, win2k) Or you can let the application populate the buffer pool, having slower access first time the data block is accessed. The advantage is that data blocks never used, won't be loaded, and won't take up valuable memory space. <br />
You might have several indexes associated with the table. You can choose to drop the indexes and always do a full table scan. If the table is small and full scan is fast and cheap. Even memory scans has cpu cost and can give rise to latch contention and memory latency. <br />
The alternative is to cache the index(es) and reserve space for the index(es) in the pool. Remember that the optimizer doesn't know that the table is fully cached, and will try to use index lookup, if not told otherwise by hints like /*+ full (t_name) */. Hints can be over ruled by the optimizer. Test your statements to be sure. <br />
If schema and table design fits, evaluate to convert table and index into IOT, index organized table. That can save memory (and disk) space. <br />
If the table (and index?) is frequently updated, reserve extra space for block copies until the db_writer wakes up and cleans out the dirty blocks. <br />
<br />
Oracle document A76992-01 (EE doc for 8.1.6/7) has following guidelines. <br />
Identifying Segments to Put into the keep and recycle Buffer Pools <br />
A good candidate for a segment to put into the recycle buffer pool is a segment that is at least twice the size of the default buffer pool and has incurred at least a few percent of the total I/Os in the system. <br />
A good candidate for a segment to put into the keep pool is a segment that is smaller than 10% of the size of the default buffer pool and has incurred at least 1% of the total I/Os in the system. <br />
The trouble with these rules is that it can sometimes be difficult to determine the number of I/Os per segment if a tablespace has more than one segment. One way to solve this problem is to sample the I/Os that occur over a period of time by selecting from v$session_wait to determine a statistical distribution of I/Os per segment. <br />
select file#, count(block#), count (distinct file# || block#) <br />
from v$bh <br />
group by file# ; <br />
<br />
For monitoring buffer pool usage, I have created 3 views in sys schema, to help me do a quick check, now and then. I created the views due to complexity and runtime problems (slow running) with a single select. And this works nice. <br />
A view for getting an objects buffer_ pool defaults: <br />
create or replace view oci_buffer_pools <br />
as <br />
select table_name object, buffer_pool from dba_tables <br />
where buffer_pool is not null <br />
union <br />
select table_name object, buffer_pool from dba_tab_partitions <br />
where buffer_pool is not null <br />
union <br />
select table_name object, buffer_pool from dba_tab_subpartitions <br />
where buffer_pool is not null <br />
union <br />
select index_name object, buffer_pool from dba_indexes <br />
where buffer_pool is not null <br />
union <br />
select index_name object, buffer_pool from dba_ind_partitions <br />
where buffer_pool is not null <br />
union <br />
select index_name object, buffer_pool from dba_ind_subpartitions <br />
where buffer_pool is not null <br />
/ <br />
A view to select the objects and types in the buffer cache: <br />
create or replace view oci_block_header as <br />
-- For performance, queries against this view should use cost based optimizer <br />
select b.indx, -- Entry in buffer cache <br />
b.hladdr, -- ADDR of v$latch_children entry for cache buffer chain latch <br />
b.ts# tblspace, -- Tablespace id <br />
b.file# fileid, -- File id <br />
b.dbablk blockid, -- Block id <br />
b.obj objid, -- Object id <br />
u.name owner, -- Object owner <br />
o.name object_name, -- Object name <br />
o.subname subobject_name, -- Subobject name <br />
decode (o.type#, <br />
1, 'INDEX', <br />
2, 'TABLE', <br />
3, 'CLUSTER', <br />
19, 'TABLE PARTITION', <br />
20, 'INDEX PARTITION', <br />
21, 'LOB', <br />
34, 'TABLE SUBPARTITION', <br />
35, 'INDEX SUBPARTITION', <br />
39, 'LOB PARTITION', <br />
40, 'LOB SUBPARTITION', <br />
'UNDEFINED' <br />
) object_type -- Object type <br />
from x$bh b, obj$ o, user$ u <br />
where b.obj = o.dataobj# <br />
and o.owner# = u.user# <br />
/ <br />
And a view to merge the information together and give me the information I want. <br />
create or replace view oci_buffer_cache_use <br />
as <br />
select a.owner, a.object_name, a.object_type, count(a.object_name) blocks#, b.buffer_pool <br />
from oci_block_header a, oci_buffer_pools b <br />
where owner not in ( 'SYS', 'SYSTEM', 'PERFSTAT') <br />
and a.object_name = b.object <br />
group by b.buffer_pool, a.owner, a.object_type, a.object_name <br />
-- having count(a.object_name) > 100 -- if you don't want small objects <br />
/ <br />
<br />
As a curiosity, I will mention that I once tried to cache a very large table (25+ mill rows) on a Sun 6500, running Solaris 7 and 8.1.6EE. <br />
A full table scan would not populate the keep pool. I tried to alter table t_name cache and that didn't help. I never figured why. I had to create a small procedure, selecting rows by primary key lookup in a loop. As all rows were very similar and no empty columns, calculations showed that I could advance primary key number by 400, hereby moving two third of a block forward, select that row and repeat. Hereby having a fair chance of hitting every data lock at least once. The table and index was partitioned in 5 hash partitions. By running 5 procedures parallel, the table loading finished in approx. 16 minutes. <br />
Converting the table to IOT with 5 hash partitions, the same loading procedure lasted only 9 minutes and saved ½ Gbyte memory space.<br />
<br />
Update Feb 2006 – Phillipe Ebersohl<br />
I use mainly Oracle 9iR2 versions. I had a set of tables defined in a couple of schemas, namely V2 and DALIM_UPDATE, and I use a KEEP buffer as suggested in the main article. But, using:<br />
SELECT * FROM sys.oci_buffer_cache_use <br />
WHERE BUFFER_POOL IN('KEEP', 'DEFAULT') <br />
ORDER BY BUFFER_POOL DESC, blocks# DESC ; <br />
<br />
I had the unpleasant surprise to see some tables and indexes of my DALIM_UPDATE schema appearing as being in the KEEP buffer. Investigation showed up that the object_name was not associated to the object_owner in the OCI_BUFFER_POOLS view. I corrected it as follows:<br />
CREATE OR REPLACE VIEW OCI_BUFFER_POOLS AS <br />
select t.owner object_owner, table_name object, buffer_pool from dba_tables t <br />
where buffer_pool is not null <br />
union <br />
select tabp.table_owner , table_name object, buffer_pool <br />
from dba_tab_partitions tabp where buffer_pool is not null <br />
union <br />
select tabsp.table_owner, table_name object, buffer_pool <br />
from dba_tab_subpartitions tabsp where buffer_pool is not null <br />
union <br />
select i.owner, index_name object, buffer_pool <br />
from dba_indexes i where buffer_pool is not null <br />
union <br />
select ip.index_owner, index_name object, buffer_pool <br />
from dba_ind_partitions ip where buffer_pool is not null <br />
union <br />
SELECT ips.index_owner, index_name object, buffer_pool <br />
from dba_ind_subpartitions ips where buffer_pool is not null <br />
; <br />
<br />
CREATE OR REPLACE VIEW OCI_BUFFER_CACHE_USE AS <br />
select a.owner, a.object_name, a.object_type, count(a.object_name) blocks#, b.buffer_pool <br />
from oci_block_header a, oci_buffer_pools b <br />
where owner not in ( 'SYS', 'SYSTEM', 'PERFSTAT') <br />
and a.object_name = b.OBJECT AND a.owner = b.object_owner <br />
group by b.buffer_pool, a.owner, a.object_type, a.object_name <br />
-- having count(a.object_name) > 100 -- if you don't want small objects <br />
; <br />
Why and when should one tune?<br />
One of the biggest responsibilities of a DBA is to ensure that the Oracle database is tuned properly. The Oracle RDBMS is highly tunable and allows the database to be monitored and adjusted to increase its performance. <br />
One should do performance tuning for the following reasons: <br />
The speed of computing might be wasting valuable human time (users waiting for response); <br />
Enable your system to keep-up with the speed business is conducted; and <br />
Optimize hardware usage to save money (companies are spending millions on hardware). <br />
Although this site is not overly concerned with hardware issues, one needs to remember than you cannot tune a Buick into a Ferrari. <br />
Where should the tuning effort be directed?<br />
Consider the following areas for tuning. The order in which steps are listed needs to be maintained to prevent tuning side effects. For example, it is no good increasing the buffer cache if you can reduce I/O by rewriting a SQL statement. <br />
Database Design (if it's not too late): <br />
Poor system performance usually results from a poor database design. One should generally normalize to the 3NF. Selective denormalization can provide valuable performance improvements. When designing, always keep the "data access path" in mind. Also look at proper data partitioning, data replication, aggregation tables for decision support systems, etc. <br />
Application Tuning: <br />
Experience showed that approximately 80% of all Oracle system performance problems are resolved by coding optimal SQL. Also consider proper scheduling of batch tasks after peak working hours. <br />
Memory Tuning: <br />
Properly size your database buffers (shared_pool, buffer cache, log buffer, etc) by looking at your wait events, buffer hit ratios, system swapping and paging, etc. You may also want to pin large objects into memory to prevent frequent reloads. <br />
Disk I/O Tuning: <br />
Database files needs to be properly sized and placed to provide maximum disk subsystem throughput. Also look for frequent disk sorts, full table scans, missing indexes, row chaining, data fragmentation, etc. <br />
Eliminate Database Contention: <br />
Study database locks, latches and wait events carefully and eliminate where possible. <br />
Tune the Operating System: <br />
Monitor and tune operating system CPU, I/O and memory utilization. For more information, read the related Oracle FAQ dealing with your specific operating system. <br />
What tools/utilities does Oracle provide to assist with performance tuning?<br />
Oracle provide the following tools/ utilities to assist with performance monitoring and tuning: <br />
ADDM (Automated Database Diagnostics Monitor) introduced in Oracle 10g <br />
TKProf <br />
Statspack <br />
Oracle Enterprise Manager - Tuning Pack (cost option) <br />
Old UTLBSTAT.SQL and UTLESTAT.SQL - Begin and end stats monitoring <br />
When is cost based optimization triggered?<br />
It's important to have statistics on all tables for the CBO (Cost Based Optimizer) to work correctly. If one table involved in a statement does not have statistics, and optimizer dynamic sampling isn't performed, Oracle has to revert to rule-based optimization for that statement. So you really want for all tables to have statistics right away; it won't help much to just have the larger tables analyzed. <br />
Generally, the CBO can change the execution plan when you: <br />
Change statistics of objects by doing an ANALYZE; <br />
Change some initialization parameters (for example: hash_join_enabled, sort_area_size, db_file_multiblock_read_count). <br />
How can one optimize %XYZ% queries?<br />
It is possible to improve %XYZ% (wildcard search) queries by forcing the optimizer to scan all the entries from the index instead of the table. This can be done by specifying hints. <br />
If the index is physically smaller than the table (which is usually the case) it will take less time to scan the entire index than to scan the entire table. <br />
Where can one find I/O statistics per table?<br />
The STATSPACK and UTLESTAT reports show I/O per tablespace. However, they do not show which tables in the tablespace has the most I/O operations. <br />
The $ORACLE_HOME/rdbms/admin/catio.sql script creates a sample_io procedure and table to gather the required information. After executing the procedure, one can do a simple SELECT * FROM io_per_object; to extract the required information. <br />
For more details, look at the header comments in the catio.sql script. <br />
My query was fine last week and now it is slow. Why?<br />
The likely cause of this is because the execution plan has changed. Generate a current explain plan of the offending query and compare it to a previous one that was taken when the query was performing well. Usually the previous plan is not available. <br />
Some factors that can cause a plan to change are: <br />
Which tables are currently analyzed? Were they previously analyzed? (ie. Was the query using RBO and now CBO?) <br />
Has OPTIMIZER_MODE been changed in INIT<sid>.ORA? <br />
Has the DEGREE of parallelism been defined/changed on any table? <br />
Have the tables been re-analyzed? Were the tables analyzed using estimate or compute? If estimate, what percentage was used? <br />
Have the statistics changed? <br />
Has the SPFILE/ INIT<sid>.ORA parameter DB_FILE_MULTIBLOCK_READ_COUNT been changed? <br />
Has the INIT<sid>.ORA parameter SORT_AREA_SIZE been changed? <br />
Have any other INIT<sid>.ORA parameters been changed? <br />
What do you think the plan should be? Run the query with hints to see if this produces the required performance. <br />
Does Oracle use my index or not?<br />
One can use the index monitoring feature to check if indexes are used by an application or not. When the MONITORING USAGE property is set for an index, one can query the v$object_usage to see if the index is being used or not. Here is an example: <br />
SQL> CREATE TABLE t1 (c1 NUMBER);<br />
Table created.<br />
<br />
SQL> CREATE INDEX t1_idx ON t1(c1);<br />
Index created.<br />
<br />
SQL> ALTER INDEX t1_idx MONITORING USAGE;<br />
Index altered.<br />
<br />
SQL><br />
SQL> SELECT table_name, index_name, monitoring, used FROM v$object_usage;<br />
TABLE_NAME INDEX_NAME MON USE<br />
------------------------------ ------------------------------ --- ---<br />
T1 T1_IDX YES NO<br />
<br />
SQL> SELECT * FROM t1 WHERE c1 = 1;<br />
no rows selected<br />
<br />
SQL> SELECT table_name, index_name, monitoring, used FROM v$object_usage;<br />
TABLE_NAME INDEX_NAME MON USE<br />
------------------------------ ------------------------------ --- ---<br />
T1 T1_IDX YES YES<br />
To reset the values in the v$object_usage view, disable index monitoring and re-enable it: <br />
ALTER INDEX indexname NOMONITORING USAGE;<br />
ALTER INDEX indexname MONITORING USAGE;<br />
Why is Oracle not using the damn index?<br />
This problem normally only arises when the query plan is being generated by the Cost Based Optimizer (CBO). The usual cause is because the CBO calculates that executing a Full Table Scan would be faster than accessing the table via the index. Fundamental things that can be checked are: <br />
USER_TAB_COLUMNS.NUM_DISTINCT - This column defines the number of distinct values the column holds. <br />
USER_TABLES.NUM_ROWS - If NUM_DISTINCT = NUM_ROWS then using an index would be preferable to doing a FULL TABLE SCAN. As the NUM_DISTINCT decreases, the cost of using an index increase thereby making the index less desirable. <br />
USER_INDEXES.CLUSTERING_FACTOR - This defines how ordered the rows are in the index. If CLUSTERING_FACTOR approaches the number of blocks in the table, the rows are ordered. If it approaches the number of rows in the table, the rows are randomly ordered. In such a case, it is unlikely that index entries in the same leaf block will point to rows in the same data blocks. <br />
Decrease the INIT<sid>.ORA parameter DB_FILE_MULTIBLOCK_READ_COUNT - A higher value will make the cost of a FULL TABLE SCAN cheaper. <br />
Remember that you MUST supply the leading column of an index, for the index to be used (unless you use a FAST FULL SCAN or SKIP SCANNING). <br />
There are many other factors that affect the cost, but sometimes the above can help to show why an index is not being used by the CBO. If from checking the above you still feel that the query should be using an index, try specifying an index hint. Obtain an explain plan of the query either using TKPROF with TIMED_STATISTICS, so that one can see the CPU utilization, or with AUTOTRACE to see the statistics. Compare this to the explain plan when not using an index. <br />
When should one rebuild an index?<br />
You can run the ANALYZE INDEX <index> VALIDATE STRUCTURE command on the affected indexes - each invocation of this command creates a single row in the INDEX_STATS view. This row is overwritten by the next ANALYZE INDEX command, so copy the contents of the view into a local table after each ANALYZE. The 'badness' of the index can then be judged by the ratio of 'DEL_LF_ROWS' to 'LF_ROWS'. <br />
How does one tune Oracle Wait event XYZ?<br />
Here are some of the wait events from V$SESSION_WAIT and V$SYSTEM_EVENT views: <br />
db file sequential read: Tune SQL to do less I/O. Make sure all objects are analyzed. Redistribute I/O across disks. <br />
buffer busy waits: Increase DB_CACHE_SIZE (DB_BLOCK_BUFFERS prior to 9i)/ Analyze contention from SYS.V$BH <br />
log buffer space: Increase LOG_BUFFER parameter or move log files to faster disks <br />
log file sync: If this event is in the top 5, you are committing too often (talk to your developers) <br />
log file parallel write: deals with flushing out the redo log buffer to disk. Your disks may be too slow or you have an I/O bottleneck. <br />
What is the difference between DBFile Sequential and Scattered Reads?<br />
Both "db file sequential read" and "db file scattered read" events signify time waited for I/O read requests to complete. Time is reported in 100's of a second for Oracle 8i releases and below, and 1000's of a second for Oracle 9i and above. Most people confuse these events with each other as they think of how data is read from disk. Instead they should think of how data is read into the SGA buffer cache. <br />
db file sequential read: <br />
A sequential read operation reads data into contiguous memory (usually a single-block read with p3=1, but can be multiple blocks). Single block I/Os are usually the result of using indexes. This event is also used for rebuilding the controlfile and reading datafile headers (P2=1). In general, this event is indicative of disk contention on index reads. <br />
db file scattered read: <br />
Similar to db file sequential reads, except that the session is reading multiple data blocks and scatters them into different discontinuous buffers in the SGA. This statistic is NORMALLY indicating disk contention on full table scans. Rarely, data from full table scans could be fitted into a contiguous buffer area, these waits would then show up as sequential reads instead of scattered reads. <br />
The following query shows average wait time for sequential versus scattered reads: <br />
prompt "AVERAGE WAIT TIME FOR READ REQUESTS"<br />
select a.average_wait "SEQ READ", b.average_wait "SCAT READ"<br />
from sys.v_$system_event a, sys.v_$system_event b<br />
where a.event = 'db file sequential read'<br />
and b.event = 'db file scattered read'<br />
<br />
How do you create an 'auto number' or 'auto increment' column similar to the offering from Microsoft's SQL Server ?<br />
A common question for users converting from MS SQL Server to Oracle is how to handle 'auto increment' or 'auto number; columns. A feature of MS SQL server for handling (typcially) primary key columns which have to be populated with meaningless sequence numbers. There is no direct equivalent in Oracle, but the combination of sequences and triggers comes close.<br />
Marco Coletti: Another option available in Oracle version 9.2 is to use the built in SQL function sys_guid() for the default column value. (13th Oct 2005)<br />
How does Oracle handle PGA memory.<br />
<<< DOWN LOAD THE PPS & PDF FROM http://www.jlcomp.demon.co.uk/faq/pat_presentation.html >>><br />
What is the relationship between Oracle's buffer cache and UNIX's buffer cache?<br />
In short there is no direct relationship between the Oracle and Unix buffer caches aside from the fact they both cache data to prevent physical IO. With regard to data stored in Oracle data files Oracle's buffer cache mostly eliminates the need for the Unix buffer cache however, with a little effort, you can get the best out of both of them.<br />
There are two aspects to consider:<br />
Reads: When data is read from disk on a cached file system it is also placed in the Unix buffer cache. In general if a block of data can not be found in Oracle's buffer cache then you are not going to find it in the Unix buffer cache either, hence the Unix cache is just an overhead. <br />
Writes: A write done through the Unix buffer cache will involve user data being written to the cache and then Unix writing the cache contents down to disk (or at least what it thinks is disk*). In an online situation it is unlikely you will notice this extra step, especially when using asynchronous IO as once the data is in the cache you can get on with your next job. However in a busy system you may notice it indirectly as lots of CPU time is spent writing cached data down to disk rather than servicing the application. You are unlikely to read the majority of data back from this cache because of Oracle’s buffer cache so it would be nice to skip this step, this is where writing direct to disk and bypassing the cache comes into play. <br />
There is a good description of asynchronous and direct IO in the the Database Performance Tuning Guide linked to at the foot of this page.<br />
In general it is probably a good idea to allow your Oracle home to sit in a file system that uses the Unix cache but put your database files into file systems that utilise direct IO (if supported) to bypass the Unix cache (save for a few oddities such as temporary tablespace data**).<br />
Eg (taken from AIX):<br />
[oracle:PROD] /u01/app/oracle/product/10.1.0 >mount<br />
node mounted mounted over vfs date options <br />
-------- --------------- --------------- ------ ------------ --------------- <br />
...<br />
/dev/lvora01 /u01 jfs2 Jun 23 15:48 rw,log=/dev/loglv01<br />
/dev/lvora02 /u02 jfs2 Jun 23 15:48 rw,dio,log=/dev/loglv02<br />
/dev/lvora03 /u03 jfs2 Jun 23 15:48 rw,dio,log=/dev/loglv03<br />
...<br />
Here /u01 uses the Unix buffer cache and contains $ORACLE_BASE (/u01/app/oracle) and certain specific oracle database files (/u01/oradata). /u02 upwards are then all set to bypass the Unix buffer cache - you can see this from the "dio" parameter under options. This could also be "cio" which means concurrent IO and utilises direct IO plus an extra tune around file inode serialisation.<br />
In summary, if your Unix system supports direct IO to bypass the Unix buffer cache then you should at least try it out. You could potentially free up CPU time and memory to be used by other processes. It is generally understood that a system with well configured file systems in regard to the Unix buffer cache can rival the performance of raw data files however, having never used a raw file system, I cannot confirm either way.<br />
* Bear in mind that there is a good chance your disks are actually logical units in a SAN which will also have its own hefty cache.<br />
** Temp is a good example of why the Unix buffer cache is useful as the data to be written is not cached by Oracle and will more often than not be read back in the near future - the buffer cache can satisfy this request. Redo logs are similar in that they are written to by LGWR and read again by ARCH.<br />
Unix File system has a buffer cache and Oracle also maintains its own cache. The buffers are being copied from Kernel Space(File system buffer cache) to User Space(Oracle Buffer Cache). This operation generates a lot of overhead. But in case of raw devices, a write to a raw device bypasses the Unix Buffer Cache , the data is transferred direct from Oracle buffer to the disk. So, write performance enhanced with the use of raw devices.<br />
http://download-west.oracle.com/docs/cd/B14117_01/server.101/b10752/ch23_os.htm#227<br />
Queries against dba_free_space are slow - is there any way to speed them up ?<br />
Original Code :<br />
SELECT<br />
TABLESPACE_NAME, SUM(BYTES) SUMB,<br />
MAX(BYTES) LARGEST,COUNT(*) CHUNKS<br />
FROM DBA_FREE_SPACE A<br />
GROUP BY TABLESPACE_NAME<br />
<br />
Modification:<br />
<br />
SELECT /*+ use_hash (A.ts A.fi) */<br />
TABLESPACE_NAME, SUM(BYTES) SUMB,<br />
MAX(BYTES) LARGEST,COUNT(*) CHUNKS<br />
FROM DBA_FREE_SPACE A<br />
GROUP BY TABLESPACE_NAME<br />
<br />
How do I change the default password of internal ?<br />
I follow below steps.<br />
1. shutdown the database.<br />
2. delete the password file<br />
3. Create new password file with ORAPWD [ ORAPWD is the oracle utility. Its Description is provided at the end of article ] <br />
orapwd file=e:\oracle\database\PWDsnap2.ora password=hello entries=30<br />
4.Startup the database.<br />
5.Enter command SVRMGRL and connect internal<br />
SVRMGR> connect internal<br />
Connected.<br />
<br />
I am able to connect internal with <br />
SVRMGR> connect internal/hello<br />
Connected.<br />
<br />
My question, why I am connecting internal without password ? It use default password for internal ? In init.ora <br />
"remote_login_passwordfile = exclusive"<br />
<br />
What's the problem – You need to enter the password that was used for creating password file when<br />
connecting to INTERNAL<br />
<br />
If your problem is that connect internal doesn't ask for password after recreating the password file - then<br />
set SQLNET.AUTHENTICATION_SERVICES = TRUE<br />
<br />
and then try svrmgrl>connect internal <br />
If you don't know the password and still want to connect internal - comment out <br />
SQLNET.AUTHENTICATION_SERVICES=TRUE in sqlnet.ora<br />
and <br />
REMOTE_LOGIN_PASSWORDFILE = EXCLUSIVE in init.ora<br />
<br />
and you will be through<br />
<br />
If you want to recreate the password then<br />
delete $ORACLE_HOME/dbs/orapwSID file and <br />
as suggested above use orapwd to create a new password as :<br />
<br />
orapwd file=$ORACLE_HOME/dbs/orapwdev password=xxxxx entries=<n><br />
<br />
For Oracle 9i : <br />
1. INTERNAL is no longer supported as a USER is Oracle9i<br />
<br />
2. To connect as SYS is Oracle9i, you need to append AS SYSDBA or AS SYSOPER<br />
(always!!!)<br />
<br />
If you have direct access to that oracle9i box then you can do this:<br />
a. Authentication is left to OS:<br />
connect / as sysdba <br />
and change the SYS password<br />
b. Authentication is done via password file(orapwd deals with this)<br />
Note: this is can be accomplished when running from a client<br />
connect sys/password as sysdba <br />
<br />
Using ORAPWD Utility : <br />
<br />
1. Creating the password file with orapwd <br />
% orapwd file=${ORACLE_HOME}/dbs/orapwSID password=password entries=max_users <br />
Where: <br />
file name is the name of the file that will hold the password information. The password file location will default to the current directory unless a path is provided. <br />
password is the one for the SYS user or new password <br />
entries parameter tells Oracle how many users you will be creating in the password file. You cannot increase entries at a later, so set the value high. <br />
Name and Location of the password file: <br />
At database startup time, Oracle will only look for the password file in the ORACLE_HOME/dbs directory. The naming convention that Oracle will search for is: orapwSID. The SID must match the ORACLE_SID environment variable. If orapwSID can not be found, Oracle will look for a file named orapw. If Oracle can not find a orapw file you will get the ORA-01990 at database startup time. <br />
An important security concern is that the password file be secured. <br />
How to get started with SQL_TRACE<br />
<br />
<br />
Is it possible to run several different versions of Oracle on the same machine<br />
Yes! We can run different versions of Oracle on the same machine provided that Oracle_base_directory is same for both the versions. <br />
Oracle_base_directory is the directory where oracle is installed like C:\oracle or D:\orant <br />
Under the Base directory Oracle home directories are differentiated like <br />
C:\oracle\oracle8 <br />
C:\oracle\oracle9 <br />
In short you can have different versions installed on the same machine if they are installed in the same directory.<br />
I have deleted a trace file with the session still live. I now want the session to resume tracing, but a new file doesn't appear.<br />
From Unix, if you switch sql_trace off, delete the trace file, and switch sql_trace on again for a given session, Oracle does not appear to create a new trace file. This can be a nuisance, and there is no documented workaround. NT is a more friendly, and a new trace file will be generated every time you delete the old one.<br />
<br />
The reason why you have a problem under Unix is that the trace file is not closed - even if you set sql_trace to false. There is, however, a method for sorting the problem out using oradebug. This is not, alas, very elegant - but may be adequate in many cases where a DBA is exerting external control of the tracing.<br />
Start up SQL*Plus as sysdba, locate the process id (oracle or unix) of the target user, e.g.<br />
select spid, pid<br />
from v$process<br />
where addr = (<br />
select paddr <br />
from v$session<br />
where sid = 99<br />
);<br />
oradebug setorapid {the pid above}<br />
or<br />
oradebug setospid {the spid above}<br />
then<br />
oradebug flush -- flushes any remaining data to the trace file<br />
oradebug close_trace -- closes the trace file.<br />
The nice thing about the close_trace option is that you can do it even after you have deleted the trace file from the operating system level. In fact in most cases you may have to delete the trace file before you issue a close_trace, otherwise you may find that Oracle re-opens to write the next bit of trace information to it, and the problem simply repeats itself.<br />
There is a simple solution to this problem if you are allowed to use it in your environment.<br />
Alter session set tracefile_identifier = ‘text string’;<br />
If you can issue this command, then the current trace file is closed and a new one is opened with the ‘text string’ included as part of the file name. <br />
This is also very convenient for running a series of tests and dropping the test results into separate trace files. Bear in mind that some of the odds and ends (such as STAT lines) may end up in the wrong trace file, though.<br />
Is there a Windows version of the oerr program?<br />
In UNIX, you type oerr ora 1234 to get a description and suggested action about the Oracle error 1234. Oracle does not provide this utility in Windows. What can you do?<br />
The UNIX version oerr program is a shell script that reads the text files $ORACLE_HOME/*/mesg/*.msg, most notably $ORACLE_HOME/rdbms/oraus.msg. But the Windows version Oracle does not bundle this utility and does not have human-readable error message files; it only has the binary versions such as %ORACLE_HOME%\RDBMS\mesg\oraus.msb and its format is not published by Oracle. So you can't build your own program based on that. The following are alternatives in order of relevance to the question. <br />
1. Yong Huang wrote a freeware Perl script Windows oerr that reads the Oracle documentation Error Message pages and returns the error description and suggested action. It generally requires installing Perl and Oracle documentation locally. Minimum installation requires the Error Message part of documentation and compiled version of the Perl script (so you don't need the Perl interpreter). <br />
2. Several people made attempts to build oerr based on the UNIX message files. The only one we still have access to is Bill Pollack's ora-text at his site. It requires you to ftp oraus.msg under rdbms from UNIX to PC. <br />
3. Inside SQL*Plus or PL/SQL, you can use the sqlerrm function. For example, in SQL*PLus, set serveroutput on. Then exec dbms_output.put_line(sqlerrm(-1555)). This does not need any extra installation but gives you very limited error description and only works for the ORA facility, of course the largest facility in Oracle. <br />
4. William Rogge posted a message to an Oracle mailing list publishing an awk script that parses $ORACLE_HOME/rdbms/mesg/oraus.msg and loads its content into an Oracle table. Obviously this requires client access to the database server to get the error description. I can't find the author or his script any more.<br />
Note that not all error descriptions are exactly the same between UNIX $ORACLE_HOME/rdbms/mesg/oraus.msg and the Error Message page of documentation. Try oerr pls 123 at the UNIX prompt and then look it up in documentation. <br />
Why does it seem that a SELECT over a db_link requires a commit after execution ?<br />
Why does it seem that a SELECT over a db_link requires a commit after execution ?<br />
Because it does! When Oracle performs a distributed SQL statement Oracle reserves an entry in the rollback segment area for the two-phase commit processing. This entry is held until the SQL statement is committed even if the SQL statement is a query. A demonstration of this fact follows. The REM’s were added to the output, which is otherwise a cut and paste of the screen. The script db/obj/rbs_users is SQL to show user sessions to rollback segment assignments (transactions) and similar SQL can be found in the FAQ entry: Is there a way to detect processes that are rolling back, and can I figure out how long it will take?<br />
REM find the current session sid<br />
<br />
PFC> select * from v$mystat where rownum = 1;<br />
<br />
SID STATISTIC# VALUE<br />
---------- ---------- ----------<br />
7 0 1<br />
<br />
REM see who is using rollback, the current session should not be there<br />
<br />
PFC> @db/org/rbs_users<br />
<br />
no rows selected<br />
<br />
REM perform a remote query<br />
<br />
PFC> select count(*) from mpowel01.item_master@ut1.world;<br />
<br />
COUNT(*)<br />
----------<br />
2603<br />
<br />
REM determine whether the current session is now a transaction or not<br />
<br />
PFC> @db/org/rbs_users<br />
<br />
NAME USN EXTENTS USERNAME SID LOGON_TIM S STATUS<br />
------------ ----- ---------- ------------ ---------- --------- --------<br />
START_TIME T STATUS USED_UBLK USED_UREC<br />
-------------------- ---------------- ---------- ----------<br />
PROGRAM TERMINAL<br />
------------------------------------------------ ------------------------------<br />
ROLL01 2 20 MPOWEL01 7 05-SEP-01 ACTIVE<br />
09/05/01 12:34:36 ACTIVE 1 1<br />
sqlplus@seqdev (TNS V1-V3) ttyiR/iARS<br />
<br />
REM end the transaction<br />
<br />
PFC> commit;<br />
<br />
Commit complete.<br />
<br />
REM verify the current session no longer shows as a transaction<br />
<br />
PFC> @db/org/rbs_users<br />
<br />
no rows selected<br />
<br />
If the application code fails to issue a commit after the remote or distributed select statement then the rollback segment entry is not released. If the program stays connected to Oracle but goes inactive for a significant period of time (such as a daemon, wait for alert, wait for mailbox entry, etc…) then when Oracle needs to wrap around and reuse the extent, Oracle has to extend the rollback segment because the remote transaction is still holding its extent. This can result in the rollback segments extending to either their maximum extent limit or consuming all free space in the rbs tablespace even where there are no large transactions in the application. When the rollback segment tablespace is created using extendable files then the files can end up growing well beyond any reasonable size necessary to support the transaction load of the database. Developers are often unaware of the need to commit distributed queries and as a result often create distributed applications that cause, experience, or contribute to rollback segment related problems like ORA-01650 (unable to extend rollback). The requirement to commit distributed SQL exists even with automated undo management available with version 9 and newer. If the segment is busy with an uncommitted distributed transaction Oracle will either have to create a new undo segment to hold new transactions or extend an existing one. Eventually undo space could be exhausted, but prior to this it is likely that data would have to be discarded before the undo_retention period has expired.<br />
Note that per the Distributed manual that a remote SQL statement is one that references all its objects at a remote database so that the statement is sent to this site to be processed and only the result is returned to the submitting instance, while a distributed transaction is one that references objects at multiple databases. For the purposes of this FAQ there is no difference, as both need to commit after issuing any form of distributed query.<br />
How do I know what Oracle EVENTS are set in my database instance?<br />
<br />
To find events that are set at the SYSTEM level you can use the oradebug utility. Following is my events_system.sql script, which runs the necessary commands. Sample output is included in the comment area of the script. The event at the time of the sample was for generating a trace file when ORA-04031 errors occurred.<br />
<br />
set echo off<br />
-- Use ORADEBUG to dump list of events set at system level<br />
--<br />
-- Based on Oracle support metalink rdbms forum post 08/04/2003<br />
- by Melissa Holman Subject: Setting Event Trace<br />
-<br />
- 20030805 Mark D Powell New, capture useful functionality<br />
-<br />
- *** 2003-08-05 09:58:20.827<br />
- *** SESSION ID:(34.5167) 2003-08-05 09:58:20.781<br />
- Dump event group for level PROCESS<br />
- TC Addr Evt#(b10) Action TR Addr Arm Life<br />
- 101D9020 4031 1 101d9080 0 0<br />
-<br />
set echo on<br />
ORADEBUG SETMYPID<br />
ORADEBUG DUMP EVENTS 2<br />
ORADEBUG TRACEFILE_NAME<br />
--<br />
<br />
The first command tells oradebug to use the current session as its target. All system wide events would be inherited by the current session at process startup. The thought might occur to you that events set by the current session at the session level would also appear in the output; my testing on 9.2.0.5 shows they do not. You can verify this by setting a session level event and then running the three lines on code.<br />
The second command obviously produces the trace file while the third command displays the trace file name.<br />
A word of caution, generally system wide events are set only with the advice of Oracle support to work around internal errors or for generating debugging (trace) information. Finding information on non-trace events is difficult, but if you have any set you can scan the Oracle support site, http://metalink.oracle.com, bug database for bug reports that refer to identified events.<br />
How do I randomly select rows from a table?<br />
In Oracle 8i and 9i, the easiest way to randomly select data out of a table is to use the SAMPLE clause with a select statement. <br />
Example: <br />
SELECT emp<br />
FROM emp<br />
SAMPLE(10);<br />
In the example statement, Oracle is instructed to randomly visit 10% of the rows in the table. <br />
The restriction to this clause is that it works for single table queries only. If you include the sample clause with a multiple table query, you will get a parse error or ORA-30561: SAMPLE option not allowed in statement with multiple table references. One way around this is to create an inline view on the driving table of the query with the SAMPLE clause.<br />
Example: <br />
SELECT t1.dept, t2.emp<br />
FROM <br />
(SELECT * FROM dept sample(5)) t1,<br />
Emp t2<br />
WHERE <br />
t1.dep_id = t2.dep_id<br />
<br />
How can I maintain a history of user logins?<br />
If you want to get some idea of which users spend most time and consume most resources on the system, you don’t necessarily have to do anything subtle and devious to find out what’s been happening. There has been a (simple) audit trail built into the database for as long as I can remember. (The 8.1 – 10.1 in the banner simply covers the fact that I’ve only recently checked the following comments against those versions)<br />
The init.ora file contains an audit_trail parameter. This can take the values true, false, none, os, db (the true/false options are for backwards compatibility). If you set the value to db (or true), then you have enabled auditing in the database. Once you have restarted the database (the parameter is not modifiable online), you can decide what events you want to audit.<br />
For a quick cheap audit of connections, connect as a suitably privileged account (which typically means as a DBA), and issue the command:<br />
audit create session;<br />
If you need to turn this audit off, the corresponding command is:<br />
noaudit create session;<br />
The older syntax for the same level of audit is:<br />
audit connect;<br />
noaudit connect;<br />
<br />
<br />
With this level of audit turned on, every session that logs on (except the SYS sessions) will insert a row into the table sys.aud$ giving various details of who they are and what time they connected. When the session ends, its last action is to update this row with various session-related details, such as log-off time, and the amount of work done. To make the results more readable, Oracle has superimposed the view dba_audit_session on top of the aud$ table; the 9.2 version of this view is as follows:<br />
Name Null? Type<br />
----------------------- -------- ----------------<br />
OS_USERNAME VARCHAR2(255) Who<br />
USERNAME VARCHAR2(30)<br />
USERHOST VARCHAR2(128) Where<br />
TERMINAL VARCHAR2(255)<br />
TIMESTAMP NOT NULL DATE logon date/time<br />
ACTION_NAME VARCHAR2(27) <br />
LOGOFF_TIME DATE log off date/time<br />
LOGOFF_LREAD NUMBER v$sess_io.consistent_gets<br />
LOGOFF_PREAD NUMBER v$sess_io.physical_reads<br />
LOGOFF_LWRITE NUMBER v$sess_io.block_changes<br />
LOGOFF_DLOCK VARCHAR2(40) Number of deadlocks <br />
SESSIONID NOT NULL NUMBER<br />
RETURNCODE NOT NULL NUMBER<br />
CLIENT_ID VARCHAR2(64)<br />
SESSION_CPU NUMBER Session statistic. CPU used by this session<br />
As you can see, there is quite a lot of helpful information here – perhaps good enough for most monitoring purposes. It is also a very light-weight tool, as it requires just one insert on logon, and an index access to update one row on log off.<br />
There are a couple of administrative points. The aud$ table is in the system tablespace and is the one table in the sys schema that you are told you can delete data from. You may want to run a regular purge job to delete data that is more then N days old from this table.<br />
You might consider moving this table to a separate tablespace – but there have been reports of problems with media recovery if you do this (possibly because the recovering processes tries to insert its own audit records and can’t because the tablespace needs recovery) and it is not supported by Oracle.<br />
Finally, if you are running a physical standby database and open it in read only mode, you may find that you can’t connect to it as anyone other than sys. Auditing requires an insert/upate on aud$ - so can’t be allowed to on a read only database. You will have to remember to change the audit_trail parameter to none on your standby before you start it up.<br />
Further reading: SQL Reference Manual – AUDIT and NOAUDIT commands.<br />
Why does AUTOTRACE not show partition pruning in the explain plan ?<br />
Autotrace not showing partition pruning/elimination is bug 1426992, but, after investigation Oracle has decided that this is not an optimiser bug, but a bug in SQL*Plus. You can, with a bit of knowledge of your data and a little experimentation, deduce that partition pruning is taking place from the output of autotrace, but there are much easier ways !<br />
The following demonstration shows the failings in autotraceand demonstrates a couple of other methods of determining whether or not your partitions are being pruned - or not.<br />
Autotrace<br />
First of all, create a simple table range partitioned over 6 different partitions, and fill it with some test data extracted from ALL_OBJECTS.<br />
SQL> create table tab_part (part_key number(1), some_text varchar2(500))<br />
<br />
2 partition by range (part_key) (<br />
<br />
3 partition part_1 values less than (2),<br />
<br />
4 partition part_2 values less than (3),<br />
<br />
5 partition part_3 values less than (4),<br />
<br />
6 partition part_4 values less than (5),<br />
<br />
7 partition part_5 values less than (6),<br />
<br />
8 partition part_6 values less than (MAXVALUE) );<br />
Table created.<br />
<br />
<br />
SQL> insert /*+ append */ into tab_part<br />
<br />
2 select mod(rownum, 10), object_name<br />
<br />
3 from all_objects;<br />
24683 rows created.<br />
<br />
<br />
SQL> commit;<br />
Commit complete.<br />
Once the table has been filled, analyse it and see how the data has been spread over the various partitions. The first and last partitions have more data in them that the remaining four, hence the differing totals.<br />
SQL> analyze table tab_part compute statistics;<br />
Table analyzed.<br />
<br />
<br />
SQL> select partition_name, num_rows<br />
<br />
2 from user_tab_partitions<br />
<br />
3 where table_name = 'TAB_PART'<br />
<br />
4 order by partition_name;<br />
PARTITION_NAME NUM_ROWS<br />
<br />
------------------------------ ----------<br />
<br />
PART_1 4937<br />
<br />
PART_2 2469<br />
<br />
PART_3 2469<br />
<br />
PART_4 2468<br />
<br />
PART_5 2468<br />
<br />
PART_6 9872<br />
6 rows selected.<br />
Now that we have a table to work with, we shall see what autotrace has to say about partition elimination. First, however, note how many logical reads a full scan of the entire table needs : <br />
SQL> set autotrace on<br />
SQL> select count(*) from tab_part;<br />
COUNT(*)<br />
<br />
----------<br />
<br />
24683<br />
<br />
<br />
Execution Plan<br />
<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=42 Card=1)<br />
<br />
1 0 SORT (AGGREGATE)<br />
<br />
2 1 PARTITION RANGE (ALL)<br />
<br />
3 2 TABLE ACCESS (FULL) OF 'TAB_PART' (Cost=42 Card=24683)<br />
<br />
<br />
Statistics<br />
<br />
----------------------------------------------------------<br />
<br />
0 recursive calls<br />
<br />
0 db block gets<br />
<br />
135 consistent gets<br />
<br />
0 physical reads<br />
<br />
0 redo size<br />
<br />
381 bytes sent via SQL*Net to client<br />
<br />
499 bytes received via SQL*Net from client<br />
<br />
2 SQL*Net roundtrips to/from client<br />
<br />
0 sorts (memory)<br />
<br />
0 sorts (disk)<br />
<br />
1 rows processed<br />
To read 24,683 rows of data Oracle had to perform 135 logical reads. Keep this in mind and note that the autotrace output shows a full table scan - as we would expect on an unindexed table. The next count should only look in a single partition :<br />
SQL> select count(*) from tab_part where part_key = 7;<br />
COUNT(*)<br />
<br />
----------<br />
<br />
2468<br />
<br />
<br />
Execution Plan<br />
<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=17 Card=1 Bytes=2)<br />
<br />
1 0 SORT (AGGREGATE)<br />
<br />
2 1 TABLE ACCESS (FULL) OF 'TAB_PART' (Cost=17 Card=2468 Bytes=4936)<br />
<br />
Statistics<br />
<br />
----------------------------------------------------------<br />
<br />
0 recursive calls<br />
<br />
0 db block gets<br />
<br />
49 consistent gets<br />
<br />
0 physical reads<br />
<br />
0 redo size<br />
<br />
380 bytes sent via SQL*Net to client<br />
<br />
499 bytes received via SQL*Net from client<br />
<br />
2 SQL*Net roundtrips to/from client<br />
<br />
0 sorts (memory)<br />
<br />
0 sorts (disk)<br />
<br />
1 rows processed<br />
This seems to have again carried out a full table scan, but as we already know that a real FTS takes 135 logical reads, the fact that only 49 were required here should indicate that something is different. Autotrace's output is not showing partition elimination. If you didn't know how many reads were required to full scan the table, you would be hard pressed to determine that partition elimination had taken place in this scan.<br />
Event 10053<br />
There are other methods by which we can obtain a true picture of the plan used by the optimiser - a 10053 trace for example would show the details. I've never had to use a 10053 trace so I'm unfortunately not in a position to explain its use, I leave this as 'an exercise for the reader' as they say :o)<br />
SQL_TRACE and TKPROF<br />
I have used SQL_TRACE and TKPROF though, so here's what shows up when SQL_TRACE is set true.<br />
<br />
<br />
SQL> set autotrace off<br />
SQL> alter session set sql_trace = true;<br />
Session altered.<br />
SQL> alter session set tracefile_identifier = 'PARTITION';<br />
Session altered.<br />
<br />
<br />
SQL> select count(*) from tab_part where part_key = 7;<br />
COUNT(*)<br />
<br />
----------<br />
<br />
2468<br />
<br />
<br />
SQL> alter session set sql_trace = false<br />
Session altered.<br />
At this point, exit from SQL*Plus and locate the trace file in USER_DUMP_DEST which has 'PARTITION' in it's name. This is the one you want to run through TKPROF. The output from this is shown below :<br />
select count(*) from tab_part where part_key = 7<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
<br />
Parse 1 0.00 0.00 0 0 0 0<br />
<br />
Execute 1 0.01 0.00 0 0 0 0<br />
<br />
Fetch 2 0.01 0.01 0 49 0 1<br />
<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
<br />
total 4 0.02 0.01 0 49 0 1<br />
Misses in library cache during parse: 0<br />
<br />
Optimizer goal: CHOOSE<br />
<br />
Parsing user id: 62 <br />
Rows Row Source Operation<br />
<br />
------- ---------------------------------------------------<br />
<br />
1 SORT AGGREGATE (cr=49 r=0 w=0 time=10353 us)<br />
<br />
2468 TABLE ACCESS FULL TAB_PART PARTITION: 6 6 (cr=49 r=0 w=0 time=6146 us)<br />
The explain plan clearly shows that partition 6 was the start and stop partition in the scan. In addition, there were 49 logical reads performed to get at the count. This is identical to what we saw above with autotrace, except we get to see that partition pruning did actually take place.<br />
Explain Plan<br />
Back in SQL*Plus, there is another method that can be used. The old faithful EXPLAIN PLAN will show how partition pruning did take place.<br />
SQL> explain plan<br />
<br />
2 set statement_id = 'Norman'<br />
<br />
3 for<br />
<br />
4 select count(*) from tab_part where part_key = 7;<br />
Explained.<br />
SQL> set lines 132<br />
<br />
SQL> set pages 10000<br />
SQL> col operation format a20<br />
<br />
SQL> col options format a15<br />
<br />
SQL> col object_name format a15<br />
<br />
SQL> col p_start format a15<br />
<br />
SQL> col p_stop format a15<br />
<br />
SQL> col level noprint<br />
SQL> select level,lpad(' ', 2*level-1)||operation as operation,<br />
<br />
2 options,<br />
<br />
3 object_name,<br />
<br />
4 partition_start as p_start,<br />
<br />
5 partition_stop as p_stop,<br />
<br />
6 cardinality<br />
<br />
7 from plan_table<br />
<br />
8 where statement_id = 'Norman'<br />
<br />
9 start with id=0<br />
<br />
10 connect by prior id=parent_id<br />
<br />
11 order by level<br />
OPERATION OPTIONS OBJECT_NAME P_START P_STOP CARDINALITY<br />
<br />
-------------------- --------------- --------------- --------------- --------------- -----------<br />
<br />
SELECT STATEMENT 1<br />
<br />
SORT AGGREGATE 1<br />
<br />
TABLE ACCESS FULL TAB_PART 6 6 2468<br />
Once again, the plan clearly shows that partition pruning takes place. The problem is that autotrace doesn't show it at all. Unless you really know how many blocks of data you have in a table and all of its partitions, you may find it difficult to determine whether or not you are seeing a 'true' plan when using partitioned tables and autotrace.<br />
Note: Do you ever suffer from the PLAN_TABLE growing too big as developers fail to delete old rows from the table? Alternatively, do you forget to delete rows from the table?<br />
Take a copy of $ORACLE_HOME/rdbms/admin/utlxplan.sql and edit it. <br />
Change this :<br />
create table PLAN_TABLE (<br />
<br />
statement_id varchar2(30), ...<br />
<br />
filter_predicates varchar2(4000));<br />
To this :<br />
create global temporary table PLAN_TABLE (<br />
<br />
statement_id varchar2(30), ...<br />
<br />
filter_predicates varchar2(4000))<br />
<br />
on commit preserve rows;<br />
Now login to SQL*Plus as SYS and :<br />
sql> @?/rdbms/admin/utlxplan_edited /* Or whatever your copy is called */<br />
sql> grant all on plan_table to public;<br />
sql> create public synonym PLAN_TABLE for SYS.PLAN_TABLE;<br />
Now when developers or DBAs use PLAN_TABLE and logout their rows will be deleted. A self-tidying PLAN_TABLE. Of course, this is no good if you want to keep rows in PLAN_TABLE between sessions. <br />
DBMS_XPLAN<br />
Under Oracle 9i (release 2 I think) there is a new PL/SQL package which you can use to show explain plans. The above statement could have its plan shown using this command :<br />
SQL> Select * from table(dbms_xplan.display(statement_id=>'Norman'));<br />
or, if this was the only statement in my PLAN_TABLE :<br />
SQL> Select * from table(dbms_xplan.display);<br />
There is much more information shown with this new feature than with a 'normal' explain plan and you don't have to worry about all that formatting either.<br />
Summary<br />
In summary, autotrace doesn't show partition elimination in Oracle up to versions 9i release 2. You should therefore be aware of this fact and use SQL_TRACE or EXPLAIN_PLAN to get at the true plan for the SQL you are trying to tune/debug.<br />
<br />
Further reading: <br />
Note: 166118.1 Partition Pruning/Elimination on Metalink. You will need a support contract to access Metalink.<br />
Bug: 1426992 SQLPlus AUTOTRACE does not show correct explain plan for partition elimination. Again on Metalink. <br />
<br />
When I rebuild an index, I see Oracle doing a sort. Why should this be necessary, why doesn't it simply read the existing index ?<br />
In the absence of the online option Oracle usually does read the existing index, but rather than read the index in key order which would use single block index IO, which in turn could be very slow for a large index, a design decision was made to perform the rebuild operation by reading the index blocks sequentially using multi-block reads. This logic is a fast full index scan, FFS, and can be seen by running an explain plan on the alter index rebuild command. My tests on 8.1.7.4 and 9.2.0.1 produced the same plan for unique and non-unique indexes of very small (16K) to reasonably large size (2.6G). Oracle access plans are subject to change with each dot release as Oracle improves its code, but the basic answer that the reason appears to be a design decision based on overall performance considerations probably will not change.<br />
<br />
Explain plan for: alter index rebuild xxx.item_trans_log_idx4<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- --------------------------------------------------<br />
1204 2919826 ALTER INDEX STATEMENT<br />
2.1 INDEX BUILD NON UNIQUE ITEM_TRANS_LOG_IDX4<br />
2919826 3.1 SORT CREATE INDEX<br />
1204 2919826 4.1 INDEX FAST FULL SCAN ITEM_TRANS_LOG_IDX4<br />
NON-UNIQUE<br />
<br />
<br />
When the online option is added to the rebuild command my testing shows that Oracle implements the rebuild using a full table scan.<br />
<br />
Explain plan for: alter index rebuild xxx.item_trans_log_idx4 online<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- --------------------------------------------------<br />
5879 2919826 ALTER INDEX STATEMENT<br />
2.1 INDEX BUILD NON UNIQUE ITEM_TRANS_LOG_IDX4<br />
2919826 3.1 SORT CREATE INDEX<br />
5879 2919826 4.1 TABLE ACCESS FULL ITEM_TRANS_LOG<br />
<br />
<br />
Another consideration is that an Index Fast Full Scan can only be used on an index where at least one of the columns is defined as NOT NULL. Otherwise it is up to the CBO to choose the access plan that it believes is best. In versions 8i and 9i it would appear that a full table scan is the preferred method when a FFS is not available. Also a full table scan has to be used in situations where the index is invalid such as immediately after an alter table move is performed.<br />
I would like to thank Mark J. Bobak for reviewing this article.<br />
<br />
Lewis, J. (2001). Practical Oracle 8i, Chapter 7: Enhanced Indexing. Upper Saddle River: NJ. Addison Wesley. pp. 128-129.<br />
Oracle Corporation (2003, December). Oracle Database Performance Tuning Guide 10g, Ch 16, Using Indexes and Clusters, Section Label: Re-creating Indexes and Ch 14, The Query Optimizer, Section Label: Fast Full Index Scans.<br />
For those with access to metalink at metalink.oracle.com the following notes might be of interest:<br />
How to Rebuild and Index Note 232819.1<br />
SCRIPT: Move indexes from one tablespace to another Note 131959.1<br />
Guidelines on When to Rebuild a B-Tree Index Note 77574.1<br />
I lock my table before truncating it, but I still get ORA-00054: resource busy ... Why ?<br />
The following quote can be found in the version 9.2 Concepts manual chapter 14: SQL, PL/SQL, and JAVA under the section titled Data Definition Language Statements: “DDL statements implicitly commit the preceding [transaction] and start a new transaction.”<br />
Truncate is a DDL statement and is contained in the list of example DDL immediately above the quote. The implicit commit that precedes the truncate command terminates the transaction began by the lock table in exclusive mode command allowing DML statements issued by other sessions after the lock table command was issued and before the truncate command was issued to access the table. The sessions issuing DML statements obtain a lock or locks on the table preventing DDL alteration during the DML activity. This behavior blocks the truncate command, which then because it is coded not to wait, immediately issues the ORA-00054 error.<br />
Example ran on Oracle version 9.2.0.4<br />
Note - the marktest table has only 6 rows contained in one data block.<br />
<br />
[step 1 from session one]<br />
<br />
12:45:31 SQL> lock table marktest in exclusive mode;<br />
<br />
Table(s) Locked.<br />
<br />
Elapsed: 00:00:00.11<br />
<br />
<br />
[Step 2 from a different session. Note time of delete, elapsed time and time upon completion]<br />
<br />
12:45:56 SQL> delete from mpowel01.marktest where rownum = 1;<br />
<br />
[session hangs waiting on lock]<br />
1 row deleted.<br />
<br />
Elapsed: 00:00:10.74 <= notice long elapsed time<br />
12:46:17 SQL> <= and termination time<br />
<br />
[Step 3 back in session one, note time of truncate is after delete was issued]<br />
<br />
12:45:41 SQL> [This enter was just to update the prompt time displayed]<br />
12:46:13 SQL> truncate table marktest;<br />
truncate table marktest<br />
*<br />
ERROR at line 1:<br />
ORA-00054: resource busy and acquire with NOWAIT specified<br />
<br />
Elapsed: 00:00:00.12<br />
12:46:18 SQL> [the truncate completes after the delete completes]<br />
<br />
The delete goes into a lock wait when first issued. Then when the truncate command is issued the delete executes, and because no commit follows the delete statement session two now has a lock on the table preventing the truncate from working. Any small table will work for duplicating this test.<br />
The behavior displayed above is a direct result of Oracle’s design decision to not require, indeed to not allowing DDL operations to be explicitly committed. By including an implicit commit both before and after every DDL operation Oracle made every DDL statement a separate transaction. This greatly reduces the likelihood of lock contention while accessing the Oracle base (dictionary) metadata tables. However, behavior as demonstrated above is the result.<br />
Note – commit and rollback statements can be issued after DDL statements are executed, but because of the implicit commit that follows all DDL statements the commit or rollback statements have no practical effect. There is nothing to commit or rollback, hence in Oracle successful DDL statements cannot be rolled back.<br />
<br />
Further reading: SQL Manual entry for the truncate table command.<br />
Can I see all the tables that are in the database if I have the DBA privilege ?<br />
The answer hinges on the word ALL. Using version 9.2 as a reference a DBA privileged user has been granted around 87 system related any type privileges such as select any table. The net effect is that a DBA privileged user has select access to any and all selectable user created objects defined to the database. The key word in the preceding sentence is user created. Excluded from these privileges are access rights to the SYS owned X$ views that the V$ views and that some RDBMS dictionary views are built on. Select privilege on the RDBMS base tables such as obj$ which house the RDBMS dictionary information are however included in the default DBA privileges. <br />
select count(*)<br />
from dba_sys_privs<br />
where grantee = 'DBA'<br />
and privilege like '%ANY%';<br />
<br />
COUNT(*)<br />
----------<br />
87<br />
<br />
The RDBMS dictionary views user_ and all_ continue to work as normal for a DBA privileged user; however since a DBA has access to all user objects the all_ views should return the same counts as the dba_ views. <br />
How do you purge old statspack snapshots automatically ?<br />
Below is a script that may be run via cron which will purge all old snapshots that exceed the specified count. This script requires no knowledge of the password for the PERFSTAT user. The script has been tested with Oracle 8.1.7 and 9.2.0.<br />
To implement this script, do the following:<br />
1) Save the script (below) as sp_purge.ksh to each Unix machine on which the Oracle instance may be located. <br />
2) If the Unix tmp directory is anything other than /tmp then you will need to modify the script accordingly.<br />
3) If your oratab file is located in any directory other than /var/opt/oracle then you will need to update the script accordingly.<br />
4) Set execute privilege on the script: chmod u+x sp_purge.ksh<br />
5) Establish a cron job to call the script. The script requires three parameters:<br />
· The name of the database in which to purge snapshots.<br />
· The maximum number of snapshots to retain.<br />
· The email recipient for success messages.<br />
Here is an example cron entry: <br />
00 19 * * 1-5 /scripts/sp_purge.ksh prod 60 mrogers@company.com >>/tmp/sp_purge_portal.log 2>&1 &<br />
This entry causes the script to run at 19:00 each weekday, to retain no more than 60 snapshots for the ‘prod’ database, and send success messages to: mrogers@company.com<br />
6) Note that this script may be invoked on any machine on which the instance may run. If the instance is not on the current machine, then a simple message to that effect will be sent to a file in the tmp directory.<br />
7) Note also that all log files are written to the tmp directory.<br />
<br />
<br />
Automatic StatsPack snapshot purge script:<br />
-----------------------CUT----------CUT----------CUT-------------------------<br />
#!/bin/ksh<br />
<br />
<br />
# Script Name: sp_purge.ksh<br />
<br />
<br />
# This script is designed to purge StatsPack snapshots.<br />
#<br />
# Parameter $1 is the name of the database.<br />
# Parameter $2 is the maximum number of snapshots to retain.<br />
# Parameter $3 is the mail recipient for success messages.<br />
#<br />
# To succeed, this script must be run on the machine on which the<br />
# instance is running.<br />
<br />
# Example for calling this script:<br />
#<br />
# sp_purge.ksh prod 30 username@mycompany.com<br />
<br />
<br />
# Script History:<br />
#<br />
# Who Date Action<br />
# --------------- ------------ --------------------------------------------<br />
# Mark J. Rogers 22-Sep-2003 Script creation.<br />
#<br />
#<br />
#<br />
<br />
tmp_dir=/tmp<br />
<br />
<br />
# Validate the parameters.<br />
<br />
if [[ $# -ne 3 ]]; then<br />
echo ""<br />
echo "*** ERROR: You must specify these parameters: "<br />
echo ""<br />
echo " 1: the name of the database"<br />
echo " 2: the maximum # of snapshots to retain"<br />
echo " 3: the mail recipient for success messages"<br />
echo ""<br />
exit 1<br />
fi<br />
<br />
grep "^${1}:" /var/opt/oracle/oratab >> /dev/null<br />
if [[ $? -ne 0 ]]; then<br />
echo ""<br />
echo "*** ERROR: The ORACLE_SID specified in parameter 1 is not a valid SID."<br />
echo " (Note that the SID is case sensitive.)"<br />
echo ""<br />
exit 1<br />
fi<br />
<br />
if [[ ! (${2} -ge 0) ]]; then<br />
echo ""<br />
echo "*** ERROR: Parameter 2 must specify the # of snapshots to retain."<br />
echo ""<br />
exit 1<br />
fi<br />
<br />
# Ensure that the instance is running on the current machine.<br />
ps -ef | grep pmon | grep $1 >> /dev/null<br />
if [[ $? -ne 0 ]]; then<br />
echo ""<br />
echo "*** ERROR: Instance $1 is not running on machine `uname -n` "<br />
echo " on `date`."<br />
echo " The instance must be running on the current machine for this"<br />
echo " script to function properly."<br />
echo ""<br />
echo " Exiting..."<br />
echo ""<br />
exit 1<br />
fi<br />
<br />
# Establish error handling for this UNIX script.<br />
<br />
function errtrap {<br />
the_status=$?<br />
echo ""<br />
echo " *** ERROR: Error message $the_status occured on line number $1."<br />
echo ""<br />
echo " *** The script is aborting."<br />
echo ""<br />
exit $the_status<br />
}<br />
<br />
trap \<br />
' \<br />
errtrap $LINENO \<br />
' \<br />
ERR<br />
<br />
# Set up the Oracle environment.<br />
<br />
export ORACLE_SID=${1}<br />
export ORAENV_ASK=NO<br />
. oraenv<br />
<br />
script_name=${0##*/}<br />
echo ""<br />
echo "Script: $script_name"<br />
echo " started on: `date`"<br />
echo " by user: `id`"<br />
echo " on machine: `uname -n`"<br />
echo ""<br />
echo "This script is designed to purge StatsPack snapshots for the "<br />
echo " $ORACLE_SID database."<br />
echo ""<br />
echo "You have requested to retain no more than $2 StatsPack snapshots."<br />
echo ""<br />
<br />
tmp_script=${tmp_dir}/sp_purge_tmp_$ORACLE_SID.ksh # script to actually purge<br />
tmp_output=${tmp_dir}/sp_purge_tmp_$ORACLE_SID.out # output to be mailed<br />
<br />
rm -f $tmp_script<br />
rm -f $tmp_output<br />
<br />
sqlplus -s <<eof_sp><br />
/ as sysdba<br />
<br />
whenever sqlerror exit failure rollback<br />
whenever oserror exit failure rollback<br />
<br />
SET SERVEROUTPUT ON<br />
SET FEEDBACK OFF<br />
<br />
VARIABLE P_SNAPS_TO_RETAIN NUMBER<br />
VARIABLE P_LOSNAPID NUMBER<br />
VARIABLE P_HISNAPID NUMBER<br />
<br />
BEGIN<br />
/* Assign values to these variables. */<br />
:P_SNAPS_TO_RETAIN := ${2};<br />
:P_LOSNAPID := -1;<br />
:P_HISNAPID := -1;<br />
END;<br />
/<br />
<br />
-- Identify the snapshot ids to purge, if any.<br />
<br />
DECLARE<br />
<br />
V_LOSNAPID NUMBER := NULL; -- Low snapshot ID to purge.<br />
V_HISNAPID NUMBER := NULL; -- High snapshot ID to purge.<br />
V_COUNT NUMBER := NULL; -- Number of snapshots current saved.<br />
V_COUNTER NUMBER := 0; -- Temporary counter variable.<br />
V_DBID NUMBER := NULL; -- Current database ID.<br />
V_INSTANCE_NUMBER NUMBER := NULL; -- Current instance number.<br />
V_SNAPS_TO_RETAIN NUMBER := :P_SNAPS_TO_RETAIN; -- Max snaps to retain.<br />
<br />
BEGIN<br />
<br />
select <br />
d.dbid,<br />
i.instance_number <br />
INTO <br />
v_DBID,<br />
V_INSTANCE_NUMBER<br />
from <br />
v\$database d,<br />
v\$instance i;<br />
<br />
select <br />
count(snap_id) <br />
into<br />
v_count<br />
from <br />
perfstat.stats\$snapshot<br />
where<br />
dbid = V_DBID AND<br />
instance_number = V_INSTANCE_NUMBER;<br />
<br />
IF V_COUNT <= V_SNAPS_TO_RETAIN THEN<br />
<br />
-- We do NOT need to perform a purge.<br />
<br />
DBMS_OUTPUT.PUT_LINE ('NOTE: There are only ' ||<br />
to_char(v_count) || ' snapshots currently saved.');<br />
<br />
ELSE<br />
<br />
-- We DO need to perform a purge.<br />
<br />
DBMS_OUTPUT.PUT_LINE ('There are currently ' ||<br />
to_char(v_count) || ' snapshots saved.');<br />
<br />
-- Obtain the low snapshot id to be purged. <br />
<br />
select <br />
min(snap_id) <br />
into<br />
V_LOSNAPID<br />
from <br />
perfstat.stats\$snapshot <br />
where <br />
dbid = V_DBID AND<br />
instance_number = V_INSTANCE_NUMBER;<br />
<br />
-- Obtain the high snapshot id to be purged. <br />
<br />
FOR V_HISNAPID_REC IN <br />
(SELECT <br />
SNAP_ID <br />
FROM <br />
perfstat.stats\$snapshot <br />
WHERE<br />
dbid = V_DBID AND<br />
instance_number = V_INSTANCE_NUMBER<br />
ORDER BY <br />
SNAP_ID DESC)<br />
LOOP<br />
V_COUNTER := V_COUNTER + 1;<br />
IF V_COUNTER > V_SNAPS_TO_RETAIN THEN<br />
V_HISNAPID := V_HISNAPID_REC.SNAP_ID;<br />
EXIT; -- Exit this LOOP and proceed to the next statement. <br />
END IF;<br />
END LOOP;<br />
<br />
:P_LOSNAPID := V_LOSNAPID; <br />
:P_HISNAPID := V_HISNAPID; <br />
<br />
END IF;<br />
<br />
END;<br />
/<br />
<br />
prompt<br />
-- Generate the specific purge script. <br />
set linesize 60<br />
spool $tmp_script<br />
begin<br />
IF (:P_LOSNAPID <> -1) THEN<br />
/* Build the script to purge the StatsPack snapshots. */<br />
dbms_output.put_line('#!/bin/ksh');<br />
dbms_output.put_line('#THIS IS THE SCRIPT TO ACTUALLY PERFORM THE PURGE');<br />
dbms_output.put_line('trap '' exit \$? '' ERR');<br />
dbms_output.put_line('sqlplus -s << SP_EOF2');<br />
dbms_output.put_line('/ as sysdba');<br />
dbms_output.put_line('whenever sqlerror exit failure rollback');<br />
dbms_output.put_line('whenever oserror exit failure rollback');<br />
dbms_output.put_line('@ \$ORACLE_HOME/rdbms/admin/sppurge.sql');<br />
dbms_output.put_line(:P_LOSNAPID);<br />
dbms_output.put_line(:P_HISNAPID);<br />
dbms_output.put_line('-- the following are needed again');<br />
dbms_output.put_line('whenever sqlerror exit failure rollback');<br />
dbms_output.put_line('whenever oserror exit failure rollback');<br />
dbms_output.put_line('commit;');<br />
dbms_output.put_line('exit');<br />
dbms_output.put_line('SP_EOF2');<br />
dbms_output.put_line('exit \$?');<br />
END IF;<br />
end;<br />
/<br />
spool off<br />
<br />
exit<br />
EOF_SP<br />
<br />
if [[ ! (-f ${tmp_script}) ]]; then<br />
echo ""<br />
echo "*** ERROR: Temporary script: ${tmp_script} does not exist."<br />
echo ""<br />
exit 1<br />
fi<br />
<br />
if [[ `cat ${tmp_script} | wc -l` -ne 0 ]]; then<br />
# Execute the newly generated StatsPack snapshot purge script. <br />
chmod u+x $tmp_script<br />
echo ""<br />
echo "Performing the purge..."<br />
echo ""<br />
$tmp_script > $tmp_output<br />
cat $tmp_output # display the output<br />
# Check the output file for a success message:<br />
trap ' ' ERR # temporarily reset error handling for the grep command<br />
grep "^Purge of specified Snapshot range complete." $tmp_output >> /dev/null<br />
if [[ $? -ne 0 ]]; then<br />
echo ""<br />
echo "*** ERROR: The purge did not complete successfully."<br />
echo " Check the log file $tmp_output."<br />
echo ""<br />
exit 1<br />
fi<br />
trap ' errtrap $LINENO ' ERR # re-establish desired error handler<br />
else<br />
# No purge script was created.<br />
echo "No snapshot purge was necessary." > $tmp_output<br />
fi<br />
<br />
echo ""<br />
echo "The ${script_name} script appears to have completed "<br />
echo " successfully on `date`."<br />
echo ""<br />
<br />
mailx \<br />
-s "sp_purge.ksh in $ORACLE_SID on `uname -n` completed successfully" \<br />
${3} \<br />
< $tmp_output<br />
<br />
# End of script sp_purge.ksh.<br />
-----------------------CUT----------CUT----------CUT-------------------------<br />
Is there any reason why the tablespace_name in user_tables and all_tables is blank for an Index Organized Table?<br />
The IOT name itself is a logical name and occupies no space. When an IOT is created it generates two segments.<br />
1. An index name SYS_IOT_TOP_<iot_object_id>.<br />
2. Optionally an overflow table named SYS_IOT_OVER_<iot_object_id>.<br />
It is these two segments which occupy space and against which tablespace is visible. To view the tablespace_name first find the object_id of the table and then search against that object_id in all_indexes. Example:<br />
<br />
-- Lets First Create a sample INDEX ORGANIZED TABLE named DELME_IOT.<br />
CREATE TABLE DELME_IOT<br />
(<br />
A INTEGER,<br />
B INTEGER NOT NULL,<br />
C INTEGER NOT NULL,<br />
D INTEGER, <br />
PRIMARY KEY (C, B)<br />
)<br />
ORGANIZATION INDEX<br />
LOGGING<br />
TABLESPACE SYSTEM<br />
PCTFREE 10<br />
INITRANS 2<br />
MAXTRANS 255<br />
STORAGE (<br />
INITIAL 64K<br />
MINEXTENTS 1<br />
MAXEXTENTS 2147483645<br />
PCTINCREASE 0<br />
FREELISTS 1<br />
FREELIST GROUPS 1<br />
BUFFER_POOL DEFAULT<br />
)<br />
NOPARALLEL;<br />
--Now we will query the tablespace name against this table.<br />
SQL> select table_name,tablespace_name from all_tables<br />
2 where table_name like 'DELME_IOT';<br />
TABLE_NAME TABLESPACE_NAME<br />
------------------------------ ------------------------------<br />
DELME_IOT<br />
--Here we cannot see the tablespace name. Lets first find the object_id for this table.<br />
SQL> select object_id, object_name from all_objects where object_name like 'DELME_IOT';<br />
OBJECT_ID OBJECT_NAME<br />
---------- ------------------------------<br />
56709 DELME_IOT<br />
-- Here we have the object_id for the table DELME_IOT. Next we will query the all_indexes table.<br />
SQL> select table_name,index_name,tablespace_name from all_indexes where index_name like '%56709%';<br />
TABLE_NAME INDEX_NAME TABLESPACE_NAME<br />
------------------------------ ------------------------------ ------------------------------<br />
DELME_IOT SYS_IOT_TOP_56709 SYSTEM<br />
--Here we can see that tablespace name against the IOT <br />
Or simply the all_indexes table can be queried directly for the tablespace name.<br />
Our PL/SQL loop to load 20M rows into the database slows down after the first 1M or so, why ?<br />
The best way to describe what is happening here is with an example. First create two tables and populate them as follows:<br />
create table SOURCE (KEY number);<br />
begin<br />
for I in 1 .. 100000 loop<br />
insert into SOURCE (KEY) values (I);<br />
end loop;<br />
commit;<br />
end;<br />
/<br />
create table DEST (KEY number);<br />
We can now time the insert of 20 million rows into the DEST table using the following PL/SQL block:<br />
declare<br />
offset number;<br />
start_time date;<br />
time_taken number;<br />
begin<br />
offset:= 0;<br />
for I in 1 .. 200 loop<br />
start_time:= sysdate;<br />
insert into DESTINATION (KEY) select KEY+offset from SOURCE;<br />
time_taken:= (sysdate-start_time)*24*60*60;<br />
dbms_output.put_line('Loop '||to_char(I)||' ending at '||to_char(sysdate,'HH24:MI:SS')||<br />
' after '||to_char(time_taken)||' seconds.');<br />
end loop;<br />
end; <br />
/<br />
This loop produces output that shows how long it takes to load 100000 rows into the table. Ideally we would like the last 100000 rows to be inserted in roughly the same amount of time as the first 1000000 rows. In this case, this is exactly what we find - that is, there is no slow down as more rows are loaded.<br />
Perhaps this is because we are inserting known data values in order? As a second attempt, we can modify the insert statement in line 9 of the above procedure to read:<br />
insert into DESTINATION (KEY) select dbms_random.random from SOURCE;<br />
Looking at the times taken for each 100000 row insert we notice that although the time is slightly longer on average than it wasfor the first test, there is still no noticeable increase in time through the iterations in the loop. The extra time taken being the overhead of the call to dbms_random.<br />
So, maybe we need an index on the DESTINATION table in order to see the effect that inserting many rows into the table has on performance. Create an index on the table as follows:<br />
create index IX1 on DESTINATION (KEY);<br />
and then rerun the PL/SQL block that we ran earlier. Perhaps surprisingly in this case, there is still no noticeable increase in the amount of time taken to load 100000 rows at the end of the procedure compared to at the beginning.<br />
As a last resort, we can run the second PL/SQL block, inserting a random number into the table. At last, we come to a point where it takes longer to insert the last 100000 rows than it did to insert the first 100000 rows. (In my test, an increase in time of over 5000%!) This indicates that the presence of an index on a column that contains values in a random order is likely to be the culprit that is slowing the inserts.<br />
The reason for this slow down can be explained if the test is modified to provide statistics on the number of reads and writes that are performed on the table and index during the load. As the table is initially empty when the load starts, there is no need to read from the disk - new blocks are simply created internally and mapped onto the data file. There are, however, disk writes as the database writer flushes dirty buffers to the disk, making space for new blocks to be created. Initially this is true for the index as well. However, after about 1 million rows have been inserted, the number of reads from the index starts to increase. In my tests, the insert of the last 100000 rows into the table required over 80000 disk reads from the index segment! (And 3 from the table.)<br />
The reason for the slow down is therefore clear. Up to about a million rows, the index is completely held within the buffer cache. In order to insert a new entry into the index, Oracle can scan a bunch of blocks that are in memory and insert the row. Once the index grows beyond the size available in the block buffers, some of the blocks needed for the index maintenance may not be in memory, and will need to be read. However, in order to red them, Oracle first needs to create space for them, by clearing out old blocks, which in turn may be needed for a later insert. This thrashing of blocks in and out of memory is the source of the performance issue.<br />
This also explains why the index did not show the same characteristics when the rows were inserted in order - once a leaf block had been written it would not be required again, and so the reads from the index were not required.<br />
In order to try to relieve the problem, the insert statement can be modified to read:<br />
insert into DESTINATION (KEY) select dbms_random.random from SOURCE order by 1;<br />
This forces each 100000 rows to be inserted in numeric order, progressing through the index from start to finish. Although this does not completely get rid of the problem, on my (rather slow) test system, the time to load 20million rows was reduced by over 28 hours (a saving of 29%)!<br />
The ideal solution is of course to drop the indexes on a table before running a large load and recreate them after the load has completed.<br />
It is worth noting that this characteristic is only observed when inserting large numbers of rows into the table using a batch system when there is little other activity on the database. Inserting a single row into a table with 20 million rows on a busy database will not take significantly more time than inserting the same row into an empty table, as the chances of the necessary blocks being in the buffer cache are equally random depending on the use that has been made of the database in the recent past. <br />
How can an end-user see whether a row is locked, and who is locking it, before they try to lock it themselves ?<br />
The short answer is: Sorry, it's not possible.<br />
For the longer answer, please continue reading.<br />
In principle, there is no scalable way to see what rows are locked or who holds a lock on a particular row. That's because there is no centralized list of locked rows anywhere in Oracle. This is by design, and is a feature, not a bug. One of the Oracle database engine's most powerful features is it's ability to scale to very large volumes of data and numbers of users. Much of it's ability to do so is integral to the locking model it employs. <br />
As previously mentioned, there is no way to determine who is locking a row without attempting to lock it yourself. To see if a particular row is locked, you may attempt to lock it. If you succeed, then no one else had it locked, but you do now. (If it's not your intent to update it, don't forget to rollback and release the lock.) If you fail to lock it, one of two things will happen. If you have chosen to attempt the lock in NO WAIT mode, then your session will return an ORA-0054 "resource busy" error, and you will not have a lock on the row(s). If you have attempted the lock in WAIT mode, (which is the default for DML statements), your session will begin waiting on the transaction which currently holds the lock on the row you are trying to lock. As soon as that transaction is commited or rolled back, your session will be freed, and will lock the row or (if the data changed and your where clause no longer applies) may fail to retrieve the row. <br />
While a session is waiting for a specific row, it's possible to identify the row being waited on, however, this information is only available while a session is actviely waiting for a lock on a row. For the duration of the wait, you may examine V$SESSION, and in particular, the following columns may be of interest: ROW_WAIT_OBJECT#, ROW_WAIT_FILE#, ROW_WAIT_BLOCK#, and ROW_WAIT_ROW#. Note that these columns are only populated for a session that is actively waiting on a specific row. Sessions not waiting on a resource, or waiting on a non-row resource, will have null values for these columns. However, this method of determining who holds locks on what rows, is cumbersome at best, and is not feasible for an end-user. Also, there is the race condition to consider. Even if you are able to implement any kind of tool or application based on the above information, it's likely that by the time you process and interpret the results, those results are out-of-date.<br />
A quick note about a MetaLink document outlining how to do this: A MetaLink analyst has cobbled together a bit of Pro*C and PL/SQL, that can identify locked rows on a specific table. A review of the code will reveal that he simply attempts to lock each row and returns those rows that failed to lock. If you read the introduction closely, the author admits that the method is "very slow and is NOT bullet-proof". I strongly recommend you not attempt implementing this method, at all, at the very least, it certainly should not be part of a larger application where the application would have to rely on the functionality. However, if you are still interested in the code used, see the hyperlink in the Further Reading section below.<br />
In my application we run into these problems, and the scripts used to see locked rows wasn't consistent either. So finally I created following objects for them (developers), which eases the pain a bit for developers.<br />
CREATE OR REPLACE FORCE VIEW SYSTEM.DB$LOCKED_OBJECTS <br />
(OBJECT_NAME, SESSION_ID, ORACLE_USERNAME, OS_USER_NAME, SQL_ACTIONS, LOCK_MODE) <br />
AS <br />
SELECT /*+ no_merge(lo) */ <br />
DO.object_name, lo.SESSION_ID, lo.oracle_username, lo.OS_USER_NAME, <br />
DECODE(locked_mode, <br />
1, 'SELECT', <br />
2, 'SELECT FOR UPDATE / LOCK ROW SHARE', <br />
3, 'INSERT/UPDATE/DELETE/LOCK ROW EXCLUSIVE', <br />
4, 'CREATE INDEX/LOCK SHARE', <br />
5, 'LOCK SHARE ROW EXCLUSIVE', <br />
6, 'ALTER TABLE/DROP TABLE/DROP INDEX/TRUNCATE TABLE/LOCK EXCLUSIVE') sql_actions, <br />
DECODE(locked_mode, 1, 'NULL', 2, 'SS - SUB SHARE', 3, 'SX - SUB EXCLUSIVE', <br />
4, 'S - SHARE', 5, 'SSX - SHARE/SUB EXCLUSIVE', 6, 'X - EXCLUSIVE') Lock_mode <br />
FROM sys.V_$LOCKED_OBJECT lo, DB$OBJECTS DO <br />
WHERE DO.object_id = lo.object_id; <br />
CREATE PUBLIC SYNONYM DB$LOCKED_OBJECTS FOR SYSTEM.DB$LOCKED_OBJECTS; <br />
GRANT SELECT ON SYSTEM.DB$LOCKED_OBJECTS TO PUBLIC; <br />
CREATE OR REPLACE FORCE VIEW SYSTEM.DB$LOCKS <br />
(OBJ_OWNER, OBJ_NAME, OBJ_TYPE, OBJ_ROWID, DB_USER,SID, LOCK_TYPE, ROW_WAIT_FILE#, ROW_WAIT_BLOCK#, ROW_WAIT_ROW#) <br />
AS <br />
SELECT owner obj_owner, <br />
object_name obj_name, <br />
object_type obj_type, <br />
dbms_rowid.rowid_create(1, row_wait_obj#, ROW_WAIT_FILE#, <br />
ROW_WAIT_BLOCK#,ROW_WAIT_ROW#) obj_rowid, <br />
a.username db_user, a.SID SID, a.TYPE lock_type, <br />
a.row_wait_file#, a.row_wait_block#, a.row_wait_row# <br />
FROM DB$OBJECTS, <br />
(SELECT /*+ no_merge(a) no_merge(b) */ <br />
a.username, a.SID, a.row_wait_obj#, a.ROW_WAIT_FILE#, <br />
a.ROW_WAIT_BLOCK#, a.ROW_WAIT_ROW#, b.TYPE <br />
FROM sys.V_$SESSION a, sys.V_$LOCK b <br />
WHERE a.username IS NOT NULL <br />
AND a.row_wait_obj# <> -1 <br />
AND a.SID = b.SID <br />
AND b.TYPE IN ('TX','TM') <br />
) a <br />
WHERE object_id = a.row_wait_obj#; <br />
CREATE PUBLIC SYNONYM DB$LOCKS FOR SYSTEM.DB$LOCKS; <br />
GRANT SELECT ON SYSTEM.DB$LOCKS TO PUBLIC; <br />
Where DB$OBJECTS is a MV that I refresh everyday at 6AM. <br />
CREATE MATERIALIZED VIEW SYSTEM.DB$OBJECTS <br />
PCTFREE 10 PCTUSED 0 INITRANS 2 MAXTRANS 255 <br />
STORAGE( <br />
MINEXTENTS 1 <br />
MAXEXTENTS UNLIMITED <br />
PCTINCREASE 0 <br />
BUFFER_POOL DEFAULT <br />
) <br />
TABLESPACE TEMPTABS <br />
NOLOGGING <br />
NOCACHE <br />
NOPARALLEL <br />
USING INDEX PCTFREE 10 INITRANS 2 MAXTRANS 255 <br />
STORAGE(BUFFER_POOL DEFAULT) <br />
REFRESH COMPLETE <br />
WITH ROWID <br />
USING DEFAULT LOCAL ROLLBACK SEGMENT <br />
DISABLE QUERY REWRITE AS <br />
SELECT * <br />
FROM DBA_OBJECTS <br />
/ <br />
With this, selects from (mainly) db$locks and db$locked_objects is really fast and my developers love it. This is all on 9202 by the way ... so as I read Mark's article, this makes finding the 'rows being waited on' much faster, I think.<br />
I have deleted 6M rows from a table - how can I reclaim the space that is now free ?<br />
The only way to make this space available for other objects to use is to use some sort of rebuild on the table.<br />
The fastest option is probably:<br />
Alter table XXXX move nologging;<br />
This will recreate the table and repack all the data. Once you have done this, you should take a backup of at least the tablespace that the table is in as the nologging command means that you could not recover the table from an earlier backup if the system crashed.<br />
To avoid the backup, you can use:<br />
alter table XXX move;<br />
After you have used the move command (in either form) you will have to rebuild all the indexes on that table. Again you can choose to do this with or without the nologging command.<br />
Alter index XXX_idx1 rebuild nologging;<br />
Alter index XXX_idx1 rebuild;<br />
In the case of indexes, it is less important to take the backup as you don't lose information if the index is accidentally destroyed - you don't need to recover them, you can always rebuild them. Don't forget that you might want to increase your sort_area_size whilst rebuilding the indexes.<br />
You shoudl also remember that if your table rows are subject to significant growth after they have been inserted you will have to work out a more subtle strategy for rebuilding the table. If you do a simple rebuild, then you will either end up wasting a lot of space, or you will find that the table starts to acquire migrated rows as time passes. The details of (and one resolution to) this problem can be found in Practical Oracle 8i.<br />
How do I clone a database from one machine to another - when the filesystems are different ?<br />
(I'm assuming you're using a hot backup of the source database to do your clone - after all, no-one should be using cold backups!) <br />
Cloning a database is normally done with the following sequence:<br />
1. Copy the datafiles to the target node <br />
2. Produce a CREATE CONTROLFILE command for the cloned database <br />
3. Start the target database in NOMOUNT mode <br />
4. Run the create controlfile command created in (2) <br />
5. Issue "recover database using backup controlfile until cancel" <br />
6. Provide the recovery processe with archived redo logs until you are happy <br />
7. Cancel the recovery and issue "alter database open resetlogs" <br />
8. Re-create or re-add any TEMPFILES used for temporary tablespaces <br />
This is all quite straightforward even if you have to rename the database, and any of the files in the database. The secret is in step (2). On the source database, you can run <br />
SQL> alter database backup controlfile to trace<br />
at which point you will get a full "create controlfile" command dumped out to the user_dump_dest directory. It will look something like <br />
CREATE CONTROLFILE REUSE DATABASE "SOURCEDB" NORESETLOGS ARCHIVELOG<br />
...<br />
LOGFILE<br />
GROUP 1 '/disk1/log1.dbf' SIZE 50M,<br />
...<br />
DATAFILE<br />
'/disk2/system01.dbf',<br />
'/disk3/rbs01.dbf',<br />
...<br />
'/diskN/last_file01.dbf'<br />
CHARACTER SET WE8ISO8859P1;<br />
For cloning this database, you will alter <br />
· The REUSE to SET <br />
· The SOURCEDB to NEWDB (that is, the new name of the target database) <br />
· The NORESETLOGS to RESETLOGS <br />
For each file listed under the DATAFILE clause, if you will be renaming the file as part of the cloning process, then you simply update the file name to the new name in the CREATE CONTROLFILE command. <br />
<br />
Further reading: You can also clone a database using the RMAN duplicate command (see the RMAN documentation for instructions)<br />
I have a two-column partition key, and my data keeps going in the wrong partition. Why ?<br />
Unfortunately, the natural assumption that many people make when specifying multiple columns in their partitioning key, is that they are creating an 'n'-dimensional partitioning of the data, where 'n' is the number of columns in the partitioning key. Consider a simple example where there are 2 columns in the partition key.<br />
SQL> create table DEMO (<br />
2 pcol1 date,<br />
3 pcol2 date,<br />
4 data varchar2(10) )<br />
5 partition by range (pcol1,pcol2)<br />
6 ( partition p_PRE_2000 values less than<br />
7 (to_date('01/01/2000','dd/mm/yyyy'), to_date('01/01/2000','dd/mm/yyyy')),<br />
8 partition p_2000_JAN_JUN values less than<br />
9 (to_date('01/07/2000','dd/mm/yyyy'), to_date('01/07/2000','dd/mm/yyyy')),<br />
10 partition p_2000_JUL_DEC values less than<br />
11 (to_date('01/01/2001','dd/mm/yyyy'), to_date('01/01/2001','dd/mm/yyyy')),<br />
12 partition p_2001_JAN_JUN values less than<br />
13 (to_date('01/07/2001','dd/mm/yyyy'), to_date('01/07/2001','dd/mm/yyyy'))<br />
14 );<br />
<br />
Table created.<br />
Its fairly obvious that the desired result here is for each partition to hold a 6 month window of data based on the columns PCOL1 and PCOL2. But the partition key columns are not dimensions, a better label would be tie-breakers. When data is presented for insertion, the columns in the partition key are evaluated "left to right" in order to determine the correct partition. Thus in the DDL above, the second partition key column (PCOL2) will only be considered when the first column is not sufficient to determine the partition. <br />
So lets see what happens when we add some data. <br />
SQL> alter session set nls_date_format = 'dd/mm/yyyy';<br />
<br />
Session altered.<br />
<br />
SQL> insert into DEMO values ('01/02/1999','01/01/1999','row1');<br />
SQL> insert into DEMO values ('01/02/2000','01/01/1999','row2');<br />
SQL> insert into DEMO values ('01/02/2001','01/08/1999','row3');<br />
SQL> insert into DEMO values ('01/02/2001','01/08/2000','row4');<br />
SQL> insert into DEMO values ('01/02/1999','01/02/1999','row5');<br />
SQL> insert into DEMO values ('01/02/2000','01/02/2000','row6');<br />
SQL> insert into DEMO values ('01/02/2001','01/08/2001','row7');<br />
SQL> insert into DEMO values ('01/02/2001','01/08/1999','row8');<br />
SQL> insert into DEMO values ('01/08/2000','01/08/2000','row9');<br />
SQL> insert into DEMO values ('01/08/1999','01/02/2001','row10');<br />
<br />
SQL> select * from DEMO partition (p_PRE_2000);<br />
<br />
PCOL1 PCOL2 DATA<br />
---------- ---------- ----------<br />
01/02/1999 01/01/1999 row1<br />
01/02/1999 01/02/1999 row5<br />
01/08/1999 01/02/2001 row10<br />
<br />
SQL> select * from DEMO partition (p_2000_JAN_JUN);<br />
<br />
PCOL1 PCOL2 DATA<br />
---------- ---------- ----------<br />
01/02/2000 01/01/1999 row2<br />
01/02/2000 01/02/2000 row6<br />
<br />
SQL> select * from DEMO partition (p_2000_JUL_DEC);<br />
<br />
PCOL1 PCOL2 DATA<br />
---------- ---------- ----------<br />
01/08/2000 01/08/2000 row9<br />
<br />
SQL> select * from DEMO partition (p_2001_JAN_JUN);<br />
<br />
PCOL1 PCOL2 DATA<br />
---------- ---------- ----------<br />
01/02/2001 01/08/1999 row3<br />
01/02/2001 01/08/2000 row4<br />
01/02/2001 01/08/2001 row7<br />
01/02/2001 01/08/1999 row8<br />
Only rows 1,5,6 appear to satisfy what the user desired, but in fact, all of the rows have been correctly located, because in all cases, the leading column of the partition key (PCOL1) was enough to determine the correct partition. To emphasise this further, consider <br />
SQL> insert into DEMO values ('01/08/1999','01/02/2005','row11');<br />
<br />
1 row created.<br />
It would appear from the original DDL that the year 2005 is out of bounds for PCOL2, but of course, this was not even considered because the value for PCOL1 is quite valid, and sufficient to determine that partiton P_PRE_2000 is the destination for this row. <br />
dba_segments and dba_tables show different numbers for blocks. What has gone wrong ?<br />
This is not a problem.<br />
The dba_segments view tells you about blocks that has been allocated (reserved) in the database for an object. The dba_tables view will be telling you how many blocks from that space had been used the last time you ran the analyze command, or used the dbms_stats package to collect statistics about the table.<br />
Note - the blocks figure recorded in the dba_tables view tells you about the point that the highwater mark for the table had reached. If you delete all the rows from a table, the highwater mark will not move, so the number of blocks used will not (from this perspective) be changed, even when you re-analyze the table. However, if you truncate the table the highwater mark will be reset to zero, and an immediate analyze would show the blocks figures back at zero.<br />
In passing - the dbms_stats package has been the 'advised'' method for collecting CBO-related statistics about object in recent versions of Oracle (probably since 8.1, but perhaps a little earlier).<br />
<br />
<br />
Further reading: FAQ: What's the difference between DELETE ROWS and TRUNCATE TABLE ?<br />
What is the time unit used in the v$ views ?<br />
In general the answer is hundredths of a second.<br />
However, there are a few exceptions (and possibly a few platform-specific oddities).<br />
In Oracle 9, the view v$enqueue_stat has been introduced to expose the number of enqueues (locks) that have been requested. The view includes the number of requests, waits, successful acquisitions, and failures. It also includes the time spent waiting - measured in milliseconds. (Humour point - according to some coder in Oracle Corp, there are 1024 milliseconds in a second; check the length of a one-second wait in the dbms_lock.sleep procedure, if you don't believe me).<br />
In late versions of Oracle 8.1, Oracle started recording time in v$session_wait, v$session_event, and v$system_event (or at least the underlying X$ objects) in micro seconds. However, the micro second time is not always exposed - some views have a column time_waited_micro, some only have the time_waited where the value is rounded to the hundredth. <br />
This does lead to some oddities in trace files, where you may find the elapsed time of an action recorded to the micro second, but the corresponding CPU time rounded to the hundredth, with the effect that the CPU time will sometimes be much larger than the elapsed time.<br />
<br />
Further reading: FAQ: Is there a way to suspend a task, or make it sleep, for a period of time ?<br />
My locally managed tablespaces seem to have lost some space according to dba_free_space. Where has it gone ?<br />
A locally managed tablespace breaks each file into uniform chunks, but reserves the first 64K (possibly the firsty 128K if you use a 32K block size) for the file header and space management tasks.<br />
In the case of an autoallocate tablespace, these chunks are 64K each, but in the case of a uniform tablespace, you can specify the chunksize when you create the tablespace. Each chunk will then be available for use as an extent (although in autoallocate tablespaces, the extent sizes will be selected from 64K, 1MB, 8MB, 64MB, 256MB, and each extent will be created from a number of adjacent 64K chunks).<br />
If you create a file that is not 64K plus an exact multiple of the chunk size, then there will be a bit of space at the end of the file which will be too small to act as a valid chunk. This will simply disappear from the dba_free_space view - which only records the total available legal chunk space for LMTs.<br />
For example, if you create a tablespace as 'space management local uniform extent 10M' with a file size of 100M, then you will find that you have 9 extents of 10M available, a 64K header, and 9.9MB 'vanished'. Add 64K to the file, and you will suddenly find that you have a whole extra 10MB extent available.<br />
Personally, I tend to keep the arithmetic simple, but not waste too much space, by working out a 'reasonable' size for the file, and then adding an over-generous 1MB for the header. . e.g, Extent size = 8M, so I start with (say) 800MB for data space, but make is 801MB for the file.<br />
What is the difference between cpu_costing in Oracle 9 and the old costing of Oracle 8 ?<br />
Oracle 9 introduces a more subtle, and thorough costing mechansim. It's a great improvement on the Oracle 7/8 version, but I think the change-over is going to be a bit like the change-over from rule-based to cost-based. If you don't understand how it works you may see some strange events.<br />
You can enable cpu_costing simply by collecting system_statistics for an appropriate period of time with the dbms_stats package. This records in the table sys.aux_stats$ values for:<br />
assumed CPU speed in MHz<br />
single block read time in milliseconds<br />
multiblock read time in milliseconds<br />
typical achieved multiblock read.<br />
These figures are used to produce three main effects.<br />
Instead of Oracle assuming that single block reads are just as expensive as multiblock reads, Oracle knows the relative speed. This is roughly equivalent to setting the parameter optimizer_index_cost_adj according to the db file xxxx read average wait times - it will encourage Oracle to use indexed access paths instead of tablescans because Oracle now understands that tablescans are more expensive than it used to think.<br />
Secondly, Oracle will use the 'achieved' average multiblock read count to calculate the number of read requests required to scan a table, instead of using an adjusted value of db_file_multiblock_read_count. Since many people use a value of db_file_multiblock_read_count that is inappropriate, the result of this is that Oracle is likely to increase the cost of, and therefore decrease the probability of, doing tablescans (and index fast full scans). Note - the recorded value is used in the calculations, but Oracle tries to use the init.ora value when actually running a tablescan.<br />
Finally, Oracle knows that (e.g.) to_char(date_col) = 'character value' costs a lot more CPU than number_col = number_value, so it may change its choice of index to use a less selective index if the consequential cost of using that index is lower on CPU usage. (Also, Oracle will re-arrange the order of non-access predicates to minimise CPU costs, but the difference in performance from this is not likely to be visible in most cases). <br />
Overall - it's definitely a good thing. In practice, you may see a much stronger bias towards indexed access paths which may impact performance.<br />
<br />
Further reading: Oracle 9.2 Performance Tuning Guide and Reference<br />
How do I find distributed queries / transactions (either issued from or connecting to this instance)?<br />
This FAQ entry will demonstrate SQL to locate sessions currently involved in distributed transactions in three forms: remote SQL issued from the instance, remote SQL issued from or to the instance, and remote transaction issued from other instances.<br />
All examples ran on 64 bit RDBMS version 8.1.7.4.<br />
Whenever a distributed query or DML statement is issued from an instance a Distributed Transaction lock, DX, is taken on the transaction and is visible through v$lock:<br />
select addr, kaddr, sid, type, id1 from v$lock where type = 'DX';<br />
<br />
no rows selected<br />
select count(*) from oper_console; -- synonym on remote table<br />
<br />
COUNT(*)<br />
--------<br />
6226<br />
select addr, kaddr, sid, type, id1 from v$lock where type = 'DX';<br />
<br />
ADDR KADDR SID TY ID1<br />
---------------- ---------------- ---------- -- ----------<br />
070000000015CC48 070000000015CC68 16 DX 29<br />
This entry will remain until the issuer either commits or rollbacks the transaction.This means that a query on a remote object is a transaction and it takes a rollback segment entry to support the two-phase commit feature (see References). But while the above query on v$lock will show distributed queries from this instance to a remote database it does not show queries from the remote instance to this instance.<br />
The only way I know to locate these queries from the (target) instance is the following query, which I based on an ASKTOM article (see references), and which requires the user to be logged on as SYS or to create a view on sys.x$k2gte and grant select on this view to the desired user(s).<br />
select<br />
username<br />
,osuser<br />
,status<br />
,sid<br />
,serial#<br />
,machine<br />
,process<br />
,terminal<br />
,program<br />
from<br />
v$session<br />
where saddr in ( select k2gtdses from sys.x$k2gte );<br />
This query will show sessions that have issued queries using remote objects and queries issued from remote instances to this instance.<br />
To find only those sessions coming from a remote source the change the where clause as follows (Eliminate DX transactions identified above):<br />
select<br />
s.username<br />
,s.osuser<br />
,s.status<br />
,s.sid<br />
,s.serial#<br />
,s.machine<br />
,s.process<br />
,s.terminal<br />
,s.program<br />
from<br />
v$session s<br />
where s.saddr in ( select x.k2gtdses from sys.x$k2gte x )<br />
and not exists ( select<br />
l.sid<br />
from v$lock l<br />
where l.type = 'DX'<br />
and l.sid = s.sid<br />
)<br />
;<br />
WARNING - During testing this query was producing incorrect results with both not in and exists clauses until I added the table name label on X$K2GTE. When X$ tables (views) are involved it is probably wise to fully qualify all tables and columns.The behavior of any X$ table is subject to change without notice with every patch set / release of the Oracle RDBMS so regular verification of query results is advisable. With a few exceptions Oracle does not document the X$ tables.<br />
<br />
Further reading: see Tom Kyte’s article on this subject at http://asktom.oracle.com/pls/ask/f?p=4950:8:::::F4950_P8_DISPLAYID:5309401983757 <br />
What happens if I drop a partition from a partitioned table whilst a query is running ?<br />
The documentation on partitioned tables explains that a select may execute and complete successfully across a drop partition even if the partition dropped is a target of the select statement. <br />
I have tested this out and found that it can work. However, if you drop a partition, then allocate the space it was using to another data segment before the query reaches it, then the query will (quite reasonably) crash.Therefore the guaranteed behaviour seems to be that the query will either be correct or the query will crash. (In fact, in a very early verion (I think 8.0.3/4) I found that some queries simply stopped when they hit the destroyed partition and returned whatever result they had accumulated up to that point.)<br />
Note - If you drop partitions at a high rate, and have a large number of partitions, you may find you get a significant parse-time overhead, because the cursors referencing a partitioned object become invalid when a partition is dropped (or subject to virtiually any other DDL-style maintenance).<br />
Why does a global index on my partitioned table work faster than a local index ?<br />
As usual, the quick (consultant's) answer is that the benefits depend on your system. How big is your data, what are your clustering factors like, are you using the features that supply the benefits, and in this case, how much work do you typical queries do anyway ?<br />
The global/local indexes problem is generic - global indexes CAN be faster than local indexes. In general, local indexes will be faster when partition elimination can take place, and the expected volume of data acquired id significant. If either of this conditions is not met, then local indexes probably won't help performance.<br />
Remember, the optimizer has to do some extra work to deal with partitioned tables. If the queries you use are very precise queries (e.g. on a nearly unique pair fo columns) then the cost of optimisation and identifying the partition may easily exceed the saving of having the partitions in the first place (The saving might be just one logical I/O, hence < 1/10,000 of a CPU second) .Where queries are very precise, a global index is quite likely to be a little more performant than a local index.<br />
This is a common Oracle trade-off between how much you win/lose and how often you make that win/loss. In the local / global indexes case you (should expect to) lose a tiny amount of performance on every query in order to win a huge amount when you do partition maintenance such as eliminating entire partitions of very old data with a quick drop partition command.<br />
Of course, if your partitions are large, and the indexes hit a large number of table rows, and the queries are over-generous in their throwaway rates; then the work saved by hitting just the one partition through exactly the correct index partition may prove to be a significant gain<br />
When I update one table from another, lots of my data turns to nulls - what have I done wrong ?<br />
The problem here is that you have probably written some SQL to update data, without remembering that sometimes the associated data may be missing. The code sample that shows this problem usually looks something like:<br />
update tableX tx<br />
set colA = (<br />
select sum(colB)<br />
from tableY ty<br />
where ty.ref_col = tx.ref_col<br />
)<br />
;<br />
But what will happen if there are no matching rows in TableY for a given row in TableX ? The subquery finds no data. You might expect this to result in Oracle returning the error "no data found". In fact Oracle simply returns a null value - so if colA originally held a value, it will disappear.<br />
To circumvent this problem, the simplest change is to modify the query so that it reads - update table X if there is any data. You do this by adding a where clause (that looks very similar to the subquery) to restrict the update to those rows for which relevant data exist. For example:<br />
update tableX tx<br />
set colA = (<br />
select sum(colB)<br />
from tableY ty<br />
where ty.ref_col = tx.ref_col<br />
)<br />
where exists (<br />
select null<br />
from tableY ty<br />
where ty.ref_col = tx.ref_col<br />
)<br />
;<br />
Note - this is not the only SQL solution for this type of update - take a look, for example, at updatable join views, which are typically twice as efficient as this subquery method.<br />
What are all these tables with names like EVT% and SMP% that have recently appeared in my system ? <br />
These tables are installed as part of Oracle Enterprise Manager (OEM). They most commonly appear if you have opted for a default database installation and the Enterprise Manager Management Server component was included. Unfortunately they have historically been dumped into the SYSTEM tablespace which is not the best location (for anything). Also in days gone by with the 2-tier client/server version of the console, if you happened to connect to a database that did not contain the objects (for example, you've logged on to the wrong schema), then the objects would be created automatically, which could leave you with multiple copies of the objects across several databases. <br />
If you do not use OEM then they can be dropped, otherwise it is best to ensure that they are within their own tablespace (an option that the more recent versions of OEM will give you during installation). <br />
How can I track structural changes to my database ?<br />
If you are worried about people doing things to your database and you not knowing, and you are running at 8i then you can use a DDL trigger and a logging table to keep a log of all DDL changes, who made them and when. The DDL trigger can be amended so that it logs only certain DDL changes - you needn't worry about logging everything.<br />
You will need a user, created somewhere so that no-one else has access - unless the DBA wishes them to have it, and a trigger owned by SYS which does the logging into a table in your user. The following shows how it all hangs together. This was taken from a session on an Oracle 8.1.7.4 database.<br />
SQL> connect system@cmdb<br />
Enter password: ******<br />
Connected.<br />
<br />
SQL> create tablespace logging<br />
2 datafile '/data2/CMDB/logging.dbf' size 201m<br />
3 extent management local<br />
4 uniform size 64k;<br />
Tablespace created.<br />
<br />
SQL> create user logging identified by something_daft<br />
2 default tablespace logging<br />
3 temporary tablespace temp<br />
4 quota 0 on system<br />
5 quota 200m on logging;<br />
User created.<br />
You will notice that I have not given the logging user the ability to connect to the database. This is not a problem as I'm simply using this user to keep my objects out of the SYS user and the SYSTEM tablespace. There is nothing stopping you from creating the logging table so that SYS owns it, but you will not be able to export it if you do. Having a separate user at least gives you that option.<br />
SQL> create table logging.ddl_log<br />
2 ( user_name varchar2(30),<br />
3 ddl_date date,<br />
4 ddl_type varchar2(30),<br />
5 object_type varchar2(18),<br />
6 owner varchar2(30),<br />
7 object_name varchar2(128)<br />
8 ) tablespace logging;<br />
Table created.<br />
Next we need to switch over to the SYS user to create our trigger. The trigger will write all its data into the logging user's ddl_log table.<br />
SQL> connect sys@cmdb<br />
Enter password: ********<br />
Connected.<br />
<br />
SQL> create or replace trigger DDLTrigger<br />
2 after DDL on database<br />
3 begin<br />
4 insert into logging.ddl_log <br />
5 values (ora_login_user,<br />
6 sysdate,<br />
7 ora_sysevent,<br />
8 ora_dict_obj_type,<br />
9 ora_dict_obj_owner,<br />
10 ora_dict_obj_name);<br />
11 exception<br />
12 when others then<br />
13 NULL;<br />
14 end;<br />
15 /<br />
Trigger created.<br />
And that is all there is to it. The following is an example where some DDL changes have been made in this test database, and the results of checking the contents of the logging.ddl_log table :<br />
SQL> connect cmdb@cmdb<br />
Enter password: ****<br />
Connected.<br />
<br />
SQL> alter table employee nologging;<br />
Table altered.<br />
<br />
SQL> grant select on location to public;<br />
Grant succeeded.<br />
<br />
SQL> alter user cmdb identified by new_password;<br />
User altered.<br />
<br />
SQL> create table test (a number);<br />
Table created.<br />
<br />
SQL> alter table test add constraint pk_test primary key (a);<br />
Table altered.<br />
<br />
SQL> insert into test (a) values (1);<br />
1 row created.<br />
<br />
SQL> commit;<br />
Commit complete.<br />
<br />
SQL> truncate table test drop storage;<br />
Table truncated.<br />
<br />
SQL> drop table test cascade constraints;<br />
Table dropped.<br />
<br />
<br />
SQL> connect system@cmdb<br />
Enter password: ******<br />
Connected.<br />
<br />
SQL> alter user cmdb identified by cmdb;<br />
User altered.<br />
<br />
SQL> column user_name format a10;<br />
SQL> column ddl_date format a20;<br />
SQL> column owner format a10<br />
SQL> column object_name format a20<br />
SQL> column object_type format a20<br />
SQL> column ddl_type format a20<br />
SQL> set lines 150<br />
SQL> set pages 50<br />
SQL> <br />
SQL> select * from logging.ddl_log;<br />
<br />
USER_NAME DDL_DATE DDL_TYPE OBJECT_TYPE OWNER OBJECT_NAME<br />
---------- -------------------- -------------------- -------------------- ---------- ---------------<br />
CMDB 10/09/2002 01:13:57 ALTER TABLE CMDB EMPLOYEE<br />
CMDB 10/09/2002 01:14:30 GRANT OBJECT PRIVILEGE CMDB LOCATION<br />
CMDB 10/09/2002 01:15:37 ALTER USER CMDB<br />
CMDB 10/09/2002 01:16:08 CREATE TABLE CMDB TEST<br />
CMDB 10/09/2002 01:16:23 CREATE INDEX CMDB PK_TEST<br />
CMDB 10/09/2002 01:16:23 ALTER TABLE CMDB TEST<br />
CMDB 10/09/2002 01:17:16 TRUNCATE TABLE CMDB TEST<br />
CMDB 10/09/2002 01:17:33 DROP TABLE CMDB TEST<br />
SYSTEM 10/09/2002 01:20:54 ALTER USER CMDB<br />
<br />
9 rows selected.<br />
From the logging.ddl_log table we can see that user CMDB made a change to the EMPLOYEE table with ALTER TABLE. CMDB then granted some privileges on the LOCATION object - which could be a view, a table, a trigger etc, we cannot tell from the logging details unfortunately, nor can we tell who was granted the privileges - whatever ones thay may have been. And so on.<br />
Notice that when CMDB created a primary key constraint using ALTER TABLE, this was logged first as a CREATE INDEX PK_TEST followed by an ALTER TABLE TEST. This is because the index was created in the background to support the constraint. Had there been an existing index which was suitable, the CREATE INDEX PK_TEST would not be seen.<br />
Things to remember<br />
You cannot, unfortunately, get the OSUSER of the person who made the changes to an object in the database. This is a problem if you have everybody logging in as user 'application' for example, you will only see that user 'application' made the change - so it will still be difficult to trace it back to a specific user. There is an option to obtain the IP address of the client PC, but that field is not available in the DDL Trigger unfortunately.<br />
The logging user has only 200 Mb of space to play with - if you are logging all changes on a fast changing database, you'll need to clear out the dross regularly to stop the tablespace filling up. You can use something like the following command to remove unwanted data :<br />
SQL> delete from logging.ddl_log where ddl_date<br />
commit;<br />
Commit complete.<br />
In my example, I simply used today's date as I don't have old data in there, you would use something like 'sysdate - 30' to delete the details of chnages made prior to 30 days ago.<br />
What will happen if the tablespace is full up, and someone tries to make some DDL changes ?<br />
SQL> alter user logging quota 0 on logging;<br />
User altered.<br />
At this point, the logging table has not used up all of it's first allocated extent, so I'll run a few 'insert into .. select * from ...' to fill the extent up. First though, I'll keep the date and time for now - I'll need it later.<br />
SQL> select sysdate from dual;<br />
<br />
SYSDATE<br />
-------------------<br />
10/09/2002 14:05:04<br />
<br />
<br />
SQL> insert into logging.ddl_log select * from logging.ddl_log;<br />
insert into logging.ddl_log<br />
<br />
( repeat until the following error occurs )<br />
<br />
ERROR at line 1:<br />
ORA-01536: space quota exceeded for tablespace 'LOGGING'<br />
Then a few more DDL changes ...<br />
SQL> alter user logging identified by logging;<br />
User altered.<br />
<br />
SQL> r<br />
1* alter user logging identified by logging<br />
User altered.<br />
<br />
SQL> r<br />
1* alter user logging identified by logging<br />
User altered.<br />
Then find out what rows have been added to the logging table after the date & time when I filled up the table :<br />
SQL> select * from logging.ddl_log<br />
2 where ddl_date > to_date('10/09/2002 14:05:04','DD/MM/YYYY HH24:MI:SS');<br />
no rows selected<br />
So, when the tablespace fills up, no DDL changes are logged to the logging table, but they do get carried out. However, if I change the trigger to remove the exception handling, the following will happen :<br />
SQL> connect sys@cmdb<br />
Enter password: ********<br />
Connected.<br />
<br />
SQL> create or replace trigger DDLTrigger<br />
2 after DDL on database<br />
3 begin<br />
4 insert into logging.ddl_log <br />
5 values (ora_login_user,<br />
6 sysdate,<br />
7 ora_sysevent,<br />
8 ora_dict_obj_type,<br />
9 ora_dict_obj_owner,<br />
10 ora_dict_obj_name);<br />
11 end;<br />
12 /<br />
Trigger created.<br />
<br />
SQL> alter user logging identified by another_password;<br />
alter user logging identified by another_password<br />
*<br />
ERROR at line 1:<br />
ORA-00604: error occurred at recursive SQL level 1<br />
ORA-01536: space quota exceeded for tablespace 'LOGGING'<br />
ORA-06512: at line 2<br />
<br />
SQL> grant create session to logging;<br />
grant create session to logging<br />
*<br />
ERROR at line 1:<br />
ORA-00604: error occurred at recursive SQL level 1<br />
ORA-01536: space quota exceeded for tablespace 'LOGGING'<br />
ORA-06512: at line 2<br />
<br />
SQL> connect logging/another_password@cmdb<br />
ERROR:<br />
ORA-01017: invalid username/password; logon denied<br />
Warning: You are no longer connected to ORACLE.<br />
So, without the exception handling, the users get to see an error message and the DDL is not carried out. Always make sure you have exception handling on your triggers to avoid causing your users problems.<br />
<br />
Further reading:<br />
The article that first pointed me in the direction of DDL triggers can be found here at DBAZine.com.<br />
More information, including a table of what triggers and events are possible, what parameters can be used for each type of trigger, and other useful information can be found in the Oracle 8i Application Developers Guide, chapter 13 - Working with system events. More importantly, it also has information on which DDL commands do not get logged.<br />
<br />
What's my best strategy for laying out an Oracle database across my available I/O devices ?<br />
<br />
Historically there have been may directives, hints and tips about Oracle and disc layout like 'separate indexes from data', 'put redo logs on their own discs'. Some suggestions have varied from pointless to dangerous, some are merely optimistic. (How many smaller systems nowadays actually have the nine discs that once was touted as the minimum requirement).<br />
This note is not a 'quick tip' suggestion - it is a generic guideline. You have to think for yourself about spreading I/O in the most effective way to reduce hotspots on your discs - if that happens to contradict one of the old rules of thumb - so be it, there are always special cases.<br />
So - when trying to work out the best (or perhaps least worst) distribution pattern for your database:<br />
a) Be aware of what your database has to do<br />
b) Understand how indexes work and what they cost<br />
c) Understand how UNDO and redo work<br />
d) Understand the possible data access mechanisms that Oracle users<br />
e) Identify the parts of your database that are necessarily read-intensive <br />
f) Identify the parts of your database that are necessarily write-intensive <br />
g.1) For administrative reasons separate parts of your system with different I/O characteristics into separate modules. (One consequence of this is that in general no tablespace will hold tables and indexes because in most cases table access behaviour can be different from index access behaviour)<br />
g.2) For administrative reasons (sub)section your system into units that can be categorised by size.<br />
g.3) For administrative reasons (sub)section your system into units that can be categorised by backup/restore requirements.<br />
g.4) For administrative reasons consider reducing the number of units generated by g1, g2, g3 above.<br />
h) Allocate estimates of physical I/O costs to each unit specified in g4<br />
i) Spread the units from g4 across the available devices with a view to balancing the I/O evenly whilst avoiding significant contention between units of significantly contradictory characteristic behaviour."<br />
Is it possible to re-set a sequence without dropping and recreating it ?<br />
Many people believe that the only way to reset a sequence is to drop it and recreate it again. This causes a number of problems for any object - functions, procedures etc - which rely of the sequence as it renders them invalid as the following small example shows :<br />
SQL> -- Create a test sequence<br />
SQL> create sequence test_seq;<br />
Sequence created.<br />
<br />
SQL> -- Create a test table<br />
SQL> create table test(a number);<br />
Table created.<br />
<br />
SQL> -- Create a trigger on the table to use the test sequence<br />
SQL> CREATE OR REPLACE TRIGGER test_bi <br />
2 BEFORE INSERT ON TEST<br />
3 FOR EACH ROW<br />
4 BEGIN<br />
5 IF (:NEW.a IS NULL) THEN<br />
6 SELECT test_seq.NEXTVAL INTO :NEW.a FROM dual;<br />
7 END IF;<br />
8 END;<br />
9 / <br />
Trigger created.<br />
<br />
SQL> -- Insert a few rows - firing the trigger and using the sequence<br />
SQL> insert into test (a) values (NULL);<br />
1 row created.<br />
<br />
SQL> r<br />
1* insert into test (a) values (NULL)<br />
1 row created.<br />
<br />
SQL> r<br />
1* insert into test (a) values (NULL)<br />
1 row created.<br />
<br />
SQL> r<br />
1* insert into test (a) values (NULL)<br />
1 row created.<br />
<br />
SQL> -- Check results<br />
SQL> select * from test;<br />
<br />
A<br />
----------<br />
1<br />
2<br />
3<br />
4<br />
<br />
SQL> commit;<br />
Commit complete.<br />
<br />
SQL> -- Check status of trigger. Note that USER_TRIGGERS shows it as ENABLED but <br />
SQL> -- USER_OBJECTS shows VALID. <br />
SQL> select status from user_objects<br />
2 where object_name = 'TEST_BI';<br />
<br />
STATUS<br />
-------<br />
VALID<br />
<br />
SQL> select status from user_triggers<br />
2 where trigger_name = 'TEST_BI';<br />
<br />
STATUS<br />
--------<br />
ENABLED<br />
<br />
SQL> -- Drop the sequence which will invalidate the trigger<br />
SQL> drop sequence test_seq;<br />
Sequence dropped.<br />
<br />
SQL> -- Note that USER_TRIGGERS still says ENABLED but USER_OBJECTS shows INVALID now !<br />
SQL> select status from user_triggers<br />
2 where trigger_name = 'TEST_BI';<br />
<br />
STATUS<br />
--------<br />
ENABLED<br />
<br />
SQL> select status from user_objects<br />
2 where object_name = 'TEST_BI';<br />
<br />
STATUS<br />
-------<br />
INVALID<br />
<br />
So now the trigger is invalid and will remains so until such time as the sequence is recreated and either the trigger manually compiled or an insert into the table is carried out. On the insert, the trigger will be recompiled and will once more become valid. This is useful, but will tend to cause a slowdown in user responses if the trigger has to be compiled on first use.<br />
SQL> create sequence test_seq;<br />
Sequence created.<br />
<br />
SQL> insert into test (a) values (NULL);<br />
1 row created.<br />
<br />
SQL> commit;<br />
Commit complete.<br />
<br />
SQL> select status from user_objects where object_name = 'TEST_BI';<br />
<br />
STATUS<br />
-------<br />
VALID<br />
As this recompiling could affect many other objects, it is advisable to avoid it. There is a way to reset a sequence back to its start value without having to drop it, and most importantly, without having to invalidate any objects.<br />
We start by finding out some information about the sequence by querying USER_SEQUENCES as follows :<br />
SQL> select sequence_name, min_value, max_value, increment_by, cache_size, last_number<br />
2 from user_sequences<br />
3* where sequence_name = 'TEST_SEQ'<br />
<br />
SEQUENCE_NAME MIN_VALUE MAX_VALUE INCREMENT_BY CACHE_SIZE LAST_NUMBER<br />
------------------------------ ---------- ---------- ------------ ---------- -----------<br />
TEST_SEQ 1 1.0000E+27 1 20 21<br />
Make a note of the increment size and min_value for later. Note that this sequence is cached, so the value in LAST_NUMBER may not be the correct value as it is not the last value 'handed out' by the sequence. The only way to get at the correct value is to select the NEXTVAL from the sequence.<br />
SQL> select test_seq.nextval from dual<br />
<br />
NEXTVAL<br />
----------<br />
2<br />
Now we have the last number, we can negate it to give a new increment for the sequence, however, because the MINVALUE is 1 we need to allow it to become zero - or the change to the sequence will fail. Simply alter the sequence and set a new INCREMENT_BY and MINVALUE as follows :<br />
SQL> alter sequence test_seq <br />
2 increment by -2<br />
3 minvalue 0;<br />
Sequence altered.<br />
The sequence is still not reset. In order to do this, we need to select its NEXTVAL again which will increment the current value (2) by -2 giving us zero.<br />
SQL> select test_seq.nextval from dual;<br />
<br />
NEXTVAL<br />
----------<br />
0<br />
Note that we had to allow the MINVALUE to be zero to allow the dummy select we did to reset the current value to zero. Why not simply set the increment to -1 instead ? Well, we could have, but then we have left ourselves with a problem because, we want to reset the sequence back to its original starting value - which is 1 - so that the first time it is used after our adjustments, the value returned is 1. If we simply set the increment to -1 we would indeed get 1 as the next value. Then we would have to monitor the sequence to find out when it was first used, and before it was used again, reset the increment to 1 again so that the sequence is once more counting upwards rather than down. In addition, if we missed the first usage, the user would get an error on subsequent usage because we are trying to reduce the value below that which we set as the minimum.<br />
Now we have successfully reset the sequence back to its original state as the next selection from it will give the value 1 and that was our original intent.<br />
NOTE we have actually changed the sequence as its original MINVALUE was 1 but now it is zero. This might cause a problem on some installations if zero is not allowed. If so, the MINVALUE will need to be adjusted after the first use of the sequence. The chance of a problem is quite minimal unless the sequence is allowed to CYCLE around when it reaches the MAXVALUE.<br />
Once again, checking the status of the trigger shows that it is still valid - it has not been affected by the changes we made to the sequence.<br />
SQL> select status from user_objects where object_name = 'TEST_BI';<br />
<br />
STATUS<br />
-------<br />
VALID<br />
Footnote The original answer to this FAQ was the link below to a document written by Howard J Rogers. Unfortunately, Oracle Australia requested that Howard take down his web site and so the document is no longer available officially. This rewrite of the FAQ answer is not a blatent copy of Howard's document, but is my own work - which addmittedly is influenced by Howard. The link has been maintained in case Howards gets permission to re-open the site.<br />
<br />
The following procedure is also offered by Peter Clark (Peter.Clark@vcint.com)<br />
CREATE or REPLACE PROCEDURE reset_sequence (sequencename IN VARCHAR2) as <br />
curr_val INTEGER;<br />
BEGIN<br />
EXECUTE IMMEDIATE 'alter sequence ' ||sequencename||' MINVALUE 0';<br />
EXECUTE IMMEDIATE 'SELECT ' ||sequencename ||'.nextval FROM dual' INTO curr_val;<br />
EXECUTE IMMEDIATE 'alter sequence ' ||sequencename||' increment by -'||curr_val;<br />
EXECUTE IMMEDIATE 'SELECT ' ||sequencename ||'.nextval FROM dual' INTO curr_val;<br />
EXECUTE IMMEDIATE 'alter sequence ' ||sequencename||' increment by 1'; <br />
END reset_sequence;<br />
How do I retrieve information about partitioned tables from the data dictionary ?<br />
Information about partitioned tables are typically stored in the data dictionary in a number of views with the word 'PART' in them; so a query like the following will list the relevant views (if you select any table privileges):<br />
select object_name<br />
from dba_objects<br />
where object_name like '%PART%'<br />
order by <br />
object_name<br />
;<br />
You will get the usual feature of lists of views with the three family prefixes, (DBA_, ALL_, USER_), which should include the following:<br />
DBA_(SUB_)PART_TABLES Identifies tables that have been (sub) partitioned<br />
DBA_TAB_(SUB_)PARTITIONS Lists (sub)partitions of tables<br />
DBA_(SUB_)PART_INDEXES Identifies indexes that have been (sub) partitioned<br />
DBA_IND_(SUB_)PARTITONS Lists (sub) partitions of indexes<br />
DBA_(SUB_)PART_KEY_COLUMNS List (sub) partitioning columns of objects<br />
DBA_(SUB_)PART_COL_STATISTICS Holds (sub)partition level statistics on columns<br />
DBA_(SUB_)PART_HISTOGRAMS Holds (sub)partition level histograms on columns<br />
DBA_LOB_(SUB_)PARTITIONS Lists (sub)partitions of lOBs<br />
DBA_(SUB_)PART_LOBS Identifies LOBs in (sub)partitioned tables<br />
Why is my import so much slower than my export ? ?<br />
If you switch on sql_trace whilst the import is running, then you will find that an import is typically executing a large number of array inserts to get your data into the database, and then creating indexes and adding constraints. In other words, it is doing a lot of very ordinary updates to the database.<br />
When imp creates indexes, it may do so in nologging mode, but otherwise, all the work you do will be generating undo and redo. This is much heavier work than simply reading the data (by tablescan) and dumping it to a flattish file, which is almost all that exp does, so imp is almost invariably slower than exp. Things may be made worse by the fact that imports tend to be large, which means that when the indexes are being created, they are scanning tables with lots of dirty blocks back from disk, so they generate even more redo doing delayed block cleanout.<br />
Note - there are always special cases - if you have lots of procedures and packages, and very little data in an exported schema, then the work done by Oracle to build the export (i.e. reconstruct your code) may be similar in complexity and volume to the work done by imp in reloading them, so some imports will not be dramatically slower than the export.<br />
Damage limitation methods:<br />
You might consider setting the commit parameter on the import. This causes imp to issue a commit every N rows, and for large imports could reduce the amount of delayed block cleanout that has to take place. This is generally NOT a good idea because it could leave you in an awkward state if something goes wrong - especially if you were using export/import to add data to existing tables.<br />
Set a large buffer size for the import, this increases the array size that Oracle can use for its array insert. It can help.<br />
Make sure that the sort_area_size is set to a large value to reduce disk activity when creating the indexes. In later versions of Oracle you could create a database logon trigger for the importing account that does an execute immediate to set the sort_area_size.<br />
An import is a big job, check the size of your redo log files. It may be that you are doing lots of redo log switches, which results in lots of checkpoints as Oracle tries to flush all the current dirty data to disk. Increasing the redo log file size may help. (Check also the log_checkpoint_interval and log_checkpoint_timeout parameters, as these could have a similar effect).<br />
If you are loading extra data on top of existing data, and therefore updating existing indexes, then the cost of those index updates may be large. You might consider dropping (some of) the indexes before the import. This may be particularly revelant for bitmap indexes which can degrade and grow quite dramatically as new data is added to the tables.<br />
On import, Oracle will always build the indexes needed to enforce enabled primary key and unique key constraints, but you could stop it from creating all the other indexes and create them manually, in parallel, by hand later on. To stop excess indexes being created, you use the indexes parameter to the imp command, and if you want to generate a script that lists the indexes that would have been created (so that you can hack it and run it under your own control) you can use the indexfile parameter.<br />
What is an object of type 'Non-existent' ?<br />
There are two issues which result in non-existent objects.<br />
If you drop an object, then it's row in obj$ (the table holding object definitions) is reclassified as type# = 10 (non-existent). If you create a new object before the database is shutdown, this object number (and row) can be re-used. This is an efficiency thing, and stops the obj$ table and its index degenerating over time in systems where objects are frequently dropped and created.<br />
The other reads is that non-existent objects are created to handle 'negative dependency' tracking.<br />
Say you have a view defined by<br />
create or replace view v1 as <br />
select * from t1;<br />
but T1 is actually a public synonym to a table in someone else's schema. <br />
The validity of this view is dependent on the fact that there is no object named T1 in your schema; so Oracle creates a non-existent object in your schema that the view is dependent on (take a look at the view user_dependencies). Then, when you create an object called T1 in your schema, this causes the elimination of the 'non-existent' object, which automatically causes the invalidation and recompilation of the view so that it now references your object T1, rather than the public synonyn T1.<br />
I am getting a database startup error about being unable to start "with new and and old " values for parameters. (9.0.1)<br />
Errors come with error numbers - if you are on a UNIX machine you can always try the oerr call to get more details about the error message, including (sometimes) a cause and action. For example, of the program returns error ORA-00381, you can try<br />
$ oerr ora 381<br />
00381, 00000, "cannot use both new and old parameters for buffer cache size specification"<br />
// *Cause: User specified one or more of { db_cache_size ,<br />
// db_recycle_cache_size, db_keep_cache_size,<br />
// db_nk_cache_size (where n is one of 2,4,8,16,32) }<br />
// AND one or more of { db_block_buffers , buffer_pool_keep ,<br />
// buffer_pool_recycle }. This is illegal.<br />
// *Action: Use EITHER the old (pre-Oracle_8.2) parameters OR the new<br />
// ones. Don't specify both. If old size parameters are specified in<br />
// the parameter file, you may want to replace them with new parameters<br />
// since the new parameters can be modified dynamically and allow<br />
// you to configure additional caches for additional block sizes.<br />
Note, the necessary file ($ORACLE_HOME/rdbms/mesg/oraus - or local language equivalent) does not exist on NT boxes, but you could try raiding your nearest friendly unix box for the file, and just search it for the error number.<br />
I assume that you have used SQL*Plus to issue a command like:<br />
alter system set db_block_buffers=4000 scope=spfile<br />
In this case, your spfile now has two lines like:<br />
*.db_block_buffers=4000<br />
*.db_cache_size=16777216 <br />
This is what the complaint is about. You need to fix the spfile, expect you can't until you've started the database, and you can't start the database until you've fixed the spfile. Rename the spfile to something else so that it doesn't get accessed by accident from now on.<br />
To fix this problem: if you have a pfile (init{SID}.ora file) that is reasonably up to date, use<br />
startup pfile={fully qualified name of pfile}<br />
create spfile from pfile;<br />
You now have a file spfile{SID}.ora sitting in the default location ($ORACLE_HOME/dbs for Unix), so copy it, and start issuing commands to patch it up for all the paramters that are currently not what you want.<br />
Second option - spfiles are 'binary' files, which in this case means that have some extra junk in them, but you can read most of the text. Copy the spfile, and edit it to extract the text (string -a is a useful command under Unix). Use the resulting file as the pfile in option 1.<br />
I am getting several hundred "buffer busy waits" on data blocks per day, even though I have increased FREELISTS on all objects - what should I do ?<br />
The questions you want to ask yourself are, presumably: which object has the problem, do I need to fix the problem, and how do I fix it. This note addresses the first two parts of that question.<br />
Do I need to fix the problem<br />
Whilst a few hundred waits per day does not look like a good number, you do need to compare it with the actual amount of work done, and check the time lost. In other words, is it likely to make any noticeable if you solve the problem ? There are two cross-checks to make. In v$system_event, is the total time waited on buffer busy waits significant compared to waits like file reads, enqueues, latches ? Then in v$sysstat - is the number of waits you have recorded tiny compared to the total number of consistent gets (say one in one million), is the time lost significant compared to the CPU used when call started.<br />
Of course, it is always a little misleading just to look at system-level figures. This can easily give you a false sense of security by hiding crticial peaks in a swamp of average behaviour. It is just possible that 99% of all your buffer busy waits take place in a critical 30 minutes of the day on one very important process; so you really need to look at regular snapshots (statspack can help) and probe down to the session level (v$sesstat and v$session_event)<br />
Once you have decided that you do need to investigate further, the next step is to identify the problem object. Note - when one session suffers buffer busy waits, it isn't necessarily a problem caused by that session; it may be another session that is doing something completely different; however it is often the case the these waits show up because multiple sessions are doing the same sort of thing at the same time, so you will often find that you can identify the cause by researching the effect.<br />
In 8.1 there is generally no better way than watching a session that you think will suffer a number of these waits. If you check v$session_event joined to v$session, you may notice that a particular program suffers most of the waits. If this is the case, then pick one of those sessions and fire event 10046 at level 8 at it, for example using:<br />
sqlplus "/ as sysdba"<br />
select spid, pid<br />
from v$process<br />
where addr = (<br />
select paddr <br />
from v$session<br />
where sid = 99<br />
);<br />
oradebug setorapid {the pid above}<br />
or<br />
oradebug setospid {the spid above}<br />
then<br />
oradebug event 10046 trace name context forever, level 8<br />
Wait a while, then look at the trace file. You may find some buffer busy waits recorded in the trace file, and be able to use the p1 and p2 values (file number and block number) in a query against dba_extents to identify the object, and the p3 value to get the reason for the wait.<br />
If you do not want to have sql_trace running, then you could simply query v$session_wait repeatedly,<br />
select sid, p1,p2,p3<br />
from v$session_wait<br />
where event = 'buffer busy waits';<br />
Steve Adams (http://www.ixora.com.au) has a more subtle approach to using v$session_wait that increases your chances of spotting a wait when using this view.<br />
<br />
Note - In Oracle 9.2, there is a view called v$segstat that can be cued to record all sorts of activity per segment - so a quick report on this dynamic view would tell you which segments were suffering most buffer busy wait. <br />
I need to delete a lot of small records from LOB-tablespace then load some larger ones. How can I avoid the fragmentation?<br />
LOB-records can be loaded only in flat free part of tablespace the same size or bigger. Small chunks scattered over all tablespace will not used. Also there is no way to defragment current LOB-tablespace online. You can avoid fragmentation by moving all data to another LOB-tablespace by clause 'ALTER TABLE .. MOVE another_blob_tspace' then drop original tablespace or by routine procedures export\truncate tablespace\reimport. <br />
Where has svrmgr gone in 9i - how do I start and stop the database now ?<br />
The desupport and removal of svrmgr .or (svrmgrl) was documented in the manual from at least version 8.1 (if not earlier), when it became possible to start and stop an Oracle instance from Sql*plus. In fact, SQL*Plus acquired many of the features of svrmgr at this time, including oradebug. <br />
You will also discover that the internal account has also disappeared, and you are now required to connect 'as sysdba' or 'as sysoper' to start and stop Oracle.<br />
There are two common strategies for use with SQL*Plus<br />
sqlplus /nolog<br />
connect / as sysdba<br />
startup<br />
or simply<br />
sqlplus "/ as sysdba"<br />
startup<br />
Note that you need to surround the "/ as sysdba" with quote marks so that the entire string will be recognised as a single item. <br />
If you are planning to use exp for transporting tablespaces and don't plan to use parameters files, this quoting mechanism is the method you will have to use to invoke exp (and imp) from the command line - note that from about 8.1.7, tablespaces can only be transported by users with the sysdba privilege, although the first releases of 8.1 allowed the privilege to ordinary DBAs.<br />
How have the log_checkpoint_interval and log_checkpoint_timeout changed from version 7 ?<br />
There was a very important change made to the log_checkpoint_timeout and log_checkpoint_interval parameters in version 8. (Possibly between 8.0 and 8.1, but I think it was probably between 7.3.4 to 8.0) <br />
In earlier versions, after log_checkpoint_interval redo blocks had been written by lgwr (or after log_checkpoint_timeout seconds had elapsed) since the last checkpoint), Oracle issued a checkpoint call and dbwr tried to write all currently dirty buffered blocks to disc. This resulted in an extreme I/O hit from time to time. <br />
To avoid the extremes, Oracle decided to keep trickling dirty blocks to disc at a higher rate than had been effected by the old 3-second idle write rate (every 3 seconds, dbwr wakes up and writes a few blocks to disc if it has had no other work in the interval). To achieve this, they changed the meaning of the two log checkpoint parameters. This change was made possible by a change in the architecture of the buffer management, which now allows Oracle to queue dirty buffers in the order that they were first made dirty. <br />
Amongst other things, Oracle already kept a low redo block address (lrba)on each buffer header for each dirty buffer. This identifies the first redo block that started the process of changing that buffered block from the state that is currently on disc to the dirty state that is in the buffer. The function of the log_checkpoint_interval is simply to limit the distance between a buffer's lrba and the addreess of the redo block that lgwr is currently writing. If the lrba is too low, then the block was first dirtied too long ago and it has to be written to disc (and its lrba set to zero). Since Oracle now queues dirty blocks in the order they were first dirtied (i.e. lrba order) it is a quick and cheap process to find such blocks.<br />
For example: if lgwr is currently writing redo block 12,100 and the log_checkpoint_interval is set to 4,000, then dbwr will be cued to write any dirty blocks with an lrba less than 8,100. This check is carried out every 3 seconds, and I believe the control files and possibly any affected data files are updated with the SCN at which this incremental checkpoint took place.<br />
Note, however, that you are no longer supposed to set these parameters. Over the last few versions, Oracle has introduced different ways of controlling the rate at which dirty blocks are written - including fast_start_io_target (8.1 Enterprise Edition), db_block_max_dirty_target, and in Oracle 9 fast_start_mttr_target.<br />
Why isn't my output from tkprof with the explain option showing row counts ?<br />
The row counts come from trace file lines which are identified as STAT lines. When a cursor is closed, the stat lines for that cursor are dumped to the trace file. consequently, if the trace file ends before the cursor is closed, the stat lines will be missing.<br />
Typically, this occurs because you have issued <br />
alter session set sql_trace true;<br />
..... execution time<br />
alter session set sql_trace false;<br />
(or one of the variants from dbms_system, dbms_support or oradebug). If this is the case, and the SQL in question is inside a pl/sql block, then the pl/sql engine will have cached the cursor, keeping it open even if the program appears to have closed it.<br />
In order to ensure that pl/sql child cursors are closed, you need to exit cleanly from the session that you are tracing.<br />
How does Oracle handle updating a row that causes it to move from one partition to another ?<br />
If you dump the contents of the undo segment or the log file after a simple update that moves a row from one partition to another you will find that Oracle takes three steps to handlethe task..<br />
Update the row in situ - which leaves it in the wrong partition<br />
Insert the corrected row into the correct partition<br />
Delete the row from the old partition<br />
Clearly this is quite heavy on overheads (deletes are always expensive anuway as the whole row has to be written to the undo segment), so the feature should be used with care, and should probably not be used to move rows at a high rate.<br />
Where is the UGA located, and how does sorting affect its use ?<br />
The UGA is the user global area, which holds session-based information.<br />
If you think about Oracle's architecture of sessions and processes, particularly in multi-threaded server (or shared server as it has been renamed for Oracle 9), you will realise that session-based information has to be handled a little carefully.<br />
When you are running dedicated servers (one session = one process) then the session information can be stored inside the process global area - i.e. in the memory space of the dedicated server.<br />
However, when you are running with MTS/shared servers, a single session can migrate from one server (Snnn) process to another. Consequently the session-based information can not be stored in the process (Snnn) memory space, or it would be lost on migration. Consequently session-based information is stored in the SGA (shared global area).<br />
In particular, when running MTS/shared servers and your session does some sorting, some of the memory allocated for sorting - specifically the amount defined by parameter sort_area_retained_size - comes from the SGA and the rest (up to sort_area_size) comes from the PGA (Snnn). This is because the sort_area_retained_size may have to be held open as the pipeline to return results to the front-end, so it has to be located where the session can find it again as the session migrate from server to server.m On the other hand, the sort_area_size is a complete throwaway, and by locating it in the PGA, Oracle can make best use of available memory without soaking the SGA.<br />
To avoid sessions grabbing too much memory in the SGA when running MTS/shared server, you can set the private_sga value in the resource_limit for the user. This ensures that any particularly greedy SQL that (for example) demands multiple allocations of sort_area_retained_size will crash rather than flushing and exhausting the SGA.<br />
I used 'set transaction use rollback segment xxx', but my query still encounters ORA-1555 on rollback segment 'yyy'.<br />
The statement 'set transaction use rollback segment xxx' does two things: <br />
1.) It begins a transaction. <br />
2.) It tells Oracle that any DML for the duration of that transaction should write it's rollback to a rollback segment named 'xxx'. <br />
However, the statement encountering the error is NOT DML. It's a select statement. A select statement will not write any data, so will not consume rollback. A select statement will (attempt to) provide a read consistent view of the data. In order to do so, it may have to refer to rollback. <br />
Depending on what other transactions modified the data you are reading, those changes may have been recorded in ANY rollback segment. If, in the course of constructing a read consistent view of the data, Oracle discovers data whose rollback has been overwritten, it will raise an ORA-1555. This error will report the name of the rollback segment from which the (now overwritten) consistent image cannot be reconstructed. <br />
If you look at it this way, it's easy to see why the name of the rollback segment reported in the ORA-1555 error doesn't match the name of the rollback segment specified in the 'set transaction ...' statement. It is a misconception to think that use of the 'set transaction use rollback segment xxx' statement can help you avoid an ORA-1555 error. It's simply not the case. Please see the items listed in the "Further reading" section below for more information about ORA-1555 and what actually can be done to avoid it. <br />
<br />
Further reading: ORA-01555 Snapshot too old - Detailed Explanation (MetaLink Registration required) <br />
Avoiding ORA-1555 Errors (Link to IxOra, Steve Adams' website) <br />
Is there a way to get the Date/Time when a table was last updated ?<br />
One option is as follows:<br />
(1) Turn the auditing on: AUDIT_TRAIL = true in init.ora <br />
(2) Restart the instance if its running.<br />
(3) Audit the table:<br />
AUDIT INSERT,SELECT,DELETE,UPDATE on TableName <br />
by ACCESS WHENEVER SUCCESSFUL<br />
(4) Get the desired information using :<br />
SELECT OBJ_NAME,ACTION_NAME ,to_char(timestamp,'dd/mm/yyyy , HH:MM:SS') <br />
from sys.dba_audit_object.<br />
What's the difference between DELETE ROWS and TRUNCATE TABLE ?<br />
Delete<br />
At the simplest level, delete scans the table and removes any rows that match the given criteria in the (optional) where clause. It generates rollback information so that the deletions can be undone should it be necessary. Index entries for the deleted rows are removed from the indexes. You must commit to make the deletions permanent.<br />
When deleting rows from a table, extents are not deallocated, so if there were 50 extents in the table before the deletion, there will still be 50 after the deletion. In addition the High Water Mark is not moved down, so it remains where it was before the deletion began. This means that any subsequent full table scans may (still) take a long time to complete - because a full table scan always scans up to the HWM. So, by example, if a select count(*) from very_large_table; took 15 minutes to complete before all the rows were deleted, you will find that it still takes about 15 mins after the deletion - because Oracle is still scanning every single block up to the HWM - even though some (or all) of the blocks may have no data in them. <br />
Truncate<br />
Truncate, on the other hand, simply moves the high water mark on the table right down to the beginning. It does this very quickly, and does not need to be committed. Once you truncate a table, there is no going back. Indexes are also truncated. There is no facility to be able to specify which rows to 'delete' as you can with the where clause on the delete command.<br />
When a table is truncated, all its extents are deallocated leaving only the extents specified when the table was originally created. So if the table was originally created with minextents 3, there will be 3 extents remaining when the tables is truncated.<br />
If you specify the reuse storage clause, then the extents are not deallocated. This saves time in the recursive SQL department if you intend to reload the table with data from an export for example, and can reduce the time it takes to do the import as there is no need to dynamically allocate any new extents.<br />
Every time when I restart the database I have to bring all my rollback segments back online manually. Why ? <br />
In order to have a private rollback segment come on-line at database startup, you must ensure that the rollback segment names are mentioned in the initSID.ora file as follows, assuming you have already created 4 rollback segments named r01, r02, r03 and r04 :<br />
rollback_segments = (r01, r02, r03, r04) <br />
Once the database is open :<br />
SQL> select segment_name,status from dba_rollback_segs;<br />
<br />
SEGMENT_NAME STATUS<br />
------------------------------ ----------------<br />
SYSTEM ONLINE<br />
R01 ONLINE<br />
R02 ONLINE<br />
R03 ONLINE<br />
R04 ONLINE<br />
<br />
SQL> <br />
You can see that the rollback segments mentions in the initSID.ora have indeed been brought on-line.<br />
You must never put the SYSTEM rollback segment name in the list - it always comes on-line, as can be seen from the above output. You must only put the names of already created rollback segments in the list. If you have a name that does not represent a rollback segment, the database will not start correctly - as shown below :<br />
SVRMGR> startup<br />
ORACLE instance started.<br />
Total System Global Area 65322004 bytes<br />
Fixed Size 76820 bytes<br />
Variable Size 47828992 bytes<br />
Database Buffers 16384000 bytes<br />
Redo Buffers 1032192 bytes<br />
Database mounted.<br />
ORA-01534: rollback segment 'OOPS' doesn't exist<br />
SVRMGR><br />
If the rollback_segments parameter is missing from iniSID.ora, then the database will, by default, use any public rollback segments that it knows about by bringing them on-line at startup. The following shows this in action :<br />
SQL> create public rollback segment public_rbs<br />
2 tablespace rbs;<br />
<br />
Rollback segment created.<br />
<br />
SQL> select segment_name,status from dba_rollback_segs;<br />
<br />
SEGMENT_NAME STATUS<br />
------------------------------ ----------------<br />
SYSTEM ONLINE<br />
PUBLIC_RBS OFFLINE<br />
R01 ONLINE<br />
R02 ONLINE<br />
R03 ONLINE<br />
R04 ONLINE<br />
<br />
6 rows selected.<br />
<br />
SQL> <br />
After the instance has been 'bounced' we find the following :<br />
SQL> select segment_name,status from dba_rollback_segs;<br />
<br />
SEGMENT_NAME STATUS<br />
------------------------------ ----------------<br />
SYSTEM ONLINE<br />
PUBLIC_RBS ONLINE<br />
R01 ONLINE<br />
R02 ONLINE<br />
R03 ONLINE<br />
R04 ONLINE<br />
<br />
6 rows selected.<br />
<br />
SQL> <br />
Private and Public Rollback Segments<br />
If you are not running Parallel Server, then a private and public rollback segment are effectively the same - except that a public one comes on-line at startup without any further action from the DBA.<br />
If you are running Parallel Server, then many instances can access a single database. Each instance can have its own private rollback segments - which are used by itself, and can also acquire a public rollback segment from the pool which is shared between all the instances.<br />
<br />
Further reading:<br />
Oracle Reference manual, chapter 1, Initialization Parameters<br />
Oracle Administrators Guide, chapter 2 - Parameters, and chapter 18 Managing Rollback Segments<br />
Oracle Concepts Manual. <br />
Is there a way to get the Date/Time when a row was last updated?<br />
As Oracle is configured out of the box there is a very quick short answer to this question: NO. For Oracle to keep track of this information on a per row basis would require a lot of overhead, which for the great majority of installations would be unnecessary and therefore wasteful.<br />
However, if you really need this information here are some possible ways of trapping this information. <br />
1 – Use triggers to capture this information on a per table basis<br />
2 - Use the Oracle audit command to capture changes<br />
3 - Search the Oracle redo logs for the information using the log miner utility<br />
Pros and Cons of each method<br />
1 – The Pro is this adds overhead only for critical to monitor objects<br />
The Con is this requires coding but it is fairly straightforward<br />
2 – The Pro is this is relatively easy to set up, as it is a built-in feature<br />
The performance overhead is usually not that noticeable<br />
The Con is that by default the audit table is stored in the system tablespace<br />
leading to the possibility of filling the system tablespace as auditing can be<br />
very costly from a storage standpoint<br />
3 – The Pro is there is no pro in my opinion to this approach; however if audit triggers<br />
and/or the Oracle auditing function are not in use then for something that happened <br />
recently that you really need to attempt to find, then this option is available<br />
The Con is this is a resource, time intensive approach <br />
How do you use triggers to capture this information? Here is an example:<br />
The basic procedure is to modify the table adding a last_modified_by and last_modified_on column. Then place before insert and before update triggers on the table. This will allow trapping the Oracle user id and date/time of the row creation and of any updates. Here is a working example trigger. MARKTEST can be any table that has the two auditing columns defined previously defined, just change the table name. Alternately a separate history table could be used to hold the audit data.<br />
set echo on<br />
create or replace trigger marktest_biu<br />
before insert or update<br />
on marktest<br />
for each row<br />
--<br />
declare<br />
--<br />
v_last_modified_by varchar2(30) ;<br />
v_last_modified_on date ;<br />
--<br />
begin<br />
--<br />
select<br />
user<br />
,sysdate<br />
into<br />
v_last_modified_by<br />
,v_last_modified_on<br />
from dual ;<br />
--<br />
:new.last_modified_by := v_last_modified_by ;<br />
:new.last_modified_on := v_last_modified_on ;<br />
--<br />
end ;<br />
/<br />
The sys_context function is a valuable potential source of information for auditing purposes especially if you have applications with imbedded Oracle user id and passwords.<br />
Using the audit command: <br />
Table MARKTEST is created then object level auditing is set using the following command<br />
> audit insert, update, delete on marktest by access;<br />
<br />
Audit succeeded.<br />
I attempted to create the table (again), access is attempted from an ID without proper privilege to the table, and then the table is updated from a user with insert privilege, the DBA_AUDIT_TRAIL is queried, and finally auditing is turned off. There is a great deal more information available than shown below.<br />
> l<br />
1 select username, timestamp, action_name, returncode<br />
2 from dba_audit_trail<br />
3* where obj_name = 'MARKTEST'<br />
> /<br />
<br />
USERNAME TIMESTAMP ACTION_NAME RETURNCODE<br />
------------------------------ --------- --------------------------- ----------<br />
MPOWEL01 01-FEB-02 CREATE TABLE 955 -- table already existed<br />
TESTID 01-FEB-02 INSERT 2004 -- 00942 issued to user<br />
MPOWEL01 01-FEB-02 INSERT 0 -- insert successful<br />
<br />
> noaudit insert, update, delete on marktest;<br />
<br />
Noaudit succeeded.<br />
<br />
Warning the auditing information is kept by default in the system tablespace and by access (row level) auditing can generate huge amounts of data very quickly possibly impacting the ability of Oracle to function. <br />
Data Miner is a topic by itself and I will not attempt to cover it here.<br />
<br />
Further Reading: <br />
See the DBA Administrator’s Guide and SQL manual for information related to auditing and SQL syntax.<br />
If you have Oracle metalink support then you can see How To Set Up Auditing Doc Id: 1020945.6 which will reference several other documents that may be of assistance such as Auditing DML (Insert, Update, Delete) Doc Id: 130146.1<br />
What is table fragmentation, and should I worry about it ?<br />
What is table fragmentation? <br />
Table fragmentation is the situation that develops when a table has been created with an INITIAL and NEXT extent sizes which are too small for the amount of data that is subsequently loaded into the table. In essence, the table ends up with a large number of extents rather than just a few.<br />
Should I worry about it? <br />
Short answer, no.<br />
Quote from Practical Oracle 8i by Jonathan Lewis, Page 150 : Let's start by reviewing one of the oldest myths, and biggest wastes of DBA time, in the Oracle book of legends. Someone once said, "You get better performance if each object is stored in a single extent." It must be well over 10 years ago that Oracle published a white paper exploding this myth, and yet the claim lingers on.<br />
The white paper referred to is (probably) How to stop defragmenting and start living: The definitive word on fragmentation by Himatsingka and Loaiza which describes the SAFE methodology for storage allocation. SAFE is Simple Algorithm For Fragmentation Elimination. In this white paper they explain how testing has shown that under normal circumstances, Oracle can quite happily handle segments which have many thousands of extents.<br />
Of course, when dropping or truncating a segment which has many extents, each extent will have to be de-allocated and the dictionary updated and this will have a degrading effect on performance. <br />
Steve Adams of www.ixora.com.au fame, goes on to clarify and extend the SAFE system a little in his article on managing extents. (See below in further reading). From Steve's article, it appears that allowing the number of extents to exceed 500 (for an 8K block) will cause cluster chaining in the dictionary. This will then have an effect on allocating and deallocating extents.<br />
Summary <br />
· Don't worry about having more than one extent in a table. <br />
· Try to keep the number of extents to a managable limit - I personally follow Steve Adams advice and keep extents around the 500 mark. <br />
· Try to keep segmnents of a similar size and 'hit rate' together in separate tablespaces. <br />
· Don't spend your life exporting and importing table to get them back into a single extent - it really isn't worth it ! <br />
<br />
Further reading:<br />
SAFE - How to stop defragmenting and start living <br />
www.ixora.co.au - Planning Extents<br />
Practical Oracle 8i - Building efficient Databases by Jonathan Lewis. Published by Addison Wesley. ISBN 0-201-71584-8<br />
How do I dump a data block ?<br />
How to determine which block to dump <br />
Assuming you didn't have to dump the block because of an error message which said something like '... corrupt file number #f block number #b' - in which case you have your file and block numbers, you can extract the information from dba_segments.<br />
How to dump the block <br />
To dump a single block : alter system dump datafile <file_id> block <block_number>; <br />
To dump a sequence of blocks : alter system dump datafile <file_id> block min <block_number> block max <block_number>; <br />
The above commands can be replaced with a filename rather than an id : alter system dump datafile 'full_path_to_filename' block <block_number>; <br />
Where does the blockdump go ? <br />
The dump file is created in the user_dump_dest location as specified in initSID.ora..<br />
And finally ... <br />
Quote from MetaLink : The capability to generate a formatted block dump is primarily intended for use by Oracle Support and Development. There is no externally published information available on interpreting the contents of a blockdump and the information is considered 'internal' information. <br />
<br />
Further reading:<br />
Dumping an Oracle 8 block - from this very web site<br />
Dumping an Oracle 7 block - and so is this !<br />
Ixora - search for Blockdump to get a list of references, hints and tips.<br />
What is an object of type 'Undefined' ?<br />
A quick look at the definition for xxx_OBJECTS shows the cause of the anomaly. As new object types are introduced (for example: partitions, nested tables, materialised views etc), the view needs to be updated to reflect the new object type. Occasionally the Oracle developers appear to miss the new object types which thus fall through the DECODE into the "UNDEFINED" tag<br />
The most commonly reported occurrence of this is appears to be materialised views in 8i. <br />
SQL> set long 5000<br />
SQL> select text<br />
2 from dba_views<br />
3 where view_name = 'DBA_OBJECTS'<br />
4 /<br />
<br />
TEXT<br />
--------------------------------------------------------------------------------<br />
select u.name, o.name, o.subname, o.obj#, o.dataobj#,<br />
decode(o.type#, 0, 'NEXT OBJECT', 1, 'INDEX', 2, 'TABLE', 3, 'CLUSTER',<br />
4, 'VIEW', 5, 'SYNONYM', 6, 'SEQUENCE',<br />
7, 'PROCEDURE', 8, 'FUNCTION', 9, 'PACKAGE',<br />
11, 'PACKAGE BODY', 12, 'TRIGGER',<br />
13, 'TYPE', 14, 'TYPE BODY',<br />
19, 'TABLE PARTITION', 20, 'INDEX PARTITION', 21, 'LOB',<br />
22, 'LIBRARY', 23, 'DIRECTORY', 24, 'QUEUE',<br />
28, 'JAVA SOURCE', 29, 'JAVA CLASS', 30, 'JAVA RESOURCE',<br />
32, 'INDEXTYPE', 33, 'OPERATOR',<br />
34, 'TABLE SUBPARTITION', 35, 'INDEX SUBPARTITION',<br />
39, 'LOB PARTITION', 40, 'LOB SUBPARTITION',<br />
42, 'MATERIALIZED VIEW',<br />
43, 'DIMENSION',<br />
44, 'CONTEXT', 47, 'RESOURCE PLAN',<br />
48, 'CONSUMER GROUP',<br />
51, 'SUBSCRIPTION', 52, 'LOCATION', 56, 'JAVA DATA',<br />
57, 'SECURITY PROFILE',<br />
'UNDEFINED'), -- THIS IS THE LINE TO LOOK AT<br />
o.ctime, o.mtime,<br />
to_char(o.stime, 'YYYY-MM-DD:HH24:MI:SS'),<br />
decode(o.status, 0, 'N/A', 1, 'VALID', 'INVALID'),<br />
decode(bitand(o.flags, 2), 0, 'N', 2, 'Y', 'N'),<br />
decode(bitand(o.flags, 4), 0, 'N', 4, 'Y', 'N'),<br />
decode(bitand(o.flags, 16), 0, 'N', 16, 'Y', 'N')<br />
from sys.obj$ o, sys.user$ u<br />
where o.owner# = u.user#<br />
and o.linkname is null<br />
and (o.type# not in (1 /* INDEX - handled below */,<br />
10 /* NON-EXISTENT */)<br />
or<br />
(o.type# = 1 and 1 = (select 1<br />
from sys.ind$ i<br />
where i.obj# = o.obj#<br />
and i.type# in (1, 2, 3, 4, 6, 7, 9))))<br />
and o.name != '_NEXT_OBJECT'<br />
and o.name != '_default_auditing_options_'<br />
union all<br />
select u.name, l.name, NULL, to_number(null), to_number(null),<br />
'DATABASE LINK',<br />
l.ctime, to_date(null), NULL, 'VALID','N','N', 'N'<br />
from sys.link$ l, sys.user$ u<br />
where l.owner# = u.user#<br />
Which of my indexes are redundant and can be dropped ?<br />
<br />
Many a database drag along with unused or lightly used indexes, serving long forgotten purposes. They add an unwished for extra workload to a critical resource. <br />
Two methods came to my mind targeted at nailing down passive, resource consuming indexes. This is not a 100% solution. This is not a point and click - problem fixed solution. Results have to be evaluated and (hard) decisions have to be made. The benefits and (might be) trade-offs have to be evaluated. <br />
This is the first solution, working through the buffer cache i.e. making use of the v$bh view and other sys objects. <br />
Some background: Databases are alive and evolving as long as the data they hold are of value to the company that owns the resource. Databases change over time, as they reflect the business they support. Tables, indexes, procedures, packages, reports, user interfaces and so forth are added, modified over time to reflect the business needs. As time goes by, the complexity and dependency increases. As people also have a tendency to move on in in their life, knowledge about a certain created 'devices' is not available any more. And few of us have the guts or time to do a real cleanup in the database and associated file systems, and drop or delete what we think - is no longer in use. Hence we drag around with indexes (and tables,...) that serve long forgotten purposes. <br />
Back to index of questions <br />
<br />
The idea is to exclude indexes in active use from all the indexes in the system. And then check the leftovers to see if they have any use or value (this is the more manual part). <br />
For Oracle versions prior to 9i, we can only se if an index has been in use in a given time frame. From a given number of indexes in a system, we can measure if a given index has been in use in the time frame we spend to analyze and measure. This then excludes the 'once a year' use of an index for reporting numbers from the general ledger annual report of something. And maybe we were better of, creating the index when called for, and drop it when the task has completed. <br />
First goal is to find indexes that are actually in use, for purposes other than being maintained by dml's. I have chosen two ways to solve this. This first part is through the use of buffer cache (db_block_buffers) and a following part (still in work) is throughout the use of library cache in shared pool (i.e.. v_$sql et.al.) <br />
Buffer cache solution: <br />
If an index is in use for active lookup, at least some blocks must show up in the buffer_cache, and remain there until aged out. If you don't use all tree buffer_caches, and have a newer database version, this is not very complicated to solve. Just takes some time. Create a keep or recycle pool of decent size, and target the suspect indexes for this pool. Ie. alter index X storage(buffer_pool [keep|recycle]). After a while (hours or more), check if the targeted indexes is present in the pool at all or with a large percentage of dirty blocks. If that is the case, evaluate if the index can be dropped (seems likely). <br />
If you use all tree buffer caches, or run an older version, here is a more generic solution. It goes: Select all indexes, total number of blocks in the buffer_caches, number of dirty blocks and time stamp per index and store this as a table. Create a view designed for update the table, and a stored procedure to do it. Run the stored procedure regularly to update the table. A small routine for this is supplied. The procedure updates the result table a time stamp and the increased number of blocks, if the total number or number of dirty blocks of a given index has increased. Run the procedure every say 5-15 minutes for a day more, depending on how fragile your buffer_caches is and on db usage pattern. Then check the table. Don't do any online index rebuilds, or anything that is likely to disturb your metering tool. If this cant be avoided, clear metering values in the table and/or restart this procedure. <br />
In case that the index hasn't had any blocks (recorded) in the buffer_caches, you can most likely drop it. It might be one of these 'once a period' indexes. Take appropriate action if that's the case. Or it might be that the underlying table is never accesses. Export table and all definitions, then rename or drop. Some evaluation is needed here. <br />
If the mayor part of blocks of a given index are dirty, it is likely that is is only present in the buffer cache due to dml's.. Evaluate if the index can be dropped and record the index definition, drop or invalidate the index (if large). <br />
Run the script (might have to change tablespaces). Run the procedure oci_index_cache_block_prc or supplied loop as fitting. When done, evaluate. <br />
If you have lots of schema's and/or a large complex setup, use the commented code to evaluate schemes, one by one. The code here can have negative impact on busy systems. <br />
/**************************************************************************************** <br />
This program is free ware under terms of GNU's General Public License® <br />
and Open Source Foundation® as long as the copyright notice is not removed. <br />
Sysdate: January 2002 <br />
Username: © 2002 Svend Jensen, Svend@OracleCare.Com <br />
Rem <br />
Rem ========================================================================== <br />
Rem DISCLAIMER: <br />
Rem This script is provided for Oracle DBAs "AS IS". It is NOT supported <br />
Rem by author, Oracle World Wide Technical Support nor third parties. <br />
Rem The script has been tested and appears to work as intended. <br />
Rem NO responsibility taken for working nor use, no matter the circumstances. <br />
Rem You should always run new scripts on a test instance initially. <br />
Rem =========================================================================== <br />
Rem # Improvements and extensions are welcome. <br />
******************************************************************************************/ <br />
-- connect sys/<password> [as sysdba] <br />
-- creating base table for index usage investigation<br />
-- can be global temporary if you like. But can give problems with commit.<br />
-- Created: Svend Jensen 2002<br />
-- remove old stuff<br />
drop index oci_index_cache_block_udx<br />
;<br />
drop view oci_index_cache_block_vw<br />
;<br />
drop table oci_index_cache_block_use<br />
;<br />
-- now (re)create table, view, index and procedure<br />
create table oci_index_cache_block_use<br />
tablespace tools<br />
as<br />
select /*+ all_rows */ <br />
object.owner, object.obj# index_obj#, object.index_name, object.table_obj#, object.table_name, <br />
sum(decode(bhead.dirty,'Y',1,0)) dirty#datablock, <br />
count(bhead.obj#) total#datablock, sysdate timestamp <br />
from <br />
(select /*+ all_rows */ <br />
u.name owner, o.obj#, o.name index_name, i.bo# table_obj#, o2.name table_name <br />
-- i.bo# is table obj# for index row in ind$ <br />
from obj$ o, ind$ i, obj$ o2, user$ u<br />
where o.obj# = i.obj#<br />
and o2.obj# = i.bo#<br />
and o.dataobj# = i.dataobj# <br />
and o.owner# = u.user#<br />
-- and o.owner# = (select user# from user$ where name = upper('$user_name')) -- <br />
-- uncommect if only user/schema owner to be checked, and fill in $user_name -- <br />
) <br />
object,<br />
(select objd as obj#, dirty<br />
from v_$bh <br />
where status != 'free'<br />
) <br />
bhead<br />
where object.obj# = bhead.obj#(+) <br />
group by object.owner, object.obj#, object.index_name, object.table_obj#, object.table_name<br />
order by total#datablock desc<br />
;<br />
<br />
-- for update of the base table with new or change entries found in buffer_cache<br />
create or replace view oci_index_cache_block_vw<br />
(owner, index_obj#, index_name, table_obj#,<br />
table_name, dirty#datablock,<br />
total#datablock, timestamp)<br />
as (<br />
select /*+ all_rows */ <br />
object.owner, object.obj# index_obj#, object.index_name, object.table_obj#, object.table_name, <br />
sum(decode(bhead.dirty,'Y',1,0)) dirty#datablock, <br />
count(bhead.obj#) total#datablock, sysdate timestamp <br />
from <br />
(select /*+ all_rows */ <br />
u.name owner, o.obj#, o.name index_name, i.bo# table_obj#, o2.name table_name <br />
-- i.bo# is table obj# for index row in ind$ <br />
from obj$ o, ind$ i, obj$ o2, user$ u<br />
where o.obj# = i.obj#<br />
and o2.obj# = i.bo#<br />
and o.dataobj# = i.dataobj# <br />
and o.owner# = u.user#<br />
-- and o.owner# = (select user# from user$ where name = upper('$user_name')) -- <br />
-- uncommect if only user/schema owner to be checked, and fill in $user_name -- <br />
) <br />
object,<br />
(select objd as obj#, dirty<br />
from v_$bh <br />
where status != 'free'<br />
) <br />
bhead<br />
where object.obj# = bhead.obj# -- (+): select only hits in buffer_cache <br />
group by object.owner, object.obj#, object.index_name, <br />
object.table_obj#, object.table_name<br />
) with check option<br />
;<br />
<br />
-- create index on table oci_index_cache_block_use for updating<br />
create unique index oci_index_cache_block_udx<br />
on oci_index_cache_block_use<br />
(owner, index_obj#, index_name, table_obj#, table_name)<br />
compute statistics <br />
tablespace INDX <br />
;<br />
<br />
<br />
-- create update procedure<br />
create or replace procedure oci_index_cache_block_prc<br />
as<br />
/* variables */<br />
v_owner varchar2(30) ;<br />
v_index_obj# number ;<br />
v_index_name varchar2(30) ;<br />
v_table_obj# number ;<br />
v_table_name varchar2(30) ;<br />
v_dirty#datablock number ;<br />
v_total#datablock number ;<br />
v_timestamp date ;<br />
/* cursors */<br />
cursor cur_oci_index is select * from OCI_INDEX_CACHE_BLOCK_VW ;<br />
BEGIN <br />
open cur_oci_index ;<br />
fetch cur_oci_index into v_owner, v_index_obj#, v_index_name, v_table_obj#,<br />
v_table_name, v_dirty#datablock,<br />
v_total#datablock, v_timestamp ;<br />
while cur_oci_index%FOUND<br />
LOOP<br />
update OCI_INDEX_CACHE_BLOCK_USE <br />
set total#datablock = v_total#datablock, <br />
timestamp = v_timestamp<br />
where owner = v_owner<br />
and index_obj# = v_index_obj#<br />
and index_name = v_index_name <br />
and table_obj# = v_table_obj#<br />
and table_obj# = v_table_obj# <br />
and total#datablock < v_total#datablock <br />
;<br />
<br />
update OCI_INDEX_CACHE_BLOCK_USE <br />
set dirty#datablock = v_dirty#datablock, <br />
timestamp = v_timestamp<br />
where owner = v_owner<br />
and index_obj# = v_index_obj#<br />
and index_name = v_index_name <br />
and table_obj# = v_table_obj#<br />
and table_obj# = v_table_obj# <br />
and dirty#datablock < v_dirty#datablock<br />
; <br />
<br />
fetch cur_oci_index into v_owner, v_index_obj#, v_index_name, v_table_obj#,<br />
v_table_name, v_dirty#datablock,<br />
v_total#datablock, v_timestamp ;<br />
END LOOP ;<br />
commit ;<br />
close cur_oci_index ;<br />
<br />
EXCEPTION<br />
WHEN OTHERS THEN<br />
IF cur_oci_index%ISOPEN THEN<br />
close cur_oci_index ;<br />
END IF ;<br />
dbms_output.put_line ('Others execption in oci_index_cache_block_prc '||SQLERRM) ;<br />
RAISE ;<br />
<br />
END ;<br />
/<br />
/**************** a little update loop ************************** <br />
begin <br />
for j in 1..30 loop <br />
oci_index_cache_block_prc ; <br />
dbms_lock.sleep(300) ; -- sleep 5 minutes <br />
end loop ; <br />
end ; <br />
/ <br />
*********************************************************/ <br />
<br />
End of story - have fun. <br />
<br />
Further Reading: Oracle concepts manual, http://www.oracle.com http://technet.oracle.com , http://metalink.oracle.com <br />
What is the difference between a unique index and a unique constraint?<br />
A constraint is defined by Oracle in the 8.1 Concepts manual, chapter 1, section on Integrity Constraints as being “a declarative way to define a business rule for a column of a table. An integrity constraint is a statement about a table's data that is always true.” Personally, I have always considered constraints to be referential integrity rules that govern the allowable contents of a column and in the case of a primary key (PK) and unique key (UK) constraint, in conjunction with foreign keys (FK), define the formal relationship between columns and rows in one table to another.<br />
The difference between a unique index and a UK or PK constraint starts with the fact that the constraint is a rule while the index is a database object that is used to provide improved performance in the retrieval of rows from a table. It is a physical object that takes space and is created with the DDL command: create index or as part of a create table with PK or UK constraints or an alter table command that adds these constraints (see SQL Manual).<br />
Briefly the constraints are:<br />
Not Null Column value must be present<br />
Unique Key Column(s) value(s) must be unique in table or null (see note below)<br />
Primary Key UK + Not Null which equates to every column in the key must have a value<br />
and this value is unique so the PK uniquely identifies each and every row<br />
in the table<br />
Foreign Key Restricts values in a table column to being a value found in the PK or UK<br />
Constraint on the referenced table (parent/child relationship)<br />
Check Tests the column value against an expression (rule)<br />
Technically it would be possible for a relational database management system, RDBMS, vendor to support PK and UK constraints without using an index at all. In the case of a UK or PK constraint the RDBMS could perform a full table scan to check for the presence of a key value before performing the insert but the performance cost of doing this for anything other than a very small table would be excessive probably rendering the RDBMS useless. So to the best of my knowledge every commercial RDBMS vendor that supports PK and UK constraints does so using indexes.<br />
Prior to Oracle 8 if you defined a PK or a UK constraint the Oracle RDBMS would create a unique index to support enforcement of the constraint. If an index already existed on the constrained columns Oracle would use it rather than define another index on the same columns. Starting with Oracle version 8 Oracle has the ability to enforce PK and UK constraints using non-unique indexes. The use of non-unique indexes supports deferring enforcement of the constraint until transaction commit time if the constraint is defined at creation time as deferrable. Also starting with version 8 Oracle has the ability to place constraints on tables where the existing data does not meet the requirements imposed by the constraint through use of a novalidate option (see SQL Manual).<br />
The practical difference between using a unique index to support data integrity and a UK or PK on the same columns since Oracle will build an index to support the constraint if you do not is that you can define FK constraints when the PK or UK constraint exist. Also in the case of a PK constraint Oracle will convert the columns in the constraint to be not null constrained when it is added to meet the PK requirement to uniquely identify each and every row in the table. There is no such restriction on a unique index. The PK and UK constraints along with FK constraints that reference them also provide a form of documentation on the relationships between objects. Some query tools make use of these relationships to define joins between the tables, example, Oracle Discoverer. In the absence of an entity relationship diagram, ERD, having PK, UK, and FK defined in the database can be beneficial when trying to determine how to find and how to query data.<br />
The Oracle RDBMS Data Dictionary views All/ DBA/ USER_CONSTRAINTS and ALL/ DBA/ USER_CONS_COLUMNS may be used to locate constraints on a table and the columns being constrained.<br />
If you drop or disable a PK or UK constraint that is supported by a unique index the index is dropped with the constraint. If a non-unique index is used to support the constraint the index is not dropped with the constraint. This second condition is effective only with version 8 and higher.<br />
Note – UK constraints allow the constrained column to be NULL. Nulls values are considered to be valid and do not violate the constraint. <br />
<br />
Further Reading: Oracle provides information on constraints in the Concepts manual, the Database Administrators Manual, the Application Developers Guide - Fundamentals, and the SQL manual.<br />
There are also other FAQ articles related to Constraints. <br />
I am unable to add a PK constraint to a table - what is the best way of finding the problem rows ?<br />
How can I check if I have the right indexes for the foreign key constraints on a child table ?<br />
How can I identify which index represents which primary or unique key constraint ? <br />
Is there a performance impact on the database of doing an analyze ?<br />
If you do an analyze .. estimate statistics without a sample size, the impact is usually pretty negligible as Oracle checks only about 1043 rows, so the tests for count distinct, count(*) and so on are quite cheap. However, if you do<br />
analyze table XXX compute statistics<br />
for table<br />
for all indexes<br />
for all indexed columns;<br />
on a very large table, the impact will be severe. To a large degree, the impact comes from the physical I/O that Oracle has to do to acquire its sample set, and the CPU and memory (and as a side-effect I/O) usage involved in sorting. Expect to see db file sequential read, db file direct path read, db file direct path write, buffer free waits, and write complete waits as you do a large analyze.<br />
Of course, there is then a relatively small undo/redo overhead as the data dictionary tables are updated with the new statistics. So even if you are doing a very small estimate, but you have a very large number of objects analyzed in a stream, then this part of the activity could have an impact on the rest of the system. You also have to remember that when the statistics on an object change, any cursors in the library cache that are dependent on that object become invalid so that the optimizer can generate a new execution plan - this could also have a temporary impact on performance as huge amounts of hard-parsing takes place.<br />
Having said that, there are usually not very many objects that need frequent analysis; the ones that need it usually do not need it to be a very high estimate, and then it is likely that only a handful of columns in the database need to have histograms generated..<br />
How do I associate an active session with a rollback segment ?<br />
If you are looking at this FAQ I can think of two different things you are looking for. The first is that you want to assign a session to using a specific rollback segment, which this FAQ will answer. The second is what rollback segment is a session using? This FAQ will also provide the SQL to answer that question.<br />
To assign a session to use a specific rollback segment for a transaction issue the set transaction command:<br />
set transaction use rollback segment roll02;<br />
The set transaction command must be the first statement since the prior commit, rollback, or session creation for it to work; otherwise, you will get an Oracle error: ORA-01453: SET TRANSACTION must be first statement of transaction. To make sure that the command is the first command issued it is common to see the command scripted immediately following a rollback statement.<br />
rollback;<br />
set transaction use rollback segment roll02;<br />
update big_table set fld1 = 'some value';<br />
commit;<br />
The first commit or rollback ends the transaction and the rollback segment assignment along with it. If you need to assign multiple transactions then the set command has to be re-issued after every commit. Inside pl/sql code you can use the Oracle provided package call dbms_transaction.use_rollback_segment('segname') to set the rollback segment for a transaction. <br />
To find the list of rollback segments available to the database query the dictionary view dba_rollback_segs. Additional information about rollback segments is contained in the dynamic performance views v$rollstat and v$rollname. Because v$rollstat does not contain the segment name join v$rollstat to v$rollname on the usn column for this information.<br />
select segment_name, status<br />
from dba_rollback_segs;<br />
<br />
SEGMENT_NAME STATUS<br />
------------------------------ ----------------<br />
SYSTEM ONLINE<br />
ROLL01 ONLINE<br />
ROLL02 ONLINE<br />
ROLL03 ONLINE<br />
ROLL04 ONLINE<br />
The following SQL will show sessions assigned to rollback segments. Note that only transactions are assigned to rollback segments and non-distributed transactions involve a DML operation: insert, update, or delete. So sessions that have issued only normal queries do not show up as being assigned to rollback segments since these sessions will access rollback segments only to read data from them. And if a session needs to read data changed by another session that data can be in any segment. <br />
select s.username, s.sid, rn.name, rs.extents<br />
,rs.status, t.used_ublk, t.used_urec<br />
,do.object_name<br />
from v$transaction t<br />
,v$session s<br />
,v$rollname rn<br />
,v$rollstat rs<br />
,v$locked_object lo<br />
,dba_objects do<br />
where t.addr = s.taddr<br />
and t.xidusn = rn.usn<br />
and rn.usn = rs.usn<br />
and t.xidusn = lo.xidusn(+)<br />
and do.object_id = lo.object_id;<br />
This should answer the question. <br />
Which Version of the database am I running ?<br />
When you connect to sql plus, you can see the version of Oracle Database you are connecting to. This is it.<br />
If you are not connecting through SQL*Plus, then look at v$version:<br />
select * from v$version;<br />
I've got a corrupted data file, and Oracle won't start. I don't need the data so how do I open the database and drop data file? <br />
There is no easy way as to drop a datafile of a tablespace. The only way to remove a datafile from a database is to drop the defining tablespace. There are a few steps to follow: <br />
If you are running in Noarchivelog mode <br />
1. mount the database - startup mount<br />
2. drop the datafile - alter database datafile xxx offline drop<br />
3. open the database - alter database open <br />
4. check all objects belong to that tablespace: <br />
select owner, segment_name, segment_type<br />
from dba_segments<br />
where tablespace_name='tbs_name'<br />
5. export out all the objects in that tablespace<br />
6. drop the tablespace - drop tablespace tbs_name including contents<br />
7. Delete the physical datafiles belonging to the tablespace<br />
8. Recreate the tablespace, import back the objects <br />
<br />
If you are running in Archivelog mode<br />
1. mount the database - startup mount <br />
2. drop the datafile - alter database datafile xxx offline<br />
(Note: the datafile is still part of the database and is marked only as offline in the controlfile. Just make sure you don't use the same data file name again) <br />
3. Remove the physical data file at OS level <br />
4. open the database - alter database open<br />
5. At the right time, you can <br />
export the objects belong to this tablespace, <br />
drop the tablespace, <br />
create back the tablespace with appropriate datafiles and <br />
import the objects back.<br />
<br />
Addendum (28th Jan 2002)<br />
Michael.Trothe@originenergy.com.au offers the following: There is another way. This will only work if there is no data in the file that you really require. <br />
1. mount the database <br />
2. alter the datafile offline drop <br />
3. delete the file from the sys.file$ table. <br />
This will prevent it from being recognised when you do hot backups and not allowing you to place the tablespace in to backup mode.<br />
4. open the database in restricted mode then delete any objects that have references to the missing datafile. <br />
5. shut the database down <br />
6. startup nomount the database <br />
7. recreate the control file and alter database open resetlogs. <br />
All reference to the missing datafile should be gone.<br />
Editor's note: Since this strategy involved direct modification of the data dictionary, your database would no longer be supported by Oracle Corp.<br />
<br />
Further reading: Metalink Note:111316.1 How to 'DROP' a Datafile from a Tablespace<br />
Why is Explain Plan showing a different execution path from tkprof ?<br />
There are a several possibilities, based on the fact that tkprof shows you the path that actually took place at run time and explain plan tells you the path that would be followed now. (In fact if you do tkprof explain=, you can get both, possibly contradictory, paths in the same output file).<br />
a) The user at runtime is not the user explaining, so the query is actually addressing different objects, despite the names being the same. (Or perhaps it is the same user, but a synonym or view has been redefined to point to a different object).<br />
b) The optimizer mode for the run-time session is not the same as the optimizer mode for the explain session (or any one of 50 other parameters that affect the optimizer could have changed) <br />
c) Indexes have been created or dropped between the generation of the trace file and the explain plan <br />
d) Analyze (or dbms_stats) has been run against some of the objects in the SQL between the generation of the trace file and the explain plan <br />
e) The SQL includes bind variables - explain plan will assume these are character variables, at run-time Oracle may have known that there were not, and therefore different coercion effects may have affected the availability of indexes.<br />
I can't think of any more reasons at the moment, but I'm sure that there are probably a couple of other reasons.<br />
I have taken over an existing tablespace and need some ideas for setting extent sizes when re-organizing it ?<br />
How do I rename a data file ?<br />
Datafiles can be moved or renamed using one of two methods: alter database or alter tablespace.<br />
The main difference between them is that alter tablespace only applies to datafiles that do not contain the SYSTEM tablespace, active rollback segments, or temporary segments, but the procedure can be performed while the instance is still running. The alter database method works for any datafile, but the instance must be shut down.<br />
The alter database method:<br />
1. Shut down the instance.<br />
2. Rename and/or move the datafile using operating system commands.<br />
3. Mount the database and use alter database to rename the file within the database. A fully qualified filename is required in the syntax of your operating system. For example to rename a file called 'data01.dbf ' to ' data04.dbf ' and move it to a new location at the same time (at this point in the example the instance has been shutdown) and;<br />
4. Start the instance.<br />
> svrmgrl<br />
SVRMGR> connect sys/oracle as sysdba;<br />
SVRMGR> startup mount U1;<br />
SVRMGR> alter database rename file '/u01/oracle/U1/data01.dbf ' TO '/u02/oracle/U1/data04.dbf ' ;<br />
SVRMGR> alter database open;<br />
Notice the single quotes around the fully qualified filenames and remember, the files must exist at the source and destination paths. The instance is now open using the new location and name for the datafile.<br />
The alter tablespace method:<br />
This method has the advantage that it doesn't require shutting down the instance, but it only works with non-SYSTEM tablespaces. Further, it can't be used for tablespaces that contain active rollback segments or temporary segments.<br />
1. Take the tablespace offline.<br />
2. Rename and/or move the datafile using operating system commands.<br />
3. Use the alter tablespace command to rename the file in the database.<br />
4. Bring the tablespace back online.<br />
SVRMGR> connect sys/oracle as sysdba<br />
SVRMGR> alter tablespace app_data offline;<br />
SVRMGR> alter tablespace app_date rename datafile '/u01/oracle/U1/data01.dbf ' TO '/u02/oracle/U1/data04.dbf ' ;<br />
SVRMGR> alter tablespace app_data online;<br />
The tablespace will be back online using the new name and/or location of the datafile.<br />
Both of these methodologies can be used within Oracle Enterprise Manager also.<br />
<br />
Further reading N/A<br />
This question is also addressed by the following documents:<br />
How big, in bytes, is a ROWID?<br />
There are three formats of ROWIDs.<br />
<br />
1. Short or Restricted <br />
2. Long or Extended <br />
3. Universal ROWID <br />
Short or Restricted ROWID is stored as a six-byte hexadecimal string. DBA (4 bytes) + Row (2 bytes). This format is used when the segment that contains the addressed row can be unambiguously determined, for example in row piece pointers, for normal indexes on nonpartitioned tables and for local indexes for partitioned tables. This is the only format used in Oracle 7. The external representation uses 18 characters:BBBBBBBB.RRRR.FFFF (BBBBBBBB: Block, RRRR: Row, FFFF: File). Internal Code = 69<br />
Long or Extended ROWID is stored as a ten-byte hexadecimal string. Object ID (4 bytes) + DBA (4 bytes) + Row (2 bytes). Long or Extended ROWID contains the object number, in addition to tablespace-relative DBA, and therefore completely identifies the row. It is used in columns of type ROWID and in global indexes for partitioned tables. This form is used in the user columns of ROWID type. This format was introduced in Oracle 8. The external representation uses 18 characters (too long to elaborate). Internal code = 69. <br />
UROWID is stored as a hexadecimal string of up to 3950 bytes. It represents the logical address of a row in an index-organized table and a foreign table. It has three subtypes; physical, logical(primary-key based) and remote(foreign). The internal code is 208. <br />
Note: DBA is Data Block Address <br />
What would be a simple list of events a DBA should be alerted about?<br />
Further reading: http://www.google.com/search?sourceid=navclient&q=oracle+monitoring <br />
What is the largest number of columns allowed in a table?<br />
The maximum columns allowed for Oracle 7 is 255. An increase from 255 to 1000 columns was introduced in Oracle release 8.0<br />
Is there any way to estimate how long a create table/index has left to run ?<br />
For an understanding of this progress meeting, there are a few possibilities to distinguish:<br />
- simple create table as select, without distinct, order by, nor group by<br />
- create table as select, with a sort phase (as mentioned above).<br />
- create index.<br />
For the latter two, there are three phases during the creation:<br />
- full table scan<br />
- sorting the data (needed also for grouping)<br />
- writing the output.<br />
These phases correspond to 'rows' in v$session_longops, as the index creation progresses.<br />
The creation of an index is discussed below. For table creation it is much the same (or even more simple).<br />
Suppose, an index needs to be created on the column 'name', of table 'customer'. Therefore, a sqlplus session is started. First statement is to find out the session identifier (sid) of that statement, with e.g.<br />
select s.sid<br />
, p.pid<br />
, p.spid<br />
from v$process p<br />
, v$session s<br />
where p.addr = s.paddr<br />
and s.audsid = userenv('sessionid')<br />
A second sqlplus session is used for progress monitoring. First, define the sid:<br />
define sid = <<sid above="" from="" select="" the="" value="">><br />
set verify off<br />
After that, start the create index in the first session. In the second session, type:<br />
select sid<br />
, message<br />
from v$session_longops<br />
where sid = &sid<br />
order by start_time;<br />
In the first phase, this query will return output like:<br />
SID MESSAGE<br />
--- -----------------------------------------------------------------------<br />
11 Table Scan: CONVERSIE.RB_RELATIE: 7001 out of 21460 Blocks done<br />
While the table is being read, output will be written to the temporary tablespace (if the sort_area is not sufficient).<br />
When the table scanning phase has been finished, the sort/merge begins. A repeat of the statement shows:<br />
SID MESSAGE<br />
--- -----------------------------------------------------------------<br />
11 Table Scan: CONVERSIE.RB_RELATIE: 21460 out of 21460 Blocks done<br />
11 Sort/Merge: : 2107 out of 3116 Blocks done<br />
During this phase, there is reading and writing to the temporary tablespace.<br />
In the last phase, the index entries have been sorted, and are being written to the index segment. V$session_longops shows:<br />
SID MESSAGE<br />
--- -----------------------------------------------------------------<br />
11 Table Scan: CONVERSIE.RB_RELATIE: 21460 out of 21460 Blocks done<br />
11 Sort/Merge: : 3116 out of 3116 Blocks done<br />
11 Sort Output: : 800 out of 3302 Blocks done<br />
In this last phase, the temporary tablespace is being read from, and writes occur to the tablespace in which the index is created. Note that the index appears as a temporary segment, once the last phase has been started. The segment is 'promoted' to a real index segment, after the physical writing has been done. <br />
Note that entries for a certain phase occur in v$session_longops after about 10 seconds. If a phase take shorter than that, it will not appear in the view.<br />
Note also that v$session_longops might contain data from previous sessions and/or earlier statements. In that case, one can filter on start_time.<br />
With this knowledge in mind, one can estimate how much work has to be done!<br />
<br />
Further reading: See the Oracle database reference guide (version 9.0.1), on OTN:<br />
http://download-eu.oracle.com/otndoc/oracle9i/901_doc/server.901/a90190/ch3161.htm#992382<br />
How much redo is filled in my redo logs?<br />
Finding how much percentage of current redo log is filled is bit tricky since the information is not exposed in any V$ views. We have to directly query the X$tables to get that information. The X$views we use here are x$kccle (Kernel Cache ControlfileComponent Log file Entry) and x$kcccp (Kernel Cache Checkpoint Progress).<br />
<br />
select<br />
le.leseq current_log_sequence#,<br />
100 * cp.cpodr_bno / le.lesiz percentage_full<br />
from<br />
x$kcccp cp,<br />
x$kccle le<br />
where<br />
le.leseq =cp.cpodr_seq<br />
and le.ledup != 0<br />
;<br />
<br />
<br />
<br />
CURRENT_LOG_SEQUENCE# PERCENTAGE_FULL<br />
--------------------- ---------------<br />
6 .428710938<br />
Here x$kcccp.cpodr_bno tells the current log block in the redo log file and x$kccle.lesiz gives the number of (log) blocks in that log file.<br />
How do I find out which users have the rights, or privileges, to access a given object ?<br />
Information on user object and system access privileges is contained in the rdbms data dictionary tables. For this specific question the most likely dictionary table of interest is DBA_TAB_PRIVS:<br />
Name Null? Type<br />
---------------------------- -------- ----------------------------<br />
GRANTEE NOT NULL VARCHAR2(30) <== Receiver of privilege<br />
OWNER NOT NULL VARCHAR2(30)<br />
TABLE_NAME NOT NULL VARCHAR2(30)<br />
GRANTOR NOT NULL VARCHAR2(30) <-- Giver of privilege<br />
PRIVILEGE NOT NULL VARCHAR2(40)<br />
GRANTABLE VARCHAR2(3) <-- Grantee has ability to grant privilege to others <br />
A description of the column values in available in the Oracle 8i Reference manual, but they should all be pretty obvious. Since the DBA_TAB_PRIVS dictionary table (view) contains all grants on all objects in the database this table is suitable for being queried for any Oracle object: tables, views, stored code, etc.... This also means this view is a good source for SQL to generate grant statements for tables, views, stored code, etc.... Example code will follow later.<br />
Before continuing any farther I want to note that privileges are divided into two classes: user access or DML access privileges to Oracle objects (tables, indexes, views...) and system privileges (create session, create table, create user...). In general you should restrict users to possessing only those privileges necessary for them to use their authorized applications and those privileges should be inherited through roles set up to support the application.<br />
Privileges are issued with the GRANT command revoked with the REVOKE command. Examples:<br />
GRANT select, insert, update, delete, references ON my_table TO user_joe ;<br />
REVOKE insert, delete ON my_table FROM user_joe ;<br />
GRANT create public synonym TO user_joe ;<br />
Some other useful security related dictionary views are:<br />
ALL_TAB_PRIVS All object grants where the user or public is grantee<br />
ALL_TAB_PRIVS_MADE All object grants made by user or on user owned objects<br />
ALL_TAB_PRIVS_RECD All object grants to user or public<br />
DBA_SYS_PRIVS System privileges granted to users and roles<br />
DBA_ROLES List of all roles in the database<br />
DBA_ROLE_PRIVS Roles granted to users and to other roles<br />
ROLE_ROLE_PRIVS Roles granted to other roles<br />
ROLE_SYS_PRIVS System privileges granted to roles<br />
ROLE_TAB_PRIVS Table privileges granted to roles<br />
SESSION_PRIVS All privileges currently available to user<br />
SESSION_ROLES All roles currently available to user<br />
USER_SYS_PRIVS System privileges granted to current user<br />
USER_TAB_PRIV Grants on objects where current user is grantee, grantor, or owner<br />
WARNING the three dictionary views that start with ROLE only show privileges on objects the user has privilege on.<br />
UT1> l<br />
1 select grantee,<br />
2 privilege,<br />
3 grantable "Adm",<br />
4 owner,<br />
5 table_name<br />
6 from sys.dba_tab_privs<br />
7 where grantee = upper('&usernm')<br />
8* order by grantee, owner, table_name, privilege<br />
<br />
<br />
GRANTEE PRIVILEGE Adm OWNER TABLE_NAME<br />
------------ ---------- --- ------------ -------------------------<br />
SEFIN DELETE NO SYSTEM SRW_FIELD<br />
INSERT NO SYSTEM SRW_FIELD<br />
SELECT NO SYSTEM SRW_FIELD<br />
UPDATE NO SYSTEM SRW_FIELD<br />
Note that break on grantee is in effect to suppress repeating the user name. <br />
set echo off<br />
rem<br />
rem 19980729 M D Powell New script.<br />
rem<br />
set verify off<br />
set pagesize 0<br />
set feedback off<br />
spool grt_&&owner._&&table_name..sql<br />
<br />
select 'REM grants on &&owner..&&table_name'<br />
from sys.dual ;<br />
<br />
select 'grant '||privilege||' on '||lower(owner)||'.'||<br />
lower(table_name)||' to '||grantee||<br />
decode(grantable,'YES',' with grant option',NULL)||<br />
' ;'<br />
from sys.dba_tab_privs<br />
where owner = upper('&&owner')<br />
and table_name = upper('&&table_name')<br />
order by grantee, privilege ;<br />
<br />
spool off<br />
undefine owner<br />
undefine table_name <br />
Sample output:<br />
grant INDEX on jit.wo_master to EDSJIT ;<br />
grant INSERT on jit.wo_master to EDSJIT with grant option ;<br />
grant REFERENCES on jit.wo_master to EDSJIT ;<br />
grant SELECT on jit.wo_master to EDSJIT with grant option ;<br />
The script above can be particularly useful when you are in a development environment and use export/import as means of making copies of a test bed across machines, this script comes in pretty handy to recreate the privileges bit if you have lost them for whatever reason. It is a nice piece of code to actually reverse engineer scripts from a production database.<br />
<br />
Further reading: For a list of all system privileges see the Oracle verson# SQL manual. For information on managing user privileges see the DBA Administration manual. Starting with version 7.3 see the Oracle ver# Reference Manual for information on the dictionary tables (views) and for more information on using the dictionary see the FAQ for How do I find information about a database object: table, index, constraint, view, etc... in Oracle ? <br />
<br />
How can I tune a tablescan?<br />
This particular example is typical of the generic case. We have a query like the following on a 40GB table, but the index is ‘not selective’:<br />
select<br />
{list of columns}<br />
from<br />
very_large_table t<br />
where<br />
colX = 'ABC' <br />
order by<br />
colY<br />
;<br />
The first problem with this question is the one of ambiguity – when the poster says “cost” do they really mean the figure reported in the COST column of the execution plan table, or do they mean the resource consumption (e.g. number of physical disk reads) when the query actually runs. <br />
In the former case, it is not necessarily safe to assume that there will be a direct correlation between the COST, the resources used, and the run-time. In the latter, there may not be a direct correlation between resource usage (e.g. number of physical disk I/Os) and the run-time when the query is transferred from development to production, because there may be more competition for physical resources on production, so response time may drop.<br />
So how can you tune a full table scan for very large tables? In a very real sense, you can’t. Once your are committed to a full tablescan, that’s it – you have to scan every block (and every row) in the table typically using multiblock reads. The block scans are likely to generate a lot of I/O; the row examinations will consume a lot of CPU. <br />
Of course, there are a few steps you can take to minimize the resource requirement – but their side effects need to be considered carefully. <br />
Can you make the scan times faster? <br />
In principle, yes, a little. Increasing the size of the parameter db_file_mulitblock_read_count so that the read requests are fewer and larger may help. In practice it is not always that simple. Apart from the side-effects on the optimizer’s calculation, you may find that odd interference effects from the various hardware levels actually make it hard to find an optimum setting for the parameter. Things like disk track size, disk buffer size, network buffer size, O/S stripe size and so on can result in some very odd “impedance” effects. You may have to find a best fit for your system by a tedious process of trial and error.<br />
In practice, you may find that the hardware “sees” your big tablescan starting, and adopts a read-ahead strategy that makes your tuning attempt redundant. (This is actually a problem with mixed-load SANs, they tend to get over-enthusiastic about tablescans at the cost of random I/Os – so on average their rate of throughput can look good while the end-users (and v$session_event) are complaining about slow disk response times)<br />
Would parallelism help ?<br />
In principle, yes; but only for the query itself. For a tablescan of degree N the table would effectively be broken up into N parts, allowing N separate processes to scan and extract the required data before forwarding the minimum possible data volume on to another set of processes to handle the sorting. So long as the number of slaves was not sufficient to overload the I/O subsystem, and provided the slaves didn’t end up colliding on the same disks all the time, then the speed of the scan would improve by a factor of roughly N.<br />
On the other hand, by making this query run as rapidly as possible – which means the highest degree of parallelism you can get away with – you are likely to put a lot of stress on the I/O subsystem – to you really want to do that, as it will probably slow everything else down.<br />
Can you do something to cache the table? <br />
At 40 GB, probably not (but who knows, maybe you have 128GB of memory to play with and that might be enough to let you put this table into a keep pool) and you’re still going to be using a lot of CPU looking at all those rows whatever you do about caching.<br />
Is it really true that the index has very poor selectivity?<br />
If so, why are you running a query that (apparently) is going to fetch a huge number of rows? Do you really need to fetch all those rows, or are you really after just the first few? <br />
If the volume is actually relatively low, then perhaps a “bad” index is still better than doing a tablescan – it may protect your I/O subsystem for other users. <br />
If the volume of data is high but you only want the first few rows, how about (in the example above) a “function-based index” of the form decode(colX,’ABC’,colY,null) – so holds entries for just the rows you are interested in, with the index entries in the right order for your order by clause. If you do this, you could re-write your query to force a range scan through this index, stopping after a few rows, rather than acquiring all the rows and sorting them before discarding most of them.<br />
Would partitioning (on the colX column in this case) work for you?<br />
A list partition strategy, or possibly even a hash partition strategy, could break the table into a reasonable number of much smaller chunks. You would still have to do a ‘full table scan’ but it would be a scan of just one partition, which could be much smaller, so use less resources, run more quickly, and cause much less damage. But can you partition on this column – or would it interfere with all the other functions of the system; and if it does could it still be worth the penalty? <br />
So, although you can’t really “tune” a large tablescan, there are some strategies you can investigate to see if you can find ways of reducing or restructuring the amount of work that you need to do. Whether any of them is appropriate for your system only you can choose.<br />
Performance Tuning<br />
Why is dbms_stats so much slower than Analyze?<br />
The person who last posed this question didn’t mention a version number – but the problem appears in many versions of Oracle as you make the change from one technology to the other, mainly because the activity carried out by the dbms_stats does not, by default, match the activity carried out by the analyze command. <br />
Not only does dbms_stats differ from the analyze command in its behaviour, virtually every version of Oracle has introduced a few extra parameters in the calls to dbms_stats and even changed some of the default values from previous versions, so that calls to dbms_stats that used to complete in a timely fashion in one version of Oracle suddenly take much longer after an upgrade – it’s not just the switch from analyze to dbms_stats that causes problems.<br />
In the worst case, when left to its own devices, dbms_stats in 10g will work out for itself the best sample size to use on the tables and indexes, which columns to create histograms for, the number of buckets in the histograms, and the sample size to use for those histograms. The volume of work may be much larger than anything you would choose to do yourself.<br />
The first guideline for using dbms_stats is: read the manual – or the script $ORACLE_HOME/rdbms/admin/dbmsstat.sql to see what it does, and how it changes from release to release. This gives you some chance of making it do what you used to do with the analyze command. The second guideline is to try a test run with lots of tracing (e.g. events 10046, 10033, and calls to v$session_event and v$sesstat) set so that you make sure that you can see how much work the tools do, and where they are losing time.<br />
Partitioned tables have always been a particular problem for statistics – and Oracle is constantly fiddling with the code to try and reduce the impact of collecting reasonable statistics. If you have large partitioned tables, and don’t have to collect global statistics, bear in mind that you probably know more about the data than Oracle does. The best strategy for partitioned tables (probably) is to write your own code that limits Oracle to collecting statistics on the most recently added partition and then derives reasonable table level statistics programmatically by combining the latest statistics with the existing table stats. Look very carefully at the procedure with names like get_column_stats, set_column_stats. <br />
<br />
Further reading: $ORACLE_HOME/rdbms/admin/dbmsstat.sql<br />
Can Bind Variable Peeking cause problems without a histogram on Predicate columns ?<br />
It is a well known fact that the reason for the majority of problems with the bind variable peek feature is an histogram on the column referenced in the access predicate. The histogram is certainly the main but not the only cause for a non appropriate execution plan. Another less known situation where a different value of bind variables can lead to a change of execution plan is an access predicate on a (sub)partitioned table. This is a particularly important scenario in case of range partitioned fact tables organized as rolling windows. These tables contain two types of partitions, those filled up and those pre allocated to future loads. As the optimiser statistics of both types are very different, the risk of getting the wrong execution plan in case of peeking “in the wrong partition” is relatively high.<br />
This is a true story with all kind of exciting attributes: <br />
large partitioned tables with proper collected statistics on them,<br />
nothing was changed but a nightly job performance explodes,<br />
no histogram on the access predicate column,<br />
Oracle 9i / 10g in a data warehouse environment <br />
Why would a reverse index be useful when the leading column of the index is generated from a sequence ?<br />
Background to Indexes.<br />
When you store data in an indexed table, certain columns of data are copied into the index alongside the rowid of the data row in the table. The data in the table is stored 'randomly', or at least, not necessarily in the order you put them there.<br />
The index entries, on the other hand, must be stored in order, otherwise the usability of the index is removed. If you could walk through the entries in an index, you would see that they are in order, usually ascending, but since 8i, descending also works. <br />
Entries are stored in order of their internal representation, not necessarily the same as what you see on screen when you SELECT the columns from a table. <br />
If the indexed column(s) contain character data (CHAR, NCHAR, VARCHAR2 or NVARCHR2) then the data will appear on screen exactly as it does in the index. For example, if the column contains 'ORACLE' the index entry will also be 'ORACLE'. <br />
We can use the DUMP command to show us the internal representation of any data type. This command takes four parameters. The first is the data you wish to dump, the second is the base you wish to dump it in. The default is 10 which means that DUMP will display the characters in decimal, or base 10. The other allowed values are 8 (Octal), 16 (Hexadecimal) or 17 (Characters). <br />
The third parameter is the start position in the data you wish to dump from and the final parameter is the amount of data you wish to dump. All but the first parameter have sensible defaults. <br />
Using DUMP, we can see the individual character codes for our 'ORACLE' data : <br />
SQL> select dump('ORACLE',10) from dual;<br />
<br />
DUMP('ORACLE',10)<br />
-------------------------------<br />
Typ=96 Len=6: 79,82,65,67,76,69<br />
<br />
We can prove that this is correct by converting back from decimal character codes to actual characters : <br />
SQL> select chr(79),chr(82),chr(65),chr(67),chr(76),chr(69) from dual;<br />
<br />
C C C C C C<br />
- - - - - -<br />
O R A C L E<br />
<br />
We could have used base 17 to do the same thing : <br />
SQL> select dump('ORACLE',17) from dual;<br />
<br />
DUMP('ORACLE',17)<br />
-------------------------<br />
Typ=96 Len=6: O,R,A,C,L,E<br />
<br />
Numeric columns are very much different. The internal format of a number is different from that which appears on screen after a SELECT because the internal format is converted to ASCII format so that it can be displayed. We can see this in the following, first in character format : <br />
SQL> select '1234' as "1234",<br />
2 dump('1234', 17)<br />
3 from dual;<br />
<br />
1234 DUMP('1234',17)<br />
---- ---------------------<br />
1234 Typ=96 Len=4: 1,2,3,4<br />
<br />
Then in internal format :<br />
SQL> select 1234 as "a number",<br />
2 dump(1234, 17)<br />
<br />
<br />
3 from dual;<br />
<br />
a number DUMP(1234,17)<br />
---------- --------------------<br />
1234 Typ=2 Len=3: c2,^M,#<br />
<br />
The first columns in both examples look identical, but this is only because SQLPlus has converted the internal format of the number 1,234 into the character format so that the display device (the monitor screen) is able to show it. Binary characters have a nasty tendency to disrupt character devices like computer monitors when running in text mode. <br />
<br />
Take a look at the second column in the above examples and notice the difference. In the first example we see the individual characters '1', '2', '3' and '4' while the second example shows only three bytes in the internal format of the number 1,234. Lets change the DUMP calls slightly, and do the whole lot in one command :<br />
SQL> select '1234' as "1234",<br />
2 dump('1234', 10),<br />
3 1234 as "a number",<br />
4 dump(1234, 10)<br />
5 from dual;<br />
<br />
1234 DUMP('1234',10) a number DUMP(1234,10)<br />
---- ------------------------- ---------- ----------------------<br />
1234 Typ=96 Len=4: 49,50,51,52 1234 Typ=2 Len=3: 194,13,35<br />
<br />
This time, we see the actual character codes used internally. Once again columns 2 and 4 differ. Column 4 is showing three bytes and these three bytes are the internal binary representation of the number 1,234. <br />
It is this binary representation that is used in the index entry when a number column is indexed. <br />
Take a few minutes and experiment with dumping a few other numbers - stick to integers for now as those are what sequences generate. <br />
SQL> create table test (a number);<br />
<br />
Table created.<br />
<br />
SQL> begin<br />
2 for x in 1 .. 1e6<br />
3 loop<br />
4 insert into test values (x, substr(dump(x,10), 14));<br />
5 end loop;<br />
6 end;<br />
7 /<br />
<br />
PL/SQL procedure successfully completed.<br />
<br />
If we have a look at the 'b' column of the table, we can see that each entry is ascending in a similar manner to the 'a' column. Here are the first 20 rows : <br />
SQL> col b format a20 wrap<br />
SQL> select a,b from test where a < 21;<br />
<br />
A B<br />
---------- ----------<br />
1 193,2<br />
2 193,3<br />
3 193,4<br />
4 193,5<br />
5 193,6<br />
6 193,7<br />
7 193,8<br />
8 193,9<br />
9 193,10<br />
10 193,11<br />
11 193,12<br />
12 193,13<br />
13 193,14<br />
14 193,15<br />
15 193,16<br />
16 193,17<br />
17 193,18<br />
18 193,19<br />
19 193,20<br />
20 193,21<br />
<br />
The entries are very similar and all have the same leading byte. <br />
<br />
How sequences affect indexes.<br />
As mentioned above, index entries have to be stored in order, however, the table data need not be. If your indexed column is fed by a sequence, the data will be similar to the 20 rows shown above.<br />
Similar entries will group together in the index, so the index blocks will split as necessary and new entries will end up all hitting the same block until it too fills up and splits. <br />
If you have one person running the application, this isn't too much of a problem. If the application is multi-user then it means that every user will tend to write into the same index block and buffer busy waits will be the outcome as transactions 'queue' to write data to the hottest index block around. <br />
Back in our small test, if you select more data from the test table, you will find that in the 1 million rows, there are only 4 different values for the leading byte on the internal numeric format and even worse, most of the entries in the index have the same leading byte value : <br />
SQL> select substr(b,1,3),count(*)<br />
2 from test<br />
3 group by substr(b,1,3);<br />
<br />
SUB COUNT(*)<br />
--- ----------<br />
193 99<br />
194 9900<br />
195 990000<br />
196 1<br />
<br />
I cheated and discovered that there was a comma in position 4 of every row in the table that's how I knew to use a three character length in my SUBSTR. <br />
What the above shows is that in an index of 1 million sequential entries, the vast majority have the same leading byte and so will all be trying to get into the same block in the index. <br />
<br />
How reverse indexes cure the problem.<br />
A reverse key index stores the bytes of the indexed column(s) in reverse order, so the data 'ORACLE' is actually stored in the index as 'ELCARO'. Using a reverse index on a column fed by a sequence spreads the location of sequential numbers across a wider range of leaf blocks and the problem of a single hot block is removed because the index entries are stored in reverse order.<br />
SQL> alter table test add (c varchar2(30));<br />
<br />
Table altered.<br />
<br />
SQL> update test set c = substr(dump(reverse(a),10),14);<br />
<br />
1000000 rows updated.<br />
<br />
SQL> select substr(c,1,instr(c,',')-1),count(*)<br />
2 from test<br />
3 group by substr(c,1,instr(c,',')-1)<br />
4 order by to_number(substr(c,1,instr(c,',')-1))<br />
<br />
SUB COUNT(*)<br />
--- ----------<br />
2 10102<br />
3 10101<br />
4 10101<br />
5 10101<br />
<br />
All other numbers between 6 and 95 inclusive, have 10,101 entries each.<br />
<br />
96 10101<br />
97 10101<br />
98 10101<br />
99 10101<br />
100 10101<br />
<br />
99 rows selected.<br />
<br />
This time, our 1 million row index entry has it's leading byte value spread across 99 (100 if you include a value for zero) different values, rather than just 4. In addition, the actual reversed bytes are fairly randomly scattered across each of the different values too. <br />
As more entries are added to the index, blocks will be split to accomodate the new entries in their proper location. As the data is arriving almost 'randomly' by means of the reversing of the actual data bytes for the index, the index itself will be extended to accomodate these new values. However, rather than always being stored in the same single 'hot' index block, new entries will be spread across a number of existing blocks (assuming the index has been around for a while) thus reducing contention. Of course, block splits will still occur on these blocks as new values fill up the existing block but it's happening all over the index not just in one place. <br />
This is the reason why reversing the index when its leading column is fed by a sequence reduces buffer contention, removes the hot block problem and by doing so, reduces the potential for buffer busy waits on a multi-user system.<br />
Drawbacks to Reverse Key Indexes <br />
Of course, there are drawbacks as well. By setting up a reverse key index you are increasing the clustering factor of the index. The clustering factor (from USER_INDEXES) is used by the optimiser (CBO) to determine how best to access data in an INDEX RANGE SCAN. If the clustering factor is roughly equal to BLOCKS minus FREE_BLOCKS from USER_TABLES then the chances are that a range scan will read one index block, and locate all (or nearly all) of the data rows in needs in one or more adjacent blocks in the table. <br />
On the other hand, if the clustering factor is close to NUM_ROWS in USER_TABLES then the chances are that the entries stored together in one index block are likely to be scattered throughout a wide range of table blocks - so the index range scan may not be chosen as a good method of access. <br />
Obviously the above applies to an analysed table and index.<br />
<br />
In a quick test on a table with 100,000 rows loaded using a sequence, a normal index was used for most queries returning up to 30 rows, as was the reverse keys index, however, when the number of rows went up to 1,000,000 the reverse key index was never used and a full table scan was used every time. <br />
<br />
Addendum (David Aldridge March 2005)<br />
Although the clustering_factor is usually increased by rebuilding the index as a reverse-key index, there are two balancing points to consider:<br />
i) Reverse-key indexes do not support index range scans, because "... lexically adjacent keys are not stored next to each other in a <br />
reverse-key index..." ( http://download-west.oracle.com/docs/cd/B14117_01/server.101/b10743/schema.htm#sthref988 )<br />
ii) When key values are generated from a sequence it is extremely rare that the rows identified with lexically adjacent keys have any <br />
real-world connection, so the index range scans ought never to be required by the application.<br />
Of course, neither of these statements applies when the index is a multi-column index, and the first column contains repeating values.<br />
<br />
Further reading: Oracle reference manual for your version of Oracle.<br />
<br />
Why does AUTOTRACE not show partition pruning in the explain plan ?<br />
Autotrace not showing partition pruning/elimination is bug 1426992, but, after investigation Oracle has decided that this is not an optimiser bug, but a bug in SQL*Plus. You can, with a bit of knowledge of your data and a little experimentation, deduce that partition pruning is taking place from the output of autotrace, but there are much easier ways !<br />
The following demonstration shows the failings in autotraceand demonstrates a couple of other methods of determining whether or not your partitions are being pruned - or not.<br />
<br />
Autotrace<br />
First of all, create a simple table range partitioned over 6 different partitions, and fill it with some test data extracted from ALL_OBJECTS.<br />
SQL> create table tab_part (part_key number(1), some_text varchar2(500))<br />
<br />
2 partition by range (part_key) (<br />
<br />
3 partition part_1 values less than (2),<br />
<br />
4 partition part_2 values less than (3),<br />
<br />
5 partition part_3 values less than (4),<br />
<br />
6 partition part_4 values less than (5),<br />
<br />
7 partition part_5 values less than (6),<br />
<br />
8 partition part_6 values less than (MAXVALUE) );<br />
Table created.<br />
<br />
<br />
SQL> insert /*+ append */ into tab_part<br />
<br />
2 select mod(rownum, 10), object_name<br />
<br />
3 from all_objects;<br />
24683 rows created.<br />
<br />
<br />
SQL> commit;<br />
Commit complete.<br />
Once the table has been filled, analyse it and see how the data has been spread over the various partitions. The first and last partitions have more data in them that the remaining four, hence the differing totals.<br />
SQL> analyze table tab_part compute statistics;<br />
Table analyzed.<br />
<br />
<br />
SQL> select partition_name, num_rows<br />
<br />
2 from user_tab_partitions<br />
<br />
3 where table_name = 'TAB_PART'<br />
<br />
4 order by partition_name;<br />
PARTITION_NAME NUM_ROWS<br />
<br />
------------------------------ ----------<br />
<br />
PART_1 4937<br />
<br />
PART_2 2469<br />
<br />
PART_3 2469<br />
<br />
PART_4 2468<br />
<br />
PART_5 2468<br />
<br />
PART_6 9872<br />
6 rows selected.<br />
Now that we have a table to work with, we shall see what autotrace has to say about partition elimination. First, however, note how many logical reads a full scan of the entire table needs : <br />
SQL> set autotrace on<br />
SQL> select count(*) from tab_part;<br />
COUNT(*)<br />
<br />
----------<br />
<br />
24683<br />
<br />
<br />
Execution Plan<br />
<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=42 Card=1)<br />
<br />
1 0 SORT (AGGREGATE)<br />
<br />
2 1 PARTITION RANGE (ALL)<br />
<br />
3 2 TABLE ACCESS (FULL) OF 'TAB_PART' (Cost=42 Card=24683)<br />
<br />
<br />
Statistics<br />
<br />
----------------------------------------------------------<br />
<br />
0 recursive calls<br />
<br />
0 db block gets<br />
<br />
135 consistent gets<br />
<br />
0 physical reads<br />
<br />
0 redo size<br />
<br />
381 bytes sent via SQL*Net to client<br />
<br />
499 bytes received via SQL*Net from client<br />
<br />
2 SQL*Net roundtrips to/from client<br />
<br />
0 sorts (memory)<br />
<br />
0 sorts (disk)<br />
<br />
1 rows processed<br />
To read 24,683 rows of data Oracle had to perform 135 logical reads. Keep this in mind and note that the autotrace output shows a full table scan - as we would expect on an unindexed table. The next count should only look in a single partition :<br />
SQL> select count(*) from tab_part where part_key = 7;<br />
COUNT(*)<br />
<br />
----------<br />
<br />
2468<br />
<br />
<br />
Execution Plan<br />
<br />
----------------------------------------------------------<br />
<br />
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=17 Card=1 Bytes=2)<br />
<br />
1 0 SORT (AGGREGATE)<br />
<br />
2 1 TABLE ACCESS (FULL) OF 'TAB_PART' (Cost=17 Card=2468 Bytes=4936)<br />
<br />
Statistics<br />
<br />
----------------------------------------------------------<br />
<br />
0 recursive calls<br />
<br />
0 db block gets<br />
<br />
49 consistent gets<br />
<br />
0 physical reads<br />
<br />
0 redo size<br />
<br />
380 bytes sent via SQL*Net to client<br />
<br />
499 bytes received via SQL*Net from client<br />
<br />
2 SQL*Net roundtrips to/from client<br />
<br />
0 sorts (memory)<br />
<br />
0 sorts (disk)<br />
<br />
1 rows processed<br />
This seems to have again carried out a full table scan, but as we already know that a real FTS takes 135 logical reads, the fact that only 49 were required here should indicate that something is different. Autotrace's output is not showing partition elimination. If you didn't know how many reads were required to full scan the table, you would be hard pressed to determine that partition elimination had taken place in this scan.<br />
Event 10053<br />
There are other methods by which we can obtain a true picture of the plan used by the optimiser - a 10053 trace for example would show the details. I've never had to use a 10053 trace so I'm unfortunately not in a position to explain its use, I leave this as 'an exercise for the reader' as they say :o)<br />
SQL_TRACE and TKPROF<br />
I have used SQL_TRACE and TKPROF though, so here's what shows up when SQL_TRACE is set true.<br />
<br />
<br />
SQL> set autotrace off<br />
SQL> alter session set sql_trace = true;<br />
Session altered.<br />
SQL> alter session set tracefile_identifier = 'PARTITION';<br />
Session altered.<br />
<br />
<br />
SQL> select count(*) from tab_part where part_key = 7;<br />
COUNT(*)<br />
<br />
----------<br />
<br />
2468<br />
<br />
<br />
SQL> alter session set sql_trace = false<br />
Session altered.<br />
At this point, exit from SQL*Plus and locate the trace file in USER_DUMP_DEST which has 'PARTITION' in it's name. This is the one you want to run through TKPROF. The output from this is shown below :<br />
select count(*) from tab_part where part_key = 7<br />
<br />
<br />
call count cpu elapsed disk query current rows<br />
<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
<br />
Parse 1 0.00 0.00 0 0 0 0<br />
<br />
Execute 1 0.01 0.00 0 0 0 0<br />
<br />
Fetch 2 0.01 0.01 0 49 0 1<br />
<br />
------- ------ -------- ---------- ---------- ---------- ---------- ----------<br />
<br />
total 4 0.02 0.01 0 49 0 1<br />
Misses in library cache during parse: 0<br />
<br />
Optimizer goal: CHOOSE<br />
<br />
Parsing user id: 62 <br />
Rows Row Source Operation<br />
<br />
------- ---------------------------------------------------<br />
<br />
1 SORT AGGREGATE (cr=49 r=0 w=0 time=10353 us)<br />
<br />
2468 TABLE ACCESS FULL TAB_PART PARTITION: 6 6 (cr=49 r=0 w=0 time=6146 us)<br />
The explain plan clearly shows that partition 6 was the start and stop partition in the scan. In addition, there were 49 logical reads performed to get at the count. This is identical to what we saw above with autotrace, except we get to see that partition pruning did actually take place.<br />
Explain Plan<br />
Back in SQL*Plus, there is another method that can be used. The old faithful EXPLAIN PLAN will show how partition pruning did take place.<br />
SQL> explain plan<br />
<br />
2 set statement_id = 'Norman'<br />
<br />
3 for<br />
<br />
4 select count(*) from tab_part where part_key = 7;<br />
Explained.<br />
SQL> set lines 132<br />
<br />
SQL> set pages 10000<br />
SQL> col operation format a20<br />
<br />
SQL> col options format a15<br />
<br />
SQL> col object_name format a15<br />
<br />
SQL> col p_start format a15<br />
<br />
SQL> col p_stop format a15<br />
<br />
SQL> col level noprint<br />
SQL> select level,lpad(' ', 2*level-1)||operation as operation,<br />
<br />
2 options,<br />
<br />
3 object_name,<br />
<br />
4 partition_start as p_start,<br />
<br />
5 partition_stop as p_stop,<br />
<br />
6 cardinality<br />
<br />
7 from plan_table<br />
<br />
8 where statement_id = 'Norman'<br />
<br />
9 start with id=0<br />
<br />
10 connect by prior id=parent_id<br />
<br />
11 order by level<br />
OPERATION OPTIONS OBJECT_NAME P_START P_STOP CARDINALITY<br />
<br />
-------------------- --------------- --------------- --------------- --------------- -----------<br />
<br />
SELECT STATEMENT 1<br />
<br />
SORT AGGREGATE 1<br />
<br />
TABLE ACCESS FULL TAB_PART 6 6 2468<br />
Once again, the plan clearly shows that partition pruning takes place. The problem is that autotrace doesn't show it at all. Unless you really know how many blocks of data you have in a table and all of its partitions, you may find it difficult to determine whether or not you are seeing a 'true' plan when using partitioned tables and autotrace.<br />
Note: Do you ever suffer from the PLAN_TABLE growing too big as developers fail to delete old rows from the table? Alternatively, do you forget to delete rows from the table?<br />
Take a copy of $ORACLE_HOME/rdbms/admin/utlxplan.sql and edit it. <br />
Change this :<br />
create table PLAN_TABLE (<br />
<br />
statement_id varchar2(30), ...<br />
<br />
filter_predicates varchar2(4000));<br />
To this :<br />
create global temporary table PLAN_TABLE (<br />
<br />
statement_id varchar2(30), ...<br />
<br />
filter_predicates varchar2(4000))<br />
<br />
on commit preserve rows;<br />
Now login to SQL*Plus as SYS and :<br />
sql> @?/rdbms/admin/utlxplan_edited /* Or whatever your copy is called */<br />
sql> grant all on plan_table to public;<br />
sql> create public synonym PLAN_TABLE for SYS.PLAN_TABLE;<br />
Now when developers or DBAs use PLAN_TABLE and logout their rows will be deleted. A self-tidying PLAN_TABLE. Of course, this is no good if you want to keep rows in PLAN_TABLE between sessions. <br />
DBMS_XPLAN<br />
Under Oracle 9i (release 2 I think) there is a new PL/SQL package which you can use to show explain plans. The above statement could have its plan shown using this command :<br />
SQL> Select * from table(dbms_xplan.display(statement_id=>'Norman'));<br />
or, if this was the only statement in my PLAN_TABLE :<br />
SQL> Select * from table(dbms_xplan.display);<br />
There is much more information shown with this new feature than with a 'normal' explain plan and you don't have to worry about all that formatting either.<br />
Summary<br />
In summary, autotrace doesn't show partition elimination in Oracle up to versions 9i release 2. You should therefore be aware of this fact and use SQL_TRACE or EXPLAIN_PLAN to get at the true plan for the SQL you are trying to tune/debug.<br />
<br />
Further reading: <br />
Note: 166118.1 Partition Pruning/Elimination on Metalink. You will need a support contract to access Metalink.<br />
Bug: 1426992 SQLPlus AUTOTRACE does not show correct explain plan for partition elimination. Again on Metalink.<br />
Will compressing my indexes improve performance ?<br />
Oracle introduced a compression option for indexes in Oracle 8.1. You can create an index as compressed, or rebuild it to compress it (although there are some restrictions about online rebuilds, rebuilds of partitioned indexes etc.) Typical syntax might be:<br />
create index t1_ci_1 on t1(col1, col2, col3, col4) compress 2;<br />
alter index t1_ci_1 rebuild compress 2;<br />
The benefits of compression come from the fact that a properly compressed index uses a smaller number of leaf blocks - which tends to mean that less I/O is involved when the index is used, there is a reduced amount of buffer cache flushing, and the optimizer is likely to calculate a lower cost for using that index for range scans. (There is a tiny chance that the number of branch blocks, and the index height might be reduced, too, but that is a little unlikely).<br />
But compressing indexes, especially compressing the wrong number of columns, can have negative impact on your performance. If you compress more columns than you should, the 'compressed' index may be larger than the uncompressed index. Use the validate option on the index, and check view index_stats to find out the optimum compression count. How did I know that I should compress just the first two columns of the t1_ci_1 index ? (Apart from knowing the data, that is):<br />
validate index t1_ci_1;<br />
<br />
select<br />
opt_cmpt_count, opt_cmpr_pctsave<br />
from <br />
index_stats;<br />
opt_cmpt_count opt_cmpr_pctsave<br />
-------------------------------<br />
2 50<br />
Unfortunately these two columns don't exist in 8.1, only in version 9 (possibly only 9.2). Fortunately Steve Adams has a script on his website to recommend a compression level (see www.ixora.com.au )<br />
Even if you get the 'right' number of columns compressed, there is a price to pay: The main penalties are: (a) reads and mods of a compressed index cost more CPU than they would (typically) for an equivalent uncompressed index (b) execution paths change - and you may not have predicted the changes, and some nominally cheaper paths may actually be slower. for example: Oracle may choose an index fast full scan instead of an index range scan because the compressed index is now much smaller, and your setting for parameter db_file_multiblock_read_count is large; or Oracle may choose to use an index and do a nested loop because the index is now 30% smaller, where previously it was doing a table scan and hash join.<br />
So - don't go and compress all the indexes in your schema. <br />
Think carefully about which indexes could give you significant gains, and whether you can afford some CPU loss to reduce buffer thrashing and I/O. <br />
Remember too, if the way you use an index is such that the column order doesn't matter, then perhaps you could rearrange the column order to maximise the compression. The most critical point, perhaps, is that you should avoid moving a column that is typically used with a range scan towards the front of the index.t<br />
How are the Cache Buffers Chains and Cache Buffers LRU Chains used in the Buffer Cache management ?<br />
The Cache Buffers Chains are very short chains (linked lists) that allow Oracle to locate a block very quickly if it is in the buffer. Each block hashes (by block address, tablespace number, and block type - I believe) to one of the chains. In Oracle 8 there are roughly twice as many chains available as there are buffers, so many chains are empty, and the remainder tend to have only one or two blocks in them - so scanning a chain can be very quick.<br />
Chains are covered by Cache Buffers Chains latches. Each Cache Buffers Chains latch covers around 64 - 128 chains - the commonest sizes of db_block_buffers (or db_cache_size as it should be in oracle 9) mean that are typically 512 or 1024 chains, but this varies in powers of 2 as the size of the buffer cache grows.<br />
The Cache Buffers LRU chains tell Oracle about how much use a buffer has had, and therefore allow it to decide quickly and cheaply which buffer to clear when someone wants to read a new block from disc. In fact the term LRU chain is somewhat obsolete, as Oracle 8.1 uses a touch count algorithm to decide on the popularity of a buffered block, nevertheless many details of the LRU algorithm still apply, and the chain of buffers still has blocks being 'pushed down' to be dropped off at the end if they have not been touched in the recent past.<br />
Why does a global index on my partitioned table work faster than a local index ?<br />
Partitioning is often touted as the cure-all for performance problems. "Partitioned your tables into 1,000 sections, and your queries run hundreds of times faster" is a comment I heard from one well-known "authority" in the early days of Oracle 8 - and some people still think it is true. So why, when you have partitioned your table, and start to index them, do you find that global indexes are quicker than local indexes.<br />
As usual, the quick (consultant's) answer is that the benefits depend on your system. How big is your data, what are your clustering factors like, are you using the features that supply the benefits, and in this case, how much work do you typical queries do anyway ?<br />
The global/local indexes problem is generic - global indexes CAN be faster than local indexes. In general, local indexes will be faster when partition elimination can take place, and the expected volume of data acquired id significant. If either of this conditions is not met, then local indexes probably won't help performance.<br />
Remember, the optimizer has to do some extra work to deal with partitioned tables. If the queries you use are very precise queries (e.g. on a nearly unique pair fo columns) then the cost of optimisation and identifying the partition may easily exceed the saving of having the partitions in the first place (The saving might be just one logical I/O, hence < 1/10,000 of a CPU second) .Where queries are very precise, a global index is quite likely to be a little more performant than a local index.<br />
This is a common Oracle trade-off between how much you win/lose and how often you make that win/loss. In the local / global indexes case you (should expect to) lose a tiny amount of performance on every query in order to win a huge amount when you do partition maintenance such as eliminating entire partitions of very old data with a quick drop partition command.<br />
Of course, if your partitions are large, and the indexes hit a large number of table rows, and the queries are over-generous in their throwaway rates; then the work saved by hitting just the one partition through exactly the correct index partition may prove to be a significant gain.<br />
Which Oracle features require and/or force the use of Cost Based Optimisation ?<br />
There is a list of Oracle features in the Performance Tuning Guide that are quoted as 'forcing Cost Based Optimisation' to happen. Unfortunately, this list is a little misleading. Some of the features (such as parallel tables) force CBO to kick in, others simply will not work unless CBO has been invoked. This note splits the list into the two relevant groups.<br />
The presence of the following features appears to force CBO to kick in.<br />
Index Organized Tables (IOTs)<br />
Partitioned Tables<br />
Parallel Tables<br />
The SAMPLE clause<br />
ANSI Outer joins<br />
The Rule Based Optimizer will not notice the presence of the following features - you must invoke CBO explicitly<br />
Reverse key indexes<br />
Function based indexes<br />
Bitmap indexes<br />
Bitmap Join indexes<br />
The Rule based optimiser will use the following, without invoking CBO<br />
Inline (from clause) views<br />
Partition views<br />
ANSI natural and cross joins<br />
Other comments<br />
Other functionality such as star joins, bitmap star transformations, the progress meter (v$session_longops) and hash joins are also dependent on the cost based optimiser being invoked. However I have not included them in the lists above, as they are 'action-oriented' options rather than having what might be called a sort of 'physical presence'.<br />
What is the difference between a soft parse and a hard parse ?<br />
When you submit an SQL statement for processing, it passes through a number of stages before your server process finally gets to execute it and give you back the results.<br />
First your command is checked for syntax errors - basically, is the command correctly formed ?<br />
If the syntax check is passed, it is checked for semantic errors - things like do all the objects required exist, does the user have appropriate privileges etc.<br />
The next stage is to pass the command thorough a hashing function to obtain a hash value for the statement. This hash value is used as a lookup in the library cache to see if this command has been used before. If the hash value is found in the cache, then the SQL for that command is compared with yours to see if they are identical (down to letter case etc). If so, the next step is to ensure that all objects referenced in the cached command are the same objects referenced in your new one. If so, then the parse tree and execution plan for the existing command can be used for your as well and the optimiser stage is missed out This is a soft parse.<br />
Assuming that there is not an identical SQL command in the library cache, then your command will be passed on to the Optimiser to work out the best plan of attack to get back the data you want. The optimiser builds a parse tree which will involve processing some recursive SQL. Once a parse tree has been build, an execution plan is created from it. This constitutes a hard parse.<br />
Once an execution plan has been created, or an existing one re-used, the command can be executed.<br />
In summary,<br />
1. Perform syntax check <br />
2. Perform semantic check <br />
3. Perform hash function <br />
4. Perform library cache lookup <br />
5. If hash value found then <br />
6. .....If command is identical to existing one in cache then <br />
7. ..........If the objects referenced in the cached command are the same as the ones in the new command then <br />
8. ...............This is a soft parse, go to step 11 <br />
9. This is a hard parse, build parse tree <br />
10. Build execution plan <br />
11. Execute plan. <br />
The building of the parse tree and execution plan are the two most expensive parts of the parsing, and if these have to be done, then we have a hard parse. <br />
Note : In the Oracle Performance Tuning 101 book mentioned below, the order given is that the hashing is done first and if the hash value is not found in the library cache, then the syntax and semantic checks are carried out. Tom Kyte says in his book, and on his web site, that this is not the case. The syntax and semantic checks are always carried out and then the hashing. As Tom works for Oracle, and because he gives good reasons, I'm following Tom's advice on the matter. The Oracle Concepts manual has the same description as Tom.<br />
I'm not sure how I could actually test it to find out.<br />
<br />
Further reading:<br />
Oracle training manuals from SQL and PL/SQL course.<br />
Oracle server concepts manual volume 2<br />
Beginning Oracle Programming by Sean Dillon, Tom Kyte etc al.<br />
Oracle Performance Tuning 101 by Vaidyanatha et al<br />
How do I find out which tables are accessed with a full table scan, and the number of times this occurred ?<br />
We have found a doc on metalink which shows a query on the x$bh table/view to determine which tables are being accessed by full table scans. We are interested in a count of number of times each table is accessed by a full table scan so we can find the most heavily accessed table by full table scan and possibly tune to reduce the number of full table scanning. The reason we know this can somehow be done is because we have received a report from a vendor like the following:<br />
There have been 15,402 full table scans since the start of the database. <br />
Listed below are the tables that have had over 10 full table scans.<br />
Table Owner Table Name Number of Full Table Scans<br />
ABC TABLEONE 335<br />
DEF TABLETWO 5551 <br />
Because it mentions how many occurred since startup of the database, it implies that there is/are v$ tablesor x$bh tables that were queried to get this type of information. I have checked metalink documents without success.<br />
There is no such view or X$ object as far as I know.<br />
I assume the script that you mention regarding using X$BH uses the flag value to check if the buffer header was used for scan read, with a clause like:<br />
select distinct obj <br />
from x$bh<br />
where bitand(flag,power(2,19)) = power(2,19):<br />
Perhaps joining to obj$ on the dataobj# column.<br />
This identifies objects that have been scanned (which may include index fast full scans), but doesn't tell you how often.<br />
To get an idea of how often an object has been scanned, you could dump v$sql or v$sqlarea from time to time and use explain plan on the queries involving the guilty objects to identify the SQL that includes a full scan on the object. If you find plans with full scans, you can use the executions column to tell you how many times the query was executed. This still doesn't tell you how often the scan occured, of course, as the scan may have taken place many times in one exection of the query.<br />
In Oracle 9.0 onwards, you have the extra view v$sql_plan so you don't need to dump the sql and explain it, and in 9.2 you have a view called v$sql_plan_statistics, which includes information about the number of times each line of each plan was executed, and this can allow you to get a much better estimate of the actual number of scans.<br />
Of course, under all Oracle versions, using v$sql as the source of such information is only an approximation - some executions may have been aged out of the SGA before you get to see them, and sometimes invalidations and reloads can confuse the statistics.<br />
<br />
Is is possible to flush the db_block_buffer for testing purposes<br />
There is no matching command, however there is a little trick which may be adequate.<br />
alter tablespace XXX offline;<br />
alter tablespace XXX online;<br />
When you alter the tablespace offline, any blocks which are in the buffer are invalidated, and therefore subject to rapid elimination as the buffer is re-used. In fact, even when you bring the tablespace back online, even if some blocks are still apparently buffered, they cannot be reused.<br />
Bear in mind, though, that if you are running Oracle on a file-system, and not using direct I/O as the O/S level, then the blocks may still be buffered in the file-system buffer; so your tests may still suffer from some spurious buffering benefit; especially if your code is accessing some smallish tables through tablescans. Remember that a small table is one that is no more that 2% of the size of the db_block_buffer, it is NOT, as is commonly believed, one that is only four blocks or less..<br />
If I store text in a column, can I index individual words ?.<br />
This is possible if you use the Oracle interMedia Text option. It allows you to create "text indexes" on your text columns which can then be queried using the contains() function in the where clause.<br />
How to speed up reporting of large execution plans based on PLAN_TABLE?<br />
Oracle supplies script to define PLAN_TABLE used by EXPLAIN PLAN command. The script usually resides in<br />
$ORACLE_HOME/rdbms/admin/utlxplan.sql file in Unix or<br />
#:\ORANT\RDBMSnn\ADMIN|utlxplan.sql file under MS Windows (# - is a letter specifying an assigned drive, nn specifies the Oracle version e.g. 81).<br />
<br />
The data is inserted into PLAN_TABLE by issuing <br />
EXPLAIN PLAN SET STATEMENT_ID = 'User_defined_id_string' FOR<br />
User written SQL statement; <br />
<br />
In order to retrieve the data another standard statement (or some derivation of it) is used:<br />
SELECT LPAD (' ', 2 * (level - 1)) || operation operation, options,<br />
object_owner || '.' || object_name object, DECODE (id, 0,'Cost = ' || position) pos<br />
FROM PLAN_TABLE<br />
START WITH id = 0 AND statement_id = 'User_defined_id_string'<br />
CONNECT BY PRIOR id = parent_id AND statement_id = 'User_defined_id_string';<br />
However quite often the execution of such statement requires large amount of time and computer resources, especially when explained statement is a complicated one (generates many rows in PLAN_TABLE) or when all application developers use single PLAN_TABLE (and nobody cares enough to delete an old data from it). The reason for such a behavior is simple enough, when we look at EXPLAIN PLAN of our SELECT statement:<br />
SELECT STATEMENT OPTIMIZER=CHOOSE<br />
CONNECT BY<br />
TABLE ACCESS (FULL) OF PLAN_TABLE<br />
TABLE ACCESS (BY USER ROWID) OF PLAN_TABLE<br />
TABLE ACCESS (FULL) OF PLAN_TABLE<br />
As we see the explain shows that:<br />
1. Full table scan is performed (to find a row with id = 0 and given statement_id)<br />
2. For each child row another full table scan is executed (to get a row with the same statement_id and with a parent_id of current row).<br />
In order to eliminate FTS and to enable the best possible access for each retrieved row I suggest defining 2 indexes:<br />
CREATE INDEX PLAN_TABLE$STMTID_ID ON PLAN_TABLE (STATEMENT_ID, ID);<br />
<br />
CREATE INDEX PLAN_TABLE$STMTID_PID ON PLAN_TABLE (STATEMENT_ID, PARENT_ID);<br />
(Of course you have to use TABLESPACE and STORAGE parameters as well)<br />
<br />
Now the explain plan looks much better:<br />
SELECT STATEMENT OPTIMIZER=CHOOSE<br />
CONNECT BY<br />
INDEX (RANGE SCAN) OF PLAN_TABLE$STMTID_ID<br />
TABLE ACCESS (BY USER ROWID) OF PLAN_TABLE<br />
TABLE ACCESS (BY INDEX ROWID) OF PLAN_TABLE<br />
INDEX (RANGE SCAN) OF PLAN_TABLE$STMTID_PID<br />
Selecting execution plan of 12 rows out of PLAN_TABLE with 7000 rows took (Oracle 8.1.6.0/Windows NT/P-III-350):<br />
Without indexes: 0.501 sec<br />
With both indexes defined: 0.010 sec - 50-fold improvement.<br />
It's possible to use 1 index only instead of 2 (I advise to define it on STATEMENT_ID and PARENT_ID columns) to get significant performance improvements compared with situation where no index exists.<br />
<br />
The same solution of defining an indexes on columns used in CONNECT BY PRIOR clause may be used to improve the performance of all hierarchical queries. <br />
Tuning UPDATE/DELETE statements with subqueries<br />
The tuning of UPDATE or DELETE statement referencing a single table is no different from SELECT, however some performance problem arises when a number of tables are involved. In major number of cases there are 2 tables when one of them must be updated based on some criteria from another.<br />
We are going to analyze 2 cases (both with RULE and COST based optimizers) of UPDATE statement execution but the same reasoning and methods apply to DELETE statement as well:<br />
Case 1. EMP and DEPT tables are involved. We would like to update salaries of all employees (lets give them 5% raise) working for ‘SALES’ department. There is always a possibility of using 2 statements:<br />
a. SELECT from DEPT to get a value of DEPTNO column.<br />
b. UPDATE EMP SET SAL = SAL * 1.05 WHERE DEPTNO = :DNO<br />
However lets see what happens when we try to do it by using single UPDATE statement (and Rule Based Optimizer - RBO): <br />
UPDATE EMP SET SAL = SAL * 1.05 <br />
WHERE DEPTNO IN (SELECT DEPTNO FROM DEPTWHERE DNAME = ‘SALES’)<br />
<br />
Call Count CPU Elapsed Disk Query Current Rows <br />
Parse 1 0.04 0.04 0 0 0 0 <br />
Execute 1 1.58 1.74 1 40135 2058 2006 <br />
Fetch 0 0.00 0.00 0 0 0 0 <br />
Total 2 1.62 1.78 1 40135 2058 2006 <br />
<br />
Rows Execution Plan <br />
0 UPDATE STATEMENT GOAL: CHOOSE <br />
1 UPDATE OF 'EMP' <br />
2007 NESTED LOOPS <br />
20015 TABLE ACCESS (FULL) OF 'EMP' ç a <br />
22020 TABLE ACCESS (BY INDEX ROWID) OF 'DEPT' ç c <br />
40028 INDEX (UNIQUE SCAN) OF 'PK_DEPT' (UNIQUE) ç b <br />
As we can see the optimizer performs:<br />
a. FULL TABLE SCAN of EMP table (about 20000 rows).<br />
For EACH row it :<br />
b. Accesses an INDEX of DEPT table by DEPTNO value to get a ROWID<br />
c. Accesses a DATA part of DEPT table by using ROWID retrieved from an<br />
index to get value of DNAME column.<br />
Only now it performs a final elimination of unnecessary rows by filter on<br />
DNAME column (DNAME = ‘SALES’)<br />
It’s quite obvious that it’s may be more efficient to access DEPT table first and EMP table at later stage.<br />
In order to force the optimizer to access the tables in specified order we have to supply USE_NL hint. It tells optimizer to use EMP table as an inner table and, by-product, DEPT table will be used as outer (driving) table. <br />
<br />
UPDATE /*+ USE_NL(EMP) */ EMP SET SAL = SAL * 1.05<br />
WHERE DEPTNO IN (SELECT DEPTNO FROM DEPT WHERE DNAME = ‘SALES’)<br />
<br />
Call Count CPU Elapsed Disk Query Current Rows <br />
Parse 1 0.01 0.01 0 0 0 0 <br />
Execute 1 0.91 1.68 5 7 2056 2006 <br />
Fetch 0 0.00 0.00 0 0 0 0 <br />
Total 2 0.92 1.69 5 7 2056 2006 <br />
<br />
Rows Execution Plan <br />
0 UPDATE STATEMENT GOAL: CHOOSE <br />
1 UPDATE OF 'EMP' <br />
2007 NESTED LOOPS <br />
2 TABLE ACCESS (FULL) OF 'DEPT' <br />
2007 INDEX (RANGE SCAN) OF 'EMP$DEPT' (NON-UNIQUE) <br />
Now we can compare the results:<br />
<br />
Without hint With hint Improvement(%) <br />
CPU 1.62 0.92 45 <br />
Elapsed 1.78 1.69 5 <br />
Access count 42193 2063 95 <br />
As expected there is a major performance gain.<br />
Now let’s see what happens with the cost based optimizer (CBO). <br />
UPDATE EMP SET SAL = SAL * 1.05 <br />
WHERE DEPTNO IN (SELECT DEPTNO FROM DEPT WHERE DNAME = ‘SALES’)<br />
<br />
Call Count CPU Elapsed Disk Query Current Rows <br />
Parse 1 0.01 0.01 0 0 0 0 <br />
Execute 1 0.97 1.71 0 108 2062 2006 <br />
Fetch 0 0.00 0.00 0 0 0 0 <br />
Total 2 0.98 1.72 0 108 2062 2006 <br />
<br />
Rows Execution Plan <br />
0 UPDATE STATEMENT GOAL: CHOOSE <br />
1 UPDATE OF 'EMP' <br />
2007 HASH JOIN <br />
1 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'DEPT' <br />
20014 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'EMP' <br />
As we can see the CBO performs much better then RBO (it uses HASH JOIN instead of NESTED LOOP JOIN of RBO). However it’s interesting to see what will happen when we access EMP table by index instead of performing FTS on it. In order to do it we add a hint to our statement once again and another hint to enforce index access: <br />
UPDATE /*+ USE_NL(EMP) INDEX(EMP) */ EMP<br />
SET SAL = SAL * 1.05<br />
WHERE DEPTNO IN (SELECT DEPTNO FROM DEPT WHERE DNAME = ‘SALES’)<br />
<br />
Call Count CPU Elapsed Disk Query Current Rows <br />
Parse 1 0.04 0.04 0 0 0 0 <br />
Execute 1 0.88 1.02 0 7 2056 2006 <br />
Fetch 0 0.00 0.00 0 0 0 0 <br />
Total 2 0.92 1.06 0 7 2056 2006 <br />
<br />
Rows Execution Plan <br />
0 UPDATE STATEMENT GOAL: CHOOSE <br />
1 UPDATE OF 'EMP' <br />
2007 NESTED LOOP <br />
2 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'DEPT' <br />
2007 INDEX ACCESS GOAL: ANALYZED (RANGE SCAN) OF 'EMP$DEPT' <br />
(NON-UNIQUE) <br />
2007 INDEX ACCESS GOAL: ANALYZED (RANGE SCAN) OF 'EMP$DEPT'<br />
(NON-UNIQUE) <br />
Now (using CBO) there is almost no performance difference between hinted and non-hinted statements, however we have to remember a number of facts:<br />
a. We update about 10 percent of rows in EMP table<br />
b. Our rows are very short (so the number of block in the table EMP is relatively small)<br />
My conclusion is that there is still a place to use hints when:<br />
a. FTS of updated table is performed and<br />
b. Only small percentage of rows are updated<br />
We are going to verify that conclusion in our next example.<br />
Case 2. Let’s check a bit more complicated statement.<br />
EMP table must be updated based on values in EMP_LOAD table that contains relatively small number of rows ( about 20000 rows in EMP table and 100 rows in EMP_LOAD table):<br />
RBO:<br />
UPDATE emp e <br />
SET (ename, job, mgr, hiredate, sal, comm, deptno) =<br />
(SELECT ename, job, mgr, hiredate, sal, comm, deptno<br />
FROM emp_load el<br />
WHERE e.empno = el.empno)<br />
WHERE e.empno IN (SELECT empno FROM emp_load)<br />
<br />
Call Count CPU Elapsed Disk Query Current Rows <br />
Parse 1 0.03 0.03 0 0 0 0 <br />
Execute 1 0.49 0.54 0 20321 208 100 <br />
Fetch 0 0.00 0.00 0 0 0 0 <br />
Total 2 0.52 0.57 0 20321 208 100 <br />
<br />
Rows Execution Plan <br />
0 UPDATE STATEMENT GOAL: CHOOSE <br />
1 UPDATE OF 'EMP' <br />
101 NESTED LOOPS <br />
20015 TABLE ACCESS (FULL) OF 'EMP' <br />
20114 INDEX (RANGE SCAN) OF 'EMP_LOAD_PK' (UNIQUE) <br />
0 TABLE ACCESS (BY INDEX ROWID) OF 'EMP_LOAD' <br />
0 INDEX (UNIQUE SCAN) OF 'EMP_LOAD_PK' (UNIQUE) <br />
Once again FULL TABLE SCAN raises it’s ugly head. We are going to use the previous approach and try to access EMP as an INNER table. <br />
UPDATE /*+ USE_NL(e) INDEX(e) */ emp<br />
SET (ename, job, mgr, hiredate, sal, comm, deptno) = <br />
(SELECT ename, job, mgr, hiredate, sal, comm, deptno<br />
FROM emp_load el<br />
WHERE e.empno = el.empno )<br />
WHERE e.empno IN ( SELECT empno FROM emp_load)<br />
<br />
Call Count CPU Elapsed Disk Query Current Rows <br />
Parse 1 0.02 0.02 0 0 0 0 <br />
Execute 1 0.05 0.05 0 401 204 100 <br />
Fetch 0 0.00 0.00 0 0 0 0 <br />
Total 2 0.07 0.07 0 401 204 100 <br />
<br />
Rows Execution Plan <br />
0 UPDATE STATEMENT GOAL: CHOOSE <br />
1 UPDATE OF 'EMP' <br />
101 NESTED LOOPS <br />
101 INDEX GOAL: ANALYZED (FULL SCAN) OF 'EMP_LOAD_PK' (UNIQUE) <br />
200 INDEX (RANGE SCAN) OF 'PK_EMP' (UNIQUE) <br />
0 TABLE ACCESS GOAL: ANALYZED (BY INDEX ROWID) OF 'EMP_LOAD' <br />
0 INDEX GOAL: ANALYZED (UNIQUE SCAN) OF 'EMP_LOAD_PK' <br />
(UNIQUE) <br />
CBO: <br />
UPDATE emp e <br />
SET (ename, job, mgr, hiredate, sal, comm, deptno) =<br />
(SELECT ename, job, mgr, hiredate, sal, comm, deptno<br />
FROM emp_load el<br />
WHERE e.empno = el.empno)<br />
WHERE e.empno IN (SELECT empno FROM emp_load)<br />
<br />
Call Count CPU Elapsed Disk Query Current Rows <br />
Parse 1 0.02 0.02 0 0 0 0 <br />
Execute 1 0.52 0.61 0 20321 208 100 <br />
Fetch 0 0.00 0.00 0 0 0 0 <br />
Total 2 0.54 0.63 0 20321 208 100 <br />
<br />
Rows Execution Plan <br />
0 UPDATE STATEMENT GOAL: CHOOSE <br />
1 UPDATE OF 'EMP' <br />
101 NESTED LOOPS <br />
20015 TABLE ACCESS GOAL: ANALYZED (FULL) OF 'EMP' <br />
20114 INDEX GOAL: ANALYZED (RANGE SCAN) OF 'EMP_LOAD_PK'<br />
(UNIQUE) <br />
0 TABLE ACCESS GOAL: ANALYZED (BY INDEX ROWID) OF 'EMP_LOAD' <br />
0 INDEX GOAL: ANALYZED (UNIQUE SCAN) OF 'EMP_LOAD_PK' <br />
(UNIQUE) <br />
The result is identical to the default RBO decision, so let’s try and improve it:<br />
UPDATE /*+ USE_NL(e) INDEX(e) */ emp<br />
SET (ename, job, mgr, hiredate, sal, comm, deptno) =<br />
(SELECT ename, job, mgr, hiredate, sal, comm, deptno<br />
FROM emp_load el<br />
WHERE e.empno = el.empno )<br />
WHERE e.empno IN ( SELECT empno FROM emp_load)<br />
<br />
Call Count CPU Elapsed Disk Query Current Rows <br />
Parse 1 0.02 0.02 0 0 0 0 <br />
Execute 1 0.06 0.06 0 401 204 100 <br />
Fetch 0 0.00 0.00 0 0 0 0 <br />
Total 2 0.08 0.08 0 401 204 100 <br />
<br />
Rows Execution Plan <br />
0 UPDATE STATEMENT GOAL: CHOOSE <br />
1 UPDATE OF 'EMP' <br />
101 NESTED LOOPS <br />
101 INDEX GOAL: ANALYZED (RANGE SCAN) OF 'EMP_LOAD_PK'<br />
(UNIQUE) <br />
200 INDEX GOAL: ANALYZED (RANGE SCAN) OF 'PK_EMP' (UNIQUE) <br />
0 TABLE ACCESS GOAL: ANALYZED (BY INDEX ROWID) OF<br />
'EMP_LOAD' <br />
0 INDEX GOAL: ANALYZED (UNIQUE SCAN) OF 'EMP_LOAD_PK' <br />
(UNIQUE) <br />
Now we can compare the results:<br />
<br />
Without hint With hint Improvement(%) <br />
CPU 0.54 0.08 85 <br />
Elapsed 0.63 0.08 87 <br />
Access count 20529 605 97 <br />
As we can see there is huge performance improvement in these cases just waiting to be executed.<br />
I am unable to add a PK constraint to a table - what is the best way of finding the problem rows ?<br />
We've all been there - you issue an "alter table ... add primary key" command, and sit and watch for 12 hours whilst it chugs through your "squillion" row table only to come back with "ORA-02437: cannot validate ..." or similar because you have some problem rows. And since it failed - you are no closer to resolving the problem. <br />
The solution is to break the task down into stages: <br />
· Create an NON-UNIQUE index on the appropriate table column(s) <br />
· Add the primary with the NOVALIDATE clause. This stops any more problem rows getting into the table <br />
· Alter the primary key with the VALIDATE EXCEPTIONS INTO clause. You will still see the standard 'cannot validate' error, but now the EXCEPTIONS table (cf: $ORACLE_HOME/rdbms/admin/utlexcpt.sql) contains the rowids for the problem rows <br />
· Remove the problem rows <br />
· Alter the primary key with the VALIDATE clause <br />
Dropping a tablespace seems to take a very long time - how can I speed it up ?<br />
There are two reasons why dropping a tablespace takes a lot of time. <br />
· If the tablespace contains a lot of objects, then this is a massive recursive dictionary operation that is being undertaken. (Even if the tablespace is empty, there may still be a lot of free space entries which may need to be cleaned up) <br />
· All of these dictionary operations need to be recorded in rollback, just in case the operation fails or is terminated in some way. <br />
A way to avoid this is to explicitly drop the segments in the tablespace before dropping the tablespace itself. The overall operation is faster, and if the session/instance crashes, then there will not be a massive undo operation to performed. Some sample timings are shown below: <br />
Standard 'drop' on a dictionary managed tablespace <br />
(prelim - we stick 1000 segments in "random" order in the tablespace)<br />
<br />
SQL> begin<br />
2 for i in 1 .. 500 loop<br />
3 execute immediate 'create table tab'||(i*2)||' ( x number ) tablespace sample_data '||<br />
4 'storage ( initial 16k next 16k )';<br />
5 end loop;<br />
6 for i in 1 .. 500 loop<br />
7 execute immediate 'create table tab'||(i*2-1)||' ( x number ) tablespace sample_data '||<br />
8 'storage ( initial 16k next 16k )';<br />
9 end loop;<br />
10 end;<br />
11 /<br />
<br />
PL/SQL procedure successfully completed.<br />
<br />
SQL> set timing on<br />
SQL> drop tablespace sample_data including contents;<br />
<br />
Tablespace dropped.<br />
<br />
Elapsed: 63 seconds<br />
Drop objects first then tablespace (dictionary managed) <br />
(prelim as before)<br />
<br />
SQL> drop table tab1;<br />
<br />
Table dropped.<br />
<br />
SQL> drop table tab2;<br />
<br />
Table dropped.<br />
<br />
(every 10 tables we issue)<br />
<br />
SQL> alter table SAMPLE_DATA coalesce;<br />
<br />
Tablespace altered.<br />
<br />
etc.<br />
<br />
(Total) Elapsed: 55 seconds<br />
<br />
SQL> drop tablespace sample_data;<br />
<br />
Tablespace dropped.<br />
<br />
Elapsed: 00:00:00.80<br />
<br />
NB: As pointed out by the FAQ owner, if you wish to use PL/SQL to automate this, you will not get any joy using 'alter tablespace ... coalesce' as a recursive SQL (ie within the PL/SQL routine). You would need to code <br />
<br />
execute immediate 'alter session set events ''immediate trace name drop_segments level n''';<br />
<br />
where 'n' is the tablespace ID plus 1. <br />
If you are using locally managed tablespaces, then you are insulated in some respect, namely, the tablespace (including its contents) can be dropped more quickly. The same test above gave 50 seconds for a plain 'drop tablespace' and 66 seconds for a preliminary drop of the objects. There is probably still a good case for the preliminary drop to avoid the resource pain of rolling back to tablespace drop in the event of a session crash. <br />
What is the difference between an 'index full scan' and an 'index fast full scan' ?<br />
The fast full scan was an operation that appeared in Oracle 7.3, although in that version of Oracle there was a time when it had to be invoked explicitly by the hing /*+ index_ffs(alias index) */. <br />
Under this operation, the index would not be treated like an index, it would be treated as if it were a narrow table, holding just the columns defined as part of the index. Because of this, Oracle is able to read the index segment using multiblock reads (discarding branch blocks as it goes), even using parallel query methods to read the segment in the shortest possible time. Of course, the data coming out of an index fast full scan will not be sorted.<br />
The full scan, however, simply means that Oracle walks the index from end to end, following leaf blocks in the right order. This can be quite slow, as Oracle can only use single block reads to do this (although in fact 8.1.5ish introduced the _non_contiguous_multiblock_read, and various readahead strategies). However the results do come out in index order without the need for an sort. You might notice that Oracle tends to choose this option when there is an index that happens to suit the order by clause of a query, even if it doesn't suit the where clause of the query - this may also result in a SORT ORDER BY (NO SORT) line appearing in your execution plan.<br />
I have a column with a default value, but I keep finding it set to NULL - what's going wrong ?.<br />
Because a default value is used only for insertion into a table, but you may update the value of column to NULL<br />
I have created a table with the nologging option, but any insert, update, or delete still generates redo log , why?<br />
First, it must be understood that redo generation is at the very heart of Oracle's ability to recover from virually any media failure. For that reason, the ability to disable it only exists for a small subset of commands. They are all related to serial and parallel direct-path loads. <br />
What operations allow NOLOGGING? <br />
DML statements such as insert, update, and delete will always generate redo. However, the nologging option may be utilized for the following SQL statements (from the Oracle 8i concepts manual, chapter 22): <br />
· direct load (SQL*Loader) <br />
· direct load INSERT (using APPEND hint) <br />
· CREATE TABLE ... AS SELECT <br />
· CREATE INDEX <br />
· ALTER TABLE ... MOVE PARTITION <br />
· ALTER TABLE ... SPLIT PARTITION <br />
· ALTER INDEX ... SPLIT PARTITION <br />
· ALTER INDEX ... REBUILD <br />
· ALTER INDEX ... REBUILD PARTITION <br />
· INSERT, UPDATE, and DELETE on LOBs in NOCACHE NOLOGGING mode stored out of line <br />
For the statements listed above, undo and redo logging can be almost entirely disabled. New extents are marked invalid, and data dictionary changes are still logged. Note that a table or index with the nologging attribute (which can be seen in the LOGGING column of the DBA_TABLES or DBA_INDEXES view) will default to nologging when one of the above statements is executed. <br />
What is the recoverability of objects created as NOLOGGING? <br />
Since nologging disables writing redo log entries, there is no way for Oracle to recover data related to nologging operations. In the case of a media failure subsequent to a nologging operation, Oracle will apply the redo log transactions, but, when it reaches the transactions related to the nologging operation, it will only be able to apply the extent invalidation records, since that is all that was recorded. Any subsequent attempt to access data in those extents will result in an ORA-26040 "Data block was loaded using the NOLOGGING option". The only resolution to the error is to drop and recreate the object. Note that this risk only exists until the next successful backup. Backups taken after the completion of the nologging operation will provide complete recovery.<br />
How much benefit is there in building a table/index that fits into a single extent<br />
Does number of extents matter ?<br />
Some DBA's are reluctant to allow more than a few extents in any segment with the mistaken belief that such "fragmentation" degrades performance. Within reason, the performance impact of multiple extents is almost insignificant if they are sized correctly <br />
Does extent size matter ?<br />
Yes, extent size does matter, but not greatly. Nevertheless, all extents should be a multiple of the multiblock read count. Otherwise, when a full table or index scan is performed, an extra multiblock read will be required to read the last few blocks of each extent, except probably the last one. This is because multiblock reads never span extent boundaries, even if the extents happen to be contiguous. <br />
Consider for example the table T1. It is comprised of 8 extents of 50 blocks each. The first block is the segment header, there are 389 data blocks in use, and there are 10 free blocks above the high water mark. With a multiblock read count of 16 blocks, and assuming none of the blocks are already in cache, a full table scan of this table will require 4 data block reads per extent, except the last - a total of 31 multiblock reads. <br />
If the table is rebuilt as T2 with an extent size that is an exact multiple of the multiblock read count, then the number of multiblock reads required to scan the table is minimized. Assuming the table is now comprised of 5 extents of 80 blocks each. A full table scan now requires 5 multiblock reads per extent, or a total of 25 multiblock reads.<br />
Please note that it was not the reduction in the number of extents as such that made the difference. There would be no further saving in rebuilding the table with a single extent of 400 blocks. The number of multiblock reads required to scan the table would still be 25..<br />
The number of redo copy latch misses reported in V$LATCH is a large fraction of the gets. What should I do ?<br />
The quick answer to the question is that you are probably looking at the half of the latch report that lists 'Willing to Wait' latches which means that you don't have a problem. <br />
It is important to remember that there are two sets of 'gets' and 'misses' recorded by the V$LATCH dynamic performance view. Two columns are simply named gets and waits, the other two are named immediate_gets and immediate_waits. If you look at the complete set of latch statistics for your system, you will notice that there are two high usage latches on 7.3 and 8.0 - Redo Allocation and Redo Copy. Oracle 8.1 introduces a third latch, the Redo Writing latch. <br />
It is worth looking at just the redo latches, and the gets/misses on just those latches , for example with the following SQL:<br />
column name format a15<br />
<br />
select<br />
name, <br />
gets, misses, <br />
immediate_gets, immediate_misses<br />
from v$latch<br />
where name like '%redo%'<br />
;<br />
<br />
<br />
NAME GETS MISSES IMMEDIATE_GETS IMMEDIATE_MISSES <br />
--------------- --------- --------- -------------- ---------------- <br />
redo allocation 1937729 1 0 0 <br />
redo copy 13 3 1814906 6 <br />
redo writing 645563 1 0 0 <br />
It takes only a moment, when presented with the results in this form, to realise that the Redo Copy latch is almost invariable acquired on an immediate get, and rarely accessed on a 'willing to wait' get. The fact that the ratio of misses to gets (3 to 13) is rather high is totally irrelevant. In fact it is actually the expected behaviour as Oracle only acquires the redo copy latch in this fashion when it is trying to suspend redo activity temporarily whilst doing some special log file activity.<br />
If you do see a relatively large number of gets (as opposed to immediate gets) on the redo copy latch, it may suggest that your redo log file size is too small, and you have a completely different performance problem to address anyway. In this case, check the alert log for indications of 'checkpoint not complete' and check the system statistics for excessive writes to disc due to checkpointing.<br />
<br />
Further reading: For a detailed description of the actvity associated with redo, the best reference site is Steve Adams' site http://www.ixora.com.au <br />
What's the quickest way of deleting all (or a large fraction of) the data in a table ?<br />
The answer depends on the circumstances, but if you need to delete all the rows in a table then the truncate command is the way to go as this command minimizes the writing of redo and undo (rollback) entries. The table and any indexes have their high water mark, stored in the header block, reset and except for the dictionary or bitmap updates for freed extents beyond the initial extent the job is done. If the table is to be reloaded and will need to reuse all or nearly all of the space it held then the reuse option can be used to eliminate even the need to update the dictionary space management tables or bitmap header for locally managed objects:<br />
truncate table owner.tablename [drop storage| reuse storage]<br />
The drop storage clause is the default and does not need to be provided.<br />
When all the data cannot be deleted then your options are limited by whether you can get exclusive access to the table to perform the task or the table has to be kept available at all times and how your applications use the table. The next set of approaches all involve the same idea: Instead of deleting 90% of the data, extract the 10% of the data that is good to a new table and discard the original.<br />
In the case where the in use applications only need insert access to the target table then if access can be stopped for a very short window the following approach can be used: <br />
When no FK to target and no insert or delete triggers exist then<br />
1- rename target to target_old<br />
2- Re-Create the target table<br />
3- recreate indexes, re-establish PK and UK constraints, and re-apply grants<br />
4- allow access<br />
5- extract good data from target_old and insert it into the recreated target<br />
6- truncate the target_old table<br />
7- drop the target_old table<br />
If there is an insert trigger but no delete trigger or FK referencing the table then the re-insertion of the data needs to happen before the users are allowed access to the table since it is not desirable to have the trigger fire for data it has already processed, but as long as the time to select and insert this data is satisfactory then a create tables as select can be used for the re-create table step. This eliminates a step from the list above, but the ordering of the list above is designed to minimize the time the table needs to be unavailable for use.<br />
If the target table has FK references to other tables then since all existing rows either meet the requirement or already violate it because the novalidate option was used in creating the FK constraints then they can be disabled in those cases where the good data is copied prior to allowing access to the table. This can save a fair amount of work and the constraints should again be re-enabled using the novalidate option to save performing unnecessary work. The existence of FK's referencing the target table complicates matters in that either the necessary deletes in the referencing tables must be done in a separate process allowing the FK to be disabled, or the deletes must take place against the target, and the re-naming/re-creating method cannot be used.<br />
If the table is partitioned then native Oracle parallel DML can be used so that each partition has a delete job dedicated to it. This can greatly reduce clock time at the expense of system resources. For a normal table the delete can be logically partitioned to manually duplicate the parallel delete process be running multiple delete statements, each of which has its where clause limited to a specified set of rowid values. If you take this approach then I recommend that you create the indexes for this table with delete job number of transaction work areas itl, such as initrans 4. This will help to ensure each delete job can obtain an existing itl entry so that they do not have to dynamically allocate any, or worse have to wait because the space to allocate an itl is unavailable. When multiple delete jobs are ran the first job should have an unbounded lower range and the last job an unbounded upper range to ensure no section of the target table is missed. It is probably advisable to generate the rowids rather than hardcode them. Sample code to find the boundary rowids follows at the end of the FAQ. It would be fairly easy to change the dbms_output statements to an insert of a single row into a delete job control table that the delete jobs reference in their where clauses to control the delete ranges.<br />
Still another concept is to partition the table such that all rows to be dropped exist in the same partition. This method requires that the data lend itself to partitioning on the columns that are used to determine that the data can be deleted and this is often not practical. If the data requires global indexes or a few rows within the partition key range end up not being valid for deletion then the benefit of portioning the table to support deletes is lost. Still the method should not be overlooked.<br />
set serveroutput on<br />
declare<br />
-- counters<br />
v_ctr pls_integer := 0 ;<br />
v_ct_rows pls_integer := 0 ;<br />
v_rowid rowid ;<br />
v_rows_per_job pls_integer := 0 ;<br />
--<br />
cursor c_get_rowids is<br />
select rowid<br />
from po_blanket_hist ;<br />
--<br />
-- pl/sql table to hold boundry rowids<br />
--<br />
type t_rowids is table of varchar2(18)<br />
index by binary_integer ;<br />
t_rows t_rowids ;<br />
I binary_integer ;<br />
--<br />
-- get total row count<br />
--<br />
begin<br />
select count(*)<br />
into v_ct_rows<br />
from po_blanket_hist ;<br />
--<br />
-- In this example we are going to use 4 jobs<br />
--<br />
v_rows_per_job := floor(v_ct_rows / 4) ;<br />
--<br />
-- While data read, test, save rowid<br />
--<br />
open c_get_rowids ;<br />
I := 1 ;<br />
loop<br />
fetch c_get_rowids into v_rowid ;<br />
exit when c_get_rowids%notfound ; -- quit when done<br />
v_ctr := v_ctr + 1 ; -- count rows returned<br />
if v_ctr = v_rows_per_job<br />
then t_rows(I) := v_rowid ; -- mod takes longer than using a counter<br />
v_ctr := 0 ;<br />
I := I + 1 ;<br />
end if ;<br />
end loop ;<br />
--<br />
close c_get_rowids ;<br />
--<br />
-- This is where we could save the rowids to a job control row<br />
--<br />
dbms_output.put_line('Row Count is '||v_ct_rows) ;<br />
dbms_output.put_line('Rows per job '||v_rows_per_job) ;<br />
dbms_output.put_line('First Rowid is '||t_rows(1)) ;<br />
dbms_output.put_line('Second Rowid is '||t_rows(2)) ;<br />
dbms_output.put_line('Third Rowid is '||t_rows(3)) ;<br />
dbms_output.put_line('Forth Rowid is '||t_rows(4)) ;<br />
end ;<br />
/<br />
<br />
Row Count is 32695051<br />
Rows per job 8173762<br />
First Rowid is AAAAb+AFQAAALHWAAO<br />
Second Rowid is AAAAb+AFgAAAMiNAA5<br />
Third Rowid is AAAAb+AGQAAADZnAAL<br />
Forth Rowid is AAAAb+AGQAAAKCMABB<br />
<br />
PL/SQL procedure successfully completed.<br />
<br />
<br />
Further reading: If you have complex delete logic then the section on views for performance in Jonathan Lewis's book Practical Oracle 8i might be of interest. You can also see Metalink document id 19890.996 and 89799.996, which are forum threads. Oracle support did not offer anything not mentioned in this FAQ.<br />
I have done a truncate on a table, and it takes hours to run - what is going on ? <br />
Whilst the truncate command is normally instantaneous, a quick consideration of what it is doing reveals why they sometimes take a long time. The 'truncate' command must do two things in order for a table to be reduced to "zero" size (ie a single extent with a reset high water mark). <br />
· Free up any used extents <br />
· Reset the HWM on the remaining single extent <br />
We can presume that moving or resetting the high water mark is a relatively painless operation, given that the database is always doing this as tables grow. However, if your table is in hundreds/thousands of extents, then freeing them up can take some time. In particular, when using dictionary managed tablespaces, the two system tables FET$ and UET$ need to be updated. For example, if we perform <br />
SQL> select extents from user_segments where segment_name = 'BLAH'<br />
2 /<br />
<br />
EXTENTS<br />
----------<br />
15<br />
<br />
SQL> alter session set sql_trace = true;<br />
<br />
Session altered.<br />
<br />
SQL> truncate table blah;<br />
<br />
Table truncated.<br />
and then look at the trace file, we'll see <br />
select length <br />
from<br />
fet$ where file#=:1 and block#=:2 and ts#=:3<br />
<br />
insert into fet$ (file#,block#,ts#,length) <br />
values<br />
(:1,:2,:3,:4)<br />
<br />
delete from uet$ <br />
where<br />
ts#=:1 and segfile#=:2 and segblock#=:3 and ext#=:4<br />
<br />
delete from fet$ <br />
where<br />
file#=:1 and block#=:2 and ts#=:3<br />
and these operations are SERIAL, namely, only one session can be performing this at a time. So if you have anything on your database that could be "attacking" FET$ and UET$ (for example, sorting in a permanent tablespace, dropping/adding objects frequently), then you will get these kinds of problems occurring. <br />
<br />
Further reading: Some versions of 8.0 would crash when a truncate is terminated with Ctrl-C, so take care<br />
How do you write a query that ignores the effects of upper and lower case ?<br />
Some databases have an "ignore case" flag that can be set for the entire database. Oracle does not, and thus case-insensitive queries have long caused problems, not with coding them, but with their performance (since indexes are typically not used to determine the result). <br />
Its relatively straightforward to create a case-insensitive query: <br />
SQL> select *<br />
2 from EMP<br />
3 where upper(ENAME) = upper(:b1)<br />
but of course (by default) the "UPPER(ENAME)" cannot take advantage of an index that may have been defined on the ENAME column. <br />
Enter 8i, where the concept of a function-based index is now possible. Before you rush off and try to create them, take note of the following: <br />
· You must have the system privelege query rewrite to create function based indexes on tables in your own schema. <br />
· You must have the system privelege global query rewrite to create function based indexes on tables in other schemas <br />
· For the optimizer to use function based indexes, the following session or system variables must be set: <br />
QUERY_REWRITE_ENABLED=TRUE<br />
QUERY_REWRITE_INTEGRITY=TRUSTED <br />
· You must be using the Cost Based Optimiser (which means analyzing your tables/indexes) <br />
and then its just a case of creating the index in the conventional way: <br />
create index UPPER_ENAME_IX on ENAME ( UPPER(ENAME) ) ;<br />
Note that this index will NOT be used for case-SENSITIVE queries. You could always have two indexes, one on ENAME and one on UPPER(ENAME), but it would probably be more efficient to just have the function-based one and code: <br />
SQL> select *<br />
2 from EMP<br />
3 where upper(ENAME) = upper(:b1)<br />
4 and ENAME = :b1<br />
for the times where you do not want case-insenstivity. <br />
<br />
Further reading: Querying the dictionary for function-based indexes<br />
How much space does a number take up.<br />
Oracle uses an internal 'base 100 encoding' format for storing numbers, which stores two digits of precision per byte, an extra byte that holds both the sign and the 'mantissa' indicating where to put the decimal point in relation to the digits of precision, and for negative numbers a terminating byte holding the value 0x66. <br />
Consequently the actual space used by a number depends on the number of significant digits that appear in the number and the sign of the number, so the numbers 1, 100, 10,000 and 1,000,000 all take two bytes, and -1, -100, -10,000, -1,000,000 will take 3 bytes , whereas 1,234,567 will require 5 bytes, and -1,234,567 will require 6 bytes.<br />
<br />
Value Representation Bytes Stored <br />
1 1 x power(100,0) c1, 2 <br />
100 1 x power(100,1) c2, 2 <br />
10,000 1 x power(100,2) c3, 2 <br />
1,000,000 1 x power(100,3) c4, 2 <br />
-1 -1 x power(100,0) 3e, 64, 66 <br />
-100 -1 x power(100,1) 3d, 64, 66 <br />
-10,000 -1 x power(100,2) 3c, 64, 66 <br />
-1,000,000 -1 x power(100,3) 3b, 64, 66 <br />
1,234,567 1.234,567 x power(100,3) c4, 2, 18, 2e, 44 <br />
-1,234,567 -1.234,567 x power(100,3) 3b, 64, 4e, 38, 22, 66 <br />
Although there are special cases, as indicated by the values for 1, 100, 10,000, and 1,000,000 above, the rules tell us that the typical N-digit positive number will have require 1 + ceil(N/2) bytes, and a negative number will required 2 + ceil(N/2) bytes, where ceil() is found by rounding up to the next integer where necessary.<br />
For example, most of the (six digit) numbers from 100,000 to 199,999 will encode to 4 bytes i.e.1 + (6/2), whereas most of the (nine digit) numbers from 100,000,000 to 100,099,999 will encode to 6 bytes i.e.1 + ceil(9/2) = 1 + 5. You can run the following SQL (as SYS because of the reference to the X$ object) to verify the second result:<br />
select len, count(*) <br />
from <br />
(<br />
select <br />
vsize(rownum + 99999999) len <br />
from x$ksmmem<br />
where rownum <= 100000<br />
)<br />
group by len<br />
;<br />
Giving results:<br />
LEN COUNT(*)<br />
--------- ---------<br />
2 1<br />
4 9<br />
5 990<br />
6 99000<br />
Don't forget that when stored in a row, each numeric column will also require the standard one-byte row-length overhead<br />
Is there a good way of counting the number of rows in a table ?<br />
You can use the count(*) function to get row counts from Oracle and with Oracle rdbms version 8.1.7+ it would appear that the cost based optimizer, CBO, optimizes unqualified select count statements for you where a primary key constraint exists. The example table is allocated at 200M and the PK index is 20M: For reference the ANSI aggregate function of min, max, count, sum, and avg ignore null values in their input sets and have the form of:<br />
function( ALL | DISTINCT column_name | * ) <br />
Examples - Starting with a full table count on a table with a primary key, PK, constraint defined.<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select count(*)<br />
3 from item_master<br />
4 /<br />
<br />
Explained.<br />
<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- -----------------------------------------------------<br />
123 1 SELECT STATEMENT<br />
1 2.1 SORT AGGREGATE<br />
123 379426 3.1 INDEX FAST FULL SCAN ITEM_MASTER_PRIME UNIQUE<br />
<br />
<br />
3 rows selected.<br />
The CBO was able to utilize a fast full scan of the PK index to perform the count rather than read the full table sequentially which is what the RULE based optimizer did for the same query. Also notice the cardinality column where Oracle tells you how many rows it thinks it will process in this step; the actual rows returned will probably be different so running explain plans is not a good substitute for running select counts:<br />
DDC2> select count(*) from item_master;<br />
<br />
COUNT(*)<br />
----------<br />
387835<br />
<br />
1 row selected.<br />
If you are running rule based a full table scan is employed to solve the unqualified full count but with version 7.3 on, a hint provided in the SQL overrides the database and session optimizer settings, even if no statistics exist on the table, so you can use an index hint on the PK to force use of the index. Under version 7 the CBO would also full scan the table to solve this query, but again you can hint the SQL as long as statistics existed on the table for version 7.0 - 7.2 and with or without statistics in version 7.3. <br />
Select /*+ INDEX(t t_pk) */ count(*)<br />
From table_name t<br />
<br />
t = label for the table in the from clause and t_pk is the primary key or unique index name<br />
The index fast full scan option was not available until version 8 and the hint is INDEX_FFS. (addendum from Zach Friese zfriese@earthlink.net in fact the fast full scan was introduced in Oracle 7.3 - see page 5-30, Oracle7 Server Tuning).<br />
In the case where a where clause needs to be provided to obtain the row count for a specific condition then Oracle's ability to optimize the query depends on the where clause columns being indexed just like with any normal query. The PK for this table consists of item_no and plant. The plant is the second column in the key. Remember that in version 7 that Oracle can only use a concatenated index when the leading column of the index is referenced in the where clause but observe:<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select count(*)<br />
3 from item_master<br />
4 where plant = '12'<br />
5 /<br />
<br />
Explained.<br />
<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- -----------------------------------------------------<br />
123 1 SELECT STATEMENT<br />
1 2.1 SORT AGGREGATE<br />
123 63238 3.1 INDEX FAST FULL SCAN ITEM_MASTER_PRIME UNIQUE<br />
<br />
<br />
3 rows selected.<br />
The CBO is still able to scan the PK to count the number of plant 12 entries. Under RULE a full table scan is once again used:<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select /*+ RULE */ count(*)<br />
3 from item_master<br />
4 where plant = '12'<br />
5 /<br />
<br />
Explained.<br />
<br />
Elapsed: 00:00:00.04<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- --------------------------------------------------<br />
SELECT STATEMENT<br />
2.1 SORT AGGREGATE<br />
3.1 TABLE ACCESS FULL ITEM_MASTER<br />
The CBO and RULE base optimizers can both use indexes to solve count requests for leading indexed columns even those that are not part of the PK or a UK constraint. In the example product_code is the leading column of item_master_idx6 and may be null.<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select count(*)<br />
3 from item_master<br />
4 where product_code = '12'<br />
5 /<br />
<br />
Explained.<br />
<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- ----------------------------------------------------<br />
3 1 SELECT STATEMENT<br />
1 2.1 SORT AGGREGATE<br />
3 234 3.1 INDEX RANGE SCAN ITEM_MASTER_IDX6 NON-UNIQUE<br />
<br />
<br />
3 rows selected.<br />
<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select /*+ RULE */ count(*)<br />
3 from item_master<br />
4 where product_code = :1<br />
5 /<br />
<br />
Explained.<br />
[Note the rule hint so the cost and cardinality column of the explain plan table are now null, but the plan is the same because under RULE if the index exists we use it. Also note that in this case using bind variables instead of constants does not prevent the optimizer from using the index under either optimizer approach.]<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- ----------------------------------------------------<br />
SELECT STATEMENT<br />
2.1 SORT AGGREGATE<br />
3.1 INDEX RANGE SCAN ITEM_MASTER_IDX6 NON-UNIQUE<br />
<br />
<br />
3 rows selected.<br />
If you need a count based on the where condition referencing a non-indexed column even if is constrained to be not null then the optimizers will have no option but to full scan the table:<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select count(*)<br />
3 from item_master<br />
4 where family_cd is null<br />
5 /<br />
<br />
Explained.<br />
<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- --------------------------------------------------<br />
1296 1 SELECT STATEMENT<br />
1 2.1 SORT AGGREGATE<br />
1296 1 3.1 TABLE ACCESS FULL ITEM_MASTER<br />
<br />
3 rows selected.<br />
The column family_cd is constrained to be not null but does not appear in any index. Logically if the column is constrained to be not null then it has a value for every row in the table so the result of a count of nulls is zero, but the optimizer is still going to count. <br />
In the case where you want the not null count for a null allowed column then you can make the following change:<br />
From:<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select count(*)<br />
3 from item_master<br />
4 where family_cd is not null<br />
5 /<br />
<br />
Explained.<br />
<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- --------------------------------------------------<br />
1296 1 SELECT STATEMENT<br />
1 2.1 SORT AGGREGATE<br />
1296 379426 3.1 TABLE ACCESS FULL ITEM_MASTER<br />
<br />
<br />
To:<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select count(family_cd)<br />
3 from item_master<br />
4 /<br />
<br />
Explained.<br />
<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- -----------------------------------------------------<br />
123 1 SELECT STATEMENT<br />
1 2.1 SORT AGGREGATE<br />
123 379426 3.1 INDEX FAST FULL SCAN ITEM_MASTER_PRIME UNIQUE<br />
By removing the where clause condition test and counting the specific column the CBO can use the PK index, but the RULE based optimized solved the above using a full table scan.<br />
The above covers the case where the column is not-null but what about when it is or can be null. I am pretty sure that at one time if you ran a query of the form select count(*) from table where [nullable] column is null OR is not null then the optimizer would perform a full table scan even if the column was indexed. But with version 8.1.7+ if at least one column in the index that the where clause column is the leading column of is constrained to being not null then the CBO can use it to satisfy the query:<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select count(*)<br />
3 from item_master<br />
4 where product_code is not null<br />
5 /<br />
<br />
Explained.<br />
<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- --------------------------------------------------------<br />
132 1 SELECT STATEMENT<br />
1 2.1 SORT AGGREGATE<br />
132 2799 3.1 INDEX FAST FULL SCAN ITEM_MASTER_IDX6 NON-UNIQUE<br />
<br />
<br />
3 rows selected.<br />
<br />
DDC2> EXPLAIN PLAN SET statement_id = 'mpowel01' FOR<br />
2 select count(*)<br />
3 from item_master<br />
4 where product_code is null<br />
5 /<br />
<br />
Explained.<br />
<br />
DDC2> set echo off<br />
<br />
COST CARDINALITY QUERY_PLAN<br />
---------- ----------- --------------------------------------------------------<br />
132 1 SELECT STATEMENT<br />
1 2.1 SORT AGGREGATE<br />
132 376627 3.1 INDEX FAST FULL SCAN ITEM_MASTER_IDX6 NON-UNIQUE<br />
<br />
<br />
3 rows selected.<br />
<br />
DDC2> select count(*) from item_master where product_code is null;<br />
<br />
COUNT(*)<br />
----------<br />
385100<br />
<br />
1 row selected.<br />
<br />
DDC2> select count(*) from item_master where product_code is not null;<br />
<br />
COUNT(*)<br />
----------<br />
2744<br />
<br />
1 row selected.<br />
<br />
DDC2> select count(*) from item_master;<br />
<br />
COUNT(*)<br />
----------<br />
387844<br />
<br />
1 row selected. <br />
[This count differs from the one at the top of the page because this is production and activity is taking place, but the null and non-null product_code counts add to the current total rows so Oracle was able to use the index.]<br />
If all the columns of a normal index are capable of being null then expect the optimizer to full table scan. However bitmap indexes store nulls so the CBO can use bitmap indexes to solve null and non-null query requests including count(*). The RULE base optimizer does not recognize bit-map indexes.<br />
<br />
Further reading: Is there a simple way to produce a report of all tables in the database with current number of rows ?<br />
<br />
Why do some queries go very fast and others very slowly when the optimizer_mode is choose ? <br />
Are composite indexes more useful than single column indexes ? <br />
Is is possible to allow Microsoft Query receive the output from the dbms_output package ? If not, what are the alternatives ? <br />
Is there any way to identify which init.ora parameters are relevant to tuning the database ? <br />
How do I decide on a good value for pga_aggregate_target in 9.2 ? <br />
I have a query that I run on two identical databases - on one it is very quick on the other it is very slow - what's up ? <br />
Is there any way of identifying how parameters in the Oracle INIT.ORA control file can be used for tuning the database including: (1) Memory usage (2) CPU usage (3) Disk usage <br />
What is the fastest way of dumping a database table to a flat file - is utl_file a good idea ? <br />
How do I get global histograms onto partitioned tables in Oracle 8.1 - apparently they can only be generated properly in v9. <br />
In documentation about tuning SQL, I see references to parse trees. What is a parse tree ? <br />
Are there any benefits or drawbacks to regular reboot of a machine running an Oracle database ? <br />
Is there any way to keep track of when datafiles are resized. <br />
Should I worry about having some relationship between extent sizes and db_file_multiblock_read_count ? <br />
How can I apply an Oracle database (application) change to many equal instances located across the country on a single date ?<br />
We're looking for he most efficient way to apply our database releases to dozens of Oracle instances on the same day. <br />
Are there any good strategies for backing up a data warehouse ? <br />
Why is the Comment_Text Column on my Sys.Dba_Audit_Trail Table is always empty ? <br />
I am getting error while restarting the database (9.0.1) after changing the parameter db_block_buffers in SPFILE<sid>.ora file. The error was "could not start the database with old and new parameter". What is wrong ? <br />
I have audit_trail=db - but want to capture the program name from v$session as well. How can I do this ? <br />
Why do I keep getting Oracle error - ORA-1002: FETCH OUT OF SEQUENCE in my batch code ? <br />
I lock my table before truncating it, but I still get ORA-00054: resource busy ... Why ? <br />
How do I determine the degree of fragmentation of a table ? <br />
How do I determine how much space I could reclaim by deleting a chunk of data from a table (and cleaning up afterwards) <br />
How do I decide on a sensible size for my redo log buffer <br />
I created a user with the wrong default tablespace - How can I move his entire schema from one tablespace to another. <br />
I have created a trigger on a table, and this seems to have wrecked the performance of some bulk inserts - why ? <br />
v$sysstat shows the number of sorts that have occurred. How do I find the number of hash joins that have occurred ? <br />
I have a table partitioned on column timestamp, with a primary key on column id. How do I create a locally partitioned index for this PK ? <br />
How do I calculate suitable values for PCTFREE and PCTUSED for a table. <br />
What is the best way to move a 500GB from 32 bit 8.1.7.4 to 64 bit 9.2 ? exp/imp, mig, SQLLDR, or CTAS ? <br />
Is it only committed transactions that are written into the redo log ? <br />
I wish to update certain portions inside long column in 8.1.7. If this were a varchar2 field I would use the replace function. How do I achieve the same thing in with a long ? <br />
<br />
<br />
PL/SQL<br />
<br />
Is there a way to read the names of a set of files with a given extension from a directory as if from a SQL cursor?<br />
In 10g there is a new procedure hidden away in the undocumented DBMS_BACKUP_RESTORE package. It's called searchfiles, which is a bit <br />
of a giveaway and appears to have been introduced for the new backup features in 10g, as RMAN now needs to know about files in the recovery destination.<br />
<br />
Calling this procedure populates an in memory table called x$krbmsft, which is one of those magic x$ tables, the only column which is of relevance to us <br />
is fname_krbmsft which is the fully qualified path and file name. This x$ table acts in a similar fashion to a global temporary table in that its contents<br />
can only be seen from the calling session. So two sessions can call searchfiles and each can only see the results of their call (which is extremely useful).<br />
<br />
The code sample below will only really run as sys, due to the select from x$krbmsft, it's just intended as a demo. The first two parameters in the call to <br />
searchfiles are IN OUT so must be defined as variables, even though the second parameter is of no consequence to us and should be left as NULL. <br />
Even though they are IN OUT, testing shows they don't appear to change. The first parameter is the string to search for, in much the same format as <br />
you would pass in a call to dir (Windows) or ls (Unix).<br />
<br />
DECLARE<br />
pattern VARCHAR2(1024) := 'C:\temp\*.csv'; -- /tmp/*.csv <br />
ns VARCHAR2(1024); <br />
<br />
BEGIN<br />
<br />
SYS.DBMS_OUTPUT.ENABLE(1000000);<br />
<br />
SYS.DBMS_BACKUP_RESTORE.searchFiles(pattern, ns);<br />
<br />
FOR each_file IN (SELECT FNAME_KRBMSFT AS name FROM X$KRBMSFT) LOOP<br />
DBMS_OUTPUT.PUT_LINE(each_file.name);<br />
END LOOP;<br />
<br />
END;<br />
/<br />
<br />
This procedure appears to raise no exceptions, passing an invalid search string, such as a non-existant path or one with no permissions, simply results in <br />
an empty x$krbmsft. However, if the database parameter db_recovery_file_dest is not set, you will get ORA-19801.<br />
<br />
Interestingly, this procedure recursively searches sub directories found in the search string. So passing a string of 'C:\windows' (for example) populates <br />
x$krbmsft with not only the files found in that directory but all the files found in all directories beneath, such as C:\windows\system32. <br />
As x$krbmsft is an in memory table, you have been warned! Calling this procedure on a directory with thousands of sub directories and files has the potential <br />
to consume large amounts of memory (or more likely just generate an exception).<br />
<br />
<br />
Further reading: http://www.ixora.com.au/scripts/prereq.htm (shows how to create views on x$ fixed tables)<br />
Further work: http://www.chrispoole.co.uk/<br />
The way forward is to wrap this functionality in a package, perhaps using directory objects instead and checking access priviliges, creating a view based on <br />
x$krbmsft, maybe even allowing/disallowing subdirectory traversal and limiting memory usage search size. I see another project coming, stay tuned for XUTL_FINDFILES!<br />
<br />
How do I send an email from PL/SQL, possibly with attachments?<br />
It is often desirable for a long running PL/SQL program to be able to send an email to a designated recipient, possibly including a file of results. Although this can be done by wrapping the procedure in a shell-script or BAT file, it is more elegant to send the email direct from the PL/SQL procedure. Oracle versions 8.1.6 and later include the UTL_SMTP package for doing this, but you need to know a bit about SMTP to use it, and it doesn't lend itself to easily sending attachments. Is there a simple way to send an email from PL/SQL, with attachments, without having to understand how SMTP works? <br />
How do I delete an O/S file from within PL/SQl<br />
One 'near-soultion' is to use the utl_file package to re-open the file for writing (without the append option), and then close the file without writing to it. This recovers most of the disk space, but still leaves the file on the system as an empty O/S file.<br />
Another approach is to write a short piece of Java, which can then be called from PL/SQL. Java currently offers far more flexibility than PL/SQL when dealing with O/S files, for example you could use Java to invoke and load a directory listing from PL/SQL so that you know what files exist for deletion. (See further reading).<br />
A pure simple PL/SQL solution, however, appears to exist in the dbms_backup_restore package. This is virtually undocumented (other than in the script dbmsbkrs.sql) in 8.1.7, but contains a simple deletefile() procedure.<br />
begin<br />
dbms_backup_restore.deletefile('/tmp/temp.txt');<br />
end;<br />
/<br />
This appears to work as required with no side-effects.<br />
Update for 9.2<br />
In version 9.2, the utl_file package has been enhanced with the fremove() procedure for deleting a file.<br />
<br />
Updated 24th Jan 2005<br />
I have received an email from S Kumar pointing out that the call to dbms_backup_restore.deletefile() always gives the message: “PL/SQL procedure successfully completed” even if the path or file is not present or file or path name is invalid or if open. So we can not depend on this package's output.<br />
<br />
Further reading: Expert One-on-One: Oracle (Tom Kyte) for examples of using Java from PL/SQL packages, in particular a routine to call Java to perform a directory listing.<br />
Can I check that the old password before changing to a new password ?<br />
The Quick Answer<br />
The quick answer is to use SQL*Plus as the following examples show :<br />
SQL> connect norman/********<br />
Connected.<br />
<br />
SQL> password<br />
Changing password for NORMAN<br />
Old password: ****<br />
New password: ******<br />
Retype new password: ******<br />
ERROR:<br />
ORA-28008: invalid old password<br />
Password unchanged<br />
The example above shows that the use of an incorrect old password has caused the changing of norman's new password to fail.<br />
The next example shows the password being changed correctly when the correct old password is supplied :<br />
SQL> password<br />
Changing password for NORMAN<br />
Old password: ********<br />
New password: ******<br />
Retype new password: ******<br />
Password changed<br />
So far, so good except if you run the password utility as a DBA user, you don't get prompted for the old password as follows :<br />
SQL> connect system/************<br />
Connected.<br />
<br />
SQL> password norman<br />
Changing password for norman<br />
New password: ******<br />
Retype new password: ******<br />
Password changed<br />
<br />
The Long-Winded Answer<br />
Getting back to the original question, which was "In a change password procedure can I check that the old password has been entered correctly before changing to a new password ", one solution is the following procedure.<br />
This needs to be created in the SYSTEM user (or similar) and the user needs to be granted select on sys.dba_users, and alter user privileges directly (not through a role) otherwise it doesn't work.<br />
Here's the code :<br />
create or replace procedure change_password(iUsername in dba_users.username%type,<br />
iOldPassword in dba_users.password%type,<br />
iNewPassword in dba_users.password%type)<br />
as<br />
-- Variables. <br />
vOldEncryptedPassword dba_users.password%type := null; <br />
vOldConfirmPassword dba_users.password%type := null; <br />
<br />
-- Constants. <br />
cUsernameMissing constant varchar2(100) := 'Username must be supplied.';<br />
cInvalidUsername constant varchar2(100) := 'Username does not exist in this database.';<br />
<br />
cOldPasswordMissing constant varchar2(100) := 'Old password must be supplied.';<br />
cOldPasswordMismatch constant varchar2(100) := 'Old password mismatch - password not changed.';<br />
cNewPasswordMissing constant varchar2(100) := 'New password must be supplied.';<br />
<br />
cPasswordChangeFailed constant varchar2(100) := 'ERROR: failed to change password.';<br />
<br />
-- Internal helper procedures and functions. <br />
function GetPassword(iUsername in dba_users.username%type)<br />
return varchar2<br />
as<br />
vResult dba_users.password%type := null;<br />
begin<br />
select password<br />
into vResult<br />
from dba_users<br />
where username = upper(iUsername);<br />
<br />
return vResult;<br />
exception<br />
when others then<br />
return null;<br />
end;<br />
<br />
procedure AlterPassword(iUsername in dba_users.username%type,<br />
iPassword in dba_users.password%type,<br />
iValues in boolean default false)<br />
as<br />
vSQL varchar2(1000) := 'alter user ' || iUsername ||<br />
' identified by ';<br />
begin<br />
if (iValues) then<br />
vSQL := vSQL || 'values ''' || iPassword || '''';<br />
else<br />
vSQL := vSQL || iPassword;<br />
end if; <br />
<br />
execute immediate vSQL;<br />
<br />
exception<br />
-- All that can go wrong is a malformed SQL statement <br />
-- Famous last words .....<br />
when others then<br />
raise;<br />
end;<br />
<br />
<br />
<br />
begin<br />
---------------------------------------------------------------------------------------<br />
-- Parameter validation first is always a good idea :o) --<br />
---------------------------------------------------------------------------------------<br />
<br />
-- Username must be supplied and be present in the database.<br />
-- We save the password for later on as we will need it.<br />
if (iUsername is null) then<br />
raise_application_error(-20001, cUsernameMissing);<br />
return;<br />
else<br />
vOldEncryptedPassword := GetPassword(iUsername);<br />
<br />
if (vOldEncryptedPassword is null) then<br />
raise_application_error(-20001, cInvalidUsername);<br />
return;<br />
end if; <br />
end if;<br />
<br />
-- Old password must be supplied. We will be using this later to <br />
-- validate the changing of the password.<br />
if (iOldPassword is null) then<br />
raise_application_error(-20001, cOldPasswordMissing);<br />
return;<br />
end if;<br />
<br />
-- New password must also be supplied - for obvious reasons.<br />
if (iNewPassword is null) then<br />
raise_application_error(-20001, cNewPasswordMissing);<br />
return;<br />
end if;<br />
<br />
---------------------------------------------------------------------------------------<br />
-- We should now be in posession of a set of valid parameters, lets have some fun ! --<br />
---------------------------------------------------------------------------------------<br />
<br />
-- We obtained the current encrypted password above, so now we can change the existing <br />
-- password to the old one, and check to see if it is the same. If so, we are able to <br />
-- continue by changing to the new password. If not, we reset the old password to the <br />
-- encrypted on, and bale out.<br />
<br />
<br />
-- First of all, change the password to its current value - strange, but needed. <br />
begin<br />
AlterPassword(iUsername, iOldPassword);<br />
exception<br />
when others then<br />
-- Cannot change the user's password to its existing value - bale out. <br />
-- So far, we have changed nothing so no corrective actions required.<br />
raise_application_error(-20001, cPasswordChangeFailed);<br />
end;<br />
<br />
-- Next, fetch the 'new' old password and compare it with the 'old' old password.<br />
vOldConfirmPassword := GetPassword(iUsername);<br />
<br />
if (vOldEncryptedPassword <> vOldConfirmPassword) then<br />
begin<br />
-- Old password doesn't match, reset to the old one and bale out.<br />
AlterPassword(iUsername, vOldEncryptedPassword, true);<br />
raise_application_error(-20001, cOldPasswordMismatch);<br />
return;<br />
exception<br />
when others then<br />
raise;<br />
end; <br />
end if;<br />
<br />
-- Old password matches, change password to the new one. <br />
begin<br />
AlterPassword(iUsername, iNewPassword);<br />
exception<br />
when others then<br />
raise;<br />
end;<br />
end;<br />
/ <br />
Examples of use<br />
The following examples are running in user SYSTEM and are attempting to change the password for user FRED. Fred's current password is also fred. Just to prove it all :<br />
SQL> connect fred/fred<br />
connected<br />
<br />
SQL>connect system/************<br />
connected<br />
First an attempt to use the wrong old password :<br />
SQL> exec change_password('fred','trouble','wilma')<br />
BEGIN change_password('fred','trouble','wilma'); END;<br />
*<br />
ERROR at line 1:<br />
ORA-20001: Old password mismatch - password not changed.<br />
ORA-06512: at "SYSTEM.CHANGE_PASSWORD", line 127<br />
ORA-06512: at line 1<br />
Next, not supplying a valid username :<br />
SQL> exec change_password('nobody', 'something', 'new');<br />
BEGIN change_password('nobody', 'something', 'new'); END;<br />
*<br />
ERROR at line 1:<br />
ORA-20001: Username does not exist in this database.<br />
ORA-06512: at "SYSTEM.CHANGE_PASSWORD", line 78<br />
ORA-06512: at line 1<br />
Finally, a working example<br />
SQL> exec change_password('fred','fred','barney');<br />
PL/SQL procedure successfully completed.<br />
<br />
SQL>connect fred/barney<br />
connected<br />
I'm sure you can amend the above code to suit your system. Note that it uses execute immediate internall. If you are using an older version of Oracle, you will need to change this to use the dbms_sql package instead.<br />
<br />
<br />
</sid></sid></password></block_number></block_number></block_number></file_id></block_number></file_id></iot_object_id></iot_object_id></eof_sp></n></index></sid></sid></sid></sid></sid>KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0tag:blogger.com,1999:blog-8692881074449232849.post-90244029534106671142009-10-06T08:18:00.000-07:002009-10-06T08:18:30.908-07:00Materialised Views1.0 Materialised Views: A bit of background<br />
I confess a weakness for Materialised Views. It strikes me that they are amongst the cleverest, neatest, most productive features Oracle has ever come up with, and they continue to impress me every time I use them, even though they’ve been around for many, many Oracle versions (including a stint in ancient versions as something known as a ‘snapshot table’).<br />
If you’ve not really worked with materialised views before, that older name for them gives a better idea to their true nature and function than the much sexier-sounding ‘materialised view’ one: they are snapshots of data in ‘real’ tables as at a particular point of time. What’s more (and their real reason for existing), the snapshot that is stored can be the result of a complex join or a set of aggregations of data. So, for example, a very simple example might be:<br />
create materialized view emp_dept_mv <br />
as <br />
select e.empno, e.ename, d.dname, d.loc, e.sal <br />
from scott.emp e, scott.dept d <br />
where e.deptno=d.deptno;<br />
Note firstly that the object uses American spelling, so whereas I refer to them as ‘materialised’, the SQL syntax demands you use a ‘z’ in their names. Second, what you have just created here is a new table, in fact -but one which stores the result of a join between two other tables. You could actually have achieved almost exactly the same thing by issuing this command instead:<br />
create table emp_dept_join <br />
as <br />
select e.empno, e.ename, d.dname, d.loc, e.sal <br />
from scott.emp e, scott.dept d <br />
where e.deptno=d.deptno;<br />
In either case, the select part of the syntax could get an awful lot more complex than the trivial example I have shown here, and hence the ‘snapshot table’ you create in either case would represent the stored results of more and more costly queries. That, in effect, is the principle reason for wanting to use materialised views in the first place: they store the results of costly queries, so that you can then select directly from the stored results in the future, without the database having to bear the costs associated with re-performing the original query.<br />
But, you might reasonably ask, if create table as select syntax can effectively snapshot complex query results, why was there felt to be a need for some new exotic segment type called a materialised view? What can it do that good old CTAS cannot?<br />
Well, the answer to that comes in two parts. First, with a CTAS table copy, you have to know the copy exists, and write your SQL to use it directly. That is, if a user issues a query along the lines of ‘select e.empno, d.loc from emp, dept...’, then the fact that there is an emp_dept_join table in my database is of no consequence: the user is not asking to query from that summarisation, so the query is resolved by re-performing the expensive join on the original, ‘real’, tables. Practically, that would mean that unless you re-wrote your entire application to be aware of the existence of summary CTAS tables, they’d never actually get used. A materialised view, however, can be used even if a user writes SQL which appears to reference the real tables: the database’s optimiser weighs up the costs of re-performing the joins (and possible aggregations) on the base tables, versus the costs of querying from the materialised view which stores the prior results of performing those operations... and if it’s cheaper to query from the materialised view, the optimiser will re-write the SQL issued by the user, automatically and transparently: materialised views could therefore be employed without re-writing a single line of application code. Very nice!<br />
The second reason why materialised views are much more functional than CTAS-copied tables is to do with the fact that any snapshot or copy of a table or tables is bound, sooner or later to get out-of-date and will therefore, presumably, need to be brought back up-to-date somehow. With a CTAS-table, the only thing you can really do is this:<br />
truncate table emp_dept_join;<br />
<br />
insert into emp_dept_join<br />
select e.empno, e.ename, d.dname, d.loc, e.sal<br />
from scott.emp e, scott.dept d<br />
where e.deptno=d.deptno;<br />
At this point, of course, you are re-performing the original costly query -and you’re having to remember to do it manually. If the base tables on which your copy is based are subject to a reasonable amount of DML, you’d be having to re-perform this sort of synchronisation so often, you probably would have been better off not having created the copy table in the first place! And if you forget to re-synchronise the copy with its base tables, then users whose SQL statements make use of the copy table will be receiving inaccurate results.<br />
By contrast, a Materialised View can refresh itself, automatically, and (under certain circumstances) the nature of that refresh can be relatively trivial. If you create another ‘exotic’ segment called a materialised view log, then the materialised view can capture the DML that gets performed to the original base tables: the materialised view log is a bit like a mini-online redo log that stores changes made only to the table its built on. At some point, the contents of the view log can then be ‘replayed’ to the materialised view itself, thus bringing it completely up-to-date.<br />
Just to show you these two key functions of materialised views in action, here’s a simple example. First, a few necessary preliminaries:<br />
drop user scott cascade; <br />
User dropped.<br />
<br />
@?/rdbms/admin/utlsampl <br />
Disconnected from Oracle Database 10g Enterprise Edition <br />
Release 10.1.0.2.0 - Production With the Partitioning, OLAP and Data Mining options<br />
<br />
sqlplus / as sysdba<br />
<br />
SQL*Plus: Release 10.1.0.2.0 - Production on Mon Feb 14 08:53:11 2005 <br />
Copyright (c) 1982, 2004, Oracle. All rights reserved.<br />
<br />
grant create materialized view, query rewrite to scott;<br />
Grant succeeded.<br />
<br />
grant select on dba_objects to scott;<br />
Grant succeeded.<br />
<br />
grant advisor to scott;<br />
Grant succeeded.<br />
<br />
connect scott/tiger<br />
Connected. <br />
Here, I’m clearing out Scott from my database so that I can be sure I’m working with a clean slate. I then re-create Scott using the utlsampl.sql script provided by Oracle in the \rdbms\admin directory. I then finally grant the minimum set of privileges necessary to Scott so that the rest of what follows in this article will work.<br />
I will pause at this point to confess that I have been guilty in the past -and in the first drafts of this paper- of simply granting the DBA role to Scott. It certainly makes sure that the examples in the articles work, but it is monstrously bad practice for any DBA to follow. I should be setting a better example than that, so I have resolved that in this and all future papers I will make the effort to demonstrate the smallest set of privileges needed to make things work. In this case, the privileges fall into three groups. The grant of create materialized view and query rewrite are fundamental to what follows. Without them, 99% of this article would be meaningless. The second grant, of select on DBA_OBJECTS is admittedly a bit odd, and not particularly good DBA practice -but it will let me create a large table with lots of rows and low cardinality ofr some columns very quickly. In a sense, it’s just a cosmetic grant of a privilege for mere convenience, and if you had an alternative table of similar size and cardinality qualities to use instead, you could manage perfectly well without it. The third privilege grant is of advisor, and that is a brand new, 10g-specific privilege. If you didn’t do it on your database, then just one (though very important) piece of the article concerning new 10g functionality won’t work for you, but the bulk of the article about materialised view functionality in general would still make sense.<br />
In any case, my system and object privilege grants have been made, and I’m ready to begin in earnest:<br />
drop table ob purge; <br />
drop table ob purge<br />
*<br />
ERROR at line 1: <br />
ORA-00942: table or view does not exist<br />
<br />
drop materialized view ob_sum; <br />
drop materialized view ob_sum<br />
*<br />
ERROR at line 1: <br />
ORA-12003: materialized view "SCOTT"."OB_SUM" does not exist <br />
I’ve thrown in a couple of drop object DDL statements there as one further way of ensuring I’m starting with a clean slate. They cause errors the first time you work through this article, of course, because the first time through, no such objects exist. Second and subsequent work-throughs would proceed without error at this point.<br />
create table ob as select * from dba_objects; <br />
Table created. <br />
<br />
alter table ob add (constraint ob_pk primary key(object_id));<br />
Table altered.<br />
<br />
SQL> create materialized view log on ob with rowid<br />
2 (owner, object_name, subobject_name, object_id) including new values;<br />
Materialized view log created.<br />
<br />
SQL> create materialized view ob_sum<br />
2 refresh on commit fast<br />
3 enable query rewrite<br />
4 as<br />
5 select owner, count(*) as counter from ob<br />
6 group by owner;<br />
<br />
Materialized view created. <br />
I start out by creating a copy of a suitably large table: DBA_OBJECTS usually has several thousand records in it, so a copy of it called OB should be sufficient for my purposes. I add a primary key constraint to the copied table, and then add on a materialised view log (so that the eventual materialised view will be able to refresh itself by re-applying base table DML that’s been captured in the log). Finally, I create the materialised view itself, including a demand that it identifies itself as being available for use if the optimiser is considering rewriting any user queries. Line 2 of that piece of SQL also indicates that if a piece of DML is performed on the base table OB, then the materialised view is to bring itself up-to-date with that change as soon as it has been committed. Note that the materialised view merely records the count of the number of objects owned by each user. It contains no joins to other tables, in other words, but it does store the results of a fairly hefty piece of summarisation, aggregation, grouping and sorting.<br />
We’ll be able to see what all of that means in practical terms in just a moment, once I have given Scott the ability to inspect the execution plans the optimiser generates for his queries. That means I have first to run the utlxplan.sql script that Oracle supplies (in /rdbms/admin) to create a PLAN_TABLE in Scott’s schema:<br />
SQL> @?\rdbms\admin\utlxplan.sql <br />
Table created. <br />
Now I can see what execution plans various simple queries will generate: SQL> set autotrace trace explain<br />
SQL> select count(*) from ob 2 where owner='SCOTT';<br />
<br />
Execution Plan<br />
----------------------------------------------------------<br />
0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=3 Card=1 Bytes=30)<br />
1 0 SORT (AGGREGATE)<br />
2 1 MAT_VIEW REWRITE ACCESS (FULL) OF 'OB_SUM' (MAT_VIEW REWRITE) (Cost=3 Card=1 Bytes=30)<br />
Note that my query says to select from OB. Yet what does the execution plan reveal? It shows that we are really accessing the OB_SUM materialized view -and a minor new feature in 10g is that the reason the query is going to use a completely different segment than the one it actually asked to use is that there has been a MAT_VIEW REWRITE. This is proof, in other words, that users can issue SQL that thinks it’s going to read from one table, but transparently end up looking at another, because the optimiser deems it cheaper to do so. That was the first of the two reasons why materialised views are such wonderful segments, you’ll recall.<br />
SQL> insert into ob (owner, object_name, subobject_name, object_id) <br />
2 values ('SCOTT','TESTINSERT','NONE',98365); <br />
<br />
1 row created. <br />
At this point, I’ve just inserted one new record into the base table, OB. I haven’t yet committed that insert. Remembering that I told my materialised view only to bring itself up-to-date when I commit DML performed on the base table, this must mean that my materialised view should now be considered out-of-date. Although it is configurable (though explaining how is outside the scope of this article), materialised views cannot generally be candidates for a bit of query rewriting by the optimiser if they are indeed known to be out-of-date. My materialised view has therefore just been ruled out of consideration for future query rewrites by my little bit of base table DML, and I can test that this is the case easily enough:<br />
SQL> select count(*) from ob 2 where owner='SCOTT'; <br />
<br />
Execution Plan<br />
----------------------------------------------------------<br />
0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=129 Card=1 Bytes=17)<br />
1 0 SORT (AGGREGATE)<br />
2 1 TABLE ACCESS (FULL) OF 'OB' (TABLE) (Cost=129 Card=6 Bytes=102) <br />
This is exactly the same query as I issued before, but this time the execution plan reveals that we’re going to select records out of the OB table itself, and not use the OB_SUM materialised view. This is proof therefore that an out-of-date materialised view is dismissed by the optimiser for query rewrite purposes. Incidentally, you can now see why, when the materialised view is known to be fully up-to-date, the optimiser is so keen on making use of it: the query previously reported a total cost of ‘3’, whereas if the base table has to be used, the cost shoots up to 129. That means my silly, simple, materialised view is nevertheless performing about 40 times faster than the base table: now you understand how these things can be so useful!<br />
SQL> commit; <br />
Commit complete. <br />
<br />
select count(*) from ob 2 where owner='SCOTT';<br />
<br />
Execution Plan<br />
----------------------------------------------------------<br />
0 SELECT STATEMENT Optimizer=ALL_ROWS (Cost=3 Card=1 Bytes=30)<br />
1 0 SORT (AGGREGATE)<br />
2 1 MAT_VIEW REWRITE ACCESS (FULL) OF 'OB_SUM' (MAT_VIEW REWRITE) (Cost=3 Card=1 Bytes=30)<br />
Finally, I commit my bit of pending DML, and then immediately re-issue exactly the same query as before. The execution plan this time reveals that the materialised view is back in business. That’s an indication that the commit must have caused the materialised view to bring itself back up-to-date, at which point it’s a candidate for query rewrites once more. A more direct proof of the second of the key features of materialised views -this ability of theirs to refresh themselves- can be seen easily like so:<br />
SQL> set autotrace off <br />
SQL> select count(*) from ob where owner='SCOTT'; <br />
<br />
COUNT(*)<br />
----------<br />
9<br />
<br />
SQL> select counter from ob_sum where owner='SCOTT';<br />
<br />
COUNTER<br />
----------<br />
9 <br />
So far, at least, the two table’s count of the number of objects owned by Scott is synchronised.<br />
SQL> insert into ob (owner, object_name, subobject_name, object_id)<br />
2 values ('SCOTT','TESTINSERT','NONE',98379);<br />
<br />
1 row created.<br />
<br />
SQL> select count(*) from ob where owner='SCOTT';<br />
<br />
COUNT(*)<br />
----------<br />
10<br />
<br />
SQL> select counter from ob_sum where owner='SCOTT';<br />
<br />
COUNTER<br />
----------<br />
9<br />
But as a result of the insert of one new record, the OB base table’s count increases to 10, whereas the count of objects in the materialised view OB_SUM remains at 9. The materialised view is thus visibly ’stale’.<br />
SQL> commit; <br />
Commit complete. <br />
<br />
SQL> select count(*) from ob where owner='SCOTT';<br />
<br />
COUNT(*)<br />
----------<br />
10<br />
<br />
SQL> select counter from ob_sum where owner='SCOTT';<br />
<br />
COUNTER<br />
----------<br />
10<br />
One commit statement later, however, and the two objects’ counts agree once more. As demanded by the materialised view’s definition, the commit causes the earlier insert to OB to be replayed to OB_SUM, thus re-synchronising the two objects. OB_SUM is considered ‘fresh’ once more, and hence (as we saw) available for query rewrites.<br />
I’ve taken quite a lot of time and space to explain the fundamentals of what materialised views are and how they work because, without that level of basic understanding, hardly any of the incremental (but very nice) enhancements that we get in 10g would make sense! Armed with that foundation-level knowledge, therefore, let’s now look specifically at what has changed in 10g. <br />
2.0 DBMS_ADVISOR<br />
Creating materialised views looks simple enough, at least when you consider the examples I used earlier. But it isn’t actually that easy. Consider this simple example:<br />
SQL> create materialized view emp_dept_mv<br />
2 build immediate<br />
3 refresh fast<br />
4 enable query rewrite<br />
5 as<br />
6 select e.ename, d.dname<br />
7 from emp e, dept d<br />
8 where e.deptno=d.deptno;<br />
from emp e, dept d<br />
*<br />
ERROR at line 7: <br />
ORA-12052: cannot fast refresh materialized view SCOTT.EMP_DEPT_MV<br />
This is not the world’s most complex select statement, I think you will agree! Yet it obviously contains sufficient complexity to prevent a fast-refreshable materialised view being created based on it. This, indeed, has been one bugbear that has bedevilled attempts to implement materialised views in previous Oracle versions: their seemingly irrational dislike of being fast-refreshed (which, of course, they have to be able to do if they are to be kept nice and fresh at minimal cost). All sorts of simple factors would trip them up and result in this sort of error, and it was something of an acquired skill to work out which ones were interfering at any one time.<br />
Fortunately, that’s practically a thing of the past in 10g. We have a new package DBMS_ADVISOR and a new procedure within that package called TUNE_MVIEW, which will not only make it clear why a proposed materialised view can’t be created, but will give you the SQL statements actually needed to create it! Here’s how I might sort out that troublesome materialised view above:<br />
SQL> var hjr varchar2(30) begin<br />
2 dbms_advisor.tune_mview(:hjr,<br />
3 'create materialized view emp_dept_mv build immediate refresh fast enable<br />
4 query rewrite as select e.ename, d.dname from emp e,<br />
5 dept d where e.deptno=d.deptno');<br />
6 end;<br />
7 / <br />
<br />
PL/SQL procedure successfully completed. <br />
The first line here simply declares a new bind variable, called hjr (you can call it anything you like, of course!). That’s to act as the first of the TUNE_MVIEWS parameters: it becomes the unique identifier for the explanations about how to make this materialised view work when you inspect the relevant table. After that, you simply invoke the TUNE_MVIEW procedure supplying the bind variable identifier and the exact text of the proposed ‘create materialized view’ SQL statement. Oracle thinks about it for a moment or two, and then tells you the procedure finished successfully. At this point, you can inspect its work like so:<br />
set long 3000<br />
set pagesize 3000<br />
set linesize 132<br />
<br />
select * from user_tune_mview; <br />
<br />
OWNER TASK_NAME ACTION_ID<br />
------------------------------ ------------------------------ ----------<br />
SCRIPT_TYPE<br />
--------------<br />
STATEMENT<br />
--------------------------------------------------------------------------------<br />
SCOTT TASK_5 7<br />
IMPLEMENTATION<br />
CREATE MATERIALIZED VIEW LOG ON "SCOTT"."DEPT" WITH ROWID<br />
<br />
SCOTT TASK_5 8<br />
IMPLEMENTATION<br />
ALTER MATERIALIZED VIEW LOG FORCE ON "SCOTT"."DEPT" ADD ROWID<br />
<br />
SCOTT TASK_5 9<br />
IMPLEMENTATION<br />
CREATE MATERIALIZED VIEW LOG ON "SCOTT"."EMP" WITH ROWID<br />
<br />
SCOTT TASK_5 10<br />
IMPLEMENTATION<br />
ALTER MATERIALIZED VIEW LOG FORCE ON "SCOTT"."EMP" ADD ROWID<br />
<br />
SCOTT TASK_5 11<br />
IMPLEMENTATION<br />
CREATE MATERIALIZED VIEW SCOTT.EMP_DEPT_MV build immediate REFRESH FAST WITH RO<br />
WID ENABLE QUERY REWRITE AS SELECT SCOTT.EMP.ROWID C1, SCOTT.DEPT.ROWID C2, "SCO<br />
TT"."DEPT"."DNAME" M1, "SCOTT"."EMP"."ENAME" M2 FROM SCOTT.EMP, SCOTT.DEPT WHERE<br />
SCOTT.DEPT.DEPTNO = SCOTT.EMP.DEPTNO<br />
<br />
SCOTT TASK_5 12<br />
UNDO<br />
DROP MATERIALIZED VIEW SCOTT.EMP_DEPT_MV<br />
<br />
6 rows selected. <br />
First of all, note how I set the SQL*Plus “LONG” environment variable to a suitably large figure: the STATEMENT column in this view returns a CLOB, and therefore the text will be truncated unless you do that before you start. The other ‘set’ commands stop lines wrapping and page breaks cluttering up the display of the query results.<br />
Secondly, notice how the new USER_TUNE_MVIEW view (there’s a DBA_ equivalent, naturally) contains (in this case) five SQL statements which need to be issued to make the proposed materialised view actually workable. There is also a sixth statement, which shows how you would drop the materialised view after it gets created! In theory, you could simply cut and paste each of the statements in turn (bar the last one, of course!) into a SQL session, and that would get the materialised view created.<br />
But third: notice that these statements are all related to each other by being identified with the one ‘task name’ -in my case, they are all part of TASK_5. I can use that information as an input to yet another new 10g procedure that will generate a SQL script consisting of these individual SQL statements for me, like so:<br />
SQL> connect / as sysdba <br />
Connected. <br />
<br />
SQL> create directory MYDIR as 'C:\';<br />
Directory created.<br />
<br />
SQL> grant read, write on directory mydir to scott;<br />
Grant succeeded.<br />
<br />
SQL> connect scott/tiger<br />
Connected.<br />
<br />
SQL> exec dbms_advisor.create_file (dbms_advisor.get_task_script('TASK_5'),'MYDIR','FIXME.TXT')<br />
PL/SQL procedure successfully completed.<br />
The first thing I did here was to make a quick detour to SYS’s schema, so that I could create a new directory object. That will allow the database to write a text file out to the file system. I then granted a couple of object privileges to Scott so that he can actually make use of the new directory object. Obviously, if a directory object already exists in your database, to which you have appropriate read and write access, then there’s no need to create a new one just to get this process working.<br />
With the directory object in place, I logged back on as Scott and invoked the DBMS_ADVISOR package once more, but this time its CREATE_FILE procedure. That takes three essential arguments; the task name I mentioned earlier, the name of the directory object the SQL script is to be written to, and the desired name of the SQL script itself. When this is executed, Oracle sweeps up all SQL statements found in the USER_TUNE_MVIEW view that share the specified task name, and outputs them all to a text file that looks a bit like this:<br />
host type c:\fixme.txt <br />
<br />
Rem SQL Access Advisor: Version 10.1.0.1 - Production <br />
Rem <br />
Rem Username: SCOTT <br />
Rem Task: TASK_5 <br />
Rem <br />
<br />
set feedback 1<br />
set linesize 80<br />
set trimspool on set tab off set pagesize 60<br />
<br />
whenever sqlerror CONTINUE<br />
<br />
CREATE MATERIALIZED VIEW LOG ON<br />
"SCOTT"."DEPT"<br />
WITH ROWID ;<br />
<br />
ALTER MATERIALIZED VIEW LOG FORCE ON<br />
"SCOTT"."DEPT"<br />
ADD ROWID<br />
;<br />
<br />
CREATE MATERIALIZED VIEW LOG ON<br />
"SCOTT"."EMP"<br />
WITH ROWID ;<br />
<br />
ALTER MATERIALIZED VIEW LOG FORCE ON<br />
"SCOTT"."EMP"<br />
ADD ROWID<br />
;<br />
<br />
CREATE MATERIALIZED VIEW SCOTT.EMP_DEPT_MV<br />
REFRESH FAST WITH ROWID<br />
ENABLE QUERY REWRITE<br />
AS SELECT SCOTT.EMP.ROWID C1, SCOTT.DEPT.ROWID C2, "SCOTT"."DEPT"."DNAME" M1<br />
"SCOTT"."EMP"."ENAME"<br />
M2 FROM SCOTT.EMP, SCOTT.DEPT WHERE SCOTT.DEPT.DEPTNO = SCOTT.EMP.DEPTNO;<br />
<br />
whenever sqlerror EXIT SQL.SQLCODE<br />
<br />
begin<br />
dbms_advisor.mark_recommendation('TASK_5',1,'IMPLEMENTED');<br />
end;<br />
/ <br />
There’s nothing particularly clever here: the SQL statements have been wrapped up nicely and formatted beautifully, of course. But otherwise, there’s nothing much here you couldn’t have gotten by simply cutting and pasting direct from USER_TUNE_MVIEW... except for that last statement, which will change the SCRIPT_TYPE column in the USER_TUNE_MVIEW column from IMPLEMENTATION to IMPLEMENTED -thus allowing you to distinguish between SQL statements in that view which still need to be performed and those which have been performed. Of course, if you choose the cut-and-paste straight from the view technqiue, there’s nothing to stop you executing something like that last line yourself and achieving exactly the same outcome. In my case, having created a file called FIXME.TXT, I can execute it as a SQL script like so:<br />
SQL> @C:\fixme.txt <br />
<br />
Materialized view log created.<br />
Materialized view log altered.<br />
Materialized view log created.<br />
Materialized view log altered.<br />
Materialized view created.<br />
<br />
PL/SQL procedure successfully completed. <br />
And suddenly, what had been impossible to create at the start of this section has clearly been created without fuss:<br />
SQL> select * from emp_dept_mv;<br />
<br />
C1 C2 M1 M2<br />
------------------ ------------------ -------------- ----------<br />
AAAPxoAAEAAAAAcAAA AAAPxmAAEAAAAAMAAB RESEARCH SMITH<br />
AAAPxoAAEAAAAAcAAB AAAPxmAAEAAAAAMAAC SALES ALLEN<br />
AAAPxoAAEAAAAAcAAC AAAPxmAAEAAAAAMAAC SALES WARD<br />
AAAPxoAAEAAAAAcAAD AAAPxmAAEAAAAAMAAB RESEARCH JONES<br />
AAAPxoAAEAAAAAcAAE AAAPxmAAEAAAAAMAAC SALES MARTIN<br />
AAAPxoAAEAAAAAcAAF AAAPxmAAEAAAAAMAAC SALES BLAKE<br />
AAAPxoAAEAAAAAcAAG AAAPxmAAEAAAAAMAAA ACCOUNTING CLARK<br />
AAAPxoAAEAAAAAcAAH AAAPxmAAEAAAAAMAAB RESEARCH SCOTT<br />
AAAPxoAAEAAAAAcAAI AAAPxmAAEAAAAAMAAA ACCOUNTING KING<br />
AAAPxoAAEAAAAAcAAJ AAAPxmAAEAAAAAMAAC SALES TURNER<br />
AAAPxoAAEAAAAAcAAK AAAPxmAAEAAAAAMAAB RESEARCH ADAMS<br />
AAAPxoAAEAAAAAcAAL AAAPxmAAEAAAAAMAAC SALES JAMES<br />
AAAPxoAAEAAAAAcAAM AAAPxmAAEAAAAAMAAB RESEARCH FORD<br />
AAAPxoAAEAAAAAcAAN AAAPxmAAEAAAAAMAAA ACCOUNTING MILLER <br />
It might not look pretty -but at least it exists!<br />
You know, it is quite often the little, insignficant new features which go to make a new Oracle version a must-have upgrade... and my belief is that DBMS_ADVISOR.TUNE_MVIEW is one of them for 10g. I myself have spent many a long hour, frustrated and irritated, at the intractable mysteries of getting materialised views to behave... and now here we have something that will write the precise SQL statements needed to make it happen for you. That is, in a word, wonderful, if you ask me.<br />
In practice, it means those many sites who have not implemented materialised views as well or as comprehensively as they could have done because of the complexity of doing so now have no excuse... and can expect a performance dividend accordingly.<br />
And one final word: remember that the before you can make use of the DBMS_ADVISOR package, you need to be granted the new 10g ADVISOR system privilege (it’s number -263 in the SYSTEM_PRIVILEGE_MAP view if you’re interested!)<br />
3.0 Miscellaneous Enhancements<br />
3.1 REWRITE_OR_ERROR Hint<br />
The other enhancements to materialised views that 10g brings are, by comparison, and in my estimation, much less significant, so I won’t spend too much time discussing them here. One of the nice ones is this:<br />
select /*+ REWRITE_OR_ERROR */ count(*) from ob where owner='SCOTT';<br />
That’s a new hint which requires of the optimiser that it either re-writes a query so that it definitely uses the materialised view, or it raises an error and aborts the whole thing. Thus, using one of my earlier examples, I get this:<br />
SQL> select /*+ REWRITE_OR_ERROR */ count(*) from ob where owner='SCOTT';<br />
<br />
COUNT(*)<br />
----------<br />
10<br />
<br />
1 row selected.<br />
<br />
...and because of the hint there, I know without even looking that this query must have been re-written to use the OB_SUM materialised view. A finalbit of proof: <br />
SQL> insert into ob (owner, object_name, subobject_name, object_id)<br />
2 values ('SCOTT','TESTINSERT','NONE',98380);<br />
<br />
1 row created.<br />
<br />
SQL> select /*+ REWRITE_OR_ERROR */ count(*) from ob where owner='SCOTT'; <br />
select /*+ REWRITE_OR_ERROR */ count(*) from ob where owner='SCOTT'<br />
*<br />
ERROR at line 1: ORA-30393: a query block in the statement did not rewrite<br />
<br />
Disconnected from Oracle Database 10g Enterprise Edition Release 10.1.0.2.0 - Production <br />
With the Partitioning, OLAP and Data Mining options<br />
<br />
<br />
Here, I’ve inserted another new row into the OB base table. That means (as we saw earlier) the materialised view is now stale and cannot be used for query rewrite purposes. When I therefore re-issue the hinted select statement, this time it raises an error... and unceremoniously dumps me back to the operating system whilst it’s at it, you might note. I’m not convinced that the hint should actually cause a session to die like that -to my way of thinking, that's a bit drastic!- but at least the base table wasn’t actually used for the query!<br />
That is a fairly major new feature in some respects: what we have here is, for the first time, a mechanism which actively prevents the use of the base table for the query (in 9i, we only had a mechanism to prevent the use of the materialised view). Given that materialised views are likely to be created as summaries and aggregates of huge tables in a data warehouse, the ability to prevent an inadvertent access to the huge base tables is capable of saving a massive amount of unwanted/unnecessary work on the database. Definitely a good thing, therefore. Just be warned that if a query could potentially be re-written so that it refers to two or more materialised views, and the optimiser successfully uses just one of those materialised views, that counts as a successful re-write, and thus the error isn’t triggered... even though by not accessing the other materialised view, the query is now (presumably) doing massively expensive access to at least one base table. Still: half a loaf is better than no bread whatever, so let’s not complain.<br />
3.2 More Fast Refreshes<br />
As I’ve mentioned elsewhere in this article, it has always been trickier than it perhaps should have been to persuade a materialised view to fast refresh. All sorts of little things would stop them being able to do so: in 8i, for example, if you used the SQL keyword UNION in your materialised view’s defining select statement, that was enough to kill fast refreshing in its tracks.<br />
That particular restriction on when you could fast refresh was lifted in 9i, and 10g continues the trend of removing obstacles which prevent fast refreshes from being possible. For example, a materialised view that joins a table to itself can now be fast refreshable; as can a materialised view built on a select statement that uses an inline view in the FROM clause of the defining select statement. And so on and on... I won’t bore you with the complete list of new occasions when fast refreshes are possible, because it’s tedious in the extreme, and they’re all accompanied by lots of caveats and provisos in any case.<br />
The general point, I think, should simply be left at this: more materialised views will be able to fast refresh in 10g than in any previous Oracle version. That’s good enough for me, at any rate.<br />
3.3 Reliable and Cheaper Refreshes<br />
Refreshing a materialised view is, almost by definition, an expensive job: we create them in the first place because the query they represent is considered expensive, yet refreshing them must inevitably require us to re-execute precisely that same expensive query! This can be extremely bad news indeed when you set about creating multiple summarisations of your base data -in other words, when you create multiple materialised views on the same base tables- because each and every one of them, when they refresh, will have to perform what we can assume will be massively expensive queries.<br />
At least, that was the way things were in 9i. Fortunately, the expense issue is somewhat alleviated in 10g because it is now possible for a materialised view to be refreshed by having the query it is built on re-written so that the data can come from other materialised views rather than direct from the expensive-to-query base tables.<br />
The subtlety of this point is easily lost, so I’ll reiterate it in case you missed it! A materialised view is usually created as a selection of columns from base tables, but the optimiser in 10g is willing to re-writing the very query that defines the contents of materialised view so that fresh data can be sourced from other materialised views, not the base tables themselves.<br />
If a table has several materialised views built on it, for example, each a different form of summarisation compared with the rest (imagine summarising sales data by day, week, month, quarter, state, region, store, salesman and so on), then potentially only one of them has to be refreshed by going back to the base table itself: all the others could conceivably get their fresh data from that one source-refreshed materialised view. If so, that would make refreshing the bulk of the materialised views enormously cheaper. Excellent news!<br />
Naturally, there’s a catch! To know that a summarisation of sales data by week (say) can give you the data needed to refresh a summarisation of sales by month, you have to know that weeks are related to months. Oracle has no such chronological understanding innately -but you’ve been able to tell it about such relationships for many versions by creating objects called dimensions. Trouble is, dimensions are not enforced in the database: they merely declare that a relationship exists: nobody and nothing actually bothers to check that it’s true. Similarly, it has long been possible to create foreign and primary key constraints which aren’t actually enabled, but which are merely flagged as being ‘reliable’ (by setting the RELY flag for them, in fact!). That is another form of merely declaring a relationship or quality to exist, rather than having the database prove it for certain. Well, the catch with materialised views being able to refresh themselves from other materialised views is that -by default- they will ignore any relationship that is merely declared to exist. Unless it’s strictly enforced (and dimensions, for example, simply cannot be), then the optimiser won’t use that relationship to determine whether a query rewrite for refresh purposes can take place.<br />
Put into simple words, that means a materialised view of sales by month would not be able to use a materialised view summarising sales by week as its source for a refresh because the dimension telling Oracle that weeks and months are related is ignored by the optimiser for the refresh operation.<br />
That is good in one sense. An enforced relationship is known, absolutely, to be accurate and reliable. A trusted relationship is only as reliable as the DBA that set the RELY flag! If a materialised view were to use a trusted relationship as the basis of determining where it should get its fresh data from, and that relationship was in fact badly defined, or violated left, right and centre by the data, then the contents of the refreshed materialised view would end up logically corrupt. Using only enforced relationships means that can’t happen.<br />
But the downside is that to definitely enforce a primary key, for example, you’re going to need an index: and in a Data Warehouse, that’s not going to be a trivial little index. It’s going to be a thumping great monster that consume huge amounts of disk space, and lots of CPU cycles as it is created and maintained. And as for dimensions: well, they are only ever trusted relationships, so the default, strict policy means they can never influence the refresh mechanism.<br />
Fortunately, this behaviour is configurable, otherwise the new feature of materialised views refreshing other materialised views would be a pretty lame one. So, although the use of only genuinely-enforced relationships is the default behaviour in 10g, it can be changed like so:<br />
create materialized view SALESMV <br />
refresh complete on demand <br />
using trusted constraints <br />
as select weekno, sum(sales) <br />
from sales <br />
group by weekno;<br />
...and the magic words there are USING TRUSTED CONSTRAINTS. That’s the opposite of the default setting, which is USING ENFORCED CONSTRAINTS.<br />
Trying to sum up a subtle and complex point in as few words as possible, therefore: ENFORCED CONSTRAINTS is the default setting, and means materialised view refreshes only use totally, 100% reliable information as the source of their data when they get refreshed. But some relationships (dimensions are the classic example) can never be used to influence the query rewriting process with this set. And other relationships may be very costly to genuinely enforce (indexes on primary keys, for example). You gain reliability at the cost of performance, basically.<br />
If you use TRUSTED CONSTRAINTS, by contrast, then things like dimensions really can be used to influence the rewriting process, and performance will probably be better without massive indexes (for example) slowing you down. Rewrites are more likely to happen at this setting, too, so maintenance of the materialised views themselves will be quicker and cheaper. But this setting does mean, remember, that the contents of the materialised views are only as good as the competence of the DBA who declares relationships to be trustworthy in the first place. If he or she gets it wrong, your materialised views are effectively corrupt and they’ll need to be rebuilt from scratch at some point (which is, of course, a very expensive operation).KrChowdaryhttp://www.blogger.com/profile/00666737573542809533noreply@blogger.com0