Fast way to copy data into a table

I have this test script I use to remind myself of the fastest way I’ve found to copy data into an Oracle table.  It shows three things:

  1. Fast way to create a new table populating with data
  2. Fast way to create an index on a large table
  3. Fast way to insert a lot of data into a table

The essence of this is that you want everything done nologging, in parallel, and using direct path.

In this example we have two tables – source and target.  We are copying data from source to target.  Here is how we create a new table called target with all the data in source:

create table target parallel 4 nologging as 
select /*+parallel(source 4) */ * from source;

The parallel 4 makes the inserts into the target table be done with 4 parallel processes.  Nologging prevents these changes from going to the redo logs for recovery.  The hint – parallel (source 4) – causes the source table to be queried with four parallel processes.  What makes this so fast is that both the select and insert are done in parallel.  Also, in this “nologging create table as select” scenario the database uses direct path inserts which loads the blocks up in bulk rather than one row at a time.

Next I build an index on the target table:

create index target_idx on target (KEY1) parallel 4 nologging;

This is fast because again you are not logging to the redo logs and you are using 4 parallel processes.

Lastly, and most important this is how to insert a lot of data into an existing table very quickly:

ALTER SESSION ENABLE PARALLEL DML;

insert into target select /*+parallel(source 4) */* from source;

This assumes your target table was created with parallel 4 just as in our previous example.  The alter session command is essential because without it the database will not insert the data in parallel.  If you do the alter session and the target table has parallel greater than 1 then your insert will be in parallel and it will use direct path load.  But, you also have to make sure your query is done with equal parallelism.  In this example the target table was parallel 4 so the parallel hint on the select makes the select parallel 4.

Here is the plan for the insert statement:

The key here is the LOAD AS SELECT which tells you it is using direct path inserts.  Also, the PARALLEL_… on the LOAD tells you it is loading in parallel.  Lastly the PX BLOCK ITERATOR tells you it is querying the table in parallel.  You can’t have any of these missing or else you won’t have the fastest way to do the insert: parallel read and write and direct path insert.

– Bobby

Posted in Uncategorized | 5 Comments

Script to show change in query execution

Click here for a simple but useful script that shows how the execution of a given query has changed over time. (current version).

This script just queries DBA_HIST_SQLSTAT for a given sql_id. A sql_id corresponds to a single SQL query.  I’ve changed the units to milliseconds because I think in terms of disk reads which take a few milliseconds each.

Here is the text of the query:

select ss.sql_id,
sn.END_INTERVAL_TIME,
ss.executions_delta,
ELAPSED_TIME_DELTA/(executions_delta*1000) "Elapsed Average ms",
CPU_TIME_DELTA/(executions_delta*1000) "CPU Average ms",
IOWAIT_DELTA/(executions_delta*1000) "IO Average ms",
BUFFER_GETS_DELTA/executions_delta "Average buffer gets",
DISK_READS_DELTA/executions_delta "Average disk reads"
from DBA_HIST_SQLSTAT ss,DBA_HIST_SNAPSHOT sn
where ss.sql_id = '7gnz0zx8070n9'
and ss.snap_id=sn.snap_id
and executions_delta > 0
order by ss.snap_id,ss.sql_id;

I’ve hard coded the sql_id as 7gnz0zx8070n9 but you would change that to the sql_id of the query you want to investigate.  You can get the sql_id from an AWR report.

Typical output:

SQL_ID        END_INTERVAL_TIME         EXECUTIONS_DELTA Elapsed Average ms CPU Average ms IO Average ms Average buffer gets Average disk reads
------------- ------------------------- ---------------- ------------------ -------------- ------------- ------------------- ------------------
7gnz0zx8070n9 14-MAY-12 12.00.04.795 PM               11         84726.4826     29198.7186    1606.64882          37011.7273         2950.72727
7gnz0zx8070n9 14-MAY-12 01.00.31.734 PM               18         176928.853     40828.9464    11324.6743          288660.056         15410.1111
7gnz0zx8070n9 14-MAY-12 02.00.05.723 PM               11         213522.486      58663.698    16163.4455          576307.545         24678.5455
7gnz0zx8070n9 14-MAY-12 03.00.43.561 PM               27         224199.385     61645.2061    7544.87026              898836         4479.62963
7gnz0zx8070n9 14-MAY-12 04.00.04.411 PM                5         104387.372     60219.0786     7351.9572           1546004.2             4287.6
7gnz0zx8070n9 21-MAY-12 12.28.46.943 PM               15         72550.5727     28886.7986     3299.5736               67194         5001.86667
7gnz0zx8070n9 21-MAY-12 01.09.43.427 PM                9         166778.289     47970.0616    12468.9748          266338.111               7342
7gnz0zx8070n9 21-MAY-12 02.00.54.605 PM                7         335266.608     68207.2234    15403.1189          457152.143         17968.8571
7gnz0zx8070n9 21-MAY-12 03.00.42.256 PM               12         395749.327     57734.2631    25077.6642            701764.5              19920
7gnz0zx8070n9 21-MAY-12 04.01.00.514 PM               23         271734.101     66244.0698    7713.01161          2645183.43         3724.08696
7gnz0zx8070n9 21-MAY-12 05.00.30.643 PM                6         93118.3805     58891.7195      8212.002          1142748.67               3662

In this example we compare the May 14 run to the May 21 run of the same query.  The output shows two things:

1 – The query ran about the same number of times both weeks

2 – The run time, CPU and disk I/O per execution were about the same

If the number of executions for one week over the next were different that would indicate a change in query volume.  If the per execution times were different that would indicate a change in the way each query ran.

Pretty simple, but helpful when tracking down whether a query is behaving the same today as the last time it ran.

– Bobby

Posted in Uncategorized | Leave a comment

Script to extract plans from AWR

I use this script all the time to get the plans for a given SQL statement I’ve identified from an AWR report:

getplans.sql

Recently I’ve found out that you can do something like this with the delivered awr script $ORACLE_HOME/rdbms/admin/awrsqrpt.sql

But, I like my version because it dumps out a text version of the plans and it shows me the last AWR snapshot that had the given plan.  This is helpful if you want to tell when the plan changed.

Here is how you run it:

SQL> @getplans j3h4j56k6j

Where j3h4j56k6j is a sql_id you’ve found in your AWR report.

– Bobby

Posted in Uncategorized | 1 Comment

No detailed documentation for AWR report

One of my coworkers asked a really good question:
“Where is the documentation that explains all the sections of the AWR report?”

The answer is that there is no comprehensive section-by-section documentation of an AWR report.

But, there is a note on Oracle’s support site that gives examples of how to use AWR reports:

FAQ: How to Use AWR reports to Diagnose Database Performance Issues [ID 1359094.1]

The AWR report contains the difference between two snapshots of the V$ views.  So, to understand what is listed on a section of the AWR report you must find the V$ view it is based on.  The snapshots of the V$ views are visible in the DBA_HIST views.

Here is an example of a mapping of a section of an AWR report to the V$ view:

“SQL ordered by Elapsed Time” -> DBA_HIST_SQLSTAT -> V$SQL

You find this mapping by reviewing the DBA_HIST and V$ views and comparing what they contain with what the AWR report contains.  You make a guess based on the contents of the report and the descriptions of the columns.

I had discussed this and appreciate the input from the people on these two forum threads:

Orafaq thread

OTN thread

– Bobby

Posted in Uncategorized | 3 Comments

Oracle Avengers

I couldn’t help notice the Oracle product placement in The Avengers movie as I watched it for the second time in two weeks with my eleven year old daughter.  I was wondering what sort of systems they were supposed to be, but Oracle has a page on their site all about this:

http://www.oracle.com/us/theavengers/index.html – NO LONGER EXISTS

It is good to know that the superheroes of the world run on Oracle 🙂

– Bobby

Posted in Uncategorized | Leave a comment

Updated resources page

I’ve updated the “Resources” page on my blog to include all my old presentation, papers, and scripts that I’ve published before.  I used to have a free Geocities blog but that disappeared.  I’ve been waiting to see if this new one – which I’m paying a small amount for – is stable before loading my old stuff.  It looks good.

Here is my resources page. You  can also find this off the menu on my blog.

– Bobby

Posted in Uncategorized | Leave a comment

Updated Intro to SQL tuning paper with example script and log

I’ve done an update on my introduction to SQL tuning paper.

Updated paper as of 05/29/2012

Example SQL script and its log – in zip with paper

I’ve added cardinality hint and breakup of a SQL query into smaller pieces with global temporary tables holding the results.  Hopefully I’ve cleaned up a few things.  Also put section headers to make the outline clear – intro, join order, join method, access method, conclusion.

– Bobby

Posted in Uncategorized | Leave a comment

Updated “about” page on my blog

I updated my very generic “about” page on my blog.

Check it out and get to know me and my blog better:

https://www.bobbydurrettdba.com/about/

– Bobby

Posted in Uncategorized | Leave a comment

Submitted two talks to the East Cost Oracle Users Conference

Submitted two talks to the East Cost Oracle Users Conference:

http://www.eastcoastoracle.org/ssl/presentations.asp – NO LONGER EXISTS

First one is my Exadata talk that I gave at work.  Second is new Intro to SQL Tuning talk I’m preparing for my coworkers.  I should know some time in July if one of these is accepted.

Here are the abstracts:

Exadata Distinctives

This presentation focuses on the new features of Exadata for improving Oracle query performance.  It explains Exadata smart scans. We include diagrams that compare and contrast normal Oracle database servers and the flow of data from disk through memory buffers and networking to the end user with these same things on Exadata.  In particular we show that smart scans bypass the high speed memory of the buffer cache on the database server but use the lower speed flash memory on the cell servers as a form of buffer cache which has tradeoffs in terms of the sorts of queries that work best in each case.

This talk includes observations from real world experience with a large custom written datawarehouse and an implementation of OBIA including tuning this packaged PeopleSoft datawarehouse with minimal application change.  The query tuning focuses on encouraging SQL queries to use full scans to take advantage of the automatic tuning features of the smart scans.  It also shows how to bypass the Exadata features where normal Oracle query processing is more efficient.

The presentation concludes with notable concerns for any new adopter of Exadata including issues with tables that have more than 255 columns, performance problems of sort segments in RAC databases with many nodes, and stability issues that come with a new platform.

Introduction to Oracle SQL Tuning

At the core of SQL tuning are three fundamental concepts – join order, join method, and access method. This presentation will explain these three concepts and show that tuning SQL boils down to making the proper choice in each category. It is hard to imagine an area of SQL tuning that doesn’t touch on one or more of these three areas and in any case the presenter knows from experience that tuning the join order, join methods, and access methods consistently leads to fantastic results.

We will demonstrate how to tune an Oracle SQL query by examining and manipulating the join order, join methods, and access methods used in the query’s execution plan. First you must query the tables to determine the true number of rows returned from each table
and from each likely join. Then you identify which join order, join method, or access method chosen by the optimizer is inefficient. Then you use one of the suggested
methods to change the optimizer’s choice to be the efficient one. These methods include
gathering optimizer statistics, hints, setting init parameters, and adjusting the parallel
degree. Lastly, you must test your changed plan to make sure the new combination of join order, join methods and access methods actually improve the performance.

– Bobby

Posted in Uncategorized | 2 Comments

New SQL Tuning paper draft

I’m working on a paper titled “Introduction to SQL Tuning”.  I’d like to have this be the start of a training program at work for DBAs and developers.  I’ve completed a draft of a paper on this topic.  If anyone out there wants to read it and give me feedback here is the current version:

https://www.bobbydurrettdba.com/uploads/introtosqltuningdraft05212012.pdf

You can reach me at bobby@bobbydurrettdba.com

– Bobby

Posted in Uncategorized | Leave a comment