11g stats not gathered on empty subpartitions

See this testcase run on 11.2.0.3:

https://www.bobbydurrettdba.com/uploads/emptystats.zip

If you have a subpartitioned table – at least of the type that I tested – the Oracle delivered stats job will leave the statistics empty (NULL) for empty subpartitions instead of setting them to zero.  If you manually gather statistics on the same table you will have zero stats on the empty subpartitions.

I can’t see any evidence of this causing a problem but it is disconcerting to see NULL stats when you are expecting the job to gather them.

– Bobby

Posted in Uncategorized | Leave a comment

Maximum active sessions exceeded

Here is a new script and example output:

https://www.bobbydurrettdba.com/uploads/maxactive.zip

Use this query if you are using resource manager and have set a maximum number of active sessions.  This SQL reports the instance number and resource consumer group that have exceeded the maximum.

Here is the text of the query:

select
act.inst_id,act.resource_consumer_group,act.num_active,
pd.active_sess_pool_p1 max_active
from
(select inst_id,resource_consumer_group,count(*) num_active
from gv$session 
where status='ACTIVE' and
type='USER' and
program not like '%(P%)'
group by inst_id,resource_consumer_group) act,
DBA_RSRC_PLAN_DIRECTIVES pd,
gv$rsrc_plan pl
where
pd.plan=pl.name and
act.inst_id=pl.inst_id and
pl.is_top_plan='TRUE' and
act.resource_consumer_group=pd.group_or_subplan and
act.num_active > pd.active_sess_pool_p1;

– Bobby

P.S.  Modified query on 4/3/2012 – wasn’t excluding parallel query slaves correctly

Posted in Uncategorized | Leave a comment

FINAL_BLOCKING_INSTANCE and FINAL_BLOCKING_SESSION

Discovered these columns on the v$session and gv$session views:

FINAL_BLOCKING_INSTANCE and FINAL_BLOCKING_SESSION

These appear to be new in 11.2.  I checked an 11.1.0.7 instance and they weren’t there.  They were there in an 11.2.0.1 instance.  Interestingly an early version of the 11.2 documentation didn’t have them.  I have Oracle document E10820-03 on my hard drive and it didn’t have a reference to these columns, but the most current 11.2 docs do.

Anyway, I used these to determine which Oracle session was holding library cache locks.  In earlier versions I don’t believe that the v$/gv$session views were that helpful with library cache locks.  They worked with regular locks only.  So, check these out when you have a locking situation and don’t know which SID is blocking the others.

Current v$session documentation here:

https://docs.oracle.com/cd/E11882_01/server.112/e40402/dynviews_3016.htm#REFRN30223

– Bobby

Posted in Uncategorized | Leave a comment

Reducing size of connection pool to improve web application performance

I attended an Oracle sponsored day yesterday on Oracle’s engineered systems.  The main speaker was Tom Kyte who is a widely known Oracle performance expert.  After his talk was complete he graciously agreed to talk with me about performance issues with a web site that has hundreds of sessions to the database in a connection pool.  Tom said that the connection pool should be roughly twice the number of CPU cores and not many times that amount as they are in our case.  He recommended this Oracle produced youtube video that demonstrates that fewer sessions in a session pool produce better throughput:

http://www.youtube.com/watch?v=xNDnVOCdvQ0&feature=youtube_gdata_player

It isn’t immediately obvious to me why this is the case.

Tom Kyte recommended changing the application to use a smaller pool of connections or if that’s not possible to use shared servers.  In my case shared servers seems like a better option because it would take more effort to modify the application tier.  We could set the number of real connections to twice the number of cpus and allow the web servers to spin up hundereds of shared server connections instead of decidated server connections.  We need to test it to prove it out of course.  No guarantees until you test it in your own environment.

– Bobby

Posted in Uncategorized | 4 Comments

GATHER AUTO overrides preferences

If you use DBMS_STATS.GATHER_SCHEMA_STATS or DBMS_STATS.GATHER_DATABASE_STATS with options=> ‘GATHER AUTO’ then the preferences for the tables whose statistics are gathered are ignored.

See this testcase which includes a SQL*Plus script and log demonstrating that GATHER_SCHEMA_STATS  ignores a preference.  I’ve seen the same thing with GATHER_DATABASE_STATS but it wasn’t as quick to build a test case.

So, if you depend on table preferences don’t use GATHER AUTO.

– Bobby

Posted in Uncategorized | Leave a comment

Usergroup presentations

These are two Oracle Database usergroup presentations that I have done related to Oracle performance tuning.  Each has a Powerpoint presentation and a Word document.

First usergroup presentation – Profile of waitsand cpu

First usergroup paper – Profile of waits and cpu

Second usergroup paper – SQL tuning and cardinality

Second usergroup presentation – SQL tuning and cardinality

– Bobby

 

Posted in Uncategorized | Leave a comment

cell single block physical read

We starting seeing a ton of these waits on an Exadata system:

cell single block physical read

Normally the predominant wait is

cell smart table scan

Simple table full scans were doing the single block reads, whereas they normally would do smart scans.

Oracle support had us run this command to identify cell servers with “quarantined” plans:

cellcli -e list quarantine

They identified a cell server with several of these plans and had us run this command in cellcli:

drop quarantine all

Ideally you would patch up your cell servers, etc to the current release and this would reduce the number of quarantined plans.

Oracle note 1349167.1 contains an example of why a cell server would crash and cause a plan to be quarantined.

Evidently if you have a certain number of quarantined plans (6 I think) this fact becomes “visible” to the database servers and they start using single block reads instead of the multiblock smart scans.  Dropping the quarantined plans reduces the number of plans below the threshhold and the database goes back to doing smart scans.

– Bobby

Posted in Uncategorized | 15 Comments

Session cursor cache bug results in high latch waits?

We are hitting this bug:

Bug 6510615  REF CURSOR opened by PLSQL does not use session cursor cache

An application I work on does a ton of executions of packages with cursor variables returning ref cursors.  Apparently the session cursor cache normally reduces the use of the library cache latch – see Oracle note 62143.1 – Troubleshooting: Tuning the Shared Pool and Tuning Library Cache Latch Contention

During a peak time we saw a bunch of library cache latch waits (and cpu usage).  The top SQL was a query that is run as a REF CURSOR.  I don’t think that patching for the bug will resolve the issues completely – if the system is maxed out it will still be maxed out.  But, fixing the bug should reduce the use of the latch and allow the system to handle more concurrent access.

To prove this out I tested a simple ref cursor by running it 10,000 times against 10.2.0.3 and 11.2.0.1.  In 10.2.0.3 the cursor wasn’t cached.

10.2.0.3:

session cursor cache hits 4

11.2.0.1

session cursor cache hits 10004

I ran the same ref cursor test with 11.2.0.1 with these parameters:

ALTER SESSION SET SESSION_CACHED_CURSORS = 1000;

session cursor cache hits 9999

Elapsed: 00:00:01.00

ALTER SESSION SET SESSION_CACHED_CURSORS = 0;

Elapsed: 00:00:01.01

session cursor cache hits 0

So, in the 11.2.0.1 test turning off cached cursors added a small amount of run time and elminated the cache hits.

So, all this proves is that there is some advantage to session cached cursors and that this advantage is missing on 10.2.0.3 when using ref cursors and cursor variables.

Code to run a bunch of times with ref cursor:

DECLARE

TYPE CurType IS REF CURSOR;     
v_Cursor CurType;
TYPE RecType IS RECORD
(
DUMMY VARCHAR2(1)
);
Rec RecType;
innerloop number;
outerloop number;
BEGIN
innerloop := 0;
outerloop := 0;
LOOP
  outerloop := outerloop + 1;
  EXIT WHEN outerloop > 100000;
  TEST_PACKAGE.TEST_PROC(v_Cursor);
  LOOP
    FETCH v_Cursor into Rec;
    EXIT WHEN v_Cursor%NOTFOUND;
    innerloop := innerloop + 1;
  END LOOP;
END LOOP;
dbms_output.put_line('innerloop = '||innerloop);
dbms_output.put_line('outerloop = '||outerloop);
END;
/

Decided to do this from Java since the real app is java.

Results weren’t much different.

Here is the java:

CallableStatement call = 
thisConxn.prepareCall ("{call TEST_PACKAGE.TEST_PROC(?)}");
call.registerOutParameter (1, OracleTypes.CURSOR);
for (int i=0;i <= 10000; i++) {
   call.execute ();
   ResultSet theseResults = (ResultSet)call.getObject (1);
   while (theseResults.next()) {
   }
   theseResults.close();
}
call.close();
Here is the package:
CREATE OR REPLACE PACKAGE TEST_PACKAGE IS
TYPE CurType IS REF CURSOR;     
PROCEDURE TEST_PROC
(
  p_Cursor OUT CurType
);
END TEST_PACKAGE;
/
show errors
CREATE OR REPLACE PACKAGE BODY TEST_PACKAGE IS
PROCEDURE TEST_PROC
(
  p_Cursor OUT CurType
) IS
BEGIN
 OPEN p_Cursor FOR
 SELECT DUMMY FROM DUAL;
END TEST_PROC;
END TEST_PACKAGE;
/
Posted in Uncategorized | 3 Comments