The opt_estimate hint is one of many that shouldn’t be used in end-user code and isn’t officially documented. Nevertheless – like so many other hints – it’s a hint that is hard to ignore when you see it floating around the code generated by the Oracle software. This note is prompted by a twitter question from fellow Oak Table member Stefan Koehler asking whether the hint’s index_filter parameter worked. Checking my library I knew the answer was yes – so after a quick exchange on twitter I said I’d write up a short note about my example, and this is it.
Although the hint is not one that you should use it’s worth writing this note as a reminder of the significance to index range scans of the access predicates and filter predicates that Oracle reports in the predicate section of an execution plan.
When a query does an index range scan it’s going to walk through a (logically) consecutive set of index leaf blocks looking at each individual index entry in turn (and those index entries will be correctly “sorted” within the leaf block) to see if it should use the rowid it finds there to visit the table. For “perfect” use of an index Oracle may be able to identify the starting and ending positions it needs in the index and know that it should use every rowid in between to visit the table – there will no “wasted”examinations of index entries on the way. In a query involving a multi-column index and multiple predicates, however, Oracle might have to use predicates on the first column(s) of the index to identify the starting and ending positions, but use further predicates on later columns in the index to decide whether or not to use each index entry to visit the table.
The predicates that Oracle can use to identify the range of leaf blocks it should visit are called access predicates, and the predicates that Oracle can use to further eliminate rowids as it walks along the leaf blocks are called filter predicates.
The simplest way to demonstrate this is with a query of the form: “Index_Column1 = … and Index_Column3 = …”, and that’s what I’ll be using in my model:
rem
rem Script: opt_est_ind_filter.sql
rem Author: Jonathan Lewis
rem
rem Last tested
rem 11.2.0.4
rem 10.2.0.5
rem
create table t1
nologging
as
with generator as (
select
rownum id
from dual
connect by
level <= 1e4 -- > comment to bypass WordPress format issue
)
select
rownum id,
mod(rownum - 1,100) n1,
rownum n2,
mod(rownum - 1, 100) n3,
lpad(rownum,10,'0') v1,
lpad('x',100,'x') padding
from
generator v1,
generator v2
where
rownum <= 1e6 -- > comment to bypass WordPress formatting issue
;
create index t1_i1 on t1(n1,n2,n3) nologging;
begin
dbms_stats.gather_table_stats(
ownname => user,
tabname => 'T1',
method_opt => 'for all columns size 1'
);
end;
/
select leaf_blocks from user_indexes where index_name = 'T1_I1';
The number of leaf blocks in the index was 3,062.
I’ve defined n1 and n3 to match, and for any value between 0 and 99 there are 10,000 rows in the table where n1 and n3 hold that value. However, in the absence of a column group defined on (n1, n3), the optimizer is going to use its standard “no correlation” arithmetic to decide that there are 10,000 possible combinations of n1 and n3, and 100 rows per combination. Let’s see what this does for a simple query:
set autotrace traceonly explain
select count(v1)
from t1
where n1 = 0 and n3 = 0
;
set autotrace off
--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 17 | 134 (1)| 00:00:01 |
| 1 | SORT AGGREGATE | | 1 | 17 | | |
| 2 | TABLE ACCESS BY INDEX ROWID| T1 | 100 | 1700 | 134 (1)| 00:00:01 |
|* 3 | INDEX RANGE SCAN | T1_I1 | 100 | | 34 (3)| 00:00:01 |
--------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("N1"=0 AND "N3"=0)
filter("N3"=0)
The plan shows an index range scan where n3=0 is used as a filter predicate and n1=0 (with a tiny bit of extra accuracy from the n3=0) is used as the access predicate, and the optimizer has calculated that 100 rowids will be retrieved from the index and used to find 100 rows in the table.
The cost of the range scan is 34: The optimizer’s estimate is that the scale of the initial access to the index will be due to the predicate n1 = 0 which is responsible for 1% of the index – giving us 3,062/100 leaf blocks (rounded up). Added to that there will be a little extra cost for the trip down the blevel of the index and a little extra for the CPU usage.
Now let’s tell the optimizer that its cardinality estimate is out by a factor of 25 (rather than 100 we actually know it to be) in one of two different ways:
prompt ============================
prompt index_scan - scale_rows = 25
prompt ============================
select
/*+
qb_name(main)
index(@main t1(n1, n2, n3))
opt_estimate(@main index_scan t1, t1_i1, scale_rows=25)
*/
count(v1)
from t1
where n1 = 0 and n3 = 0
;
prompt ==============================
prompt index_filter - scale_rows = 25
prompt ==============================
select
/*+
qb_name(main)
index(@main t1(n1, n2, n3))
opt_estimate(@main index_filter t1, t1_i1, scale_rows=25)
*/
count(v1)
from t1
where n1 = 0 and n3 = 0
;
In both examples I’ve hinted the index to stop the optimizer from switching to a tablescan; but in the first case I’ve told Oracle that the entire index range scan has to be scaled up by a factor of 25 while in the second case I’ve told Oracle that its estimate due to the final filter has to be scaled up by a factor of 25. How does this affect the costs and cardinalities of the plans:
============================
index_scan - scale_rows = 25
============================
--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 17 | 3285 (1)| 00:00:17 |
| 1 | SORT AGGREGATE | | 1 | 17 | | |
| 2 | TABLE ACCESS BY INDEX ROWID| T1 | 100 | 1700 | 3285 (1)| 00:00:17 |
|* 3 | INDEX RANGE SCAN | T1_I1 | 2500 | | 782 (2)| 00:00:04 |
--------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("N1"=0 AND "N3"=0)
filter("N3"=0)
==============================
index_filter - scale_rows = 25
==============================
--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 17 | 2537 (1)| 00:00:13 |
| 1 | SORT AGGREGATE | | 1 | 17 | | |
| 2 | TABLE ACCESS BY INDEX ROWID| T1 | 100 | 1700 | 2537 (1)| 00:00:13 |
|* 3 | INDEX RANGE SCAN | T1_I1 | 2500 | | 34 (3)| 00:00:01 |
--------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("N1"=0 AND "N3"=0)
filter("N3"=0)
In both cases the cardinality estimate has gone up by a factor of 25 for the index range scan. Notice, though, that the optimizer is now suffering from cognitive dissonance – it “knows” that it’s going to get 2,500 rowids to use to visit the table, it “knows” there are no extra predicates to eliminate rows from the table when it gets there, but it also “knows” that it’s going to find only 100 rows. Messing around with opt_estimate() and cardinality() hints is difficult even for fairlyl trivial cases – like all hinting, it takes more than one or two hints to achieve the result you want.
More significantly for the purposes of this note, are the costs. When we use the index_filter parameter the optimizer still thinks it’s going to access the same number of leaf blocks and the only correction it has to make is the number of rowids it finds in those blocks – so the index range scan cost hasn’t changed (though I suppose in some cases it might change slightly due to increased CPU costs). When we use the index_scan parameter the optimizer scales up its estimate of the number of leaf blocks (hence cost), which we can see in the figures 782 / 25 = 31.28. (Without going into the trace file and checking exact details that’s close enough to the previously reported 34 for me to think it’s allowing for 25 times the number of leaf blocks plus a chunk more CPU)
Conclusion
As I said at the outset, opt_estimate() really isn’t a hint you should be playing with, but I hope that this note has helped to shed some light on the significance of access predicates and filter predicates in relation to the costs of index range scans.
Footnote
There were two significant details in the notes I had in my script. First was the frequency of the expression “it looks as if” – which is my shorthand for “I really ought to do some more tests before I publish any conclusions”; second was that my most recent testing had been on 10.2.0.5 (where the results were slightly different thanks to sampling in the statistics). Given that Stefan Koehler had mentioned 11.2.0.3 as his version I ran up an instance of 11.1.0.7 – and found that the index_filter example didn’t scale up the cardinality – so maybe his problem is a version problem.