In the previous article in this series on the opt_estimate() hint I mentioned the “query_block” option for the hint. If you can identify a specify query block that becomes an “outline_leaf” in an execution plan (perhaps because you’ve deliberately given an query block name to an inline subquery and applied the no_merge() hint to it) then you can use the opt_estimate() hint to tell the optimizer how many rows will be produced by that query block (each time it starts). The syntax of the hint is very simple:
opt_estimate(@{query block name} query_block rows={number of rows})
As with other options for the hint, you can use scale_rows=, min=, max= as alternatives (the last seems to be used in the code generated by Oracle for materialized view refreshes) but the simple “rows=N” is likely to be the most popular. In effect it does the same as the “non-specific” version of the cardinality() hint – which I’ve suggested from time to time as a way of telling the optimizer the size of a data set in a materialized CTE (“with” subquery), e.g.
set serveroutput off
with demo as (
select /*+
qb_name(mat_cte)
materialize
cardinality(@mat_cte 11)
-- opt_estimate(@mat_cte query_block rows=11)
*/
distinct trunc(created) date_list
from all_objects
)
select * from demo
;
select * from table(dbms_xplan.display_cursor);
Regardless of whether you use the opt_estimate() or cardinality() hint above, the materialized temporary table will be reported with 11 rows. (Note that in this case where the hint is inside the query block it applies to the “@mat_cte” isn’t necessary).
In the previous article I generated some data with a script called opt_est_gby.sql to show you the effects of the group_by and having options of the opt_estimate() hint and pointed out that there were case where you might also want to include the query_block option as well. Here’s a final example query showing the effect, with the scale_rows feature after creating a table t2 as a copy of t1 but setting pctfree 75 (to make a tablescan more expensive) and creating an index on t2(id):
create table t2 pctfree 75 as select * from t1;
create index t2_i1 on t2(id);
select
t2.n1, t1ct
from
t2,
(
select /*+
qb_name(main)
opt_estimate(@main group_by scale_rows=4)
opt_estimate(@main having scale_rows=0.4)
opt_estimate(@main query_block scale_rows=0.5)
*/
mod(n1,10), count(*) t1ct
from t1
group by
mod(n1,10)
having
count(*) > 100
) v1
where
t2.id = v1.t1ct
;
--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 8 | 168 | 27 (8)| 00:00:01 |
| 1 | NESTED LOOPS | | 8 | 168 | 27 (8)| 00:00:01 |
| 2 | NESTED LOOPS | | 8 | 168 | 27 (8)| 00:00:01 |
| 3 | VIEW | | 8 | 104 | 10 (10)| 00:00:01 |
|* 4 | FILTER | | | | | |
| 5 | HASH GROUP BY | | 8 | 32 | 10 (10)| 00:00:01 |
| 6 | TABLE ACCESS FULL | T1 | 3000 | 12000 | 9 (0)| 00:00:01 |
|* 7 | INDEX RANGE SCAN | T2_I1 | 1 | | 1 (0)| 00:00:01 |
| 8 | TABLE ACCESS BY INDEX ROWID| T2 | 1 | 8 | 2 (0)| 00:00:01 |
--------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
4 - filter(COUNT(*)>100)
7 - access("T2"."ID"="V1"."T1CT")
I’ve inlined the last query (with the two opt_estimate() hints) that I used in the previous article, and added a third opt_estimate() hint to that inline view. In this case I didn’t have to add a no_merge() hint because the numbers worked in my favour but to be safe in a production environment that’s a hint that I should have included.
You may recall that the hash group by on its own resulted in a prediction of 200 rows, and with the having clause the prediction dropped to 10 rows (standard 5%). With my three opt_estimate() hints in place I should see the effects of the following arithmetic:
group by 200 * 4 = 800 having 5% of 800 * 0.4 = 16 query block 16 * 0.5 = 8
As you can see, the cardinality prediction for the VIEW operation is, indeed, 8 – so the combination of hints has worked. It’s just a shame that we can’t see the three individual steps in the arithmetic as we walk the plan.
A Warning
As always I can only repeat – hinting is not easy; and “not easy” usually translates to “not stable / not safe” (and thanks to a Freudian slip while typing: “not sage”. You probably don’t know how do it properly, except in the very simplest cases, and we don’t really know how Oracle is interpreting the hints (particularly the undocumented ones). Here’s an example of how puzzling even the opt_estimate(query_block) hint can be – as usual starting with some data:
rem rem Script: opt_estimate_2.sql rem Author: Jonathan Lewis rem Dated: Aug 2017 rem create table t1 as select * from all_objects; create table t2 as select * from all_objects;
As you can see, I’ve been a bit lazy with this example (which I wrote a couple of years ago) and it uses all_objects as a convenient source of data. Unfortunately this means you won’t necessarily be able to reproduce exactly the results I’m about to show you, which I did on a small instance of 12.2.0.1. I’m going to examine four versions of a simple query which
- restricts the rows from t1,
- finds the unique set of object_types in that subset of t1
- then joins to t2 by object_type
select
/*+
qb_name(main)
*/
t2.object_id, t2.object_name, created
from (
select /*+ qb_name(inline) */
distinct object_type
from t1
where
created >= date'2017-03-01'
) v1,
t2
where
t2.object_type = v1.object_type
;
select
/*+
qb_name(main)
merge(@inline)
*/
t2.object_id, t2.object_name, created
from (
select /*+ qb_name(inline) */
distinct object_type
from t1
where
created >= date'2017-03-01'
) v1,
t2
where
t2.object_type = v1.object_type
;
select
/*+
qb_name(main)
opt_estimate(@inline query_block rows=14)
*/
t2.object_id, t2.object_name, created
from (
select /*+ qb_name(inline) */
distinct object_type
from t1
where
created >= date'2017-03-01'
) v1,
t2
where
t2.object_type = v1.object_type
;
select
/*+
qb_name(main)
merge(@inline)
opt_estimate(@inline query_block rows=14)
*/
t2.object_id, t2.object_name, created
from (
select /*+ qb_name(inline) */
distinct object_type
from t1
where
created >= date'2017-03-01'
) v1,
t2
where
t2.object_type = v1.object_type
;
The first version is my unhinted baseline (where, in my case, Oracle doesn’t use complex view merging), the second forces complex view merging of the inline aggregate view, then queries 3 and 4 repeat queries 1 and 2 but tell the optimizer that the number of distinct object_type values is 14 (roughly half the actual in may case). But there is an oddity in the last query – I’ve told the optimizer how many rows it should estimate for the inline view but I’ve also told it to get rid of the inline view and merge it into the outer query block; so what effect is that going to have? My hope would be that the hint would have to be ignored because it’s going to apply to a query block that doesn’t exist in the final plan and that makes it irrelevant and unusable. Here are the four execution plans:
-----------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 61776 | 4464K| 338 (7)| 00:00:01 |
|* 1 | HASH JOIN | | 61776 | 4464K| 338 (7)| 00:00:01 |
| 2 | VIEW | | 27 | 351 | 173 (9)| 00:00:01 |
| 3 | HASH UNIQUE | | 27 | 486 | 173 (9)| 00:00:01 |
|* 4 | TABLE ACCESS FULL| T1 | 59458 | 1045K| 164 (4)| 00:00:01 |
| 5 | TABLE ACCESS FULL | T2 | 61776 | 3680K| 163 (4)| 00:00:01 |
-----------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("T2"."OBJECT_TYPE"="V1"."OBJECT_TYPE")
4 - filter("CREATED">=TO_DATE(' 2017-03-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
--------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
--------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 61776 | 5308K| | 1492 (2)| 00:00:01 |
| 1 | VIEW | VM_NWVW_1 | 61776 | 5308K| | 1492 (2)| 00:00:01 |
| 2 | HASH UNIQUE | | 61776 | 5489K| 6112K| 1492 (2)| 00:00:01 |
|* 3 | HASH JOIN RIGHT SEMI| | 61776 | 5489K| | 330 (5)| 00:00:01 |
|* 4 | TABLE ACCESS FULL | T1 | 59458 | 1045K| | 164 (4)| 00:00:01 |
| 5 | TABLE ACCESS FULL | T2 | 61776 | 4403K| | 163 (4)| 00:00:01 |
--------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("T2"."OBJECT_TYPE"="OBJECT_TYPE")
4 - filter("CREATED">=TO_DATE(' 2017-03-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
-----------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 32032 | 2314K| 338 (7)| 00:00:01 |
|* 1 | HASH JOIN | | 32032 | 2314K| 338 (7)| 00:00:01 |
| 2 | VIEW | | 14 | 182 | 173 (9)| 00:00:01 |
| 3 | HASH UNIQUE | | 14 | 252 | 173 (9)| 00:00:01 |
|* 4 | TABLE ACCESS FULL| T1 | 59458 | 1045K| 164 (4)| 00:00:01 |
| 5 | TABLE ACCESS FULL | T2 | 61776 | 3680K| 163 (4)| 00:00:01 |
-----------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("T2"."OBJECT_TYPE"="V1"."OBJECT_TYPE")
4 - filter("CREATED">=TO_DATE(' 2017-03-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
--------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes |TempSpc| Cost (%CPU)| Time |
--------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 14 | 1232 | | 1492 (2)| 00:00:01 |
| 1 | VIEW | VM_NWVW_1 | 14 | 1232 | | 1492 (2)| 00:00:01 |
| 2 | HASH UNIQUE | | 14 | 1274 | 6112K| 1492 (2)| 00:00:01 |
|* 3 | HASH JOIN RIGHT SEMI| | 61776 | 5489K| | 330 (5)| 00:00:01 |
|* 4 | TABLE ACCESS FULL | T1 | 59458 | 1045K| | 164 (4)| 00:00:01 |
| 5 | TABLE ACCESS FULL | T2 | 61776 | 4403K| | 163 (4)| 00:00:01 |
--------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access("T2"."OBJECT_TYPE"="OBJECT_TYPE")
4 - filter("CREATED">=TO_DATE(' 2017-03-01 00:00:00', 'syyyy-mm-dd hh24:mi:ss'))
The first plan tells us that most of the rows in t1 have created > 1st March 2017 and there are (estimated) 27 distinct values for object_type; and there are 61,776 rows in t2 (which is basically the same as t1), and none of them are eliminated by the join on object_type from the inline view.
The second plan (with the forced complext view merging) shows Oracle changing the view with “distinct” into a (right) semi-join between t2 and t1 with the internal view name of VM_NWVW_1 – and the cardinality is correct.
The third plan shows that my hint telling the optimizer to assume the original inline view produces 14 rows has been accepted and, not surprisingly, when we claim that we have roughly half the number of object_type values the final estimate of rows in the join is roughly halved.
So what happens in the fourth plan when our hint applies to a view that no longer exists? I think the optimizer should have discarded the hint as irrelevant the moment it merged the view. Unfortunately it seems to have carried the hint up into the merged view and used it to produce a wildly inaccurate estimate for the final cardinality. If this had been a three-table join this is the sort of error that could make a sensible hash join into a third table become an unbelievably stupid nested loop join. If you had thought you were doing something incredibly clever with (just) the one opt_estimate() hint, the day might come when a small change in the statistics resulted in the optimizer using a view merge strategy you’d never seen before and producing a catastrophic execution plan in (say) an overnight batch that then ran “forever”.
Hinting is hard, you really have to be extremely thorough in your hints and make sure you cover all the options that might appear. And then you might still run into something that looks (as this does) like a bug.
Footnote
Here’s a closing thought: even if you manage to tell the optimizer exactly how many rows will come out of a query block to be joined to the next table in the query, you may still get a very bad plan unless you can also tell the optimizer how many distinct values of the join column(s) there are in that data set. Which means you may also have to learn all about the (even more undocumented) column_stats() hint.