One of the most irritating features of solving problems for clients is that the models I build to confirm my diagnosis and test my solutions often highlight further anomalies, or make me ask questions that might produce some useful answers to future problems.
Recently I had cause to ask myself if Oracle would push a filter subquery into the second tablescan of a hash join – changing a plan from this:
filter hash join table access full t1 table access full t2 table access by rowid t3 index range scan t3_i1
to this:
hash join table access full t1 filter table access full t2 table access by rowid t3 index range scan t3_i1
or, perhaps more likely, to this:
hash join table access full t1 table access full t2 table access by rowid t3 index range scan t3_i1
The final variation here is an example where the FILTER operation itself is swallowed up in line 3 of the plan, twisting the body of the plan in a way that makes the “first child first” rule of thumb lead to an incorrect interpretation. I’ve discussed this pattern of behaviour before, but in the earlier cases the “missing filter” has either applied to an index or to the first table of the hash join.
The type of query where the the strategy for pushing a filter subquery into the second table of a hash join might be appropriate would be something like the following (although in this simple case we’d probably expect Oracle to unnest the subquery and turn it into a semi-join):
select
t1.n1,
t2.n1
from
t1, t2
where
mod(t1.n1,100) = 0
and t2.id = t1.id -- join condition with a possible order t1 -> t2
and exists (
select -- subquery that could be pushed against t2
null
from t3
where t3.id = t2.n1
)
;
The benefit of using a filter subquery and pushing it would only appear in specific circumstances – you would would need the number of executions of the subquery to be significantly larger AFTER the hash join than BEFORE in order for the early subquery filter to be a good idea.
Since there are always special cases that can be improved by carefully selected optimisation strategies I created three tables to find out what plans I could produce by blocking unnesting and trying to push the filter subquery. Here’s the code I used for the tables:
create table t1 nologging
as
with generator as (
select --+ materialize
rownum id
from dual
connect by
level <= 1e4
)
select
rownum id,
rownum n1,
rpad('x',100) padding
from
generator v1,
generator v2
where
rownum <= 1e5
;
create table t2 nologging as
select * from t1;
create table t3 nologging as
select * from t1;
create index t3_i1 on t3(id);
-- gather stats if needed (version dependent) with no histograms
With this data in place I can experiment with hinting the path I want to see; there are two basically two parts to the hints I need, the first in the main query to control the join: /*+ leading (t1 t2) use_hash(t2) no_swap_join_inputs(t2) */, the second in the subquery /*+ no_unnest push_subq */. So here are a couple of plans – first without the push_subq hint:
Plan hash value: 2281699686
-------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | Cost (%CPU)| A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
-------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 926 (100)| 1000 |00:00:00.94 | 5409 | | | |
|* 1 | FILTER | | 1 | | | 1000 |00:00:00.94 | 5409 | | | |
|* 2 | HASH JOIN | | 1 | 1000 | 425 (5)| 1000 |00:00:00.91 | 3295 | 1888K| 1888K| 1502K (0)|
|* 3 | TABLE ACCESS FULL| T1 | 1 | 1000 | 214 (6)| 1000 |00:00:00.03 | 1614 | | | |
| 4 | TABLE ACCESS FULL| T2 | 1 | 100K| 209 (3)| 100K|00:00:00.23 | 1681 | | | |
|* 5 | INDEX RANGE SCAN | T3_I1 | 1000 | 1 | 1 (0)| 1000 |00:00:00.02 | 2114 | | | |
-------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter( IS NOT NULL)
2 - access("T2"."ID"="T1"."ID")
3 - filter(MOD("T1"."N1",100)=0)
5 - access("T3"."ID"=:B1)
In the absence of the push_subq hint the optimizer has taken the hash join (operations 2 – 4) and filtered late (operations 1 and 5).
When I included the push_subq hint this is what I got in 11.2.0.4:
Plan hash value: 2281699686
-------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | Cost (%CPU)| A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
-------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 424 (100)| 1000 |00:00:00.94 | 5409 | | | |
|* 1 | FILTER | | 1 | | | 1000 |00:00:00.94 | 5409 | | | |
|* 2 | HASH JOIN | | 1 | 1000 | 423 (5)| 1000 |00:00:00.91 | 3295 | 1888K| 1888K| 1535K (0)|
|* 3 | TABLE ACCESS FULL| T1 | 1 | 1000 | 214 (6)| 1000 |00:00:00.03 | 1614 | | | |
| 4 | TABLE ACCESS FULL| T2 | 1 | 5000 | 209 (3)| 100K|00:00:00.23 | 1681 | | | |
|* 5 | INDEX RANGE SCAN | T3_I1 | 1000 | 1 | 1 (0)| 1000 |00:00:00.02 | 2114 | | | |
-------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter( IS NOT NULL)
2 - access("T2"."ID"="T1"."ID")
3 - filter(MOD("T1"."N1",100)=0)
5 - access("T3"."ID"=:B1)
The plan hasn’t changed!
Clearly the shape of the plan hasn’t changed, the numbers for Starts and A-rows haven’t changed, the Buffers haven’t changed, the Time hasn’t changed – in fact the session stats for the two queries were virtually identical. Subquery pushing has clearly NOT taken place. But take a look at the E-rows and Cost: operation 4 in the “pushed” plan reports E-Rows = 5,000 which is the classic 5% for an existence subquery when compared with the E-rows = 100K in the first plan; the cost of the hash join is slightly smaller, and the cost of the whole query has halved – but the run-time engine is doing the same amount of work and following the same plan. The optimizer seems to have pushed the arithmetic, without pushing the subquery!
I could force subquery pushing to take place if I reversed the join order – and all I have to do is change the main hint to /*+ leading (t2 t1) use_hash(t1) no_swap_join_inputs(t1) */ to see this happen; here’s the resulting plan:
------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | Cost (%CPU)| A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 424 (100)| 1000 |00:00:02.02 | 104K| | | |
|* 1 | HASH JOIN | | 1 | 1000 | 423 (5)| 1000 |00:00:02.02 | 104K| 5984K| 2337K| 5601K (0)|
|* 2 | TABLE ACCESS FULL| T2 | 1 | 5000 | 209 (3)| 100K|00:00:01.31 | 102K| | | |
|* 3 | INDEX RANGE SCAN| T3_I1 | 100K| 1 | 1 (0)| 100K|00:00:00.58 | 101K| | | |
|* 4 | TABLE ACCESS FULL| T1 | 1 | 1000 | 214 (6)| 1000 |00:00:00.03 | 1681 | | | |
------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("T2"."ID"="T1"."ID")
2 - filter( IS NOT NULL)
3 - access("T3"."ID"=:B1)
4 - filter(MOD("T1"."N1",100)=0)
You can see (as I implied earlier on) that it was a bad idea to push the subquery with this data set; the subquery has now run 100,000 times adding an extra 1.08 seconds of CPU to the run-time activity; but I’m only trying to establish a principle, so I’m not worried about that. Perhaps, having got subquery pushing in this plan, I could change that no_swap_join_inputs(t1) hint to a swap_join_inputs(t1) to see the plan I want with lines 2 and 3 below line 4 – and here’s what I get when I do:
------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | Cost (%CPU)| A-Rows | A-Time | Buffers | OMem | 1Mem | Used-Mem |
------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 424 (100)| 1000 |00:00:01.97 | 104K| | | |
|* 1 | HASH JOIN | | 1 | 1000 | 423 (5)| 1000 |00:00:01.97 | 104K| 1888K| 1888K| 1499K (0)|
|* 2 | TABLE ACCESS FULL| T1 | 1 | 1000 | 214 (6)| 1000 |00:00:00.02 | 1614 | | | |
|* 3 | TABLE ACCESS FULL| T2 | 1 | 5000 | 209 (3)| 100K|00:00:01.28 | 103K| | | |
|* 4 | INDEX RANGE SCAN| T3_I1 | 100K| 1 | 1 (0)| 100K|00:00:00.56 | 101K| | | |
------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - access("T2"."ID"="T1"."ID")
2 - filter(MOD("T1"."N1",100)=0)
3 - filter( IS NOT NULL)
4 - access("T3"."ID"=:B1)
So we can get where we want to be by starting backwards and reversing the join order! You might notice, by the way, that in the last two plans the optimizer “thinks” it will have to run the subquery 5,000 (or possibly 100,000) times, but the cost of the query is still less than the initial case where the optimizer thought it would have to run the subquery just 1,000 times. (You can see these numbers by looking at the E-rows that feed the filter operation.)
Summary
In this particular case it doesn’t make sense to force the plan I’ve managed to achieve – when filter subqueries are involved the patterns in the data can make a huge difference to performance – but in demonstrating that I can get to a plan that I want I’ve had to work through the option of starting with the wrong join order and then swapping sides on the hash join, and I’ve demonstrated in passing that there is a curious costing anomaly that could affect the optimizer’s choice in more complex executions plans.
Reference script: filter_hash.sql
