A new optimizer feature that appears in 23c (probably not 21c) was the ability to push group by clauses into union all set operations. This will happen unhinted, but can be hinted with the highly memorable [no_]push_gby_into_union_all() hint that appeared in 23.1.0.0 according to v$sql_hint. and the feature can be disabled by setting the (equally memorable) hidden parameter _optimizer_push_gby_into_union_all to false.
From a couple of simple experiments it looks as if the hint should be used to identify query blocks where you want an aggregation (group by) that appears “outside” a union all (inline) view to happen “inside” the view. Here’s a trivial demonstration that I’ve run on 23.3
rem
rem Script: push_gby_ua.sql
rem Author: Jonathan Lewis
rem Dated: Nov 2023
rem
rem Last tested
rem 23.3.0.0
rem
create table t1
as
select *
from all_Objects
where rownum <= 50000
;
set linesize 156
set pagesize 60
set trimspool on
set tab off
set serveroutput off
alter session set statistics_level = all;
select
/*+
-- qb_name(main)
push_gby_into_union_all(@sel$2)
no_push_gby_into_union_all(@sel$3)
*/
owner, count(*)
from (
select /* qb_name(u1) */ owner from t1 where owner = 'SYS'
union all
select /* qb_name(u2) */ owner from t1 where owner = 'PUBLIC'
)
group by owner
/
select * from table(dbms_xplan.display_cursor(format=>'allstats last cost outline alias hint_report qbregistry qbregistry_graph '));
All I’ve done here is create a table that copies 50,000 rows from the view all_objects, then executed a query that reports the number of objects for owners SYS and PUBLIC by selecting the two sets of objects separately and aggregating a union all of those sets.
For maximum visibility I’ve shown the positive and negative versions of the hint – the aggregation doesn’t have to apply to all the branches of the view and it’s not unknown for the optimizer to make the wrong choices if it hasn’t managed to produce a good cost estimate.
I had to know that the two subqueries inside the view would be named sel$2 and sel$3 by default, and I’ve used these names in the hints. In general it can be quite hard to work out the internal names of query blocks and you’ll note that I had used the qb_name() hint for the two subqueries (and the main query block) but then turned those hints into comments because the qbregistry_graph format I’d used in my call to dbms_xplan can’t cope with user-supplied query block names (yet). [Correction: I must have done something wrong when I was using the qb_name hints. I’ve re-run the tests and the graph comes out correctly]
Here’s the execution plan (with some of the bits removed) that I got from 23.3 for this test:
----------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | Cost (%CPU)| A-Rows | A-Time | Buffers | Reads |
----------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 295 (100)| 2 |00:00:00.03 | 2232 | 1114 |
| 1 | HASH GROUP BY | | 1 | 15 | 295 (4)| 2 |00:00:00.03 | 2232 | 1114 |
| 2 | VIEW | | 1 | 3334 | 293 (3)| 9288 |00:00:00.03 | 2232 | 1114 |
| 3 | UNION-ALL | | 1 | 3334 | 293 (3)| 9288 |00:00:00.03 | 2232 | 1114 |
| 4 | SORT GROUP BY NOSORT| | 1 | 1 | 147 (3)| 1 |00:00:00.02 | 1116 | 1114 |
|* 5 | TABLE ACCESS FULL | T1 | 1 | 3333 | 147 (3)| 39724 |00:00:00.01 | 1116 | 1114 |
|* 6 | TABLE ACCESS FULL | T1 | 1 | 3333 | 147 (3)| 9287 |00:00:00.01 | 1116 | 0 |
----------------------------------------------------------------------------------------------------------------
Outline Data
-------------
/*+
BEGIN_OUTLINE_DATA
IGNORE_OPTIM_EMBEDDED_HINTS
OPTIMIZER_FEATURES_ENABLE('23.1.0')
DB_VERSION('23.1.0')
ALL_ROWS
OUTLINE_LEAF(@"SEL$FC1F66D1")
OUTLINE_LEAF(@"SEL$3")
OUTLINE_LEAF(@"SET$22FBD6DA")
PUSH_GBY_INTO_UNION_ALL(@"SEL$2")
OUTLINE_LEAF(@"SEL$1")
OUTLINE(@"SEL$2")
OUTLINE(@"SET$1")
NO_ACCESS(@"SEL$1" "from$_subquery$_001"@"SEL$1")
USE_HASH_AGGREGATION(@"SEL$1" GROUP_BY)
FULL(@"SEL$3" "T1"@"SEL$3")
FULL(@"SEL$FC1F66D1" "T1"@"SEL$2")
END_OUTLINE_DATA
*/
Predicate Information (identified by operation id):
---------------------------------------------------
5 - filter("OWNER"='SYS')
6 - filter("OWNER"='PUBLIC')
Hint Report (identified by operation id / Query Block Name / Object Alias):
Total hints for statement: 2
---------------------------------------------------------------------------
4 - SEL$FC1F66D1
- push_gby_into_union_all(@sel$2)
6 - SEL$3
- no_push_gby_into_union_all(@sel$3)
It’s interesting to note that the Hint Report tells us that both my hints were valid (and used); but the Ouline Data echoes only one of them (the “positive” push_gby_into_union_all). Because I’ve used the same table twice it’s not instantly clear that the optimizer has pushed the subquery that I had specified but if you check the Predicate Information you can confirm that the SYS data has been aggregated inside the union all and the PUBLIC data has been passed up to the union all operator without aggregation. (In the absence of the hints both data sets would have been aggregated early.)
Here, in comparison, is the plan (slightly reduced, and with the qbregistry options removed) that I got from 19.11.0.0
----------------------------------------------------------------------------------------------------------------------------------------
| Id | Operation | Name | Starts | E-Rows | Cost (%CPU)| A-Rows | A-Time | Buffers | Reads | OMem | 1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | | 265 (100)| 2 |00:00:00.09 | 1986 | 991 | | | |
| 1 | HASH GROUP BY | | 1 | 13 | 265 (4)| 2 |00:00:00.09 | 1986 | 991 | 1422K| 1422K| 653K (0)|
| 2 | VIEW | | 1 | 7692 | 263 (3)| 48446 |00:00:00.07 | 1986 | 991 | | | |
| 3 | UNION-ALL | | 1 | | | 48446 |00:00:00.06 | 1986 | 991 | | | |
|* 4 | TABLE ACCESS FULL| T1 | 1 | 3846 | 131 (3)| 42034 |00:00:00.02 | 993 | 991 | | | |
|* 5 | TABLE ACCESS FULL| T1 | 1 | 3846 | 131 (3)| 6412 |00:00:00.01 | 993 | 0 | | | |
----------------------------------------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
4 - filter("OWNER"='SYS')
5 - filter("OWNER"='PUBLIC')
Hint Report (identified by operation id / Query Block Name / Object Alias):
Total hints for statement: 1 (E - Syntax error (1))
---------------------------------------------------------------------------
1 - SEL$1
E - push_gby_into_union_all
As you can see, 19.11 treats the hint as an error and both subqueries against t1 pass their rows up to the union all without aggregation. The 19.11 plan also gives you some idea of why it can be worth pushing the group by: 23.3 doesn’t report any memory used for either of the aggregation operations that take place while the postponed (or, rather, unpushed) aggregation in 19.11 reports 1.4M of memory used. As a general principle we might expect several small aggregations have a lower peak of memory usage than one large aggregation. There’s also a CPU benefit when Oracle doesn’t have to push lots of rows up through a couple of operations.
In fact the absence of memory-related columns in the 23.3 plan is a little suspect and I may have to examine it further. It may simply be the case that the size of the “small allocation” that doesn’t get reported in earlier versions has been increased to (best guess) 1MB; it may be that dbms_xplan in 23c has got a little bug that omits that part of the report.
Summary
Oracle 23c has a new transformation that will probably help to reduce memory and CPU consumption when it comes into play. Queries that aggregate over union all views may change plans to push the aggregation into some or all of the separate subqueries inside the union.
The feature is cost-based but you can over-ride the optimizer’s choice of which subqueries should be aggregated early with the hint [no_]push_gby_into_union_all(@qbname). The feature can also be disabled completely by setting the hidden parameter _optimizer_push_gby_into_union_all to false.
Addendum
It occurred to me that the optimizer will transform an IN-list to a list of equalities with OR, and it’s also capable of using OR-expansion then there might be cases where an aggregate based on an IN-list could go through the two steps and then benefit from this new feature, for example:
select
sts, count(*) ct
from t1
where sts in ('B','C')
group by
sts
/
-------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | | | 3 (100)| |
| 1 | SORT GROUP BY NOSORT| | 2 | 4 | 3 (0)| 00:00:01 |
| 2 | INLIST ITERATOR | | | | | |
|* 3 | INDEX RANGE SCAN | T1_I1A | 100 | 200 | 3 (0)| 00:00:01 |
-------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
3 - access(("STS"='B' OR "STS"='C'))
Alas, no. Although we can see the rewrite of the IN-list the optimizer doesn’t then use OR-expansion. And when I added the hint /*+ or_expand */ to try to push Oracle into the right direction the Hint Report told me:
Hint Report (identified by operation id / Query Block Name / Object Alias):
Total hints for statement: 1 (U - Unused (1))
---------------------------------------------------------------------------
1 - SEL$1
U - or_expand / No valid predicate for OR expansion
Maybe in the next release.
