Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[SPARK-48265][SQL] Infer window group limit batch should do constant …
…folding ### What changes were proposed in this pull request? Plan after PropagateEmptyRelation may generate double local limit ``` GlobalLimit 21 +- LocalLimit 21 ! +- Union false, false ! :- LocalLimit 21 ! : +- Project [item_id#647L] ! : +- Filter (xxxx) ! : +- Relation db.table[,... 91 more fields] parquet ! +- LocalLimit 21 ! +- Project [item_id#738L] ! +- LocalRelation <empty>, [, ... 91 more fields] ``` to ``` GlobalLimit 21 +- LocalLimit 21 - LocalLimit 21 +- Project [item_id#647L] +- Filter (xxxx) +- Relation db.table[,... 91 more fields] parquet ``` after `Infer window group limit batch` batch's `EliminateLimits` will be ``` GlobalLimit 21 +- LocalLimit least(21, 21) +- Project [item_id#647L] +- Filter (xxxx) +- Relation db.table[,... 91 more fields] parquet ``` It can't work, here miss a `ConstantFolding` ### Why are the changes needed? Fix bug ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#46568 from AngersZhuuuu/SPARK-48265. Authored-by: Angerszhuuuu <[email protected]> Signed-off-by: Wenchen Fan <[email protected]>
- Loading branch information