HIVE-29176: Wrong result when HiveAntiJoin is replacing an IS NULL filter on a nullable column #6062
Conversation
…lter on a nullable column
16ca591 to
0d9de28
Compare
|
| (4, "val_4", "val_4"), | ||
| (5, "val_5", "val_5"); | ||
|
|
||
| -- do not introduce anti-join if filtering a nullable column with IS NULL |
There was a problem hiding this comment.
One more suggestion for a test case would where the right column is not coming from a base table but from another Left Outer Join so it becomes nullable even if it was nonnullable in the base table. This is just to check if the nullable property is being properly propagated up the operators and if so, this PR's changes would work fine. If for some reason it is not getting propagated, it could uncover a hidden bug.
| return Optional.empty(); | ||
| } | ||
|
|
||
| if(filterNode.getKind() != SqlKind.IS_NULL) { |
There was a problem hiding this comment.
Since this is a quicker conditional check, it would be better to move this up a bit before line 160 so we can avoid executing the contains() if possible.
| } | ||
|
|
||
| private boolean isStrong(RexNode rexNode) { | ||
| AtomicBoolean hasCast = new AtomicBoolean(false); |
There was a problem hiding this comment.
Previously, since AtomicBoolean was used, I am wondering there might have been a good reason. Was there a concurrency issue that motivated the original implementation. Changing it to a MutableBoolean means that this variable is not thread safe.
What changes were proposed in this pull request?
Improve the logic of HiveAntiSemiJoinRule to fix the problems mentioned in HIVE-29176.
Why are the changes needed?
Wrong query plans introduced by HiveAntiSemiJoinRule.
Does this PR introduce any user-facing change?
No
How was this patch tested?
Unit tests and qfile tests.