public class TestLateralJoinCorrectness extends SubOperatorTest
DrillTest.MemWatcherdirTestWatcher, fixturelogOutcome, objectMapper, REPEAT_RULE, thrownException, TIMEOUT| Constructor and Description |
|---|
TestLateralJoinCorrectness() |
| Modifier and Type | Method and Description |
|---|---|
void |
afterTest() |
void |
beforeTest() |
static void |
setUpBeforeClass() |
static void |
tearDownAfterClass() |
void |
test_OK_NEW_SCHEMAFromLeft_EmitFromRight_PostBuildSchema()
Test to verify if OK_NEW_SCHEMA is received from left side of LATERAL post build schema phase and EMIT is
received from right side of LATERAL for each row on left side, then Lateral sends OK_NEW_SCHEMA downstream with
the output batch.
|
void |
test1RecordLeftBatchTo1RightRecordBatch()
Test to show correct IterOutcome produced by LATERAL when one record in left batch produces only 1 right batch
with EMIT outcome.
|
void |
test1RecordLeftBatchTo2RightRecordBatch()
Test to show correct IterOutcome & output produced by LATERAL when one record in left batch produces 2 right
batches with OK and EMIT outcome respectively.
|
void |
test1RecordLeftBatchToEmptyRightBatch()
Test to show that if the result of cross join between left batch and empty right batch is an empty
batch and next batch on left side result in NONE outcome, then there is no output produced and result
returned to downstream is NONE.
|
void |
testBasicLeftLateralJoin()
Test to check basic left lateral join is working correctly or not.
|
void |
testBuildSchemaEmptyLNonEmptyRBatch()
This case should never happen since With empty left there cannot be non-empty right batch, the
LateralJoinBatch.buildSchema() phase should fail() with IllegalStateException |
void |
testBuildSchemaEmptyLRBatch()
With both left and right batch being empty, the
LateralJoinBatch.buildSchema() phase
should still build the output container schema and return empty batch |
void |
testBuildSchemaNonEmptyLEmptyRBatch()
With non-empty left and empty right batch, the
LateralJoinBatch.buildSchema() phase
should still build the output container schema and return empty batch |
void |
testBuildSchemaNonEmptyLRBatch()
With both left and right batch being non-empty, the
LateralJoinBatch.buildSchema() phase
will fail with IllegalStateException since it expects first batch from right branch of LATERAL
to be always empty |
void |
testBuildSchemaWithEMITOutcome()
Test for receiving unexpected EMIT outcome during build phase on left side.
|
void |
testFillingUpOutputBatch_With2ExcludedColumns() |
void |
testFillingUpOutputBatch_WithExcludedColumns() |
void |
testFillingUpOutputBatch()
Test to show LATERAL tries to pack the output batch until it's full or all the data is consumed from left and
right side.
|
void |
testHandlingEMITFromLeft()
When multiple left batch is received with same schema but with OK_NEW_SCHEMA, then LATERAL detects that
correctly and suppresses schema change operation by producing output in same batch created with initial schema.
|
void |
testHandlingEmptyEMITAfterOK()
Test for the case when LATERAL received a left batch with OK outcome and then populate the Join output in the
outgoing batch.
|
void |
testHandlingNoneAfterOK()
Test for the case where LATERAL received a left batch with OK outcome and then populate the Join output in the
outgoing batch.
|
void |
testHandlingNonEmpty_EMITAfterOK_WithMultipleOutput()
Temporary test to validate LATERAL handling output batch getting filled without consuming full output from left
and right batch join.
|
void |
testHandlingNonEmptyEMITAfterOK()
Test for the case when LATERAL received a left batch with OK outcome and then populate the Join output in the
outgoing batch.
|
void |
testHandlingSchemaChangeForNonUnnestField()
When multiple left batches are received with different schema, then LATERAL produces output for each schema type
separately (even though output batch is not filled completely) and handles the schema change in left batch.
|
void |
testHandlingSchemaChangeForUnnestField()
When multiple left and right batches are received with different schema, then LATERAL produces output for each
schema type separately (even though output batch is not filled completely) and handles the schema change correctly.
|
void |
testHandlingUnexpectedSchemaChangeForUnnestField()
Verify if there is no schema change on left side and LATERAL still sees an unexpected schema change on right side
then it handles it correctly.
|
void |
testLateral_SchemaChange_Left_EmptyRightBatchForFirst()
Test to verify that for first left incoming if there is no right side incoming batch and then second left
incoming comes with schema change, then the schema change with empty output batch for first incoming is handled
properly.
|
void |
testLeftLateralJoin_WithAndWithoutMatching_MultipleBatch()
Test to see if there are multiple rows in left batch and for some rows right side produces batch with records
and for other rows right side produces empty batches then based on left join type we are populating the output
batch correctly.
|
void |
testLeftLateralJoin_WithAndWithoutMatching()
Test to see if there are multiple rows in left batch and for some rows right side produces batch with records
and for other rows right side produces empty batches then based on left join type we are populating the output
batch correctly.
|
void |
testLeftLateralJoin_WithMatchingAndEmptyBatch()
Test to see if there are multiple rows in left batch and for some rows right side produces multiple batches such
that some are with records and some are empty then we are not duplicating those left side records based on left
join type.
|
void |
testMultiLevelLateral_MultipleOutput()
This test generates an operator tree for multi level LATERAL by stacking 2 LATERAL and finally an UNNEST pair
(using MockRecord Batch) as left and right child of lower level LATERAL.
|
void |
testMultiLevelLateral_SchemaChange_LeftRightUnnest_NonEmptyBatch()
Test to verify in case of Multilevel lateral when a non-empty OK_NEW_SCHEMA batch post build schema phase is
received from right most UNNEST of lower LATERAL then pipeline works fine.
|
void |
testMultiLevelLateral_SchemaChange_LeftRightUnnest()
This test generates an operator tree for multi level LATERAL by stacking 2 LATERAL and finally an UNNEST pair
(using MockRecord Batch) as left and right child of lower level LATERAL.
|
void |
testMultiLevelLateral_SchemaChange_LeftUnnest()
This test generates an operator tree for multi level LATERAL by stacking 2 LATERAL and finally an UNNEST pair
(using MockRecord Batch) as left and right child of lower level LATERAL.
|
void |
testMultiLevelLateral_SchemaChange_RightUnnest()
This test generates an operator tree for multi level LATERAL by stacking 2 LATERAL and finally an UNNEST pair
(using MockRecord Batch) as left and right child of lower level LATERAL.
|
void |
testMultiLevelLateral()
This test generates an operator tree for multi level LATERAL by stacking 2 LATERAL and finally an UNNEST pair
(using MockRecord Batch) as left and right child of lower level LATERAL.
|
void |
testMultipleUnnestAtSameLevel()
This test generates an operator tree for multiple UNNEST at same level by stacking 2 LATERAL and UNNEST pair on
top of each other.
|
void |
testOK_NEW_SCHEMA_WithNoActualSchemaChange_ForNonUnnestField()
When multiple left batch is received with same schema but with OK_NEW_SCHEMA, then LATERAL rebuilds the
schema each time and sends output in multiple output batches
The schema change was only for columns which are not produced by the UNNEST or right branch.
|
void |
testOK_NEW_SCHEMA_WithNoActualSchemaChange_ForUnnestField()
When multiple left batch is received with same schema but with OK_NEW_SCHEMA, then LATERAL correctly
handles it by re-creating the schema and producing multiple batches of final output
The schema change is for columns common on both left and right side.
|
void |
testPostBuildSchema_OK_NEW_SCHEMA_NonEmptyRightBatch()
Verifies that if a non-empty batch with OK_NEW_SCHEMA is received from right side post buildSchema phase then it
is handled correctly by sending an empty batch with OK_NEW_SCHEMA and later consuming it to produce actual
output batch with some data
|
void |
testUnsupportedSelectionVector()
Test unsupported incoming batch to LATERAL with SelectionVector
|
classSetup, classTeardownescapeJsonString, finishDrillTest, initDrillTestpublic void testBuildSchemaEmptyLRBatch()
throws Exception
LateralJoinBatch.buildSchema() phase
should still build the output container schema and return empty batchExceptionpublic void testBuildSchemaNonEmptyLRBatch()
throws Exception
LateralJoinBatch.buildSchema() phase
will fail with IllegalStateException since it expects first batch from right branch of LATERAL
to be always emptyExceptionpublic void testBuildSchemaNonEmptyLEmptyRBatch()
throws Exception
LateralJoinBatch.buildSchema() phase
should still build the output container schema and return empty batchExceptionpublic void testBuildSchemaEmptyLNonEmptyRBatch()
throws Exception
LateralJoinBatch.buildSchema() phase should fail() with IllegalStateExceptionExceptionpublic void testBuildSchemaWithEMITOutcome()
throws Exception
Exceptionpublic void test1RecordLeftBatchTo1RightRecordBatch()
throws Exception
Exceptionpublic void test1RecordLeftBatchTo2RightRecordBatch()
throws Exception
Exceptionpublic void test1RecordLeftBatchToEmptyRightBatch()
throws Exception
Exceptionpublic void testFillingUpOutputBatch()
throws Exception
Exceptionpublic void testHandlingSchemaChangeForNonUnnestField()
throws Exception
Exceptionpublic void testHandlingSchemaChangeForUnnestField()
throws Exception
Exceptionpublic void testHandlingUnexpectedSchemaChangeForUnnestField()
throws Exception
Exceptionpublic void testOK_NEW_SCHEMA_WithNoActualSchemaChange_ForNonUnnestField()
throws Exception
Exceptionpublic void testOK_NEW_SCHEMA_WithNoActualSchemaChange_ForUnnestField()
throws Exception
Exceptionpublic void testHandlingEMITFromLeft()
throws Exception
Exceptionpublic void testHandlingNoneAfterOK()
throws Exception
Exceptionpublic void testHandlingEmptyEMITAfterOK()
throws Exception
Exceptionpublic void testHandlingNonEmptyEMITAfterOK()
throws Exception
Exceptionpublic void testHandlingNonEmpty_EMITAfterOK_WithMultipleOutput()
throws Exception
For this test we are updating RecordBatch.MAX_BATCH_ROW_COUNT by making it public, which might not expected
after including the BatchSizing logic
TODO: Update the test after incorporating the BatchSizing change.
Exceptionpublic void testBasicLeftLateralJoin()
throws Exception
Exceptionpublic void testLeftLateralJoin_WithMatchingAndEmptyBatch()
throws Exception
Exceptionpublic void testLeftLateralJoin_WithAndWithoutMatching()
throws Exception
Exceptionpublic void testLeftLateralJoin_WithAndWithoutMatching_MultipleBatch()
throws Exception
Exceptionpublic void testMultipleUnnestAtSameLevel()
throws Exception
Exceptionpublic void testMultiLevelLateral()
throws Exception
Exceptionpublic void testMultiLevelLateral_MultipleOutput()
throws Exception
Exceptionpublic void testMultiLevelLateral_SchemaChange_LeftUnnest()
throws Exception
Exceptionpublic void testMultiLevelLateral_SchemaChange_RightUnnest()
throws Exception
Exceptionpublic void testMultiLevelLateral_SchemaChange_LeftRightUnnest()
throws Exception
Exceptionpublic void testUnsupportedSelectionVector()
throws Exception
Exceptionpublic void test_OK_NEW_SCHEMAFromLeft_EmitFromRight_PostBuildSchema()
throws Exception
Exceptionpublic void testPostBuildSchema_OK_NEW_SCHEMA_NonEmptyRightBatch()
throws Exception
Exceptionpublic void testMultiLevelLateral_SchemaChange_LeftRightUnnest_NonEmptyBatch()
throws Exception
Exceptionpublic void testLateral_SchemaChange_Left_EmptyRightBatchForFirst()
throws Exception
Exceptionpublic void testFillingUpOutputBatch_WithExcludedColumns()
throws Exception
ExceptionCopyright © 2021 The Apache Software Foundation. All rights reserved.