org.apache.drill.exec.physical.impl.scan.v3.lifecycle

Class TestScanLifecycleTwoReaders

java.lang.Object

org.apache.drill.test.BaseTest

org.apache.drill.test.DrillTest

org.apache.drill.test.SubOperatorTest

org.apache.drill.exec.physical.impl.scan.v3.lifecycle.BaseTestScanLifecycle

org.apache.drill.exec.physical.impl.scan.v3.lifecycle.TestScanLifecycleTwoReaders

public class TestScanLifecycleTwoReaders
extends BaseTestScanLifecycle

Test two readers in succession in various cases: empty readers, normal readers, type conflicts, etc.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.drill.exec.physical.impl.scan.v3.lifecycle.BaseTestScanLifecycle

BaseTestScanLifecycle.BaseMockBatchReader, BaseTestScanLifecycle.DummySubScan, BaseTestScanLifecycle.FailingReader, BaseTestScanLifecycle.MockEarlySchemaReader, BaseTestScanLifecycle.MockEarlySchemaTypeConflictReader, BaseTestScanLifecycle.MockEmptySchemaReader, BaseTestScanLifecycle.MockLateSchemaReader, BaseTestScanLifecycle.MockLateSchemaTypeConflictReader, BaseTestScanLifecycle.MockModeConflictReader, BaseTestScanLifecycle.MockReorderedReader, BaseTestScanLifecycle.MockSingleColReader, BaseTestScanLifecycle.MockThreeColReader, BaseTestScanLifecycle.NoDataReader, BaseTestScanLifecycle.SingleReaderFactory, BaseTestScanLifecycle.TwoReaderFactory

Nested classes/interfaces inherited from class org.apache.drill.test.DrillTest

DrillTest.MemWatcher

Field Summary

Fields inherited from class org.apache.drill.exec.physical.impl.scan.v3.lifecycle.BaseTestScanLifecycle

CONFLICT_SCHEMA, SCHEMA

Fields inherited from class org.apache.drill.test.SubOperatorTest

dirTestWatcher, fixture

Fields inherited from class org.apache.drill.test.DrillTest

logOutcome, objectMapper, REPEAT_RULE, thrownException, TIMEOUT

Constructor Summary

Constructors

Constructor and Description
TestScanLifecycleTwoReaders()

Method Summary

Modifier and Type	Method and Description
void	testColumnReorderingAB() SELECT * FROM (a, b) then (b,a).
void	testColumnReorderingBA() SELECT * FROM (b,a) then (a, b).
void	testEarlySchemaTypeConflict() SELECT * from two readers: both (a, b), but with a conflicting type for the column b.
void	testExpandingSchemaAllowingSchemaChange() SELECT * FROM two readers, one (a, b), the other (a, b, c).
void	testExpandingSchemaDisallowingSchemaChange() SELECT * FROM two readers, one (a, b), the other (a, b, c).
void	testLateSchemaTypeConflict() SELECT * from two readers: both (a, b), but with a conflicting type for the column b.
void	testModeConflict() SELECT * from two readers: both (a, b), but with a conflicting type for the column b.
void	testNullThenValidReader()
void	testShrinkingSchema() SELECT * FROM two readers, one with (a, b), the other with just (a).
void	testShrinkingSchemaWithConflict() Shrinking schema, as above.
void	testSpecifiedColumnOrder() SELECT , b FROM (b,a) then (a, b).
void	testTwoNullReaders() Two null readers: neither provides a valid scan schema.
void	testTwoValidReaders()
void	testValidThenNullReader()

Methods inherited from class org.apache.drill.exec.physical.impl.scan.v3.lifecycle.BaseTestScanLifecycle

buildScan, simpleExpected, verifyEmptyReader, verifyStandardReader

Methods inherited from class org.apache.drill.test.SubOperatorTest

classSetup, classTeardown

Methods inherited from class org.apache.drill.test.DrillTest

escapeJsonString, finishDrillTest, initDrillTest

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TestScanLifecycleTwoReaders

public TestScanLifecycleTwoReaders()

Method Detail

testTwoNullReaders

public void testTwoNullReaders()

Two null readers: neither provides a valid scan schema.

testNullThenValidReader

public void testNullThenValidReader()

testValidThenNullReader

public void testValidThenNullReader()

testTwoValidReaders

public void testTwoValidReaders()

testShrinkingSchema

public void testShrinkingSchema()

SELECT * FROM two readers, one with (a, b), the other with just (a). The scan will reuse the type of b from the first reader when filling the missing column in the second reader. Since the schema shrinks, the result is the same whether schema change is allowed or not.

testShrinkingSchemaWithConflict

public void testShrinkingSchemaWithConflict()

Shrinking schema, as above. Explicit projection:

 SELECT a, b FROM (a) then (a,b)
 

But choose a missing column type (the default Nullable INT) in the first reader that will conflict with the actual column type (VARCHAR) in the second.

testExpandingSchemaAllowingSchemaChange

public void testExpandingSchemaAllowingSchemaChange()

SELECT * FROM two readers, one (a, b), the other (a, b, c). With schema change enabled, the third column shows up only in the second batch, forcing a schema change downstream.

testExpandingSchemaDisallowingSchemaChange

public void testExpandingSchemaDisallowingSchemaChange()

SELECT * FROM two readers, one (a, b), the other (a, b, c). With schema change disabled, the third column is ignored in the second reader: the first reader "fixes" projection to just (a, b) even though projection started as wildcard.

We can argue if this behavior is correct. We know most operators, and no ODBC or JDBC client can handle schema change. So, our other options are do the schema change anyway (the above test), fail, or this choice, which is to try to muddle though. The correct solution is to provide a schema (tested elsewhere.)

testEarlySchemaTypeConflict

public void testEarlySchemaTypeConflict()

SELECT * from two readers: both (a, b), but with a conflicting type for the column b. The reader fail will with a type conflict.

testLateSchemaTypeConflict

public void testLateSchemaTypeConflict()

SELECT * from two readers: both (a, b), but with a conflicting type for the column b. The reader fail will with a type conflict.

testModeConflict

public void testModeConflict()

SELECT * from two readers: both (a, b), but with a conflicting type for the column b. The reader fail will with a type conflict. We could possibly try to handle a NULLABLE --> NON NULLABLE conflict, but there does not yet appear to be a compelling case for doing so.

testColumnReorderingAB

public void testColumnReorderingAB()

SELECT * FROM (a, b) then (b,a). The order of the first table drives the order of the results.

testColumnReorderingBA

public void testColumnReorderingBA()

SELECT * FROM (b,a) then (a, b). The order of the first table drives the order of the results.

testSpecifiedColumnOrder

public void testSpecifiedColumnOrder()

SELECT , b FROM (b,a) then (a, b). The order of the project list drives the order of the results.