org.apache.drill.exec.physical.impl.scan

Class TestScanOrchestratorEarlySchema

java.lang.Object

org.apache.drill.test.BaseTest

org.apache.drill.test.DrillTest

org.apache.drill.test.SubOperatorTest

org.apache.drill.exec.physical.impl.scan.TestScanOrchestratorEarlySchema

public class TestScanOrchestratorEarlySchema
extends SubOperatorTest

Test the early-schema support of the scan orchestrator. "Early schema" refers to the case in which the reader can provide a schema when the reader is opened. Examples: CSV, HBase, MapR-DB binary, JDBC.

The tests here focus on the scan orchestrator itself; the tests assume that tests for lower-level components have already passed.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.drill.test.DrillTest

DrillTest.MemWatcher

Field Summary

Fields inherited from class org.apache.drill.test.SubOperatorTest

dirTestWatcher, fixture

Fields inherited from class org.apache.drill.test.DrillTest

logOutcome, objectMapper, REPEAT_RULE, thrownException, TIMEOUT

Constructor Summary

Constructors

Constructor and Description
TestScanOrchestratorEarlySchema()

Method Summary

Modifier and Type	Method and Description
void	testColumnReordering() Verify that different table column orders are projected into the SELECT order, preserving vectors, so no schema change for column reordering.
void	testEarlySchemaSelectAll() Test SELECT a, b FROM table(a, b)
void	testEarlySchemaSelectAllReorder() Test SELECT b, a FROM table(a, b)
void	testEarlySchemaSelectExtra() Test SELECT a, b, c FROM table(a, b) c will be null
void	testEarlySchemaSelectExtraCustomType() Test SELECT a, b, c FROM table(a, b) c will be null of type VARCHAR
void	testEarlySchemaSelectNone() Test SELECT - FROM table(a, b)
void	testEarlySchemaSelectSubset() Test SELECT a FROM table(a, b)
void	testEarlySchemaWildcard() Test SELECT * from an early-schema table of (a, b)
void	testEmptySchema() Test SELECT * from an early-schema table of () (that is, a schema that consists of zero columns.
void	testEmptySchemaExtra() Test SELECT a from an early-schema table of () (that is, a schema that consists of zero columns.
void	testModeSmoothing()
void	testTypeSmoothing() Test the ability of the scan scanner to "smooth" out schema changes by reusing the type from a previous reader, if known.
void	testTypeSmoothingExplicit() The projection mechanism provides "type smoothing": null columns prefer the type of previously-seen non-null columns.

Methods inherited from class org.apache.drill.test.SubOperatorTest

classSetup, classTeardown

Methods inherited from class org.apache.drill.test.DrillTest

escapeJsonString, finishDrillTest, initDrillTest

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TestScanOrchestratorEarlySchema

public TestScanOrchestratorEarlySchema()

Method Detail

testEarlySchemaWildcard

public void testEarlySchemaWildcard()

Test SELECT * from an early-schema table of (a, b)

testEarlySchemaSelectAll

public void testEarlySchemaSelectAll()

Test SELECT a, b FROM table(a, b)

testEarlySchemaSelectAllReorder

public void testEarlySchemaSelectAllReorder()

Test SELECT b, a FROM table(a, b)

testEarlySchemaSelectExtra

public void testEarlySchemaSelectExtra()

Test SELECT a, b, c FROM table(a, b) c will be null

testEarlySchemaSelectExtraCustomType

public void testEarlySchemaSelectExtraCustomType()

Test SELECT a, b, c FROM table(a, b) c will be null of type VARCHAR

testEarlySchemaSelectSubset

public void testEarlySchemaSelectSubset()

Test SELECT a FROM table(a, b)

testEarlySchemaSelectNone

public void testEarlySchemaSelectNone()

Test SELECT - FROM table(a, b)

testEmptySchema

public void testEmptySchema()

Test SELECT * from an early-schema table of () (that is, a schema that consists of zero columns.

testEmptySchemaExtra

public void testEmptySchemaExtra()

Test SELECT a from an early-schema table of () (that is, a schema that consists of zero columns.

testTypeSmoothingExplicit

public void testTypeSmoothingExplicit()

The projection mechanism provides "type smoothing": null columns prefer the type of previously-seen non-null columns.

 SELECT a, b ...

 Table 1: (a: BIGINT, b: VARCHAR)
 Table 2: (a: BIGINT)
 Table 3: (b: VARCHAR)
 

The result in all cases should be (a : BIGINT, b: VARCHAR)

testTypeSmoothing

public void testTypeSmoothing()

Test the ability of the scan scanner to "smooth" out schema changes by reusing the type from a previous reader, if known. That is, given three readers:
(a, b)
(b)
(a, b)
Then the type of column a should be preserved for the second reader that does not include a. This works if a is nullable. If so, a's type will be used for the empty column, rather than the usual nullable int.

Detailed testing of type matching for "missing" columns is done in #testNullColumnLoader().

As a side effect, makes sure that two identical tables (in this case, separated by a different table) results in no schema change.

testModeSmoothing

public void testModeSmoothing()

testColumnReordering

public void testColumnReordering()

Verify that different table column orders are projected into the SELECT order, preserving vectors, so no schema change for column reordering.