org.apache.drill.exec.physical.resultSet.impl

Class TestResultSetLoaderUnions

java.lang.Object

org.apache.drill.test.BaseTest

org.apache.drill.test.DrillTest

org.apache.drill.test.SubOperatorTest

org.apache.drill.exec.physical.resultSet.impl.TestResultSetLoaderUnions

public class TestResultSetLoaderUnions
extends SubOperatorTest

Tests the result set loader support for union vectors. Union vectors are only lightly supported in Apache Drill and not supported at all in commercial versions. They have may problems: both in code and in theory. Most operators do not support them. But, JSON uses them, so they must be made to work in the result set loader layer.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.drill.test.DrillTest

DrillTest.MemWatcher

Field Summary

Fields inherited from class org.apache.drill.test.SubOperatorTest

dirTestWatcher, fixture

Fields inherited from class org.apache.drill.test.DrillTest

logOutcome, objectMapper, REPEAT_RULE, thrownException, TIMEOUT

Constructor Summary

Constructors

Constructor and Description
TestResultSetLoaderUnions()

Method Summary

Modifier and Type	Method and Description
void	testListofListofScalar() The semantics of the ListVector are such that it allows multi-dimensional lists.
void	testRepeatedListOfUnion() The repeated list type is way off in the weeds in Drill.
void	testSimpleList() Test for the case of a list defined to contain exactly one type.
void	testSimpleListDynamic() Test a simple list created dynamically at load time.
void	testSimpleListMultiTypes() Try to create a simple (non-expandable) list while specifying two types.
void	testSimpleListNoTypes() Try to create a simple (non-expandable) list without giving a member type.
void	testUnionAddTypes()
void	testUnionBasics()
void	testUnionOverflow()
void	testVariantListDynamic() Test a variant list created dynamically at load time.
void	testVariantListWithMap() Dynamically add a map to a list that also contains scalars.

Methods inherited from class org.apache.drill.test.SubOperatorTest

classSetup, classTeardown

Methods inherited from class org.apache.drill.test.DrillTest

escapeJsonString, finishDrillTest, initDrillTest

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TestResultSetLoaderUnions

public TestResultSetLoaderUnions()

Method Detail

testUnionBasics

public void testUnionBasics()

testUnionAddTypes

public void testUnionAddTypes()

testUnionOverflow

public void testUnionOverflow()

testSimpleList

public void testSimpleList()

Test for the case of a list defined to contain exactly one type. Relies on the row set tests to verify that the single type model works for lists. Here we test that the ResultSetLoader put the pieces together correctly.

testSimpleListDynamic

public void testSimpleListDynamic()

Test a simple list created dynamically at load time. The list must include a single type member.

testSimpleListNoTypes

public void testSimpleListNoTypes()

Try to create a simple (non-expandable) list without giving a member type. Expected to fail.

testSimpleListMultiTypes

public void testSimpleListMultiTypes()

Try to create a simple (non-expandable) list while specifying two types. Expected to fail.

testVariantListDynamic

public void testVariantListDynamic()

Test a variant list created dynamically at load time. The list starts with no type, at which time it can hold only null values. Then we add a Varchar, and finally an Int.

This test is superficial. There are many odd cases to consider.

• Write nulls to a list with no type. (This test ensures that adding a (nullable) scalar "does the right thing."
• Add a map to the list. Maps carry no "bits" vector, so null list entries to that point are lost. (For maps, we could go straight to a union, with just a map, to preserve the null states. This whole area is a huge mess...)
• Do the type transitions when writing to a row. (The tests here do the transition between rows.)

The reason for the sparse coverage is that Drill barely supports lists and unions; most code is just plain broken. Our goal here is not to fix all those problems, just to leave things no more broken than before.

testVariantListWithMap

public void testVariantListWithMap()

Dynamically add a map to a list that also contains scalars. Assumes that testVariantListDynamic() passed.

testListofListofScalar

public void testListofListofScalar()

The semantics of the ListVector are such that it allows multi-dimensional lists. In this way, it is like a (slightly more normalized) version of the repeated list vector. This form allows arrays to be null.

This test verifies that the (non-repeated) list vector can be used to create multi-dimensional arrays in the result set loader layer. However, the rest of Drill does not support this functionality at present, so this test is more of a proof-of- concept than a necessity.

testRepeatedListOfUnion

public void testRepeatedListOfUnion()

The repeated list type is way off in the weeds in Drill. It is not fully supported and the semantics are very murky as a result. It is not clear how such a structure fits into SQL or into an xDBC client. Still, it is part of Drill at present and must be supported in the result set loader.

This test models using the repeated list as a 2D array of UNION.