org.apache.drill.exec.physical.impl.xsort

Class TestSortImpl

java.lang.Object

org.apache.drill.test.BaseTest

org.apache.drill.test.DrillTest

org.apache.drill.exec.physical.impl.xsort.TestSortImpl

public class TestSortImpl
extends DrillTest

Tests the external sort implementation: the "guts" of the sort stripped of the Volcano-protocol layer. Assumes the individual components are already tested.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TestSortImpl.DataGenerator Crude-but-effective data generator that produces pseudo-random data that can be easily verified.
static class	TestSortImpl.DataValidator Validate a sort output batch based on the expectation that the key is an ordered sequence of integers, split across multiple batches.
static class	TestSortImpl.SortTestFixture Handy fixture to hold a sort, a set of input row sets (batches) and the output set of row sets (batches.) Pumps the input into the sort and harvests the output.

Nested classes/interfaces inherited from class org.apache.drill.test.DrillTest

DrillTest.MemWatcher

Field Summary

Fields

Modifier and Type	Field and Description
BaseDirTestWatcher	dirTestWatcher

Fields inherited from class org.apache.drill.test.DrillTest

logOutcome, objectMapper, REPEAT_RULE, thrownException, TIMEOUT

Constructor Summary

Constructors

Constructor and Description
TestSortImpl()

Method Summary

Modifier and Type	Method and Description
static SortImpl	makeSortImpl(OperatorFixture fixture, String sortOrder, String nullOrder) Create the sort implementation to be used by test.
void	runJumboBatchTest(OperatorFixture fixture, int rowCount) Set up and run a test for "jumbo" batches, and time the run.
void	runLargeSortTest(OperatorFixture fixture, TestSortImpl.DataGenerator dataGen, TestSortImpl.DataValidator validator) Run a full-blown sort test with multiple input batches.
void	runWideRowsTest(OperatorFixture fixture, int colCount, int rowCount) Run a test using wide rows.
void	testEmptyInput() Test for an input with a schema, but only an empty input batch.
void	testLargeBatch() Hit the sort with the largest possible batch size to ensure nothing is lost at the edges.
void	testModerateBatch() Most tests have used small row counts because we want to probe specific bits of interest.
void	testNullInput() Test for null input (no input batches).
void	testSingleBatch() Degenerate case: two (unsorted) rows in single batch
void	testSingleRow() Degenerate case: single row in single batch.
void	testSpill() Force the sorter to spill, and verify that the resulting data is correct.
void	testTwoBatches() Degenerate case, one row in each of two (unsorted) batches.
void	testWideRows() Test wide rows with the stock copier.

Methods inherited from class org.apache.drill.test.DrillTest

escapeJsonString, finishDrillTest, initDrillTest

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

dirTestWatcher

public final BaseDirTestWatcher dirTestWatcher

Constructor Detail

TestSortImpl

public TestSortImpl()

Method Detail

makeSortImpl

public static SortImpl makeSortImpl(OperatorFixture fixture,
                                    String sortOrder,
                                    String nullOrder)

Create the sort implementation to be used by test.

Parameters:

fixture - operator fixture

sortOrder - sort order as specified by Order.Ordering

nullOrder - null order as specified by Order.Ordering

Returns:

the initialized sort implementation, ready to do work

testNullInput

public void testNullInput()
                   throws Exception

Test for null input (no input batches). Note that, in this case, we never see a schema.

Throws:

Exception

testEmptyInput

public void testEmptyInput()
                    throws Exception

Test for an input with a schema, but only an empty input batch.

Throws:

Exception

testSingleRow

public void testSingleRow()
                   throws Exception

Degenerate case: single row in single batch.

Throws:

Exception

testSingleBatch

public void testSingleBatch()
                     throws Exception

Degenerate case: two (unsorted) rows in single batch

Throws:

Exception

testTwoBatches

public void testTwoBatches()
                    throws Exception

Degenerate case, one row in each of two (unsorted) batches.

Throws:

Exception

runLargeSortTest

public void runLargeSortTest(OperatorFixture fixture,
                             TestSortImpl.DataGenerator dataGen,
                             TestSortImpl.DataValidator validator)

Run a full-blown sort test with multiple input batches. Because we want to generate multiple inputs, we don't create them statically. Instead, we generate them on the fly using a data generator. A matching data validator verifies the output. Here, we are focusing on overall test flow. Separate, detailed, unit tests have already probed the details of each sort component and data type, so we don't need to repeat that whole exercise here; using integer keys is sufficient.

Parameters:

fixture - the operator test fixture

dataGen - input batch generator

validator - validates output batches

runJumboBatchTest

public void runJumboBatchTest(OperatorFixture fixture,
                              int rowCount)

Set up and run a test for "jumbo" batches, and time the run.

Parameters:

fixture - operator test fixture

rowCount - number of rows to test

testModerateBatch

public void testModerateBatch()
                       throws Exception

Most tests have used small row counts because we want to probe specific bits of interest. Try 1000 rows just to ensure things work

Throws:

Exception

testLargeBatch

public void testLargeBatch()
                    throws Exception

Hit the sort with the largest possible batch size to ensure nothing is lost at the edges.

Throws:

Exception

runWideRowsTest

public void runWideRowsTest(OperatorFixture fixture,
                            int colCount,
                            int rowCount)

Run a test using wide rows. This stresses the "copier" portion of the sort and allows us to test the original generated copier and the revised "generic" copier.

Parameters:

fixture - operator test fixture

colCount - number of data (non-key) columns

rowCount - number of rows to generate

testWideRows

public void testWideRows()
                  throws Exception

Test wide rows with the stock copier.

Throws:

Exception

testSpill

public void testSpill()
               throws Exception

Force the sorter to spill, and verify that the resulting data is correct. Uses a specific property of the sort to set the in-memory batch limit so that we don't have to fiddle with filling up memory. The point here is not to test the code that decides when to spill (that was already tested.) Nor to test the spilling mechanism itself (that has also already been tested.) Rather it is to ensure that, when those components are integrated into the sort implementation, that the whole assembly does the right thing.

Throws:

Exception