Package org.apache.drill.test

Provides a variety of test framework tools to simplify Drill unit tests and ad-hoc tests created while developing features.

See: Description

Interface Summary

Interface	Description
DrillTestWrapper.TestServices
QueryBuilder.VectorQueryReader<T,V>	VectorQueryReader function interface

Class Summary

Class	Description
BaseDirTestWatcher	Overview
BaseFixture	Base class for "fixtures." Provides the basics such as the Drill configuration, a memory allocator and so on.
BaseTest	Contains patchers that must be executed at the very beginning of test runs.
BaseTestQuery	deprecated Use ClusterTest instead.
BaseTestQuery.ClassicTestServices
BaseTestQuery.SilentListener
BufferingQueryEventListener	Drill query event listener that buffers rows into a producer-consumer queue.
BufferingQueryEventListener.QueryEvent
ClientFixture	Represents a Drill client.
ClientFixture.ClientBuilder
ClientFixture.StatementParser	Very simple parser for semi-colon separated lists of SQL statements which handles quoted semicolons.
ClusterFixture	Test fixture to start a Drillbit with provide options, create a client, and execute queries.
ClusterFixture.FixtureTestServices	Shim class to allow the TestBuilder class to work with the cluster fixture.
ClusterFixtureBuilder	Build a Drillbit and client with the options provided.
ClusterFixtureBuilder.RuntimeOption
ClusterMockStorageFixture
ClusterTest	Base class for tests that use a single cluster fixture for a set of tests.
ConfigBuilder	Builds a DrillConfig for use in tests.
DirTestWatcher	This class is used to create consistently named and safe temp directories for unit tests.
Drill2130CommonHamcrestConfigurationTest
DrillAssert
DrillTest
DrillTest.MemWatcher
DrillTestWrapper	Encapsulates the options for a Drill unit test, as well as the execution methods to perform the tests and validation of results.
ExampleTest	Example test case using the Drill cluster fixture.
FieldDef	Basic representation of a column parsed from a query profile.
HadoopUtils
LegacyOperatorTestBuilder	Deprecated Use OperatorTestBuilder instead.
LogFixture	Establishes test-specific logging without having to alter the global logback-test.xml file.
LogFixture.LogFixtureBuilder	Builds the log settings to be used for a test.
LogFixture.LogSpec	Memento for a logger name and level.
OperatorFixture	Test fixture for operator and (especially) "sub-operator" tests.
OperatorFixture.Builder	Builds an operator fixture based on a set of config options and system/session options.
OperatorFixture.MockExecutorState
OperatorFixture.MockFragmentContext	Provide a simplified test-time code generation context that uses the same code generation mechanism as the full Drill, but provide test-specific versions of various other services.
OperatorFixture.MockOperatorContext
OperatorTestBuilder
OperatorTestBuilderTest
OperatorTestBuilderTest.MockPhysicalOperator
PhysicalOpUnitTestBase
PhysicalOpUnitTestBase.BatchIterator
PhysicalOpUnitTestBase.MockExecutorFragmentContext	Note
PhysicalOpUnitTestBase.MockPhysicalOperator	Note
PrintingResultsListener
PrintingUtils	This class contains utility methods to run lambda functions with the necessary LogFixture boilerplate to print results to stdout for debugging purposes.
ProfileParser	Parses a query profile and provides access to various bits of the profile for diagnostic purposes during tests.
ProfileParser.Aggregator
ProfileParser.CostPrinter	Print out the tree showing a comparison of estimated vs.
ProfileParser.FindOpVisitor
ProfileParser.FragInfo	Information for a fragment, including the operators in that fragment and the set of minor fragments.
ProfileParser.MinorFragInfo	Information about a minor fragment as parsed from the profile.
ProfileParser.OperatorProfile	Detailed information about each operator within a minor fragment for a major fragment.
ProfileParser.OperatorSummary	Information about an operator definition: the plan-time information that appears in the plan portion of the profile.
ProfileParser.TimePrinter
ProfileParser.TreePrinter	Print the operator tree for analysis.
ProfileParser.TreeVisitor	Visit a tree of operator definitions to support printing, analysis and other tasks.
QueryBatchIterator	Iterator over batches returned from a query.
QueryBuilder	Builder for a Drill query.
QueryBuilder.PlanMatcher	Collects expected and non-expected query patterns.
QueryBuilder.QuerySummary	Summary results of a query: records, batches, run time.
QueryBuilder.QuerySummaryFuture	The future used to wait for the completion of an async query.
QueryBuilder.SummaryOnlyQueryEventListener	Listener used to retrieve the query summary (only) asynchronously using a QueryBuilder.QuerySummaryFuture.
QueryResultSet	Returns query results as an iterator over row sets.
QueryRowSetIterator	Converts an incoming set of record batches into an iterator over a set of row sets.
QueryRowSetReader
QueryTestUtil	Utilities useful for tests that issue SQL queries.
RestClientFixture	Represents a client for the Drill Rest API.
RestClientFixture.Builder	A builder for the rest client.
StatementParser	Very simple parser for semi-colon separated lists of SQL statements which handles quoted semicolons.
SubDirTestWatcher	This is a TestWatcher which is used to create and delete sub directories before and after unit tests respectively.
SubDirTestWatcher.Builder	This is a builder for a SubDirTestWatcher.
SubOperatorTest
TestBuilder
TestGracefulShutdown
TestTools
UserExceptionMatcher	Matcher for UserException that matches if expected type and actual type are the same, and expected message is contained in the actual message.

Enum Summary

Enum	Description
BaseDirTestWatcher.DirType	An enum used to represent the directories mapped to the dfs.root and dfs.tmp workspaces respectively.
BufferingQueryEventListener.QueryEvent.Type
FieldDef.Type
FieldDef.TypeHint
TestTools.FileSource

Package org.apache.drill.test Description

Provides a variety of test framework tools to simplify Drill unit tests and ad-hoc tests created while developing features. Key components include:

• ClusterFixtureBuilder: Builder pattern to create an embedded Drillbit, or cluster of Drillbits, using a specified set of configuration, session and system options.
• ClusterFixture: The cluster created by the builder.
• ClientFixture: A facade to the Drill client that provides convenience methods for setting session options, running queries and so on. A client is associated with a cluster. If tests desire, multiple clients can be created for a single cluster, though most need just one client. A builder exists for clients, but most tests get the client directly from the cluster.
• QueryBuilder: a builder pattern for constructing and running any form of query (SQL, logical or physical) and running the query in a wide variety of ways (just count the rows, return the results as a list, run using a listener, etc.)
• QuerySummary: a summary of a query returned from running the query. Contains the query ID, the row count, the batch count and elapsed run time.
• ProfileParser: A simple tool to load a query profile and provide access to the profile structure. Also prints the key parts of the profile for diagnostic purposes.
• LogFixture: Allows per-test changes to log settings to, say, send a particular logger to the console for easier debugging, or to suppress logging of a deliberately created failure.
• BaseDirTestWatcher: Creates temporary directories which are used for the following aspects of drill in unit tests:
  • The drill.tmp-dir property: A temp directory (BaseDirTestWatcher.getTmpDir()) is created and configured as the tmp directory for drill bits.
  • The drill.exec.sys.store.provider.local.path property: A temp directory (BaseDirTestWatcher.getStoreDir()) is created and configured as the store directory for drillbits.
  • The dfs.default workspace: A temp directory (BaseDirTestWatcher.getRootDir() is created and configured as the directory for this workspace.
  • The dfs.root workspace: dfs.root is configured to use the same temp directory ad dfs.default.
  • The dfs.tmp workspace: A temp directory (BaseDirTestWatcher.getDfsTestTmpDir()) is created and configured as the directory for this workspace.
  By default a BaseDirTestWatcher is used in tests that extend BaseTestQuery and tests that use the ClusterFixture.

Usage

A typical test using this framework looks like this:

   @org.junit.Rule
  public final BaseDirTestWatcher dirTestWatcher = new BaseDirTestWatcher();

   @Test
  public void exampleTest() throws Exception {
    createEmployeeCsv(dirTestWatcher.getRootDir());

    // Configure the cluster. One Drillbit by default.
    FixtureBuilder builder = ClusterFixture.builder(dirTestWatcher)
        // Set up per-test specialized config and session options.
        .configProperty(ExecConstants.SYS_STORE_PROVIDER_LOCAL_ENABLE_WRITE, true)
        .configProperty(ExecConstants.REMOVER_ENABLE_GENERIC_COPIER, true)
        .sessionOption(ExecConstants.MAX_QUERY_MEMORY_PER_NODE_KEY, 3L * 1024 * 1024 * 1024)
        .maxParallelization(1);

    // Launch the cluster and client.
    try (ClusterFixture cluster = builder.build();
         ClientFixture client = cluster.clientFixture()) {

      // Run a query (using the mock data source) and print a summary.
      String sql = "SELECT id_i FROM dfs.`employee.csv` ORDER BY id_i";
      QuerySummary summary = client.queryBuilder().sql(sql).run();
      assertEquals(1_000_000, summary.recordCount());
      System.out.println(String.format("Sorted %,d records in %d batches.", summary.recordCount(), summary.batchCount()));
      System.out.println(String.format("Query Id: %s, elapsed: %d ms", summary.queryIdString(), summary.runTimeMs()));
      client.parseProfile(summary.queryIdString()).print();
    }
  }
 

Typical usage for the logging fixture:

  @Test
 public void myTest() {
   LogFixtureBuilder logBuilder = LogFixture.builder()
          .toConsole()
          .disable() // Silence all other loggers
          .logger(ExternalSortBatch.class, Level.DEBUG);
   try (LogFixture logs = logBuilder.build()) {
     // Test code here
   }
 }