org.apache.drill.exec.physical.impl.scan.v3.schema

Class SchemaBasedTracker

java.lang.Object

org.apache.drill.exec.physical.impl.scan.v3.schema.AbstractSchemaTracker

org.apache.drill.exec.physical.impl.scan.v3.schema.SchemaBasedTracker

All Implemented Interfaces:

ScanSchemaTracker

public class SchemaBasedTracker
extends AbstractSchemaTracker

Simple "tracker" based on a defined, fixed schema. The only resolution needed is to identify which columns, if any, are implicit. Readers are ignorant of this state: readers may still produce a subset of the defined columns and so missing columns may be needed. However, readers cannot add to the set of output columns, nor change their types.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.drill.exec.physical.impl.scan.v3.schema.ScanSchemaTracker

ScanSchemaTracker.ProjectionType

Field Summary

Fields inherited from class org.apache.drill.exec.physical.impl.scan.v3.schema.AbstractSchemaTracker

errorContext, isResolved, schema

Constructor Summary

Constructors

Constructor and Description
SchemaBasedTracker(TupleMetadata definedSchema, CustomErrorContext errorContext)

Method Summary

Modifier and Type	Method and Description
void	applyEarlyReaderSchema(TupleMetadata readerSchema) If a reader can define a schema before reading data, apply that schema to the scan schema.
void	applyReaderSchema(TupleMetadata readerOutputSchema, CustomErrorContext errorContext) Once a reader has read a batch, the reader will have provided a type for each projected column which the reader knows about.
ProjectedColumn	columnProjection(String colName) Return the projection for a column, if any.
void	expandImplicitCol(ColumnMetadata resolved, ImplicitColumnMarker marker) Drill defines a wildcard to expand not just reader columns, but also partition columns.
ProjectionFilter	projectionFilter(CustomErrorContext errorContext) Set up a projection filter using the defined schema
int	schemaVersion() Gives the output schema version which will start at some arbitrary positive number.
void	validateProjection(TupleMetadata projection) Validate a projection list (provided as an argument) against a defined schema already held by this tracker.

Methods inherited from class org.apache.drill.exec.physical.impl.scan.v3.schema.AbstractSchemaTracker

applyImplicitCols, checkResolved, errorContext, internalSchema, isResolved, missingColumns, outputSchema, projectionType, readerInputSchema, resolveMissingCols, validateProjection

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SchemaBasedTracker

public SchemaBasedTracker(TupleMetadata definedSchema,
                          CustomErrorContext errorContext)

Method Detail

validateProjection

public void validateProjection(TupleMetadata projection)

Validate a projection list (provided as an argument) against a defined schema already held by this tracker. Ensures that, when we have both a defined schema and projection list, that they are consistent.

Parameters:

projection - the parsed projection list

applyEarlyReaderSchema

public void applyEarlyReaderSchema(TupleMetadata readerSchema)

Description copied from interface: ScanSchemaTracker

If a reader can define a schema before reading data, apply that schema to the scan schema. Allows the scan to report its output schema before the first batch of data if the scan schema becomes resolved after the early reader schema.

projectionFilter

public ProjectionFilter projectionFilter(CustomErrorContext errorContext)

Set up a projection filter using the defined schema

Parameters:

errorContext - the reader-specific error context to use if errors are found

Returns:

a filter used to decide which reader columns to project during reading

applyReaderSchema

public void applyReaderSchema(TupleMetadata readerOutputSchema,
                              CustomErrorContext errorContext)

Description copied from interface: ScanSchemaTracker

Once a reader has read a batch, the reader will have provided a type for each projected column which the reader knows about. For a wildcard projection, the reader will have added all the columns that it found. This call takes the reader output schema and merges it with the current scan schema to resolve dynamic types to concrete types and to add newly discovered columns.

The process can raise an exception if the reader projects a column that it shouldn't (which is not actually possible because of the way the ResultSetLoader works.) An error can also occur if the reader provides a type different than that already defined in the scan schema by a defined schema, a provided schema, or a previous reader in the same scan. In such cases, the reader is expected to have converted its input type to the specified type, which was presumably selected because the reader is capable of the required conversion.

Parameters:

readerOutputSchema - the actual schema produced by a reader when reading a record batch

errorContext - the reader-specific error context to use if errors are found

expandImplicitCol

public void expandImplicitCol(ColumnMetadata resolved,
                              ImplicitColumnMarker marker)

Description copied from interface: ScanSchemaTracker

Drill defines a wildcard to expand not just reader columns, but also partition columns. When the implicit column handlers sees that the query has a wildcard (by calling #isProjectAll()), the handler then determines which partition columns are needed and calls this method to add each one.

schemaVersion

public int schemaVersion()

Description copied from interface: ScanSchemaTracker

Gives the output schema version which will start at some arbitrary positive number.

If schema change is allowed, the schema version allows detecting schema changes as the scan schema moves from one resolved state to the next. Each schema will have a unique, increasing version number. A schema change has occurred if the version is newer than the previous output schema version.

Specified by:

schemaVersion in interface ScanSchemaTracker

Overrides:

schemaVersion in class AbstractSchemaTracker

Returns:

the schema version. The absolute number is not important, rather an increase indicates one or more columns were added at the top level or within a map at some nesting level

columnProjection

public ProjectedColumn columnProjection(String colName)

Description copied from interface: ScanSchemaTracker

Return the projection for a column, if any.