org.apache.drill.exec.record.metadata

Interface ColumnMetadata

All Superinterfaces:

Propertied

All Known Implementing Classes:

AbstractColumnMetadata, AbstractMapColumnMetadata, DictColumnMetadata, DynamicColumn, MapColumnMetadata, PrimitiveColumnMetadata, ProjectedColumn, RepeatedListColumnMetadata, VariantColumnMetadata

public interface ColumnMetadata
extends Propertied

Metadata description of a column including names, types and structure information.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	ColumnMetadata.StructureType Rough characterization of Drill types into metadata categories.

Field Summary

Fields

Modifier and Type	Field and Description
static String	BLANK_AS_NULL Convert blanks to null values (if the column is nullable), or fill with the default value (non-nullable.)
static String	BLANK_AS_PROP Indicates how to handle blanks.
static String	BLANK_AS_ZERO Convert blanks for numeric fields to 0.
static int	DEFAULT_ARRAY_SIZE
static String	DEFAULT_VALUE_PROP Default value represented as a string.
static String	EXCLUDE_FROM_WILDCARD Indicates whether to project the column in a wildcard (*) query.
static String	EXPECTED_CARDINALITY_PROP Predicted number of elements per array entry.
static String	EXPECTED_WIDTH_PROP Expected (average) width for variable-width columns.
static String	FORMAT_PROP Optional format to use when converting to/from string values.
static String	IMPLICIT_COL_TYPE Indicates that a provided schema column is an implicit column (one defined by Drill rather than the reader.) Allows the implicit schema to reify partition names, say, as reader-specific names.
static String	IMPLICIT_FILENAME File name implicit column type.
static String	IMPLICIT_FILEPATH File path implicit column type.
static String	IMPLICIT_FQN Fully-qualified name implicit column type.
static String	IMPLICIT_PARTITION_PREFIX Prefix for partition directories.
static String	IMPLICIT_SUFFIX File suffix implicit column type.

Fields inherited from interface org.apache.drill.exec.record.metadata.Propertied

DRILL_PROP_PREFIX

Method Summary

Modifier and Type	Method and Description
void	bind(TupleMetadata parentTuple)
ColumnMetadata	childSchema() Schema of inner dimension for MULTI_ARRAY columns.
ColumnMetadata	cloneEmpty() Create an empty version of this column.
String	columnString() Converts column metadata into string representation accepted by the table schema parser.
ColumnMetadata	copy()
DateTimeFormatter	dateTimeFormatter() Returns the formatter to use for date/time values.
Object	decodeDefaultValue() Returns the default value decoded into object form.
String	defaultValue() Returns the default value for this column in String literal representation.
int	dimensions()
MaterializedField	emptySchema()
int	expectedElementCount() Returns the expected array cardinality for array columns, or 1 for non-array columns.
int	expectedWidth() Get the expected width for a column.
String	format()
boolean	isArray()
boolean	isDict()
boolean	isDynamic() Reports if the column is dynamic.
boolean	isEquivalent(ColumnMetadata other) Report whether one column is equivalent to another.
boolean	isMap()
boolean	isMultiList() Determine if the schema represents a column with a LIST type with UNION elements.
boolean	isNullable()
boolean	isVariableWidth()
boolean	isVariant()
TypeProtos.MajorType	majorType()
TypeProtos.DataMode	mode()
String	name()
int	precision()
int	scale()
MaterializedField	schema()
void	setDefaultValue(String value) Sets the default value property using the string-encoded form of the value.
void	setExpectedElementCount(int childCount) For an array column, specify the expected average array cardinality.
void	setExpectedWidth(int width) For variable-width columns, specify the expected column width to be used when allocating a new vector.
void	setFormat(String value)
ColumnMetadata.StructureType	structureType()
TupleMetadata	tupleSchema() Schema for TUPLE columns.
TypeProtos.MinorType	type()
String	typeString() Converts type metadata into string representation accepted by the table schema parser.
Object	valueFromString(String value)
String	valueToString(Object value)
VariantMetadata	variantSchema() Schema for VARIANT columns.

Methods inherited from interface org.apache.drill.exec.record.metadata.Propertied

booleanProperty, booleanProperty, hasProperties, intProperty, intProperty, pluginPrefix, properties, property, property, removeProperty, setBooleanProperty, setIntProperty, setProperties, setProperty

Field Detail

EXPECTED_CARDINALITY_PROP

static final String EXPECTED_CARDINALITY_PROP

Predicted number of elements per array entry. Default is taken from the often hard-coded value of 10.

See Also:

Constant Field Values

DEFAULT_VALUE_PROP

static final String DEFAULT_VALUE_PROP

Default value represented as a string.

See Also:

Constant Field Values

EXPECTED_WIDTH_PROP

static final String EXPECTED_WIDTH_PROP

Expected (average) width for variable-width columns.

See Also:

Constant Field Values

FORMAT_PROP

static final String FORMAT_PROP

Optional format to use when converting to/from string values.

See Also:

Constant Field Values

BLANK_AS_PROP

static final String BLANK_AS_PROP

Indicates how to handle blanks. Must be one of the valid values defined in AbstractConvertFromString. Normally set on the converter by the plugin rather than by the user in the schema.

See Also:

Constant Field Values

BLANK_AS_NULL

static final String BLANK_AS_NULL

Convert blanks to null values (if the column is nullable), or fill with the default value (non-nullable.)

See Also:

Constant Field Values

BLANK_AS_ZERO

static final String BLANK_AS_ZERO

Convert blanks for numeric fields to 0. For non-numeric fields, convert to null (for nullable) or the default value (for non-nullable). Works best if non-numeric fields are declared as nullable.

See Also:

Constant Field Values

EXCLUDE_FROM_WILDCARD

static final String EXCLUDE_FROM_WILDCARD

Indicates whether to project the column in a wildcard (*) query. Special columns may be excluded from projection. Certain "special" columns may be available only when explicitly requested. For example, the log reader has a "_raw" column which includes the entire input line before parsing. This column can be requested explicitly:
SELECT foo, bar, _raw FROM ...
but the column will not be included when using the wildcard:
SELECT * FROM ...

Marking a column (either in the provided schema or the reader schema) will prevent that column from appearing in a wildcard expansion.

See Also:

Constant Field Values

DEFAULT_ARRAY_SIZE

static final int DEFAULT_ARRAY_SIZE

See Also:

Constant Field Values

IMPLICIT_COL_TYPE

static final String IMPLICIT_COL_TYPE

Indicates that a provided schema column is an implicit column (one defined by Drill rather than the reader.) Allows the implicit schema to reify partition names, say, as reader-specific names. For example, dir0 might be reified as year, etc.

Available when the underlying reader supports implicit columns. The value is the defined implicit column name (not the name set via system/session options.) Using the defined name makes the provided schema immune from runtime changes to column names.

As the result of adding this feature, any column not tagged as implicit is a reader column, even if that column happens to have the same (currently selected runtime) name as an implicit column.

See Also:

Constant Field Values

IMPLICIT_FQN

static final String IMPLICIT_FQN

Fully-qualified name implicit column type.

See Also:

Constant Field Values

IMPLICIT_FILEPATH

static final String IMPLICIT_FILEPATH

File path implicit column type.

See Also:

Constant Field Values

IMPLICIT_FILENAME

static final String IMPLICIT_FILENAME

File name implicit column type.

See Also:

Constant Field Values

IMPLICIT_SUFFIX

static final String IMPLICIT_SUFFIX

File suffix implicit column type.

See Also:

Constant Field Values

IMPLICIT_PARTITION_PREFIX

static final String IMPLICIT_PARTITION_PREFIX

Prefix for partition directories. dir0 is the table root folder, dir1 the first subdirectory, and so on. Directories that don't exist in the actual file path take a NULL value.

See Also:

Constant Field Values

Method Detail

structureType

ColumnMetadata.StructureType structureType()

tupleSchema

TupleMetadata tupleSchema()

Schema for TUPLE columns.

Returns:

the tuple schema

variantSchema

VariantMetadata variantSchema()

Schema for VARIANT columns.

Returns:

the variant schema

childSchema

ColumnMetadata childSchema()

Schema of inner dimension for MULTI_ARRAY columns. If an array is 3D, the outer column represents all 3 dimensions. outer.childSchema() gives another MULTI_ARRAY for the inner 2D array. outer.childSchema().childSchema() gives a column of some other type (but repeated) for the 1D array.

Sorry for the mess, but it is how the code works and we are not in a position to revisit data type fundamentals.

Returns:

the description of the (n-1) st dimension.

schema

MaterializedField schema()

emptySchema

MaterializedField emptySchema()

name

String name()

type

TypeProtos.MinorType type()

majorType

TypeProtos.MajorType majorType()

mode

TypeProtos.DataMode mode()

dimensions

int dimensions()

isNullable

boolean isNullable()

isArray

boolean isArray()

isVariableWidth

boolean isVariableWidth()

isMap

boolean isMap()

isVariant

boolean isVariant()

isDict

boolean isDict()

isDynamic

boolean isDynamic()

Reports if the column is dynamic. A dynamic column is one with a "type to be named later." It is valid for describing a dynamic schema, but not for creating vectors; to create a vector the column must be resolved to a concrete type. The context should make it clear if any columns can be dynamic.

Returns:

true if the column does not yet have a concrete type, false if the column type is concrete

isMultiList

boolean isMultiList()

Determine if the schema represents a column with a LIST type with UNION elements. (Lists can be of a single type (with nullable elements) or can be of unions.)

Returns:

true if the column is of type LIST of UNIONs

isEquivalent

boolean isEquivalent(ColumnMetadata other)

Report whether one column is equivalent to another. Columns are equivalent if they have the same name, type and structure (ignoring internal structure such as properties.)

setExpectedWidth

void setExpectedWidth(int width)

For variable-width columns, specify the expected column width to be used when allocating a new vector. Does nothing for fixed-width columns.

Parameters:

width - the expected column width

expectedWidth

int expectedWidth()

Get the expected width for a column. This is the actual width for fixed- width columns, the specified width (defaulting to 50) for variable-width columns.

Returns:

the expected column width of the each data value. Does not include "overhead" space such as for the null-value vector or offset vector

setExpectedElementCount

void setExpectedElementCount(int childCount)

For an array column, specify the expected average array cardinality. Ignored for non-array columns. Used when allocating new vectors.

Parameters:

childCount - the expected average array cardinality. Defaults to 1 for non-array columns, 10 for array columns

expectedElementCount

int expectedElementCount()

Returns the expected array cardinality for array columns, or 1 for non-array columns.

Returns:

the expected value cardinality per value (per-row for top-level columns, per array element for arrays within lists)

setFormat

void setFormat(String value)

format

String format()

dateTimeFormatter

DateTimeFormatter dateTimeFormatter()

Returns the formatter to use for date/time values. Only valid for date/time columns.

Returns:

setDefaultValue

void setDefaultValue(String value)

Sets the default value property using the string-encoded form of the value. The default value is used for filling a vector when no real data is available.

Parameters:

value - the default value in String representation

defaultValue

String defaultValue()

Returns the default value for this column in String literal representation.

Returns:

the default value in String literal representation, or null if no default value has been set

decodeDefaultValue

Object decodeDefaultValue()

Returns the default value decoded into object form. This is the same as:

decodeValue(defaultValue());
 

Returns:

the default value decode as an object that can be passed to the ColumnWriter#setObject() method.

valueToString

String valueToString(Object value)

valueFromString

Object valueFromString(String value)

cloneEmpty

ColumnMetadata cloneEmpty()

Create an empty version of this column. If the column is a scalar, produces a simple copy. If a map, produces a clone without child columns.

Returns:

empty clone of this column

precision

int precision()

scale

int scale()

bind

void bind(TupleMetadata parentTuple)

copy

ColumnMetadata copy()

typeString

String typeString()

Converts type metadata into string representation accepted by the table schema parser.

Returns:

type metadata string representation

columnString

String columnString()

Converts column metadata into string representation accepted by the table schema parser.

Returns:

column metadata string representation