org.apache.drill.exec.store.parquet

Class ParquetGroupScan

java.lang.Object

org.apache.drill.exec.physical.base.AbstractBase

org.apache.drill.exec.physical.base.AbstractGroupScan

org.apache.drill.exec.physical.base.AbstractFileGroupScan

org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata<ParquetMetadataProvider>

org.apache.drill.exec.store.parquet.AbstractParquetGroupScan

org.apache.drill.exec.store.parquet.ParquetGroupScan

All Implemented Interfaces:

Iterable<PhysicalOperator>, GraphValue<PhysicalOperator>, FileGroupScan, FragmentLeaf, GroupScan, HasAffinity, Leaf, PhysicalOperator, Scan

public class ParquetGroupScan
extends AbstractParquetGroupScan

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.drill.exec.store.parquet.AbstractParquetGroupScan

AbstractParquetGroupScan.RowGroupScanFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>

Nested classes/interfaces inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata

AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>>

Field Summary

Fields inherited from class org.apache.drill.exec.store.parquet.AbstractParquetGroupScan

entries, mappings, readerConfig, rowGroups

Fields inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata

columns, files, fileSet, filter, matchAllMetadata, maxRecords, metadataProvider, nonInterestingColumnsMetadata, partitionColumns, partitions, segments, tableMetadata, usedMetastore

Fields inherited from class org.apache.drill.exec.physical.base.AbstractBase

INIT_ALLOCATION, initialAllocation, MAX_ALLOCATION, maxAllocation

Fields inherited from interface org.apache.drill.exec.physical.base.GroupScan

ALL_COLUMNS

Constructor Summary

Constructors

Constructor and Description
ParquetGroupScan(StoragePluginRegistry engineRegistry, String userName, List<ReadEntryWithPath> entries, StoragePluginConfig storageConfig, FormatPluginConfig formatConfig, List<SchemaPath> columns, org.apache.hadoop.fs.Path selectionRoot, org.apache.hadoop.fs.Path cacheFileRoot, ParquetReaderConfig readerConfig, LogicalExpression filter, TupleMetadata schema)
ParquetGroupScan(String userName, FileSelection selection, ParquetFormatPlugin formatPlugin, List<SchemaPath> columns, ParquetReaderConfig readerConfig, LogicalExpression filter, MetadataProviderManager metadataProviderManager)
ParquetGroupScan(String userName, FileSelection selection, ParquetFormatPlugin formatPlugin, List<SchemaPath> columns, ParquetReaderConfig readerConfig, MetadataProviderManager metadataProviderManager)

Method Summary

Modifier and Type	Method and Description
ParquetGroupScan	clone(FileSelection selection)
GroupScan	clone(List<SchemaPath> columns) Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .
protected AbstractParquetGroupScan	cloneWithFileSelection(Collection<org.apache.hadoop.fs.Path> filePaths)
protected ParquetTableMetadataProviderBuilder<?>	defaultTableMetadataProviderBuilder(MetadataProviderManager source) Returns TableMetadataProviderBuilder instance which may provide metadata without using Drill Metastore.
AnalyzeInfoProvider	getAnalyzeInfoProvider() Returns AnalyzeInfoProvider instance which will be used when running ANALYZE statement.
org.apache.hadoop.fs.Path	getCacheFileRoot()
protected Collection<CoordinationProtos.DrillbitEndpoint>	getDrillbits()
StoragePluginConfig	getEngineConfig()
protected AbstractParquetGroupScan.RowGroupScanFilterer<?>	getFilterer() Returns holder for metadata values which provides API to filter metadata and build new group scan instance using filtered metadata.
ParquetFormatConfig	getFormatConfig()
PhysicalOperator	getNewWithChildren(List<PhysicalOperator> children) Regenerate with this node with a new set of children.
protected List<String>	getPartitionValues(LocationProvider locationProvider)
org.apache.hadoop.fs.Path	getSelectionRoot() Returns path to the selection root.
ParquetRowGroupScan	getSpecificScan(int minorFragmentId)
protected boolean	supportsFileImplicitColumns()
protected ParquetTableMetadataProviderBuilder<?>	tableMetadataProviderBuilder(MetadataProviderManager source) Returns TableMetadataProviderBuilder instance based on specified MetadataProviderManager source.
String	toString()

Methods inherited from class org.apache.drill.exec.store.parquet.AbstractParquetGroupScan

applyAssignments, applyFilter, applyLimit, canPushdownProjects, getEntries, getFiles, getMaxParallelizationWidth, getOperatorAffinity, getReadEntries, getReaderConfig, getReaderConfigForSerialization, getRowGroupsMetadata, modifyFileSelection, pruneForPartitions, pruneRowGroupsForFiles, supportsFilterPushDown

Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata

checkMetadataConsistency, getColumns, getColumnValueCount, getDigest, getFileSet, getFilesMetadata, getFilter, getFilterPredicate, getFilterPredicate, getFilterString, getMaxRecords, getMetadataProvider, getNextOrEmpty, getNonInterestingColumnsMetadata, getPartitionColumns, getPartitionsMetadata, getPartitionValue, getScanStats, getSchema, getSegmentsMetadata, getTableMetadata, getTypeForColumn, hasFiles, init, isAllDataPruned, isGroupScanFullyMatchesFilter, isImplicitOrPartCol, isMatchAllMetadata, limitMetadata, pruneForPartitions, setFilter, setFilterForRuntime, supportsLimitPushdown, usedMetastore

Methods inherited from class org.apache.drill.exec.physical.base.AbstractFileGroupScan

supportsPartitionFilterPushdown

Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScan

accept, enforceWidth, getDistributionAffinity, getInitialAllocation, getMaxAllocation, getMinParallelizationWidth, getOperatorType, getScanStats, isDistributed, isExecutable, iterator

Methods inherited from class org.apache.drill.exec.physical.base.AbstractBase

accept, getCost, getOperatorId, getSVMode, getUserName, isBufferedOperator, setCost, setMaxAllocation, setOperatorId

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.drill.exec.physical.base.GroupScan

enforceWidth, getMinParallelizationWidth, getScanStats, isDistributed

Methods inherited from interface org.apache.drill.exec.physical.base.PhysicalOperator

accept, getCost, getInitialAllocation, getMaxAllocation, getOperatorId, getOperatorType, getSVMode, getUserName, isBufferedOperator, isExecutable, setCost, setMaxAllocation, setOperatorId

Methods inherited from interface org.apache.drill.common.graph.GraphValue

accept

Methods inherited from interface java.lang.Iterable

forEach, iterator, spliterator

Methods inherited from interface org.apache.drill.exec.physical.base.HasAffinity

getDistributionAffinity

Constructor Detail

ParquetGroupScan

public ParquetGroupScan(StoragePluginRegistry engineRegistry,
                        String userName,
                        List<ReadEntryWithPath> entries,
                        StoragePluginConfig storageConfig,
                        FormatPluginConfig formatConfig,
                        List<SchemaPath> columns,
                        org.apache.hadoop.fs.Path selectionRoot,
                        org.apache.hadoop.fs.Path cacheFileRoot,
                        ParquetReaderConfig readerConfig,
                        LogicalExpression filter,
                        TupleMetadata schema)
                 throws IOException

Throws:

IOException

ParquetGroupScan

public ParquetGroupScan(String userName,
                        FileSelection selection,
                        ParquetFormatPlugin formatPlugin,
                        List<SchemaPath> columns,
                        ParquetReaderConfig readerConfig,
                        MetadataProviderManager metadataProviderManager)
                 throws IOException

Throws:

IOException

ParquetGroupScan

public ParquetGroupScan(String userName,
                        FileSelection selection,
                        ParquetFormatPlugin formatPlugin,
                        List<SchemaPath> columns,
                        ParquetReaderConfig readerConfig,
                        LogicalExpression filter,
                        MetadataProviderManager metadataProviderManager)
                 throws IOException

Throws:

IOException

Method Detail

getFormatConfig

public ParquetFormatConfig getFormatConfig()

getEngineConfig

public StoragePluginConfig getEngineConfig()

getSelectionRoot

public org.apache.hadoop.fs.Path getSelectionRoot()

Description copied from interface: GroupScan

Returns path to the selection root. If this GroupScan cannot provide selection root, it returns null.

Specified by:

getSelectionRoot in interface GroupScan

Overrides:

getSelectionRoot in class AbstractGroupScan

Returns:

path to the selection root

getCacheFileRoot

public org.apache.hadoop.fs.Path getCacheFileRoot()

getSpecificScan

public ParquetRowGroupScan getSpecificScan(int minorFragmentId)

getNewWithChildren

public PhysicalOperator getNewWithChildren(List<PhysicalOperator> children)

Description copied from interface: PhysicalOperator

Regenerate with this node with a new set of children. This is used in the case of materialization or optimization.

clone

public GroupScan clone(List<SchemaPath> columns)

Description copied from interface: GroupScan

Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .

Specified by:

clone in interface GroupScan

Overrides:

clone in class AbstractGroupScan

clone

public ParquetGroupScan clone(FileSelection selection)
                       throws IOException

Specified by:

clone in interface FileGroupScan

Overrides:

clone in class AbstractFileGroupScan

Throws:

IOException

toString

public String toString()

Overrides:

toString in class Object

cloneWithFileSelection

protected AbstractParquetGroupScan cloneWithFileSelection(Collection<org.apache.hadoop.fs.Path> filePaths)
                                                   throws IOException

Specified by:

cloneWithFileSelection in class AbstractParquetGroupScan

Throws:

IOException

getFilterer

protected AbstractParquetGroupScan.RowGroupScanFilterer<?> getFilterer()

Description copied from class: AbstractGroupScanWithMetadata

Returns holder for metadata values which provides API to filter metadata and build new group scan instance using filtered metadata.

Specified by:

getFilterer in class AbstractParquetGroupScan

getDrillbits

protected Collection<CoordinationProtos.DrillbitEndpoint> getDrillbits()

Specified by:

getDrillbits in class AbstractParquetGroupScan

supportsFileImplicitColumns

protected boolean supportsFileImplicitColumns()

Specified by:

supportsFileImplicitColumns in class AbstractGroupScanWithMetadata<ParquetMetadataProvider>

getPartitionValues

protected List<String> getPartitionValues(LocationProvider locationProvider)

Specified by:

getPartitionValues in class AbstractGroupScanWithMetadata<ParquetMetadataProvider>

getAnalyzeInfoProvider

public AnalyzeInfoProvider getAnalyzeInfoProvider()

Description copied from interface: GroupScan

Returns AnalyzeInfoProvider instance which will be used when running ANALYZE statement.

Specified by:

getAnalyzeInfoProvider in interface GroupScan

Overrides:

getAnalyzeInfoProvider in class AbstractGroupScan

Returns:

AnalyzeInfoProvider instance

defaultTableMetadataProviderBuilder

protected ParquetTableMetadataProviderBuilder<?> defaultTableMetadataProviderBuilder(MetadataProviderManager source)

Description copied from class: AbstractGroupScanWithMetadata

Returns TableMetadataProviderBuilder instance which may provide metadata without using Drill Metastore.

Specified by:

defaultTableMetadataProviderBuilder in class AbstractParquetGroupScan

Parameters:

source - metadata provider manager

Returns:

TableMetadataProviderBuilder instance

tableMetadataProviderBuilder

protected ParquetTableMetadataProviderBuilder<?> tableMetadataProviderBuilder(MetadataProviderManager source)

Description copied from class: AbstractGroupScanWithMetadata

Returns TableMetadataProviderBuilder instance based on specified MetadataProviderManager source.

Specified by:

tableMetadataProviderBuilder in class AbstractGroupScanWithMetadata<ParquetMetadataProvider>

Parameters:

source - metadata provider manager

Returns:

TableMetadataProviderBuilder instance