Package org.apache.drill.exec.store.hive
Class HiveDrillNativeParquetScan
java.lang.Object
org.apache.drill.exec.physical.base.AbstractBase
org.apache.drill.exec.physical.base.AbstractGroupScan
org.apache.drill.exec.physical.base.AbstractFileGroupScan
org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata<ParquetMetadataProvider>
org.apache.drill.exec.store.parquet.AbstractParquetGroupScan
org.apache.drill.exec.store.hive.HiveDrillNativeParquetScan
- All Implemented Interfaces:
Iterable<PhysicalOperator>
,GraphValue<PhysicalOperator>
,FileGroupScan
,FragmentLeaf
,GroupScan
,HasAffinity
,Leaf
,PhysicalOperator
,Scan
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.drill.exec.store.parquet.AbstractParquetGroupScan
AbstractParquetGroupScan.RowGroupScanFilterer<B extends AbstractParquetGroupScan.RowGroupScanFilterer<B>>
Nested classes/interfaces inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata
AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>>
-
Field Summary
Fields inherited from class org.apache.drill.exec.store.parquet.AbstractParquetGroupScan
entries, mappings, readerConfig, rowGroups
Fields inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata
columns, files, fileSet, filter, limit, matchAllMetadata, metadataProvider, nonInterestingColumnsMetadata, partitionColumns, partitions, segments, tableMetadata, usedMetastore
Fields inherited from class org.apache.drill.exec.physical.base.AbstractBase
INIT_ALLOCATION, initialAllocation, MAX_ALLOCATION, maxAllocation, userName
Fields inherited from interface org.apache.drill.exec.physical.base.GroupScan
ALL_COLUMNS
-
Constructor Summary
ConstructorDescriptionHiveDrillNativeParquetScan
(String userName, List<SchemaPath> columns, HiveStoragePlugin hiveStoragePlugin, List<HiveMetadataProvider.LogicalInputSplit> logicalInputSplits, Map<String, String> confProperties, ParquetReaderConfig readerConfig) HiveDrillNativeParquetScan
(String userName, List<SchemaPath> columns, HiveStoragePlugin hiveStoragePlugin, List<HiveMetadataProvider.LogicalInputSplit> logicalInputSplits, Map<String, String> confProperties, ParquetReaderConfig readerConfig, LogicalExpression filter) HiveDrillNativeParquetScan
(StoragePluginRegistry engineRegistry, String userName, HiveStoragePluginConfig hiveStoragePluginConfig, List<SchemaPath> columns, List<ReadEntryWithPath> entries, HivePartitionHolder hivePartitionHolder, Map<String, String> confProperties, ParquetReaderConfig readerConfig, LogicalExpression filter, TupleMetadata schema) -
Method Summary
Modifier and TypeMethodDescriptionclone
(List<SchemaPath> columns) Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .clone
(FileSelection selection) protected AbstractParquetGroupScan
cloneWithFileSelection
(Collection<org.apache.hadoop.fs.Path> filePaths) protected HiveParquetTableMetadataProvider.Builder
ReturnsTableMetadataProviderBuilder
instance which may provide metadata without using Drill Metastore.protected Collection<CoordinationProtos.DrillbitEndpoint>
protected AbstractParquetGroupScan.RowGroupScanFilterer<?>
Returns holder for metadata values which provides API to filter metadata and build new group scan instance using filtered metadata.getNewWithChildren
(List<PhysicalOperator> children) Regenerate with this node with a new set of children.getPartitionValues
(LocationProvider locationProvider) getSpecificScan
(int minorFragmentId) protected boolean
protected HiveParquetTableMetadataProvider.Builder
ReturnsTableMetadataProviderBuilder
instance based on specifiedMetadataProviderManager
source.toString()
Methods inherited from class org.apache.drill.exec.store.parquet.AbstractParquetGroupScan
applyAssignments, applyFilter, applyLimit, canPushdownProjects, getEntries, getFiles, getMaxParallelizationWidth, getOperatorAffinity, getReadEntries, getReaderConfig, getReaderConfigForSerialization, getRowGroupsMetadata, modifyFileSelection, pruneForPartitions, pruneRowGroupsForFiles, supportsFilterPushDown
Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata
checkMetadataConsistency, getColumns, getColumnValueCount, getDigest, getFileSet, getFilesMetadata, getFilter, getFilterPredicate, getFilterPredicate, getFilterString, getLimit, getMetadataProvider, getNextOrEmpty, getNonInterestingColumnsMetadata, getPartitionColumns, getPartitionsMetadata, getPartitionValue, getScanStats, getSchema, getSegmentsMetadata, getTableMetadata, getTypeForColumn, hasFiles, init, isAllDataPruned, isGroupScanFullyMatchesFilter, isImplicitOrPartCol, isMatchAllMetadata, limitMetadata, pruneForPartitions, setFilter, setFilterForRuntime, supportsLimitPushdown, usedMetastore
Methods inherited from class org.apache.drill.exec.physical.base.AbstractFileGroupScan
supportsPartitionFilterPushdown
Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScan
accept, enforceWidth, getAnalyzeInfoProvider, getDistributionAffinity, getInitialAllocation, getMaxAllocation, getMinParallelizationWidth, getOperatorType, getScanStats, getScanStats, getSelectionRoot, isDistributed, isExecutable, iterator
Methods inherited from class org.apache.drill.exec.physical.base.AbstractBase
accept, getCost, getOperatorId, getSVMode, getUserName, isBufferedOperator, setCost, setMaxAllocation, setOperatorId
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface org.apache.drill.common.graph.GraphValue
accept
Methods inherited from interface org.apache.drill.exec.physical.base.GroupScan
enforceWidth, getAnalyzeInfoProvider, getMinParallelizationWidth, getScanStats, getScanStats, getSelectionRoot, isDistributed
Methods inherited from interface org.apache.drill.exec.physical.base.HasAffinity
getDistributionAffinity
Methods inherited from interface java.lang.Iterable
forEach, iterator, spliterator
Methods inherited from interface org.apache.drill.exec.physical.base.PhysicalOperator
accept, getCost, getInitialAllocation, getMaxAllocation, getOperatorId, getOperatorType, getSVMode, getUserName, isBufferedOperator, isExecutable, setCost, setMaxAllocation, setOperatorId
-
Constructor Details
-
HiveDrillNativeParquetScan
public HiveDrillNativeParquetScan(StoragePluginRegistry engineRegistry, String userName, HiveStoragePluginConfig hiveStoragePluginConfig, List<SchemaPath> columns, List<ReadEntryWithPath> entries, HivePartitionHolder hivePartitionHolder, Map<String, String> confProperties, ParquetReaderConfig readerConfig, LogicalExpression filter, TupleMetadata schema) throws IOException- Throws:
IOException
-
HiveDrillNativeParquetScan
public HiveDrillNativeParquetScan(String userName, List<SchemaPath> columns, HiveStoragePlugin hiveStoragePlugin, List<HiveMetadataProvider.LogicalInputSplit> logicalInputSplits, Map<String, String> confProperties, ParquetReaderConfig readerConfig) throws IOException- Throws:
IOException
-
HiveDrillNativeParquetScan
public HiveDrillNativeParquetScan(String userName, List<SchemaPath> columns, HiveStoragePlugin hiveStoragePlugin, List<HiveMetadataProvider.LogicalInputSplit> logicalInputSplits, Map<String, String> confProperties, ParquetReaderConfig readerConfig, LogicalExpression filter) throws IOException- Throws:
IOException
-
-
Method Details
-
getHiveStoragePluginConfig
-
getHivePartitionHolder
-
getConfProperties
-
getSpecificScan
-
getNewWithChildren
Description copied from interface:PhysicalOperator
Regenerate with this node with a new set of children. This is used in the case of materialization or optimization. -
clone
- Specified by:
clone
in interfaceFileGroupScan
- Overrides:
clone
in classAbstractFileGroupScan
- Throws:
IOException
-
clone
Description copied from interface:GroupScan
Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .- Specified by:
clone
in interfaceGroupScan
- Overrides:
clone
in classAbstractGroupScan
-
toString
-
getFilterer
Description copied from class:AbstractGroupScanWithMetadata
Returns holder for metadata values which provides API to filter metadata and build new group scan instance using filtered metadata.- Specified by:
getFilterer
in classAbstractParquetGroupScan
-
getDrillbits
- Specified by:
getDrillbits
in classAbstractParquetGroupScan
-
cloneWithFileSelection
protected AbstractParquetGroupScan cloneWithFileSelection(Collection<org.apache.hadoop.fs.Path> filePaths) throws IOException - Specified by:
cloneWithFileSelection
in classAbstractParquetGroupScan
- Throws:
IOException
-
supportsFileImplicitColumns
protected boolean supportsFileImplicitColumns()- Specified by:
supportsFileImplicitColumns
in classAbstractGroupScanWithMetadata<ParquetMetadataProvider>
-
getPartitionValues
- Specified by:
getPartitionValues
in classAbstractGroupScanWithMetadata<ParquetMetadataProvider>
-
tableMetadataProviderBuilder
protected HiveParquetTableMetadataProvider.Builder tableMetadataProviderBuilder(MetadataProviderManager source) Description copied from class:AbstractGroupScanWithMetadata
ReturnsTableMetadataProviderBuilder
instance based on specifiedMetadataProviderManager
source.- Specified by:
tableMetadataProviderBuilder
in classAbstractGroupScanWithMetadata<ParquetMetadataProvider>
- Parameters:
source
- metadata provider manager- Returns:
TableMetadataProviderBuilder
instance
-
defaultTableMetadataProviderBuilder
protected HiveParquetTableMetadataProvider.Builder defaultTableMetadataProviderBuilder(MetadataProviderManager source) Description copied from class:AbstractGroupScanWithMetadata
ReturnsTableMetadataProviderBuilder
instance which may provide metadata without using Drill Metastore.- Specified by:
defaultTableMetadataProviderBuilder
in classAbstractParquetGroupScan
- Parameters:
source
- metadata provider manager- Returns:
TableMetadataProviderBuilder
instance
-