Class EasyGroupScan
java.lang.Object
org.apache.drill.exec.physical.base.AbstractBase
org.apache.drill.exec.physical.base.AbstractGroupScan
org.apache.drill.exec.physical.base.AbstractFileGroupScan
org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata<TableMetadataProvider>
org.apache.drill.exec.store.dfs.easy.EasyGroupScan
- All Implemented Interfaces:
Iterable<PhysicalOperator>
,GraphValue<PhysicalOperator>
,FileGroupScan
,FragmentLeaf
,GroupScan
,HasAffinity
,Leaf
,PhysicalOperator
,Scan
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata
AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B extends AbstractGroupScanWithMetadata.GroupScanWithMetadataFilterer<B>>
-
Field Summary
Fields inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata
columns, files, fileSet, filter, limit, matchAllMetadata, metadataProvider, nonInterestingColumnsMetadata, partitionColumns, partitions, segments, tableMetadata, usedMetastore
Fields inherited from class org.apache.drill.exec.physical.base.AbstractBase
INIT_ALLOCATION, initialAllocation, MAX_ALLOCATION, maxAllocation, userName
Fields inherited from interface org.apache.drill.exec.physical.base.GroupScan
ALL_COLUMNS
-
Constructor Summary
ConstructorDescriptionEasyGroupScan
(String userName, List<org.apache.hadoop.fs.Path> files, StoragePluginConfig storageConfig, FormatPluginConfig formatConfig, StoragePluginRegistry engineRegistry, List<SchemaPath> columns, org.apache.hadoop.fs.Path selectionRoot, TupleMetadata schema, int limit) EasyGroupScan
(String userName, FileSelection selection, EasyFormatPlugin<?> formatPlugin, List<SchemaPath> columns, org.apache.hadoop.fs.Path selectionRoot, int minWidth, MetadataProviderManager metadataProvider) EasyGroupScan
(String userName, FileSelection selection, EasyFormatPlugin<?> formatPlugin, List<SchemaPath> columns, org.apache.hadoop.fs.Path selectionRoot, MetadataProviderManager metadataProviderManager) -
Method Summary
Modifier and TypeMethodDescriptionvoid
applyAssignments
(List<CoordinationProtos.DrillbitEndpoint> incomingEndpoints) boolean
canPushdownProjects
(List<SchemaPath> columns) GroupScan should check the list of columns, and see if it could support all the columns in the list.clone
(List<SchemaPath> columns) Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .clone
(FileSelection selection) protected FileTableMetadataProviderBuilder<?>
ReturnsTableMetadataProviderBuilder
instance which may provide metadata without using Drill Metastore.ReturnsAnalyzeInfoProvider
instance which will be used when running ANALYZE statement.List<org.apache.hadoop.fs.Path>
getFiles()
Returns a collection of file names associated with this GroupScan.Returns holder for metadata values which provides API to filter metadata and build new group scan instance using filtered metadata.int
int
At minimum, the GroupScan requires these many fragments to run.getNewWithChildren
(List<PhysicalOperator> children) Regenerate with this node with a new set of children.Get the list of Endpoints with associated affinities that this operator has preference for.getPartitionValues
(LocationProvider locationProvider) getScanStats
(PlannerSettings settings) org.apache.hadoop.fs.Path
Returns path to the selection root.getSpecificScan
(int minorFragmentId) boolean
hasFiles()
Return true if this GroupScan can return its selection as a list of file names (retrieved by getFiles()).void
modifyFileSelection
(FileSelection selection) protected boolean
boolean
Checks whether this group scan supports filter push down.protected FileTableMetadataProviderBuilder<?>
ReturnsTableMetadataProviderBuilder
instance based on specifiedMetadataProviderManager
source.toString()
Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScanWithMetadata
applyFilter, applyLimit, checkMetadataConsistency, getColumns, getColumnValueCount, getDigest, getFileSet, getFilesMetadata, getFilter, getFilterPredicate, getFilterPredicate, getFilterString, getLimit, getMetadataProvider, getNextOrEmpty, getNonInterestingColumnsMetadata, getPartitionColumns, getPartitionsMetadata, getPartitionValue, getScanStats, getSegmentsMetadata, getTableMetadata, getTypeForColumn, init, isAllDataPruned, isGroupScanFullyMatchesFilter, isImplicitOrPartCol, isMatchAllMetadata, limitMetadata, pruneForPartitions, setFilter, setFilterForRuntime, supportsLimitPushdown, usedMetastore
Methods inherited from class org.apache.drill.exec.physical.base.AbstractFileGroupScan
supportsPartitionFilterPushdown
Methods inherited from class org.apache.drill.exec.physical.base.AbstractGroupScan
accept, enforceWidth, getDistributionAffinity, getInitialAllocation, getMaxAllocation, getOperatorType, getScanStats, isDistributed, isExecutable, iterator
Methods inherited from class org.apache.drill.exec.physical.base.AbstractBase
accept, getCost, getOperatorId, getSVMode, getUserName, isBufferedOperator, setCost, setMaxAllocation, setOperatorId
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
Methods inherited from interface org.apache.drill.common.graph.GraphValue
accept
Methods inherited from interface org.apache.drill.exec.physical.base.GroupScan
enforceWidth, getScanStats, isDistributed
Methods inherited from interface org.apache.drill.exec.physical.base.HasAffinity
getDistributionAffinity
Methods inherited from interface java.lang.Iterable
forEach, iterator, spliterator
Methods inherited from interface org.apache.drill.exec.physical.base.PhysicalOperator
accept, getCost, getInitialAllocation, getMaxAllocation, getOperatorId, getOperatorType, getSVMode, getUserName, isBufferedOperator, isExecutable, setCost, setMaxAllocation, setOperatorId
-
Constructor Details
-
EasyGroupScan
public EasyGroupScan(String userName, List<org.apache.hadoop.fs.Path> files, StoragePluginConfig storageConfig, FormatPluginConfig formatConfig, StoragePluginRegistry engineRegistry, List<SchemaPath> columns, org.apache.hadoop.fs.Path selectionRoot, TupleMetadata schema, int limit) throws IOException - Throws:
IOException
-
EasyGroupScan
public EasyGroupScan(String userName, FileSelection selection, EasyFormatPlugin<?> formatPlugin, List<SchemaPath> columns, org.apache.hadoop.fs.Path selectionRoot, MetadataProviderManager metadataProviderManager) throws IOException - Throws:
IOException
-
EasyGroupScan
public EasyGroupScan(String userName, FileSelection selection, EasyFormatPlugin<?> formatPlugin, List<SchemaPath> columns, org.apache.hadoop.fs.Path selectionRoot, int minWidth, MetadataProviderManager metadataProvider) throws IOException - Throws:
IOException
-
-
Method Details
-
getWorkIterable
-
getSelectionRoot
public org.apache.hadoop.fs.Path getSelectionRoot()Description copied from interface:GroupScan
Returns path to the selection root. If this GroupScan cannot provide selection root, it returns null.- Specified by:
getSelectionRoot
in interfaceGroupScan
- Overrides:
getSelectionRoot
in classAbstractGroupScan
- Returns:
- path to the selection root
-
getMinParallelizationWidth
public int getMinParallelizationWidth()Description copied from interface:GroupScan
At minimum, the GroupScan requires these many fragments to run. Currently, this is used inSimpleParallelizer
- Specified by:
getMinParallelizationWidth
in interfaceGroupScan
- Overrides:
getMinParallelizationWidth
in classAbstractGroupScan
- Returns:
- the minimum number of fragments that should run
-
getMaxParallelizationWidth
public int getMaxParallelizationWidth() -
getScanStats
- Specified by:
getScanStats
in interfaceGroupScan
- Overrides:
getScanStats
in classAbstractGroupScan
-
hasFiles
public boolean hasFiles()Description copied from interface:GroupScan
Return true if this GroupScan can return its selection as a list of file names (retrieved by getFiles()).- Specified by:
hasFiles
in interfaceGroupScan
- Overrides:
hasFiles
in classAbstractGroupScanWithMetadata<TableMetadataProvider>
-
getFiles
Description copied from interface:GroupScan
Returns a collection of file names associated with this GroupScan. This should be called after checking hasFiles(). If this GroupScan cannot provide file names, it returns null.- Specified by:
getFiles
in interfaceGroupScan
- Overrides:
getFiles
in classAbstractGroupScanWithMetadata<TableMetadataProvider>
- Returns:
- collection of files paths
-
getFileSelection
-
modifyFileSelection
- Specified by:
modifyFileSelection
in interfaceFileGroupScan
- Overrides:
modifyFileSelection
in classAbstractGroupScanWithMetadata<TableMetadataProvider>
-
supportsFileImplicitColumns
protected boolean supportsFileImplicitColumns()- Specified by:
supportsFileImplicitColumns
in classAbstractGroupScanWithMetadata<TableMetadataProvider>
-
supportsFilterPushDown
public boolean supportsFilterPushDown()Description copied from interface:GroupScan
Checks whether this group scan supports filter push down.- Specified by:
supportsFilterPushDown
in interfaceGroupScan
- Overrides:
supportsFilterPushDown
in classAbstractGroupScan
- Returns:
true
if this group scan supports filter push down,false
otherwise
-
getPartitionValues
- Specified by:
getPartitionValues
in classAbstractGroupScanWithMetadata<TableMetadataProvider>
-
getNewWithChildren
Description copied from interface:PhysicalOperator
Regenerate with this node with a new set of children. This is used in the case of materialization or optimization. -
getOperatorAffinity
Description copied from interface:HasAffinity
Get the list of Endpoints with associated affinities that this operator has preference for.- Specified by:
getOperatorAffinity
in interfaceHasAffinity
- Overrides:
getOperatorAffinity
in classAbstractGroupScan
- Returns:
- List of EndpointAffinity objects.
-
applyAssignments
-
getSpecificScan
-
getStorageConfig
-
getFormatConfig
-
toString
-
clone
Description copied from interface:GroupScan
Returns a clone of GroupScan instance, except that the new GroupScan will use the provided list of columns .- Specified by:
clone
in interfaceGroupScan
- Overrides:
clone
in classAbstractGroupScan
-
clone
- Specified by:
clone
in interfaceFileGroupScan
- Overrides:
clone
in classAbstractFileGroupScan
- Throws:
IOException
-
canPushdownProjects
Description copied from interface:GroupScan
GroupScan should check the list of columns, and see if it could support all the columns in the list.- Specified by:
canPushdownProjects
in interfaceGroupScan
- Overrides:
canPushdownProjects
in classAbstractGroupScan
-
getSchema
- Overrides:
getSchema
in classAbstractGroupScanWithMetadata<TableMetadataProvider>
-
getAnalyzeInfoProvider
Description copied from interface:GroupScan
ReturnsAnalyzeInfoProvider
instance which will be used when running ANALYZE statement.- Specified by:
getAnalyzeInfoProvider
in interfaceGroupScan
- Overrides:
getAnalyzeInfoProvider
in classAbstractGroupScan
- Returns:
AnalyzeInfoProvider
instance
-
getFilterer
Description copied from class:AbstractGroupScanWithMetadata
Returns holder for metadata values which provides API to filter metadata and build new group scan instance using filtered metadata.- Specified by:
getFilterer
in classAbstractGroupScanWithMetadata<TableMetadataProvider>
-
tableMetadataProviderBuilder
protected FileTableMetadataProviderBuilder<?> tableMetadataProviderBuilder(MetadataProviderManager source) Description copied from class:AbstractGroupScanWithMetadata
ReturnsTableMetadataProviderBuilder
instance based on specifiedMetadataProviderManager
source.- Specified by:
tableMetadataProviderBuilder
in classAbstractGroupScanWithMetadata<TableMetadataProvider>
- Parameters:
source
- metadata provider manager- Returns:
TableMetadataProviderBuilder
instance
-
defaultTableMetadataProviderBuilder
protected FileTableMetadataProviderBuilder<?> defaultTableMetadataProviderBuilder(MetadataProviderManager source) Description copied from class:AbstractGroupScanWithMetadata
ReturnsTableMetadataProviderBuilder
instance which may provide metadata without using Drill Metastore.- Specified by:
defaultTableMetadataProviderBuilder
in classAbstractGroupScanWithMetadata<TableMetadataProvider>
- Parameters:
source
- metadata provider manager- Returns:
TableMetadataProviderBuilder
instance
-