org.apache.drill.exec.metastore.analyze

Class FileMetadataInfoCollector

java.lang.Object

org.apache.drill.exec.metastore.analyze.FileMetadataInfoCollector

All Implemented Interfaces:

MetadataInfoCollector

public class FileMetadataInfoCollector
extends Object
implements MetadataInfoCollector

Implementation of MetadataInfoCollector for file-based tables.

Constructor Summary

Constructors

Constructor and Description
FileMetadataInfoCollector(BasicTablesRequests basicRequests, TableInfo tableInfo, FormatSelection selection, PlannerSettings settings, Supplier<org.apache.calcite.rel.core.TableScan> tableScanSupplier, List<SchemaPath> interestingColumns, MetadataType metadataLevel, int segmentColumnsCount)

Method Summary

Modifier and Type	Method and Description
List<MetadataInfo>	getAllMetaToHandle() Returns list of all metadata info instances which should be handled either producing analyze or when fetching from the Metastore.
static FileSelection	getExpandedFileSelection(FileSelection fileSelection) Returns FileSelection file selection based on specified file selection with expanded file statuses.
List<MetadataInfo>	getFilesInfo() Returns list of files metadata info which should be fetched from the Metastore.
static List<org.apache.hadoop.fs.FileStatus>	getFileStatuses(FileSelection fileSelection, DrillFileSystem fs) Returns list of FileStatus file statuses obtained from specified FileSelection file selection.
List<MetadataInfo>	getMetadataToRemove() Returns list of all metadata info which corresponds to top-level segments and should be removed from the Metastore.
org.apache.calcite.rel.core.TableScan	getPrunedScan() Returns TableScan instance which will be used when produced incremental analyze.
List<MetadataInfo>	getRowGroupsInfo() Returns list of row groups metadata info which should be fetched from the Metastore.
org.apache.drill.shaded.guava.com.google.common.collect.Multimap<Integer,MetadataInfo>	getSegmentsInfo() Returns list of segments metadata info which should be fetched from the Metastore.
boolean	isOutdated() Returns true if table metadata is outdated.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileMetadataInfoCollector

public FileMetadataInfoCollector(BasicTablesRequests basicRequests,
                                 TableInfo tableInfo,
                                 FormatSelection selection,
                                 PlannerSettings settings,
                                 Supplier<org.apache.calcite.rel.core.TableScan> tableScanSupplier,
                                 List<SchemaPath> interestingColumns,
                                 MetadataType metadataLevel,
                                 int segmentColumnsCount)
                          throws IOException

Throws:

IOException

Method Detail

getRowGroupsInfo

public List<MetadataInfo> getRowGroupsInfo()

Description copied from interface: MetadataInfoCollector

Returns list of row groups metadata info which should be fetched from the Metastore.

Specified by:

getRowGroupsInfo in interface MetadataInfoCollector

Returns:

list of row groups metadata info

getFilesInfo

public List<MetadataInfo> getFilesInfo()

Description copied from interface: MetadataInfoCollector

Returns list of files metadata info which should be fetched from the Metastore.

Specified by:

getFilesInfo in interface MetadataInfoCollector

Returns:

list of files metadata info

getSegmentsInfo

public org.apache.drill.shaded.guava.com.google.common.collect.Multimap<Integer,MetadataInfo> getSegmentsInfo()

Description copied from interface: MetadataInfoCollector

Returns list of segments metadata info which should be fetched from the Metastore.

Specified by:

getSegmentsInfo in interface MetadataInfoCollector

Returns:

list of segments metadata info

getAllMetaToHandle

public List<MetadataInfo> getAllMetaToHandle()

Description copied from interface: MetadataInfoCollector

Returns list of all metadata info instances which should be handled either producing analyze or when fetching from the Metastore.

Specified by:

getAllMetaToHandle in interface MetadataInfoCollector

Returns:

list of all metadata info

getMetadataToRemove

public List<MetadataInfo> getMetadataToRemove()

Description copied from interface: MetadataInfoCollector

Returns list of all metadata info which corresponds to top-level segments and should be removed from the Metastore.

Specified by:

getMetadataToRemove in interface MetadataInfoCollector

Returns:

list of all metadata info which should be removed

getPrunedScan

public org.apache.calcite.rel.core.TableScan getPrunedScan()

Description copied from interface: MetadataInfoCollector

Returns TableScan instance which will be used when produced incremental analyze. Table scan will contain minimal selection required for obtaining correct metadata.

Specified by:

getPrunedScan in interface MetadataInfoCollector

Returns:

TableScan instance

isOutdated

public boolean isOutdated()

Description copied from interface: MetadataInfoCollector

Returns true if table metadata is outdated.

Specified by:

isOutdated in interface MetadataInfoCollector

Returns:

true if table metadata is outdated

getFileStatuses

public static List<org.apache.hadoop.fs.FileStatus> getFileStatuses(FileSelection fileSelection,
                                                                    DrillFileSystem fs)
                                                             throws IOException

Returns list of FileStatus file statuses obtained from specified FileSelection file selection. Specified file selection may be expanded fully if it wasn't expanded before.

Parameters:

fileSelection - file selection

fs - file system

Returns:

list of FileStatus file statuses

Throws:

IOException

getExpandedFileSelection

public static FileSelection getExpandedFileSelection(FileSelection fileSelection)
                                              throws IOException

Returns FileSelection file selection based on specified file selection with expanded file statuses.

Parameters:

fileSelection - file selection

Returns:

expanded file selection

Throws:

IOException