org.apache.drill.exec.physical.impl.scan.file

Class FileScanFramework

java.lang.Object

org.apache.drill.exec.physical.impl.scan.framework.ManagedScanFramework

org.apache.drill.exec.physical.impl.scan.file.FileScanFramework

All Implemented Interfaces:

ScanOperatorEvents

Direct Known Subclasses:

ColumnsScanFramework

public class FileScanFramework
extends ManagedScanFramework

The file scan framework adds into the scan framework support for implicit reading from DFS splits (a file and a block). Since this framework is file-based, it also adds support for file metadata (AKA implicit columns. The file scan framework brings together a number of components:

• The set of options defined by the base framework.
• The set of files and/or blocks to read.
• The file system configuration to use for working with the files or blocks.
• The factory class to create a reader for each of the files or blocks defined above. (Readers are created one-by-one as files are read.)
• Options as defined by the base class.

The framework iterates over file descriptions, creating readers at the moment they are needed. This allows simpler logic because, at the point of reader creation, we have a file system, context and so on.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FileScanFramework.FileReaderFactory Iterates over the splits for the present scan.
static class	FileScanFramework.FileRowSetContext
static class	FileScanFramework.FileScanBuilder Options for a file-based scan.
static interface	FileScanFramework.FileSchemaNegotiator The file schema negotiator adds no behavior at present, but is created as a placeholder anticipating the need for file-specific behavior later.
static class	FileScanFramework.FileSchemaNegotiatorImpl Implementation of the file-level schema negotiator.

Nested classes/interfaces inherited from class org.apache.drill.exec.physical.impl.scan.framework.ManagedScanFramework

ManagedScanFramework.ReaderFactory, ManagedScanFramework.ScanFrameworkBuilder

Field Summary

Fields inherited from class org.apache.drill.exec.physical.impl.scan.framework.ManagedScanFramework

builder, context, readerFactory, scanOrchestrator

Constructor Summary

Constructors

Constructor and Description
FileScanFramework(FileScanFramework.FileScanBuilder builder)

Method Summary

Modifier and Type	Method and Description
protected void	configure()
DrillFileSystem	fileSystem()
protected SchemaNegotiatorImpl	newNegotiator()
protected org.apache.hadoop.mapred.FileSplit	nextSplit()
boolean	open(ShimBatchReader shimBatchReader)
FileScanFramework.FileScanBuilder	options()

Methods inherited from class org.apache.drill.exec.physical.impl.scan.framework.ManagedScanFramework

bind, close, context, errorContext, nextReader, outputSchema, scanOrchestrator

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileScanFramework

public FileScanFramework(FileScanFramework.FileScanBuilder builder)

Method Detail

options

public FileScanFramework.FileScanBuilder options()

configure

protected void configure()

Overrides:

configure in class ManagedScanFramework

nextSplit

protected org.apache.hadoop.mapred.FileSplit nextSplit()

newNegotiator

protected SchemaNegotiatorImpl newNegotiator()

Overrides:

newNegotiator in class ManagedScanFramework

open

public boolean open(ShimBatchReader shimBatchReader)

Overrides:

open in class ManagedScanFramework

fileSystem

public DrillFileSystem fileSystem()