org.apache.drill.exec.store.easy.json

Class JSONFormatPlugin

java.lang.Object

org.apache.drill.exec.store.dfs.easy.EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

org.apache.drill.exec.store.easy.json.JSONFormatPlugin

All Implemented Interfaces:

FormatPlugin

public class JSONFormatPlugin
extends EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	JSONFormatPlugin.JSONFormatConfig

Nested classes/interfaces inherited from class org.apache.drill.exec.store.dfs.easy.EasyFormatPlugin

EasyFormatPlugin.EasyFormatConfig, EasyFormatPlugin.EasyFormatConfigBuilder

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_NAME
static String	OPERATOR_TYPE

Fields inherited from class org.apache.drill.exec.store.dfs.easy.EasyFormatPlugin

formatConfig

Constructor Summary

Constructors

Constructor and Description
JSONFormatPlugin(String name, DrillbitContext context, org.apache.hadoop.conf.Configuration fsConf, StoragePluginConfig storageConfig)
JSONFormatPlugin(String name, DrillbitContext context, org.apache.hadoop.conf.Configuration fsConf, StoragePluginConfig config, JSONFormatPlugin.JSONFormatConfig formatPluginConfig)

Method Summary

Modifier and Type	Method and Description
String	getReaderOperatorType()
RecordReader	getRecordReader(FragmentContext context, DrillFileSystem dfs, FileWork fileWork, List<SchemaPath> columns, String userName) Return a record reader for the specific file format, when using the original ScanBatch scanner.
org.apache.drill.exec.store.RecordWriter	getRecordWriter(FragmentContext context, EasyWriter writer)
StatisticsRecordWriter	getStatisticsRecordWriter(FragmentContext context, EasyWriter writer)
String	getWriterOperatorType()
boolean	isStatisticsRecordWriter(FragmentContext context, EasyWriter writer)
DrillStatsTable.TableStatistics	readStatistics(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path statsTablePath)
boolean	supportsPushDown() Does this plugin support projection push down? That is, can the reader itself handle the tasks of projecting table columns, creating null columns for missing table columns, and so on?
boolean	supportsStatistics()
void	writeStatistics(DrillStatsTable.TableStatistics statistics, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path statsTablePath)

Methods inherited from class org.apache.drill.exec.store.dfs.easy.EasyFormatPlugin

easyConfig, frameworkBuilder, getConfig, getContext, getFsConf, getGroupScan, getGroupScan, getMatcher, getName, getOptimizerRules, getReaderBatch, getScanStats, getStorageConfig, getWriter, getWriterBatch, initScanBuilder, isBlockSplittable, isCompressible, newBatchReader, supportsAutoPartitioning, supportsFileImplicitColumns, supportsLimitPushdown, supportsRead, supportsWrite, useEnhancedScan

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.drill.exec.store.dfs.FormatPlugin

getGroupScan, getGroupScan

Field Detail

DEFAULT_NAME

public static final String DEFAULT_NAME

See Also:

Constant Field Values

OPERATOR_TYPE

public static final String OPERATOR_TYPE

See Also:

Constant Field Values

Constructor Detail

JSONFormatPlugin

public JSONFormatPlugin(String name,
                        DrillbitContext context,
                        org.apache.hadoop.conf.Configuration fsConf,
                        StoragePluginConfig storageConfig)

JSONFormatPlugin

public JSONFormatPlugin(String name,
                        DrillbitContext context,
                        org.apache.hadoop.conf.Configuration fsConf,
                        StoragePluginConfig config,
                        JSONFormatPlugin.JSONFormatConfig formatPluginConfig)

Method Detail

getRecordReader

public RecordReader getRecordReader(FragmentContext context,
                                    DrillFileSystem dfs,
                                    FileWork fileWork,
                                    List<SchemaPath> columns,
                                    String userName)

Description copied from class: EasyFormatPlugin

Return a record reader for the specific file format, when using the original ScanBatch scanner.

Overrides:

getRecordReader in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

Parameters:

context - fragment context

dfs - Drill file system

fileWork - metadata about the file to be scanned

columns - list of projected columns (or may just contain the wildcard)

userName - the name of the user running the query

Returns:

a record reader for this format

isStatisticsRecordWriter

public boolean isStatisticsRecordWriter(FragmentContext context,
                                        EasyWriter writer)

Overrides:

isStatisticsRecordWriter in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

getStatisticsRecordWriter

public StatisticsRecordWriter getStatisticsRecordWriter(FragmentContext context,
                                                        EasyWriter writer)
                                                 throws IOException

Overrides:

getStatisticsRecordWriter in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

Throws:

IOException

getRecordWriter

public org.apache.drill.exec.store.RecordWriter getRecordWriter(FragmentContext context,
                                                                EasyWriter writer)
                                                         throws IOException

Overrides:

getRecordWriter in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

Throws:

IOException

supportsStatistics

public boolean supportsStatistics()

Specified by:

supportsStatistics in interface FormatPlugin

Overrides:

supportsStatistics in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

readStatistics

public DrillStatsTable.TableStatistics readStatistics(org.apache.hadoop.fs.FileSystem fs,
                                                      org.apache.hadoop.fs.Path statsTablePath)
                                               throws IOException

Specified by:

readStatistics in interface FormatPlugin

Overrides:

readStatistics in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

Throws:

IOException

writeStatistics

public void writeStatistics(DrillStatsTable.TableStatistics statistics,
                            org.apache.hadoop.fs.FileSystem fs,
                            org.apache.hadoop.fs.Path statsTablePath)
                     throws IOException

Specified by:

writeStatistics in interface FormatPlugin

Overrides:

writeStatistics in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

Throws:

IOException

getReaderOperatorType

public String getReaderOperatorType()

Overrides:

getReaderOperatorType in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

getWriterOperatorType

public String getWriterOperatorType()

Overrides:

getWriterOperatorType in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

supportsPushDown

public boolean supportsPushDown()

Description copied from class: EasyFormatPlugin

Does this plugin support projection push down? That is, can the reader itself handle the tasks of projecting table columns, creating null columns for missing table columns, and so on?

Overrides:

supportsPushDown in class EasyFormatPlugin<JSONFormatPlugin.JSONFormatConfig>

Returns:

true if the plugin supports projection push-down, false if Drill should do the task by adding a project operator