org.apache.drill.exec.store.easy.json

Class JsonStatisticsRecordWriter

java.lang.Object

org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

org.apache.drill.exec.store.easy.json.JsonStatisticsRecordWriter

All Implemented Interfaces:

org.apache.drill.exec.store.StatisticsRecordCollector, StatisticsRecordWriter

public class JsonStatisticsRecordWriter
extends org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter
implements StatisticsRecordWriter

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.BigIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.BitJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.DateJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.Decimal18JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.Decimal28DenseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.Decimal28SparseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.Decimal38DenseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.Decimal38SparseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.Decimal9JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.Float4JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.Float8JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.IntervalDayJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.IntervalJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.IntervalYearJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.IntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableBigIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableBitJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableDateJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableDecimal18JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableDecimal28DenseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableDecimal28SparseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableDecimal38DenseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableDecimal38SparseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableDecimal9JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableFloat4JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableFloat8JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableIntervalDayJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableIntervalJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableIntervalYearJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableSmallIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableTimeJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableTimeStampJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableTinyIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableUInt1JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableUInt2JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableUInt4JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableUInt8JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableVar16CharJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableVarBinaryJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableVarCharJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.NullableVarDecimalJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedBigIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedBitJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedDateJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedDecimal18JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedDecimal28DenseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedDecimal28SparseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedDecimal38DenseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedDecimal38SparseJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedDecimal9JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedFloat4JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedFloat8JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedIntervalDayJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedIntervalJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedIntervalYearJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedSmallIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedTimeJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedTimeStampJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedTinyIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedUInt1JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedUInt2JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedUInt4JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedUInt8JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedVar16CharJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedVarBinaryJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedVarCharJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.RepeatedVarDecimalJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.SmallIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.TimeJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.TimeStampJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.TinyIntJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.UInt1JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.UInt2JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.UInt4JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.UInt8JsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.Var16CharJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.VarBinaryJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.VarCharJsonConverter, org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter.VarDecimalJsonConverter

Field Summary

Fields inherited from class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

gen, skipNullFields

Constructor Summary

Constructors

Constructor and Description
JsonStatisticsRecordWriter(org.apache.hadoop.conf.Configuration fsConf, FormatPlugin formatPlugin)

Method Summary

Modifier and Type	Method and Description
void	abort()
void	checkForNewPartition(int index) Check if the writer should start a new partition, and if so, start a new partition
void	cleanup()
void	endStatisticsRecord()
void	flushBlockingWriter() For a blocking writer, called after processing all the records to flush out the writes
org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter	getNewBigIntConverter(int fieldId, String fieldName, FieldReader reader)
org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter	getNewDateConverter(int fieldId, String fieldName, FieldReader reader)
org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter	getNewIntConverter(int fieldId, String fieldName, FieldReader reader)
org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter	getNewNullableBigIntConverter(int fieldId, String fieldName, FieldReader reader)
org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter	getNewNullableFloat8Converter(int fieldId, String fieldName, FieldReader reader)
org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter	getNewNullableVarBinaryConverter(int fieldId, String fieldName, FieldReader reader)
org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter	getNewVarCharConverter(int fieldId, String fieldName, FieldReader reader)
DrillStatsTable.TableStatistics	getStatistics()
boolean	hasStatistics()
void	init(Map<String,String> writerOptions) Initialize the writer.
boolean	isBlockingWriter() Returns if the writer is a blocking writer i.e.
void	startStatisticsRecord()
void	updateSchema(VectorAccessible batch) Update the schema in RecordWriter.

Methods inherited from class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

getNewBitConverter, getNewDecimal18Converter, getNewDecimal28DenseConverter, getNewDecimal28SparseConverter, getNewDecimal38DenseConverter, getNewDecimal38SparseConverter, getNewDecimal9Converter, getNewFloat4Converter, getNewFloat8Converter, getNewIntervalConverter, getNewIntervalDayConverter, getNewIntervalYearConverter, getNewNullableBitConverter, getNewNullableDateConverter, getNewNullableDecimal18Converter, getNewNullableDecimal28DenseConverter, getNewNullableDecimal28SparseConverter, getNewNullableDecimal38DenseConverter, getNewNullableDecimal38SparseConverter, getNewNullableDecimal9Converter, getNewNullableFloat4Converter, getNewNullableIntConverter, getNewNullableIntervalConverter, getNewNullableIntervalDayConverter, getNewNullableIntervalYearConverter, getNewNullableSmallIntConverter, getNewNullableTimeConverter, getNewNullableTimeStampConverter, getNewNullableTinyIntConverter, getNewNullableUInt1Converter, getNewNullableUInt2Converter, getNewNullableUInt4Converter, getNewNullableUInt8Converter, getNewNullableVar16CharConverter, getNewNullableVarCharConverter, getNewNullableVarDecimalConverter, getNewRepeatedBigIntConverter, getNewRepeatedBitConverter, getNewRepeatedDateConverter, getNewRepeatedDecimal18Converter, getNewRepeatedDecimal28DenseConverter, getNewRepeatedDecimal28SparseConverter, getNewRepeatedDecimal38DenseConverter, getNewRepeatedDecimal38SparseConverter, getNewRepeatedDecimal9Converter, getNewRepeatedFloat4Converter, getNewRepeatedFloat8Converter, getNewRepeatedIntConverter, getNewRepeatedIntervalConverter, getNewRepeatedIntervalDayConverter, getNewRepeatedIntervalYearConverter, getNewRepeatedSmallIntConverter, getNewRepeatedTimeConverter, getNewRepeatedTimeStampConverter, getNewRepeatedTinyIntConverter, getNewRepeatedUInt1Converter, getNewRepeatedUInt2Converter, getNewRepeatedUInt4Converter, getNewRepeatedUInt8Converter, getNewRepeatedVar16CharConverter, getNewRepeatedVarBinaryConverter, getNewRepeatedVarCharConverter, getNewRepeatedVarDecimalConverter, getNewSmallIntConverter, getNewTimeConverter, getNewTimeStampConverter, getNewTinyIntConverter, getNewUInt1Converter, getNewUInt2Converter, getNewUInt4Converter, getNewUInt8Converter, getNewVar16CharConverter, getNewVarBinaryConverter, getNewVarDecimalConverter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.drill.exec.store.StatisticsRecordCollector

getNewBitConverter, getNewDecimal18Converter, getNewDecimal28DenseConverter, getNewDecimal28SparseConverter, getNewDecimal38DenseConverter, getNewDecimal38SparseConverter, getNewDecimal9Converter, getNewFloat4Converter, getNewFloat8Converter, getNewIntervalConverter, getNewIntervalDayConverter, getNewIntervalYearConverter, getNewNullableBitConverter, getNewNullableDateConverter, getNewNullableDecimal18Converter, getNewNullableDecimal28DenseConverter, getNewNullableDecimal28SparseConverter, getNewNullableDecimal38DenseConverter, getNewNullableDecimal38SparseConverter, getNewNullableDecimal9Converter, getNewNullableFloat4Converter, getNewNullableIntConverter, getNewNullableIntervalConverter, getNewNullableIntervalDayConverter, getNewNullableIntervalYearConverter, getNewNullableSmallIntConverter, getNewNullableTimeConverter, getNewNullableTimeStampConverter, getNewNullableTinyIntConverter, getNewNullableUInt1Converter, getNewNullableUInt2Converter, getNewNullableUInt4Converter, getNewNullableUInt8Converter, getNewNullableVar16CharConverter, getNewNullableVarCharConverter, getNewNullableVarDecimalConverter, getNewRepeatedBigIntConverter, getNewRepeatedBitConverter, getNewRepeatedDateConverter, getNewRepeatedDecimal18Converter, getNewRepeatedDecimal28DenseConverter, getNewRepeatedDecimal28SparseConverter, getNewRepeatedDecimal38DenseConverter, getNewRepeatedDecimal38SparseConverter, getNewRepeatedDecimal9Converter, getNewRepeatedFloat4Converter, getNewRepeatedFloat8Converter, getNewRepeatedIntConverter, getNewRepeatedIntervalConverter, getNewRepeatedIntervalDayConverter, getNewRepeatedIntervalYearConverter, getNewRepeatedSmallIntConverter, getNewRepeatedTimeConverter, getNewRepeatedTimeStampConverter, getNewRepeatedTinyIntConverter, getNewRepeatedUInt1Converter, getNewRepeatedUInt2Converter, getNewRepeatedUInt4Converter, getNewRepeatedUInt8Converter, getNewRepeatedVar16CharConverter, getNewRepeatedVarBinaryConverter, getNewRepeatedVarCharConverter, getNewRepeatedVarDecimalConverter, getNewSmallIntConverter, getNewTimeConverter, getNewTimeStampConverter, getNewTinyIntConverter, getNewUInt1Converter, getNewUInt2Converter, getNewUInt4Converter, getNewUInt8Converter, getNewVar16CharConverter, getNewVarBinaryConverter, getNewVarDecimalConverter

Constructor Detail

JsonStatisticsRecordWriter

public JsonStatisticsRecordWriter(org.apache.hadoop.conf.Configuration fsConf,
                                  FormatPlugin formatPlugin)

Method Detail

init

public void init(Map<String,String> writerOptions)

Description copied from interface: StatisticsRecordWriter

Initialize the writer.

Specified by:

init in interface StatisticsRecordWriter

Parameters:

writerOptions - Contains key, value pair of settings.

updateSchema

public void updateSchema(VectorAccessible batch)

Description copied from interface: StatisticsRecordWriter

Update the schema in RecordWriter. Called at least once before starting writing the records.

Specified by:

updateSchema in interface StatisticsRecordWriter

isBlockingWriter

public boolean isBlockingWriter()

Description copied from interface: StatisticsRecordWriter

Returns if the writer is a blocking writer i.e. consumes all input before writing it out

Specified by:

isBlockingWriter in interface StatisticsRecordWriter

Returns:

TRUE, if writer is blocking. FALSE, otherwise

checkForNewPartition

public void checkForNewPartition(int index)

Description copied from interface: StatisticsRecordWriter

Check if the writer should start a new partition, and if so, start a new partition

Specified by:

checkForNewPartition in interface StatisticsRecordWriter

getNewBigIntConverter

public org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter getNewBigIntConverter(int fieldId,
                                                                                               String fieldName,
                                                                                               FieldReader reader)

Specified by:

getNewBigIntConverter in interface org.apache.drill.exec.store.StatisticsRecordCollector

Overrides:

getNewBigIntConverter in class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

getNewIntConverter

public org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter getNewIntConverter(int fieldId,
                                                                                            String fieldName,
                                                                                            FieldReader reader)

Specified by:

getNewIntConverter in interface org.apache.drill.exec.store.StatisticsRecordCollector

Overrides:

getNewIntConverter in class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

getNewDateConverter

public org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter getNewDateConverter(int fieldId,
                                                                                             String fieldName,
                                                                                             FieldReader reader)

Specified by:

getNewDateConverter in interface org.apache.drill.exec.store.StatisticsRecordCollector

Overrides:

getNewDateConverter in class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

getNewVarCharConverter

public org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter getNewVarCharConverter(int fieldId,
                                                                                                String fieldName,
                                                                                                FieldReader reader)

Specified by:

getNewVarCharConverter in interface org.apache.drill.exec.store.StatisticsRecordCollector

Overrides:

getNewVarCharConverter in class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

getNewNullableBigIntConverter

public org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter getNewNullableBigIntConverter(int fieldId,
                                                                                                       String fieldName,
                                                                                                       FieldReader reader)

Specified by:

getNewNullableBigIntConverter in interface org.apache.drill.exec.store.StatisticsRecordCollector

Overrides:

getNewNullableBigIntConverter in class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

getNewNullableVarBinaryConverter

public org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter getNewNullableVarBinaryConverter(int fieldId,
                                                                                                          String fieldName,
                                                                                                          FieldReader reader)

Specified by:

getNewNullableVarBinaryConverter in interface org.apache.drill.exec.store.StatisticsRecordCollector

Overrides:

getNewNullableVarBinaryConverter in class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

getNewNullableFloat8Converter

public org.apache.drill.exec.store.EventBasedRecordWriter.FieldConverter getNewNullableFloat8Converter(int fieldId,
                                                                                                       String fieldName,
                                                                                                       FieldReader reader)

Specified by:

getNewNullableFloat8Converter in interface org.apache.drill.exec.store.StatisticsRecordCollector

Overrides:

getNewNullableFloat8Converter in class org.apache.drill.exec.store.JSONBaseStatisticsRecordWriter

startStatisticsRecord

public void startStatisticsRecord()

Specified by:

startStatisticsRecord in interface org.apache.drill.exec.store.StatisticsRecordCollector

endStatisticsRecord

public void endStatisticsRecord()

Specified by:

endStatisticsRecord in interface org.apache.drill.exec.store.StatisticsRecordCollector

hasStatistics

public boolean hasStatistics()

Specified by:

hasStatistics in interface org.apache.drill.exec.store.StatisticsRecordCollector

getStatistics

public DrillStatsTable.TableStatistics getStatistics()

Specified by:

getStatistics in interface org.apache.drill.exec.store.StatisticsRecordCollector

flushBlockingWriter

public void flushBlockingWriter()
                         throws IOException

Description copied from interface: StatisticsRecordWriter

For a blocking writer, called after processing all the records to flush out the writes

Specified by:

flushBlockingWriter in interface StatisticsRecordWriter

Throws:

IOException

abort

public void abort()

Specified by:

abort in interface StatisticsRecordWriter

cleanup

public void cleanup()

Specified by:

cleanup in interface StatisticsRecordWriter