org.apache.drill.exec.store.kafka.decoders

Class KafkaJsonLoader

java.lang.Object

org.apache.drill.exec.store.easy.json.loader.JsonLoaderImpl

org.apache.drill.exec.store.kafka.decoders.KafkaJsonLoader

All Implemented Interfaces:

JsonLoader, ErrorFactory

public class KafkaJsonLoader
extends JsonLoaderImpl

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	KafkaJsonLoader.KafkaJsonLoaderBuilder

Nested classes/interfaces inherited from class org.apache.drill.exec.store.easy.json.loader.JsonLoaderImpl

JsonLoaderImpl.JsonLoaderBuilder

Field Summary

Fields inherited from interface org.apache.drill.exec.store.easy.json.loader.JsonLoader

JSON_LITERAL_MODE, JSON_MODE, JSON_TEXT_MODE, JSON_TYPED_MODE

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	KafkaJsonLoader(KafkaJsonLoader.KafkaJsonLoaderBuilder builder)

Method Summary

Modifier and Type	Method and Description
void	endBatch() Finish reading a batch of data.

Methods inherited from class org.apache.drill.exec.store.easy.json.loader.JsonLoaderImpl

addNullMarker, buildError, buildError, close, dataConversionError, fieldFactory, ioException, messageParseError, nullDisallowedError, options, parseError, parser, readBatch, removeNullMarker, structureError, syntaxError, syntaxError, typeConversionError, typeConversionError, typeError, unrecoverableError, unsupportedJsonTypeException, unsupportedType

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KafkaJsonLoader

protected KafkaJsonLoader(KafkaJsonLoader.KafkaJsonLoaderBuilder builder)

Method Detail

endBatch

public void endBatch()

Description copied from class: JsonLoaderImpl

Finish reading a batch of data. We may have pending "null" columns: a column for which we've seen only nulls, or an array that has always been empty. The batch needs to finish, and needs a type, but we still don't know the type. Since we must decide on one, we do the following guess Varchar, and switch to text mode.

This choices is not perfect. Switching to text mode means results will vary from run to run depending on the order that we see empty and non-empty values for this column. Plus, since the system is distributed, the decision made here may conflict with that made in some other fragment.

The only real solution is for the user to provide a schema.

Bottom line: the user is responsible for not giving Drill ambiguous data that would require Drill to predict the future.

Overrides:

endBatch in class JsonLoaderImpl