org.apache.drill.exec.store.easy.text.reader

Class HeaderBuilder

java.lang.Object

org.apache.drill.exec.store.easy.text.reader.HeaderBuilder

public class HeaderBuilder
extends Object

Text output that implements a header reader/parser. The caller parses out the characters of each header; this class assembles UTF-8 bytes into Unicode characters, fixes invalid characters (those not legal for SQL symbols), and maps duplicate names to unique names.

That is, this class is as permissive as possible with file headers to avoid spurious query failures for trivial reasons.

Field Summary

Fields

Modifier and Type	Field and Description
static String	ANONYMOUS_COLUMN_PREFIX Prefix used to create numbered columns for missing headers.
static String	COLUMN_PREFIX Prefix used to replace non-alphabetic characters at the start of a column name.
static int	MAX_HEADER_LEN Maximum Drill symbol length, as enforced for headers.

Constructor Summary

Constructors

Constructor and Description
HeaderBuilder(org.apache.hadoop.fs.Path filePath)

Method Summary

Modifier and Type	Method and Description
void	append(byte data) Appends a byte to the output character data buffer.
boolean	endEmptyField() Shortcut that lets the output know that we are closing ending a field with no data.
boolean	endField() End processing a field within a record.
void	finishRecord() Completes the processing of a given record.
String[]	getHeaders()
long	getRecordCount() Return the total number of records (across batches) processed
boolean	isFull() Indicates if the current batch is full and reading for this batch should stop.
void	startField(int index) Start processing a new field within a record.
void	startRecord() Start processing a new record.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAX_HEADER_LEN

public static final int MAX_HEADER_LEN

Maximum Drill symbol length, as enforced for headers.

See Also:

identifier documentation, Constant Field Values

COLUMN_PREFIX

public static final String COLUMN_PREFIX

Prefix used to replace non-alphabetic characters at the start of a column name. For example, $foo becomes col_foo. Used because SQL does not allow _foo.

See Also:

Constant Field Values

ANONYMOUS_COLUMN_PREFIX

public static final String ANONYMOUS_COLUMN_PREFIX

Prefix used to create numbered columns for missing headers. Typical names: column_1, column_2, ...

See Also:

Constant Field Values

Constructor Detail

HeaderBuilder

public HeaderBuilder(org.apache.hadoop.fs.Path filePath)

Method Detail

startField

public void startField(int index)

Start processing a new field within a record.

Parameters:

index - index within the record

endField

public boolean endField()

End processing a field within a record.

Returns:

true if engine should continue processing record. false if rest of record can be skipped.

endEmptyField

public boolean endEmptyField()

Shortcut that lets the output know that we are closing ending a field with no data.

Returns:

true if engine should continue processing record. false if rest of record can be skipped.

append

public void append(byte data)

Appends a byte to the output character data buffer.

Parameters:

data - current byte read

finishRecord

public void finishRecord()

Completes the processing of a given record. Also completes the processing of the last field being read.

startRecord

public void startRecord()

Start processing a new record.

getHeaders

public String[] getHeaders()

getRecordCount

public long getRecordCount()

Return the total number of records (across batches) processed

Returns:

record count

isFull

public boolean isFull()

Indicates if the current batch is full and reading for this batch should stop.

Returns:

true if the batch is full and the reader must exit to allow the batch to be sent downstream, false if the reader may continue to add rows to the current batch