Package org.apache.drill.exec.store.parquet.columnreaders

Class ColumnReader<V extends ValueVector>

java.lang.Object
org.apache.drill.exec.store.parquet.columnreaders.ColumnReader<V>
Direct Known Subclasses:
NullableFixedByteAlignedReaders.CorruptionDetectingNullableDateReader, NullableFixedByteAlignedReaders.NullableCorruptDateReader, NullableFixedByteAlignedReaders.NullableDateReader, NullableFixedByteAlignedReaders.NullableIntervalReader, VarLengthColumn
public abstract class ColumnReader<V extends ValueVector> extends Object

  • Field Details

    • DICTIONARY_ENCODINGS

      public static final Set<org.apache.parquet.column.Encoding> DICTIONARY_ENCODINGS

    • VALUE_ENCODINGS

      public static final Set<org.apache.parquet.column.Encoding> VALUE_ENCODINGS

    • vectorData

      protected DrillBuf vectorData

  • Constructor Details

    • ColumnReader

      protected ColumnReader(ParquetRecordReader parentReader, org.apache.parquet.column.ColumnDescriptor descriptor, org.apache.parquet.hadoop.metadata.ColumnChunkMetaData columnChunkMetaData, boolean fixedLength, V v, org.apache.parquet.format.SchemaElement schemaElement) throws ExecutionSetupException
      Throws:
      ExecutionSetupException

  • Method Details

    • getRecordsReadInCurrentPass

      public int getRecordsReadInCurrentPass()

    • processPagesAsync

      public Future<Long> processPagesAsync(long recordsToReadInThisPass)

    • processPages

      public void processPages(long recordsToReadInThisPass) throws IOException
      Throws:
      IOException

    • clear

      public void clear()

    • readValues

      public void readValues(long recordsToRead)

    • readField

      protected abstract void readField(long recordsToRead)

    • determineSize

      public boolean determineSize(long recordsReadInCurrentPass) throws IOException
      Determines the size of a single value in a variable column. Return value indicates if we have finished a row group and should stop reading
      Parameters:
      recordsReadInCurrentPass - records read in current pass
      Returns:
      true if we should stop reading
      Throws:
      IOException

    • readRecordsAsync

      protected Future<Integer> readRecordsAsync(int recordsToRead)

    • readRecords

      protected void readRecords(int recordsToRead)

    • readRecordsInBulk

      protected int readRecordsInBulk(int recordsToReadInThisPass) throws IOException
      Throws:
      IOException

    • recordsRequireDecoding

      protected boolean recordsRequireDecoding()

    • processPageData

      protected boolean processPageData(int recordsToReadInThisPass) throws IOException
      Throws:
      IOException

    • updatePosition

      public void updatePosition()

    • updateReadyToReadPosition

      public void updateReadyToReadPosition()

    • reset

      public void reset()

    • capacity

      public int capacity()

    • readPageAsync

      public Future<Boolean> readPageAsync()

    • readPage

      public boolean readPage() throws IOException
      Read a page. If we need more data, exit the read loop and return true.
      Returns:
      true if we need more data and page is not read successfully
      Throws:
      IOException

    • totalValuesReadAndReadyToReadInPage

      protected int totalValuesReadAndReadyToReadInPage()

    • postPageRead

      protected void postPageRead()

    • hitRowGroupEnd

      protected void hitRowGroupEnd()

    • checkVectorCapacityReached

      protected boolean checkVectorCapacityReached()

    • readIntLittleEndian

      public static int readIntLittleEndian(DrillBuf in, int offset)
      This is copied out of Parquet library, didn't want to deal with the unnecessary throws statement they had declared
      Parameters:
      in - incoming data
      offset - offset
      Returns:
      little endian integer