Package org.apache.drill.exec.store.http

Class HttpBatchReader

java.lang.Object
org.apache.drill.exec.store.http.HttpBatchReader
All Implemented Interfaces:
ManagedReader<SchemaNegotiator>
Direct Known Subclasses:
HttpCSVBatchReader, HttpXMLBatchReader
public class HttpBatchReader extends Object implements ManagedReader<SchemaNegotiator>

  • Field Details

  • Constructor Details

  • Method Details

    • open

      public boolean open(SchemaNegotiator negotiator)
      Description copied from interface: ManagedReader
      Setup the record reader. Called just before the first call to next(). Allocate resources here, not in the constructor. Example: open files, allocate buffers, etc.
      Specified by:
      open in interface ManagedReader<SchemaNegotiator>
      Parameters:
      negotiator - mechanism to negotiate select and table schemas, then create the row set reader used to load data into value vectors
      Returns:
      true if the reader is open and ready to read (possibly no) rows. false for a "soft" failure in which no schema or data is available, but the scanner should not fail, it should move onto another reader

    • buildImplicitColumns

      protected void buildImplicitColumns()

    • populateImplicitFieldMap

      protected void populateImplicitFieldMap(SimpleHttp http)

    • generatePaginationFieldMap

      protected Map<String,Object> generatePaginationFieldMap()

    • implicitColumnsAreProjected

      protected boolean implicitColumnsAreProjected()

    • buildUrl

      protected okhttp3.HttpUrl buildUrl()

    • addFilters

      protected void addFilters(okhttp3.HttpUrl.Builder urlBuilder, List<String> params, Map<String,String> filters)
      Convert equality filter conditions into HTTP query parameters Parameters must appear in the order defined in the config.

    • proxySettings

      protected HttpProxyConfig proxySettings(com.typesafe.config.Config drillConfig, okhttp3.HttpUrl url)

    • populateIndexPaginator

      protected void populateIndexPaginator()

    • next

      public boolean next()
      Description copied from interface: ManagedReader
      Read the next batch. Reading continues until either EOF, or until the mutator indicates that the batch is full. The batch is considered valid if it is non-empty. Returning true with an empty batch is valid, and is helpful on the very first batch (returning schema only.) An empty batch with a false return code indicates EOF and the batch will be discarded. A non-empty batch along with a false return result indicates a final, valid batch, but that EOF was reached and no more data is available.

      This somewhat complex protocol avoids the need to allocate a final batch just to find out that no more data is available; it allows EOF to be returned along with the final batch.

      Specified by:
      next in interface ManagedReader<SchemaNegotiator>
      Returns:
      true if more data may be available (and so next() should be called again, false to indicate that EOF was reached

    • close

      public void close()
      Description copied from interface: ManagedReader
      Release resources. Called just after a failure, when the scanner is cancelled, or after next() returns EOF. Release all resources and close files. Guaranteed to be called if open() returns normally; will not be called if open() throws an exception.
      Specified by:
      close in interface ManagedReader<SchemaNegotiator>