Package org.apache.drill.exec.store.dfs

Class FileSystemPlugin

java.lang.Object
org.apache.drill.exec.store.AbstractStoragePlugin
org.apache.drill.exec.store.dfs.FileSystemPlugin
All Implemented Interfaces:
AutoCloseable, SchemaFactory, StoragePlugin
public class FileSystemPlugin extends AbstractStoragePlugin
A Storage engine associated with a Hadoop FileSystem Implementation. Examples include HDFS, MapRFS, QuantacastFileSystem, LocalFileSystem, as well Apache Drill specific CachedFileSystem, ClassPathFileSystem and LocalSyncableFileSystem. Tables are file names, directories and path patterns. This storage engine delegates to FSFormatEngines but shares references to the FileSystem configuration and path management.

  • Constructor Details

  • Method Details

    • initializeOauthTokenTable

      public void initializeOauthTokenTable(String username)

    • getTokenRegistry

      public TokenRegistry getTokenRegistry()

    • getTokenRegistry

      public TokenRegistry getTokenRegistry(String username)
      This method returns the TokenRegistry for a given user. It is only used for testing user translation with OAuth 2.0.
      Parameters:
      username - A String of the current active user.
      Returns:
      A TokenRegistry for the given user.

    • getTokenTable

      public PersistentTokenTable getTokenTable()

    • newFormatCreator

      protected FormatCreator newFormatCreator(FileSystemConfig config, DrillbitContext context, org.apache.hadoop.conf.Configuration fsConf)
      Creates a new FormatCreator instance. To be used by subclasses to return custom formats if required. Note that this method is called by the constructor, which fields may not be initialized yet.
      Parameters:
      config - the plugin configuration
      context - the drillbit context
      Returns:
      a new FormatCreator instance

    • supportsRead

      public boolean supportsRead()
      Description copied from interface: StoragePlugin
      Indicates if Drill can read the table from this format.
      Specified by:
      supportsRead in interface StoragePlugin
      Overrides:
      supportsRead in class AbstractStoragePlugin

    • getConfig

      public StoragePluginConfig getConfig()
      Description copied from interface: StoragePlugin
      Method returns a Jackson serializable object that extends a StoragePluginConfig.
      Returns:
      an extension of StoragePluginConfig

    • getPhysicalScan

      public AbstractGroupScan getPhysicalScan(String userName, JSONOptions selection, SessionOptionManager options) throws IOException
      Description copied from interface: StoragePlugin
      Get the physical scan operator for the particular GroupScan (read) node.
      Specified by:
      getPhysicalScan in interface StoragePlugin
      Overrides:
      getPhysicalScan in class AbstractStoragePlugin
      Parameters:
      userName - User whom to impersonate when reading the contents as part of Scan.
      selection - The configured storage engine specific selection.
      options - (optional) session options
      Returns:
      The physical scan operator for the particular GroupScan (read) node.
      Throws:
      IOException

    • getPhysicalScan

      public AbstractGroupScan getPhysicalScan(String userName, JSONOptions selection, SessionOptionManager options, MetadataProviderManager metadataProviderManager) throws IOException
      Description copied from interface: StoragePlugin
      Get the physical scan operator for the particular GroupScan (read) node.
      Specified by:
      getPhysicalScan in interface StoragePlugin
      Overrides:
      getPhysicalScan in class AbstractStoragePlugin
      Parameters:
      userName - User whom to impersonate when reading the contents as part of Scan.
      selection - The configured storage engine specific selection.
      options - (optional) session options
      metadataProviderManager - manager for handling metadata providers
      Returns:
      The physical scan operator for the particular GroupScan (read) node.
      Throws:
      IOException

    • getPhysicalScan

      public AbstractGroupScan getPhysicalScan(String userName, JSONOptions selection, List<SchemaPath> columns) throws IOException
      Description copied from interface: StoragePlugin
      Get the physical scan operator for the particular GroupScan (read) node.
      Specified by:
      getPhysicalScan in interface StoragePlugin
      Overrides:
      getPhysicalScan in class AbstractStoragePlugin
      Parameters:
      userName - User whom to impersonate when reading the contents as part of Scan.
      selection - The configured storage engine specific selection.
      columns - (optional) The list of column names to scan from the data source.
      Returns:
      The physical scan operator for the particular GroupScan (read) node.
      Throws:
      IOException

    • getPhysicalScan

      public AbstractGroupScan getPhysicalScan(String userName, JSONOptions selection, List<SchemaPath> columns, SessionOptionManager options, MetadataProviderManager metadataProviderManager) throws IOException
      Description copied from interface: StoragePlugin
      Get the physical scan operator for the particular GroupScan (read) node.
      Specified by:
      getPhysicalScan in interface StoragePlugin
      Overrides:
      getPhysicalScan in class AbstractStoragePlugin
      Parameters:
      userName - User whom to impersonate when reading the contents as part of Scan.
      selection - The configured storage engine specific selection.
      columns - (optional) The list of column names to scan from the data source.
      options - (optional) session options
      metadataProviderManager - manager for handling metadata providers
      Returns:
      The physical scan operator for the particular GroupScan (read) node.
      Throws:
      IOException

    • registerSchemas

      public void registerSchemas(SchemaConfig schemaConfig, org.apache.calcite.schema.SchemaPlus parent) throws IOException
      Description copied from interface: SchemaFactory
      Register the schemas provided by this SchemaFactory implementation under the given parent schema.
      Parameters:
      schemaConfig - Configuration for schema objects.
      parent - Reference to parent schema.
      Throws:
      IOException - in case of error during schema registration

    • getFormatPlugin

      public FormatPlugin getFormatPlugin(String name)

    • getFormatPlugin

      public FormatPlugin getFormatPlugin(FormatPluginConfig config)
      If format plugin configuration is for named format plugin, will return format plugin from pre-loaded list by name. For other cases will try to find format plugin by its configuration, if not present will attempt to create one.
      Specified by:
      getFormatPlugin in interface StoragePlugin
      Overrides:
      getFormatPlugin in class AbstractStoragePlugin
      Parameters:
      config - format plugin configuration
      Returns:
      format plugin for given configuration if found, null otherwise

    • getOptimizerRules

      public Set<? extends org.apache.calcite.plan.RelOptRule> getOptimizerRules(OptimizerRulesContext optimizerContext, PlannerPhase phase)
      Description copied from interface: StoragePlugin
      An implementation of this method will return one or more specialized rules that Drill query optimizer can leverage in physical space. Otherwise, it should return an empty set.
      Returns:
      an empty set or a set of plugin specific physical optimizer rules.

    • getFsConf

      public org.apache.hadoop.conf.Configuration getFsConf()

    • initializeTokenTableForTesting

      public void initializeTokenTableForTesting()
      This function is only used for testing and creates the necessary token tables. Note that the token tables still need to be populated.

    • start

      public void start()
      Description copied from interface: StoragePlugin
      Initialize the storage plugin. The storage plugin will not be used until this method is called.
      Specified by:
      start in interface StoragePlugin
      Overrides:
      start in class AbstractStoragePlugin

    • onEnabled

      public void onEnabled()
      Description copied from interface: StoragePlugin
      Lifecycle method allowing the plugin to perform operations when it has been enabled.
      Specified by:
      onEnabled in interface StoragePlugin
      Overrides:
      onEnabled in class AbstractStoragePlugin

    • onDisabled

      public void onDisabled()
      Description copied from interface: StoragePlugin
      Lifecycle method allowing the plugin to perform operations when it has been disabled.
      Specified by:
      onDisabled in interface StoragePlugin
      Overrides:
      onDisabled in class AbstractStoragePlugin

    • close

      public void close()
      Specified by:
      close in interface AutoCloseable
      Overrides:
      close in class AbstractStoragePlugin