org.apache.drill.exec.store

Class ClassicConnectorLocator

java.lang.Object

org.apache.drill.exec.store.ClassicConnectorLocator

All Implemented Interfaces:

ConnectorLocator

public class ClassicConnectorLocator
extends Object
implements ConnectorLocator

Plugin locator for the "classic" class-path method of locating connectors. Connectors appear somewhere on the class path, and are subclasses of StoragePlugin. The connector and its configuration class must reside in Drill's default class loader.

Handles "classic" storage plugin classes which ship with Drill, or are added to Drill's class path.

The plugin registry supports access from multiple threads concurrently. This locator is therefore immutable after the init() call. The set of plugin instances and configurations (managed by the registry) changes during a run, but the set of known plugin classes is fixed.

We sometimes need to add special plugins for testing. Since we cannot add them on the fly, we must add them at (test) startup time via config properties.

Plugin Implementations

This class manages plugin implementation classes (AKA "connectors") which must derive from StoragePlugin. Each must be configured via a class derived from StoragePluginConfig. No two connectors can share a configuration class. Each connector must have exactly one associated configuration. (Actually, a single connector might handle multiple configurations, but that seems an obscure use case.)

The constructor of the connector associates the two classes, and must be of the form:

 public SomePlugin(SomePluginConfig config,
                   DrillbitContext context,
                   String pluginName) {

Classes must be on the class path. Drill often caches the class path: creating it at build time, then storing it in a file, to be read at run time. If you are developing a plugin in an IDE, and this class refuses to find the plugin, you can temporarly force a runtime class path scan via setting the ClassPathScanner.IMPLEMENTATIONS_SCAN_CACHE config property to false. The various test "fixtures" provide an easy way to set config properties per-test.

This locator ignores four categories of StoragePluginConfig classes:

• Interfaces or abstract classes
• Those that do not have the constructor described above.
• System plugins with the SystemPlugin annotation.
• Private test plugins with the PrivatePlugin annotation.

Config Properties

ExecConstants.PRIVATE_CONNECTORS (drill.exec.storage.private_connectors)

An optional list of private plugin class names. Private plugins are valid instances of StoragePlugin which have the PrivatePlugin annotation and so are not automatically loaded.

Constructor Summary

Constructors

Constructor and Description
ClassicConnectorLocator(PluginRegistryContext context)

Method Summary

Modifier and Type	Method and Description
StoragePlugins	bootstrapPlugins() Read bootstrap storage plugins ExecConstants.BOOTSTRAP_STORAGE_PLUGINS_FILE and format plugins ExecConstants.BOOTSTRAP_FORMAT_PLUGINS_FILE files for the first fresh install of Drill.
void	close()
Set<Class<? extends StoragePluginConfig>>	configClasses() Return the set of known storage plugin configuration classes for which the user can create configs.
Class<? extends StoragePlugin>	connectorClassFor(Class<? extends StoragePluginConfig> configClass) Given a configuration class, return the corresponding connector (plugin) class.
static Map<Class<? extends StoragePluginConfig>,Constructor<? extends StoragePlugin>>	constuctorsFor(Class<? extends StoragePlugin> plugin)
StoragePlugin	create(String name, StoragePluginConfig pluginConfig) Creates plugin instance with the given name and configuration pluginConfig.
StoragePlugin	get(String name) Classic storage plugins do not provide default configurations.
void	init() Initialize the locator.
List<StoragePlugin>	intrinsicPlugins() Enumerate the intrinsic plugins.
void	onUpgrade() If updatedPlugins() returned non-null, then the registry will call this method after successful update of the persistent store.
boolean	storable()
StoragePlugins	updatedPlugins() Identify plugins to be added to an existing system, typically on the first run after an upgrade.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ClassicConnectorLocator

public ClassicConnectorLocator(PluginRegistryContext context)

Method Detail

init

public void init()

Description copied from interface: ConnectorLocator

Initialize the locator. Must be called before the locator is used.

Specified by:

init in interface ConnectorLocator

constuctorsFor

public static Map<Class<? extends StoragePluginConfig>,Constructor<? extends StoragePlugin>> constuctorsFor(Class<? extends StoragePlugin> plugin)

configClasses

public Set<Class<? extends StoragePluginConfig>> configClasses()

Description copied from interface: ConnectorLocator

Return the set of known storage plugin configuration classes for which the user can create configs. Excludes system plugin configs. Used to map config classes to this locator to create plugin instances.

Specified by:

configClasses in interface ConnectorLocator

Returns:

the unuordered set of storage plugin configuration classes available from this locator. Can be null if this locator offers only system plugins

get

public StoragePlugin get(String name)

Classic storage plugins do not provide default configurations.

Specified by:

get in interface ConnectorLocator

Parameters:

name - the name of a connector class (not the name of a plugin (configuration)

Returns:

a plugin with default configuration, or null if this locator does not support such plugins

intrinsicPlugins

public List<StoragePlugin> intrinsicPlugins()

Description copied from interface: ConnectorLocator

Enumerate the intrinsic plugins. An intrinsic plugin is one which takes no configuration and which therefore cannot be disabled, and thus is always available. Example: Drill's system plugins. For an intrinsic plugin, the plugin name is also the name of the configuration.

Specified by:

intrinsicPlugins in interface ConnectorLocator

Returns:

map of intrinsic plugins which require no configuration

bootstrapPlugins

public StoragePlugins bootstrapPlugins()
                                throws IOException

Read bootstrap storage plugins ExecConstants.BOOTSTRAP_STORAGE_PLUGINS_FILE and format plugins ExecConstants.BOOTSTRAP_FORMAT_PLUGINS_FILE files for the first fresh install of Drill.

Specified by:

bootstrapPlugins in interface ConnectorLocator

Parameters:

lpPersistence - deserialization mapper provider

Returns:

bootstrap storage plugins

Throws:

IOException - if a read error occurs

updatedPlugins

public StoragePlugins updatedPlugins()

Description copied from interface: ConnectorLocator

Identify plugins to be added to an existing system, typically on the first run after an upgrade.

TODO: The current mechanism depends on deleting a file after the first run, which is unreliable. It won't, for example, correctly handle a restored ZK. A better mechanism would store a version number in the persistent store, and pass that version number into this method.

Specified by:

updatedPlugins in interface ConnectorLocator

Returns:

the set of plugin configurations to refresh in the persistent store, or null if none to update

onUpgrade

public void onUpgrade()

Description copied from interface: ConnectorLocator

If updatedPlugins() returned non-null, then the registry will call this method after successful update of the persistent store. This method can do any post-update cleanup, such as deleting the file mentioned above.

Specified by:

onUpgrade in interface ConnectorLocator

create

public StoragePlugin create(String name,
                            StoragePluginConfig pluginConfig)
                     throws ExecutionSetupException

Creates plugin instance with the given name and configuration pluginConfig. The plugin need to be present in a list of available plugins and be enabled in the configuration

Specified by:

create in interface ConnectorLocator

Parameters:

name - name of the plugin

pluginConfig - plugin configuration

Returns:

plugin client or null if plugin is disabled

Throws:

ExecutionSetupException - for all errors

storable

public boolean storable()

Specified by:

storable in interface ConnectorLocator

Returns:

true if configs for this locator should be persisted, false if these are ad-hoc or otherwise per-run connectors

connectorClassFor

public Class<? extends StoragePlugin> connectorClassFor(Class<? extends StoragePluginConfig> configClass)

Description copied from interface: ConnectorLocator

Given a configuration class, return the corresponding connector (plugin) class.

Specified by:

connectorClassFor in interface ConnectorLocator

close

public void close()

Specified by:

close in interface ConnectorLocator