org.apache.drill.yarn.zk

Class ZKClusterCoordinator

java.lang.Object

org.apache.drill.exec.coord.ClusterCoordinator

org.apache.drill.yarn.zk.ZKClusterCoordinator

All Implemented Interfaces:

AutoCloseable

public class ZKClusterCoordinator
extends ClusterCoordinator

Manages cluster coordination utilizing zookeeper.

This is a clone of the Drill class org.apache.drill.exec.coord.zk.ZKClusterCoordinator with a number of modifications:

• Removed dependency on the Drill config system. That system uses Google's Guava library version 18, which conflicts with the earlier versions used by YARN and Hadoop, which resulted in runtime undefined method exceptions.
• Instead of getting config information out of the Drill config, the parameters are instead passed directly.
• Adds support for the drillbits registered event which was neither needed nor implemented by Drill.
• Use the YARN logging system instead of Drill's.

This class should be replaced by the Drill version if/when the Guava conflicts can be resolved (and when registered Drillbit notifications are added to the Drill version.)

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.drill.exec.coord.ClusterCoordinator

ClusterCoordinator.RegistrationHandle

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.apache.commons.logging.Log	logger

Fields inherited from class org.apache.drill.exec.coord.ClusterCoordinator

listeners

Constructor Summary

Constructors

Constructor and Description
ZKClusterCoordinator(String connect, String zkRoot, String clusterId, int retryCount, int retryDelayMs, int connectTimeoutMs)

Method Summary

Modifier and Type	Method and Description
void	close()
Collection<CoordinationProtos.DrillbitEndpoint>	getAvailableEndpoints() Get a collection of available Drillbit endpoints, Thread-safe.
org.apache.curator.framework.CuratorFramework	getCurator()
Collection<CoordinationProtos.DrillbitEndpoint>	getOnlineEndPoints() Get a collection of ONLINE drillbit endpoints by excluding the drillbits that are in QUIESCENT state (drillbits that are shutting down).
<V> TransientStore<V>	getOrCreateTransientStore(TransientStoreConfig<V> config) Returns a store instance with the given configuration.
DistributedSemaphore	getSemaphore(String name, int maximumLeases)
protected org.apache.curator.x.discovery.ServiceDiscovery<CoordinationProtos.DrillbitEndpoint>	newDiscovery()
protected org.apache.curator.x.discovery.ServiceInstance<CoordinationProtos.DrillbitEndpoint>	newServiceInstance(CoordinationProtos.DrillbitEndpoint endpoint)
ClusterCoordinator.RegistrationHandle	register(CoordinationProtos.DrillbitEndpoint data)
void	start(long millisToWait) Start the cluster coordinator.
void	unregister(ClusterCoordinator.RegistrationHandle handle)
ClusterCoordinator.RegistrationHandle	update(ClusterCoordinator.RegistrationHandle handle, CoordinationProtos.DrillbitEndpoint.State state)

Methods inherited from class org.apache.drill.exec.coord.ClusterCoordinator

addDrillbitStatusListener, drillbitRegistered, drillbitUnregistered, isDrillbitInState, removeDrillbitStatusListener

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.apache.commons.logging.Log logger

Constructor Detail

ZKClusterCoordinator

public ZKClusterCoordinator(String connect,
                            String zkRoot,
                            String clusterId,
                            int retryCount,
                            int retryDelayMs,
                            int connectTimeoutMs)
                     throws IOException

Throws:

IOException

Method Detail

getCurator

public org.apache.curator.framework.CuratorFramework getCurator()

start

public void start(long millisToWait)
           throws Exception

Description copied from class: ClusterCoordinator

Start the cluster coordinator. Millis to wait is

Specified by:

start in class ClusterCoordinator

Parameters:

millisToWait - The maximum time to wait before throwing an exception if the cluster coordination service has not successfully started. Use 0 to wait indefinitely.

Throws:

Exception - in case when unable to start coordinator

close

public void close()
           throws Exception

Throws:

Exception

register

public ClusterCoordinator.RegistrationHandle register(CoordinationProtos.DrillbitEndpoint data)

Specified by:

register in class ClusterCoordinator

unregister

public void unregister(ClusterCoordinator.RegistrationHandle handle)

Specified by:

unregister in class ClusterCoordinator

getAvailableEndpoints

public Collection<CoordinationProtos.DrillbitEndpoint> getAvailableEndpoints()

Description copied from class: ClusterCoordinator

Get a collection of available Drillbit endpoints, Thread-safe. Could be slightly out of date depending on refresh policy.

Specified by:

getAvailableEndpoints in class ClusterCoordinator

Returns:

A collection of available endpoints.

getSemaphore

public DistributedSemaphore getSemaphore(String name,
                                         int maximumLeases)

Specified by:

getSemaphore in class ClusterCoordinator

getOrCreateTransientStore

public <V> TransientStore<V> getOrCreateTransientStore(TransientStoreConfig<V> config)

Description copied from class: ClusterCoordinator

Returns a store instance with the given configuration. Note that implementor might cache the instance so new instance creation is not guaranteed.

Specified by:

getOrCreateTransientStore in class ClusterCoordinator

Type Parameters:

V - value type for this store

Parameters:

config - store configuration

newServiceInstance

protected org.apache.curator.x.discovery.ServiceInstance<CoordinationProtos.DrillbitEndpoint> newServiceInstance(CoordinationProtos.DrillbitEndpoint endpoint)
                                                                                                          throws Exception

Throws:

Exception

newDiscovery

protected org.apache.curator.x.discovery.ServiceDiscovery<CoordinationProtos.DrillbitEndpoint> newDiscovery()

getOnlineEndPoints

public Collection<CoordinationProtos.DrillbitEndpoint> getOnlineEndPoints()

Description copied from class: ClusterCoordinator

Get a collection of ONLINE drillbit endpoints by excluding the drillbits that are in QUIESCENT state (drillbits that are shutting down). Primarily used by the planner to plan queries only on ONLINE drillbits and used by the client during initial connection phase to connect to a drillbit (foreman)

Specified by:

getOnlineEndPoints in class ClusterCoordinator

Returns:

A collection of ONLINE endpoints

update

public ClusterCoordinator.RegistrationHandle update(ClusterCoordinator.RegistrationHandle handle,
                                                    CoordinationProtos.DrillbitEndpoint.State state)

Specified by:

update in class ClusterCoordinator