Package org.apache.drill.yarn.appMaster

Interface ClusterController

All Superinterfaces:
RegistryHandler
All Known Implementing Classes:
ClusterControllerImpl
public interface ClusterController extends RegistryHandler
Interface which identifies the cluster controller methods that are save to call from the Dispatcher. Methods here are either designed to be called before the event threads start or after they complete. The remainder synchronized to coordinate between event threads.

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    cancelTask(int id)
    Cancels the given task, reducing the target task count.
    void
    containersAllocated(List<org.apache.hadoop.yarn.api.records.Container> containers)
    The RM has allocated one or more containers in response to container requests submitted to the RM.
    void
    containersCompleted(List<org.apache.hadoop.yarn.api.records.ContainerStatus> statuses)
    The Resource Manager reports that containers have completed with the given statuses.
    void
    containerStarted(org.apache.hadoop.yarn.api.records.ContainerId containerId)
    The NM reports that a container has successfully started.
    void
    containerStopped(org.apache.hadoop.yarn.api.records.ContainerId containerId)
    The Node Manager reports that a container has stopped.
    void
    enableFailureCheck(boolean flag)
     
    int
    getFreeNodeCount()
     
    float
    getProgress()
     
    Object
    getProperty(String key)
     
    int
    getTargetCount()
    Return the target number of tasks that the controller seeks to maintain.
    boolean
    isTaskLive(int id)
     
    void
    registerLifecycleListener(TaskLifecycleListener listener)
     
    void
    registerScheduler(Scheduler resourceGroup)
     
    void
    resizeDelta(int delta)
    Request to resize the Drill cluster by a relative amount.
    int
    resizeTo(int n)
    Request to resize the Drill cluster to the given size.
    void
    setMaxRetries(int value)
     
    void
    setProperty(String key, Object value)
     
    void
    shutDown()
    Indicates a request to gracefully shut down the cluster.
    void
    started()
    Called after the dispatcher has started YARN and other server components.
    void
    stopTaskFailed(org.apache.hadoop.yarn.api.records.ContainerId containerId, Throwable t)
    The Node Manager API reports that a request sent to the NM to stop a task has failed.
    boolean
    supportsDiskResource()
    Whether this distribution of YARN supports disk resources.
    void
    taskStartFailed(org.apache.hadoop.yarn.api.records.ContainerId containerId, Throwable t)
    The RM API reports that an attempt to start a container has failed locally.
    void
    tick(long curTime)
    Called by the timer ("pulse") thread to trigger time-based events.
    void
    updateRMStatus()
     
    void
    visit(ControllerVisitor visitor)
    Allow an observer to see a consistent view of the controller's state by performing the visit in a synchronized block.
    void
    visitTasks(TaskVisitor visitor)
    Allow an observer to see a consistent view of the controller's task state by performing the visit in a synchronized block.
    boolean
    waitForCompletion()
    Called by the main thread to wait for the normal shutdown of the controller.

    Methods inherited from interface org.apache.drill.yarn.appMaster.RegistryHandler

    completionAck, registryDown, releaseHost, reserveHost, startAck

  • Method Details

    • enableFailureCheck

      void enableFailureCheck(boolean flag)

    • registerLifecycleListener

      void registerLifecycleListener(TaskLifecycleListener listener)

    • registerScheduler

      void registerScheduler(Scheduler resourceGroup)

    • setProperty

      void setProperty(String key, Object value)

    • getProperty

      Object getProperty(String key)

    • started

      void started() throws YarnFacadeException, AMException
      Called after the dispatcher has started YARN and other server components. The controller can now begin to spin up tasks.
      Throws:
      YarnFacadeException
      AMException

    • tick

      void tick(long curTime)
      Called by the timer ("pulse") thread to trigger time-based events.
      Parameters:
      curTime -

    • containersAllocated

      void containersAllocated(List<org.apache.hadoop.yarn.api.records.Container> containers)
      The RM has allocated one or more containers in response to container requests submitted to the RM.
      Parameters:
      containers - the set of containers provided by YARN

    • containerStarted

      void containerStarted(org.apache.hadoop.yarn.api.records.ContainerId containerId)
      The NM reports that a container has successfully started.
      Parameters:
      containerId - the container which started

    • taskStartFailed

      void taskStartFailed(org.apache.hadoop.yarn.api.records.ContainerId containerId, Throwable t)
      The RM API reports that an attempt to start a container has failed locally.
      Parameters:
      containerId - the container that failed to launch
      t - the error that occurred

    • containerStopped

      void containerStopped(org.apache.hadoop.yarn.api.records.ContainerId containerId)
      The Node Manager reports that a container has stopped.
      Parameters:
      containerId -

    • containersCompleted

      void containersCompleted(List<org.apache.hadoop.yarn.api.records.ContainerStatus> statuses)
      The Resource Manager reports that containers have completed with the given statuses. Find the task for each container and mark them as completed.
      Parameters:
      statuses -

    • getProgress

      float getProgress()

    • stopTaskFailed

      void stopTaskFailed(org.apache.hadoop.yarn.api.records.ContainerId containerId, Throwable t)
      The Node Manager API reports that a request sent to the NM to stop a task has failed.
      Parameters:
      containerId - the container that failed to stop
      t - the reason that the stop request failed

    • resizeDelta

      void resizeDelta(int delta)
      Request to resize the Drill cluster by a relative amount.
      Parameters:
      delta - the amount of change. Can be positive (to grow) or negative (to shrink the cluster)

    • resizeTo

      int resizeTo(int n)
      Request to resize the Drill cluster to the given size.
      Parameters:
      n - the desired cluster size

    • shutDown

      void shutDown()
      Indicates a request to gracefully shut down the cluster.

    • waitForCompletion

      boolean waitForCompletion()
      Called by the main thread to wait for the normal shutdown of the controller. Such shutdown occurs when the admin sends a sutdown command from the UI or REST API.

    • updateRMStatus

      void updateRMStatus()

    • setMaxRetries

      void setMaxRetries(int value)

    • visit

      void visit(ControllerVisitor visitor)
      Allow an observer to see a consistent view of the controller's state by performing the visit in a synchronized block.
      Parameters:
      visitor -

    • visitTasks

      void visitTasks(TaskVisitor visitor)
      Allow an observer to see a consistent view of the controller's task state by performing the visit in a synchronized block.
      Parameters:
      visitor -

    • getTargetCount

      int getTargetCount()
      Return the target number of tasks that the controller seeks to maintain. This is the sum across all pools.

    • isTaskLive

      boolean isTaskLive(int id)

    • cancelTask

      boolean cancelTask(int id)
      Cancels the given task, reducing the target task count. Called from the UI to allow the user to select the specific task to end when reducing cluster size.
      Parameters:
      id -

    • supportsDiskResource

      boolean supportsDiskResource()
      Whether this distribution of YARN supports disk resources.
      Returns:
      True if this distribution of YARN supports disk resources. False otherwise.

    • getFreeNodeCount

      int getFreeNodeCount()