Package org.apache.drill.yarn.appMaster

Interface Scheduler

All Known Implementing Classes:
AbstractDrillbitScheduler, AbstractScheduler, BatchScheduler, DrillbitScheduler, PersistentTaskScheduler
public interface Scheduler
The scheduler describes the set of tasks to run. It provides the details required to launch each task and optionally a specification of the containers required to run the task.

Schedulers can manage batch task (which do their job and complete), or persistent tasks (which run until terminated.)

The scheduler tracks task completion (for batch tasks) and task levels (for persistent tasks.)

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static interface 
    Scheduler.TaskManager
     

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    adjust()
    Adjust the number of running tasks to better track the desired number.
    void
    change(int delta)
    Increase (positive) or decrease (negative) the number of desired tasks by the given amount.
    void
    completed(Task task)
     
    String
    getName()
     
    int[]
    getProgress()
    Return an estimate of progress given as a ratio of (work completed, total work).
    int
    getRequestTimeoutSec()
    Maximum amount of time to wait when cancelling a job in the REQUESTING state.
    ContainerRequestSpec
    getResource()
    For reporting, get the YARN resources requested by processes in this pool.
    int
    getTarget()
    Get the desired number of running tasks.
    Scheduler.TaskManager
    getTaskManager()
     
    String
    getType()
     
    boolean
    hasMoreTasks()
    If this is a batch scheduler, whether all tasks for the batch have completed.
    boolean
    isTracked()
    Whether tasks from this scheduler should incorporate app startup/shutdown acknowledgements (acks) into the task lifecycle.
    void
    limitContainerSize(org.apache.hadoop.yarn.api.records.Resource maxResource)
     
    void
    registerState(SchedulerState state)
    Register the state object that tracks tasks launched by this scheduler.
    void
    requestTimedOut()
    Informs the scheduler that a YARN resource request timed out.
    int
    resize(int level)
    Set the number of desired tasks to the given level.
    void
    setPriority(int priority)
    Controller-assigned priority for this scheduler.

  • Method Details

    • setPriority

      void setPriority(int priority)
      Controller-assigned priority for this scheduler. Used to differentiate container requests by scheduler.
      Parameters:
      priority -

    • registerState

      void registerState(SchedulerState state)
      Register the state object that tracks tasks launched by this scheduler.
      Parameters:
      state -

    • getName

      String getName()

    • getType

      String getType()

    • isTracked

      boolean isTracked()
      Whether tasks from this scheduler should incorporate app startup/shutdown acknowledgements (acks) into the task lifecycle.

    • getTaskManager

      Scheduler.TaskManager getTaskManager()

    • getTarget

      int getTarget()
      Get the desired number of running tasks.
      Returns:
      The desired number of running tasks

    • change

      void change(int delta)
      Increase (positive) or decrease (negative) the number of desired tasks by the given amount.
      Parameters:
      delta -

    • resize

      int resize(int level)
      Set the number of desired tasks to the given level.
      Parameters:
      level -
      Returns:
      the actual resize level, which may be lower than the requested level if the system cannot provide the requested level

    • completed

      void completed(Task task)

    • adjust

      void adjust()
      Adjust the number of running tasks to better track the desired number. Starts or stops tasks using the SchedulerState registered with registerState(SchedulerState).

    • getProgress

      int[] getProgress()
      Return an estimate of progress given as a ratio of (work completed, total work).
      Returns:
      Estimate of progress.

    • hasMoreTasks

      boolean hasMoreTasks()
      If this is a batch scheduler, whether all tasks for the batch have completed. If this is a persistent task scheduler, always returns false.
      Returns:
      true if the scheduler has more tasks to run, false if the scheduler has no more tasks or manages a set of long-running tasks

    • getResource

      ContainerRequestSpec getResource()
      For reporting, get the YARN resources requested by processes in this pool.
      Returns:
      The request spec.

    • limitContainerSize

      void limitContainerSize(org.apache.hadoop.yarn.api.records.Resource maxResource) throws AMException
      Throws:
      AMException

    • getRequestTimeoutSec

      int getRequestTimeoutSec()
      Maximum amount of time to wait when cancelling a job in the REQUESTING state. YARN will happily wait forever for a resource, this setting forcibly cancels the request at timeout.
      Returns:
      the number of seconds to wait for timeout. 0 means no timeout

    • requestTimedOut

      void requestTimedOut()
      Informs the scheduler that a YARN resource request timed out. The scheduler can either retry or (more productively) assume that the requested node is not available and adjust its target size downward.