Package org.apache.drill.yarn.appMaster

Class NodeInventory

java.lang.Object
org.apache.drill.yarn.appMaster.NodeInventory
public class NodeInventory extends Object
Creates an AM-side inventory of cluster nodes. Used to track node reservations (container allocations) to prevent requesting multiple containers on the same node. Tracks blacklisted nodes that have failed too often. Since YARN will discard our blacklist if we add to many nodes, tracks when a container is allocated on a blacklisted node and signals that the cluster is in a bad state.

  • Constructor Details

  • Method Details

    • isFailed

      public boolean isFailed()

    • reserve

      public void reserve(org.apache.hadoop.yarn.api.records.Container container)

    • reserve

      public void reserve(String hostName)

    • release

      public void release(org.apache.hadoop.yarn.api.records.Container container)

    • release

      public void release(String hostName)

    • blacklist

      public void blacklist(String hostName)

    • getFreeNodeCount

      public int getFreeNodeCount()
      Determine the number of free nodes in the YARN cluster. The free set is the set of all YARN nodes minus those that are allocated and those that are blacklisted. Note that a node might be both in use and blacklisted if DoY blacklists a node, but then the user starts a "stray" Drillbit on that same node.

      This number is an approximation: the set of nodes managed by YARN can change any time, and in-flight container requests will consume a node, but since the request is not yet completed, we don't know which node will be assigned, so the node does not yet appear in the in-use list.

      Returns:
      an approximation of the free node count

    • getBlacklist

      public List<String> getBlacklist()
      Return a copy of the blacklist (list of failed nodes) for use in display to the user or similar purpose.
      Returns:
      a copy of the blacklist.

    • isInUse

      public boolean isInUse(String hostName)
      Report if the given host name is in use.
      Parameters:
      hostName -
      Returns:
      true if the host is reserved (in use by a container) or blacklisted (failed.)