DefaultResourceCalculator (Apache Hadoop YARN Common 3.5.0 API)
Description: declaration: package: org.apache.hadoop.yarn.util.resource, class: DefaultResourceCalculator

Package org.apache.hadoop.yarn.util.resource

# Class DefaultResourceCalculator

java.lang.Object
org.apache.hadoop.yarn.util.resource.ResourceCalculator
org.apache.hadoop.yarn.util.resource.DefaultResourceCalculator

@Private @Unstable public class DefaultResourceCalculator extends ResourceCalculator

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `DefaultResourceCalculator()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `int`
  `compare(org.apache.hadoop.yarn.api.records.Resource unused, org.apache.hadoop.yarn.api.records.Resource lhs, org.apache.hadoop.yarn.api.records.Resource rhs, boolean singleType)`
  On a cluster with capacity `clusterResource`, compare `lhs` and `rhs`.
  `long`
  `computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available, org.apache.hadoop.yarn.api.records.Resource required)`
  Compute the number of containers which can be allocated given `available` and `required` resources.
  `float`
  `divide(org.apache.hadoop.yarn.api.records.Resource unused, org.apache.hadoop.yarn.api.records.Resource numerator, org.apache.hadoop.yarn.api.records.Resource denominator)`
  Divide resource `numerator` by resource `denominator` using specified policy (domination, average, fairness etc.); hence overall `clusterResource` is provided for context.
  `org.apache.hadoop.yarn.api.records.Resource`
  `divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, float denominator)`
  Divide-and-ceil `numerator` by `denominator`.
  `org.apache.hadoop.yarn.api.records.Resource`
  `divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, int denominator)`
  Divide-and-ceil `numerator` by `denominator`.
  `boolean`
  `fitsIn(org.apache.hadoop.yarn.api.records.Resource smaller, org.apache.hadoop.yarn.api.records.Resource bigger)`
  Check if a smaller resource can be contained by bigger resource.
  `Set<String>`
  `getInsufficientResourceNames(org.apache.hadoop.yarn.api.records.Resource required, org.apache.hadoop.yarn.api.records.Resource available)`
  Get insufficient resource names via comparing required resource and capacity resource.
  `boolean`
  `isAllInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)`
  Determine if all resources are zero.
  `boolean`
  `isAnyMajorResourceAboveZero(org.apache.hadoop.yarn.api.records.Resource resource)`
  Check if resource has any major resource types (which are all NodeManagers included) has a > 0 value.
  `boolean`
  `isAnyMajorResourceZeroOrNegative(org.apache.hadoop.yarn.api.records.Resource resource)`
  Check if resource has any major resource types (which are all NodeManagers included) a zero value or negative value.
  `boolean`
  `isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)`
  Determine if a resource is not suitable for use as a divisor (will result in divide by 0, etc)
  `org.apache.hadoop.yarn.api.records.Resource`
  `multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor)`
  Multiply resource `r` by factor `by` and normalize down using step-factor `stepFactor`.
  `org.apache.hadoop.yarn.api.records.Resource`
  `multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r, double[] by, org.apache.hadoop.yarn.api.records.Resource stepFactor)`
  Multiply resource `r` by factor `by` and normalize up using step-factor `stepFactor`.
  `org.apache.hadoop.yarn.api.records.Resource`
  `multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor)`
  Multiply resource `r` by factor `by` and normalize up using step-factor `stepFactor`.
  `org.apache.hadoop.yarn.api.records.Resource`
  `normalize(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource, org.apache.hadoop.yarn.api.records.Resource stepFactor)`
  Normalize resource `r` given the base `minimumResource` and verify against max allowed `maximumResource` using a step factor for the normalization.
  `org.apache.hadoop.yarn.api.records.Resource`
  `normalizeDown(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)`
  Get resource `r`and normalize down using step-factor `stepFactor`.
  `float`
  `ratio(org.apache.hadoop.yarn.api.records.Resource a, org.apache.hadoop.yarn.api.records.Resource b)`
  Ratio of resource `a` to resource `b`.
  `org.apache.hadoop.yarn.api.records.Resource`
  `roundDown(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)`
  Round-down resource `r` given factor `stepFactor`.
  `org.apache.hadoop.yarn.api.records.Resource`
  `roundUp(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)`
  Round-up resource `r` given factor `stepFactor`.

  ### Methods inherited from class org.apache.hadoop.yarn.util.resource.ResourceCalculator

  `compare, divideAndCeil, divideAndCeil, divideAndCeil, divideAndCeil, divideSafelyAsFloat, roundDown, roundDown, roundUp, roundUp`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### DefaultResourceCalculator

    public DefaultResourceCalculator()

- ## Method Details

  - ### compare

    public int compare(org.apache.hadoop.yarn.api.records.Resource unused, org.apache.hadoop.yarn.api.records.Resource lhs, org.apache.hadoop.yarn.api.records.Resource rhs, boolean singleType)
    Description copied from class: `ResourceCalculator`
    On a cluster with capacity `clusterResource`, compare `lhs` and `rhs`. Consider all resources unless `singleType` is set to true. When `singleType` is set to true, consider only one resource as per the `ResourceCalculator` implementation; the `DefaultResourceCalculator` considers memory and `DominantResourceCalculator` considers the dominant resource.
    Specified by:
      `compare` in class `ResourceCalculator`
    Parameters:
      `unused` - cluster capacity
      `lhs` - First `Resource` to compare
      `rhs` - Second `Resource` to compare
      `singleType` - Whether to consider a single resource type or all resource types
    Returns:
      -1 if `lhs` is smaller, 0 if equal and 1 if it is larger

  - ### computeAvailableContainers

    public long computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available, org.apache.hadoop.yarn.api.records.Resource required)
    Description copied from class: `ResourceCalculator`
    Compute the number of containers which can be allocated given `available` and `required` resources.
    Specified by:
      `computeAvailableContainers` in class `ResourceCalculator`
    Parameters:
      `available` - available resources
      `required` - required resources
    Returns:
      number of containers which can be allocated

  - ### divide

    public float divide(org.apache.hadoop.yarn.api.records.Resource unused, org.apache.hadoop.yarn.api.records.Resource numerator, org.apache.hadoop.yarn.api.records.Resource denominator)
    Description copied from class: `ResourceCalculator`
    Divide resource `numerator` by resource `denominator` using specified policy (domination, average, fairness etc.); hence overall `clusterResource` is provided for context.
    Specified by:
      `divide` in class `ResourceCalculator`
    Parameters:
      `unused` - cluster resources
      `numerator` - numerator
      `denominator` - denominator
    Returns:
      `numerator`/`denominator` using specific policy

  - ### isInvalidDivisor

    public boolean isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
    Description copied from class: `ResourceCalculator`
    Determine if a resource is not suitable for use as a divisor (will result in divide by 0, etc)
    Specified by:
      `isInvalidDivisor` in class `ResourceCalculator`
    Parameters:
      `r` - resource
    Returns:
      true if divisor is invalid (should not be used), false else

  - ### ratio

    public float ratio(org.apache.hadoop.yarn.api.records.Resource a, org.apache.hadoop.yarn.api.records.Resource b)
    Description copied from class: `ResourceCalculator`
    Ratio of resource `a` to resource `b`.
    Specified by:
      `ratio` in class `ResourceCalculator`
    Parameters:
      `a` - resource
      `b` - resource
    Returns:
      ratio of resource `a` to resource `b`

  - ### divideAndCeil

    public org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, int denominator)
    Description copied from class: `ResourceCalculator`
    Divide-and-ceil `numerator` by `denominator`.
    Specified by:
      `divideAndCeil` in class `ResourceCalculator`
    Parameters:
      `numerator` - numerator resource
      `denominator` - denominator
    Returns:
      resultant resource

  - ### divideAndCeil

    public org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, float denominator)
    Description copied from class: `ResourceCalculator`
    Divide-and-ceil `numerator` by `denominator`.
    Specified by:
      `divideAndCeil` in class `ResourceCalculator`
    Parameters:
      `numerator` - numerator resource
      `denominator` - denominator
    Returns:
      resultant resource

  - ### normalize

    public org.apache.hadoop.yarn.api.records.Resource normalize(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource, org.apache.hadoop.yarn.api.records.Resource stepFactor)
    Description copied from class: `ResourceCalculator`
    Normalize resource `r` given the base `minimumResource` and verify against max allowed `maximumResource` using a step factor for the normalization.
    Specified by:
      `normalize` in class `ResourceCalculator`
    Parameters:
      `r` - resource
      `minimumResource` - minimum value
      `maximumResource` - the upper bound of the resource to be allocated
      `stepFactor` - the increment for resources to be allocated
    Returns:
      normalized resource

  - ### roundUp

    public org.apache.hadoop.yarn.api.records.Resource roundUp(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)
    Description copied from class: `ResourceCalculator`
    Round-up resource `r` given factor `stepFactor`.
    Specified by:
      `roundUp` in class `ResourceCalculator`
    Parameters:
      `r` - resource
      `stepFactor` - step-factor
    Returns:
      rounded resource

  - ### roundDown

    public org.apache.hadoop.yarn.api.records.Resource roundDown(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)
    Description copied from class: `ResourceCalculator`
    Round-down resource `r` given factor `stepFactor`.
    Specified by:
      `roundDown` in class `ResourceCalculator`
    Parameters:
      `r` - resource
      `stepFactor` - step-factor
    Returns:
      rounded resource

  - ### multiplyAndNormalizeUp

    public org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor)
    Description copied from class: `ResourceCalculator`
    Multiply resource `r` by factor `by` and normalize up using step-factor `stepFactor`.
    Specified by:
      `multiplyAndNormalizeUp` in class `ResourceCalculator`
    Parameters:
      `r` - resource to be multiplied
      `by` - multiplier
      `stepFactor` - factor by which to normalize up
    Returns:
      resulting normalized resource

  - ### multiplyAndNormalizeUp

    public org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r, double[] by, org.apache.hadoop.yarn.api.records.Resource stepFactor)
    Description copied from class: `ResourceCalculator`
    Multiply resource `r` by factor `by` and normalize up using step-factor `stepFactor`.
    Specified by:
      `multiplyAndNormalizeUp` in class `ResourceCalculator`
    Parameters:
      `r` - resource to be multiplied
      `by` - multiplier array for all resource types
      `stepFactor` - factor by which to normalize up
    Returns:
      resulting normalized resource

  - ### multiplyAndNormalizeDown

    public org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor)
    Description copied from class: `ResourceCalculator`
    Multiply resource `r` by factor `by` and normalize down using step-factor `stepFactor`.
    Specified by:
      `multiplyAndNormalizeDown` in class `ResourceCalculator`
    Parameters:
      `r` - resource to be multiplied
      `by` - multiplier
      `stepFactor` - factor by which to normalize down
    Returns:
      resulting normalized resource

  - ### fitsIn

    public boolean fitsIn(org.apache.hadoop.yarn.api.records.Resource smaller, org.apache.hadoop.yarn.api.records.Resource bigger)
    Description copied from class: `ResourceCalculator`
    Check if a smaller resource can be contained by bigger resource.
    Specified by:
      `fitsIn` in class `ResourceCalculator`
    Parameters:
      `smaller` - smaller resource.
      `bigger` - bigger resource.
    Returns:
      if true, smaller resource can be contained by bigger resource; false otherwise.

  - ### normalizeDown

    public org.apache.hadoop.yarn.api.records.Resource normalizeDown(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)
    Description copied from class: `ResourceCalculator`
    Get resource `r`and normalize down using step-factor `stepFactor`.
    Specified by:
      `normalizeDown` in class `ResourceCalculator`
    Parameters:
      `r` - resource to be multiplied
      `stepFactor` - factor by which to normalize down
    Returns:
      resulting normalized resource

  - ### isAnyMajorResourceZeroOrNegative

    public boolean isAnyMajorResourceZeroOrNegative(org.apache.hadoop.yarn.api.records.Resource resource)
    Description copied from class: `ResourceCalculator`
    Check if resource has any major resource types (which are all NodeManagers included) a zero value or negative value.
    Specified by:
      `isAnyMajorResourceZeroOrNegative` in class `ResourceCalculator`
    Parameters:
      `resource` - resource
    Returns:
      returns true if any resource is zero.

  - ### isAnyMajorResourceAboveZero

    public boolean isAnyMajorResourceAboveZero(org.apache.hadoop.yarn.api.records.Resource resource)
    Description copied from class: `ResourceCalculator`
    Check if resource has any major resource types (which are all NodeManagers included) has a > 0 value.
    Specified by:
      `isAnyMajorResourceAboveZero` in class `ResourceCalculator`
    Parameters:
      `resource` - resource
    Returns:
      returns true if any resource is > 0

  - ### getInsufficientResourceNames

    public Set<String> getInsufficientResourceNames(org.apache.hadoop.yarn.api.records.Resource required, org.apache.hadoop.yarn.api.records.Resource available)
    Description copied from class: `ResourceCalculator`
    Get insufficient resource names via comparing required resource and capacity resource.
    Specified by:
      `getInsufficientResourceNames` in class `ResourceCalculator`
    Parameters:
      `required` - - required resource
      `available` - - available resource
    Returns:
      insufficient resource names

  - ### isAllInvalidDivisor

    public boolean isAllInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
    Description copied from class: `ResourceCalculator`
    Determine if all resources are zero.
    Specified by:
      `isAllInvalidDivisor` in class `ResourceCalculator`
    Parameters:
      `r` - resource
    Returns:
      true if all divisors are invalid (should not be used), false else