ResourceUtils (Apache Hadoop YARN API 3.5.0 API)
Description: declaration: package: org.apache.hadoop.yarn.util.resource, class: ResourceUtils

Package org.apache.hadoop.yarn.util.resource

# Class ResourceUtils

java.lang.Object
org.apache.hadoop.yarn.util.resource.ResourceUtils

public class ResourceUtils extends Object
Helper class to read the resource-types to be supported by the system.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final String`
  `EXTERNAL_VOLUME_RESOURCE_TAG`
  `static final String`
  `MAXIMUM_ALLOCATION`
  `static final String`
  `MINIMUM_ALLOCATION`
  `static final Pattern`
  `RESOURCE_REQUEST_VALUE_PATTERN`
  `static final String`
  `TAGS`
  `static final String`
  `TYPE`
  `static final String`
  `UNITS`
  `static final String`
  `YARN_IO_OPTIONAL`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static void`
  `areMandatoryResourcesAvailable(Resource res)`
  Are mandatory resources like memory-mb, vcores available?
  `static Resource`
  `createResourceFromString(String resourceStr, List<ResourceTypeInfo> resourceTypeInfos)`
  `static ResourceInformation[]`
  `createResourceTypesArray(Map<String,Long> res)`
  Create an array of `ResourceInformation` objects corresponding to the passed in map of names to values.
  `static Resource`
  `createResourceWithSameValue(long value)`
  Return a new `Resource` instance with all resource values initialized to `value`.
  `static Resource`
  `fetchMaximumAllocationFromConfig(org.apache.hadoop.conf.Configuration conf)`
  Get maximum allocation from config, *THIS WILL NOT UPDATE INTERNAL DATA.
  `static StringBuilder`
  `getCustomResourcesStrings(Resource resource)`
  `static String`
  `getDefaultUnit(String resourceType)`
  Get default unit by given resource type.
  `static Map<String,ResourceInformation>`
  `getNodeResourceInformation(org.apache.hadoop.conf.Configuration conf)`
  Function to get the resources for a node.
  `static int`
  `getNumberOfCountableResourceTypes()`
  `static int`
  `getNumberOfKnownResourceTypes()`
  `static List<ResourceInformation>`
  `getRequestedResourcesFromConfig(org.apache.hadoop.conf.Configuration configuration, String prefix)`
  From a given configuration get all entries representing requested resources: entries that match the {prefix}{resourceName}={value}[{units}] pattern.
  `static List<ResourceTypeInfo>`
  `getResourcesTypeInfo()`
  Get all resource types information from known resource types.
  `static Map<String,Integer>`
  `getResourceTypeIndex()`
  Get associate index of resource types such memory, cpu etc.
  `static Map<String,ResourceInformation>`
  `getResourceTypes()`
  Get the resource types to be supported by the system.
  `static ResourceInformation[]`
  `getResourceTypesArray()`
  `static Resource`
  `getResourceTypesMaximumAllocation()`
  Get a Resource object with for the maximum allocation possible.
  `static Resource`
  `getResourceTypesMinimumAllocation()`
  `static String`
  `getUnits(String resourceValue)`
  `static long`
  `getValue(String resourceValue)`
  `static void`
  `initializeResourcesFromResourceInformationMap(Map<String,ResourceInformation> resourceInformationMap)`
  This method is visible for testing, unit test can construct a resourceInformationMap and pass it to this method to initialize multiple resources.
  `static Resource`
  `multiplyFloor(Resource resource, double multiplier)`
  `static Resource`
  `multiplyRound(Resource resource, double multiplier)`
  `static String[]`
  `parseResourceValue(String resourceValue)`
  Extract unit and actual value from resource value.
  `static void`
  `reinitializeResources(List<ResourceTypeInfo> resourceTypeInfo)`
  Reinitialize all resource types from external source (in case of client, server will send the updated list and local resourceutils cache will be updated as per server's list of resources).
  `static void`
  `resetNodeResources()`
  `static void`
  `resetResourceTypes()`
  `static Map<String,ResourceInformation>`
  `resetResourceTypes(org.apache.hadoop.conf.Configuration conf)`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### UNITS

    public static final String UNITS
    See Also:
      - Constant Field Values

  - ### TYPE

    public static final String TYPE
    See Also:
      - Constant Field Values

  - ### TAGS

    public static final String TAGS
    See Also:
      - Constant Field Values

  - ### MINIMUM_ALLOCATION

    public static final String MINIMUM_ALLOCATION
    See Also:
      - Constant Field Values

  - ### MAXIMUM_ALLOCATION

    public static final String MAXIMUM_ALLOCATION
    See Also:
      - Constant Field Values

  - ### EXTERNAL_VOLUME_RESOURCE_TAG

    public static final String EXTERNAL_VOLUME_RESOURCE_TAG
    See Also:
      - Constant Field Values

  - ### RESOURCE_REQUEST_VALUE_PATTERN

    public static final Pattern RESOURCE_REQUEST_VALUE_PATTERN

  - ### YARN_IO_OPTIONAL

    public static final String YARN_IO_OPTIONAL
    See Also:
      - Constant Field Values

- ## Method Details

  - ### fetchMaximumAllocationFromConfig

    public static Resource fetchMaximumAllocationFromConfig(org.apache.hadoop.conf.Configuration conf)
    Get maximum allocation from config, *THIS WILL NOT UPDATE INTERNAL DATA.
    Parameters:
      `conf` - config
    Returns:
      maximum allocation

  - ### initializeResourcesFromResourceInformationMap

    @VisibleForTesting public static void initializeResourcesFromResourceInformationMap(Map<String,ResourceInformation> resourceInformationMap)
    This method is visible for testing, unit test can construct a resourceInformationMap and pass it to this method to initialize multiple resources.
    Parameters:
      `resourceInformationMap` - constructed resource information map.

  - ### getResourceTypeIndex

    public static Map<String,Integer> getResourceTypeIndex()
    Get associate index of resource types such memory, cpu etc. This could help to access each resource types in a resource faster.
    Returns:
      Index map for all Resource Types.

  - ### getResourceTypes

    public static Map<String,ResourceInformation> getResourceTypes()
    Get the resource types to be supported by the system.
    Returns:
      A map of the resource name to a ResourceInformation object which contains details such as the unit.

  - ### getResourceTypesArray

    public static ResourceInformation[] getResourceTypesArray()

  - ### getNumberOfKnownResourceTypes

    public static int getNumberOfKnownResourceTypes()

  - ### getNumberOfCountableResourceTypes

    public static int getNumberOfCountableResourceTypes()

  - ### resetResourceTypes

    @VisibleForTesting public static void resetResourceTypes()

  - ### resetResourceTypes

    @VisibleForTesting public static Map<String,ResourceInformation> resetResourceTypes(org.apache.hadoop.conf.Configuration conf)

  - ### getUnits

    public static String getUnits(String resourceValue)

  - ### parseResourceValue

    public static String[] parseResourceValue(String resourceValue)
    Extract unit and actual value from resource value.
    Parameters:
      `resourceValue` - Value of the resource
    Returns:
      Array containing unit and value. [0]=unit, [1]=value
    Throws:
      `IllegalArgumentException` - if units contain non alpha characters

  - ### getValue

    public static long getValue(String resourceValue)

  - ### getNodeResourceInformation

    public static Map<String,ResourceInformation> getNodeResourceInformation(org.apache.hadoop.conf.Configuration conf)
    Function to get the resources for a node. This function will look at the file `YarnConfiguration.NODE_RESOURCES_CONFIGURATION_FILE` to determine the node resources.
    Parameters:
      `conf` - configuration file
    Returns:
      a map to resource name to the ResourceInformation object. The map is guaranteed to have entries for memory and vcores

  - ### resetNodeResources

    @VisibleForTesting public static void resetNodeResources()

  - ### getResourceTypesMinimumAllocation

    public static Resource getResourceTypesMinimumAllocation()

  - ### getResourceTypesMaximumAllocation

    public static Resource getResourceTypesMaximumAllocation()
    Get a Resource object with for the maximum allocation possible.
    Returns:
      a Resource object with the maximum allocation for the scheduler

  - ### getDefaultUnit

    public static String getDefaultUnit(String resourceType)
    Get default unit by given resource type.
    Parameters:
      `resourceType` - resourceType
    Returns:
      default unit

  - ### getResourcesTypeInfo

    public static List<ResourceTypeInfo> getResourcesTypeInfo()
    Get all resource types information from known resource types.
    Returns:
      List of ResourceTypeInfo

  - ### reinitializeResources

    public static void reinitializeResources(List<ResourceTypeInfo> resourceTypeInfo)
    Reinitialize all resource types from external source (in case of client, server will send the updated list and local resourceutils cache will be updated as per server's list of resources).
    Parameters:
      `resourceTypeInfo` - List of resource types

  - ### getRequestedResourcesFromConfig

    public static List<ResourceInformation> getRequestedResourcesFromConfig(org.apache.hadoop.conf.Configuration configuration, String prefix)
    From a given configuration get all entries representing requested resources: entries that match the {prefix}{resourceName}={value}[{units}] pattern.
    Parameters:
      `configuration` - The configuration
      `prefix` - Keys with this prefix are considered from the configuration
    Returns:
      The list of requested resources as described by the configuration

  - ### areMandatoryResourcesAvailable

    public static void areMandatoryResourcesAvailable(Resource res)
    Are mandatory resources like memory-mb, vcores available? If not, throw exceptions. On availability, ensure those values are within boundary.
    Parameters:
      `res` - resource
    Throws:
      `IllegalArgumentException` - if mandatory resource is not available or value is not within boundary

  - ### createResourceTypesArray

    public static ResourceInformation[] createResourceTypesArray(Map<String,Long> res)
    Create an array of `ResourceInformation` objects corresponding to the passed in map of names to values. The array will be ordered according to the order returned by `getResourceTypesArray()`. The value of each resource type in the returned array will either be the value given for that resource in the `res` parameter or, if none is given, 0.
    Parameters:
      `res` - the map of resource type values
    Returns:
      an array of `ResourceInformation` instances

  - ### createResourceWithSameValue

    @Private @Unstable public static Resource createResourceWithSameValue(long value)
    Return a new `Resource` instance with all resource values initialized to `value`.
    Parameters:
      `value` - the value to use for all resources
    Returns:
      a new `Resource` instance

  - ### multiplyFloor

    public static Resource multiplyFloor(Resource resource, double multiplier)

  - ### multiplyRound

    public static Resource multiplyRound(Resource resource, double multiplier)

  - ### createResourceFromString

    @Private @Unstable public static Resource createResourceFromString(String resourceStr, List<ResourceTypeInfo> resourceTypeInfos)

  - ### getCustomResourcesStrings

    public static StringBuilder getCustomResourcesStrings(Resource resource)