NMClientAsync (Apache Hadoop Main 3.5.0 API)
Description: declaration: package: org.apache.hadoop.yarn.client.api.async, class: NMClientAsync

Package org.apache.hadoop.yarn.client.api.async

# Class NMClientAsync

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.yarn.client.api.async.NMClientAsync

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `Service`

@Public @Stable public abstract class NMClientAsync extends AbstractService
`NMClientAsync` handles communication with all the NodeManagers and provides asynchronous updates on getting responses from them. It maintains a thread pool to communicate with individual NMs where a number of worker threads process requests to NMs by using `NMClientImpl`. The max size of the thread pool is configurable through `YarnConfiguration.NM_CLIENT_ASYNC_THREAD_POOL_MAX_SIZE`. It should be used in conjunction with a CallbackHandler. For example

     class MyCallbackHandler extends NMClientAsync.AbstractCallbackHandler {
       public void onContainerStarted(ContainerId containerId,
           Map<String, ByteBuffer> allServiceResponse) {
         [post process after the container is started, process the response]
       }

       public void onContainerResourceIncreased(ContainerId containerId,
           Resource resource) {
         [post process after the container resource is increased]
       }

       public void onContainerStatusReceived(ContainerId containerId,
           ContainerStatus containerStatus) {
         [make use of the status of the container]
       }

       public void onContainerStopped(ContainerId containerId) {
         [post process after the container is stopped]
       }

       public void onStartContainerError(
           ContainerId containerId, Throwable t) {
         [handle the raised exception]
       }

       public void onGetContainerStatusError(
           ContainerId containerId, Throwable t) {
         [handle the raised exception]
       }

       public void onStopContainerError(
           ContainerId containerId, Throwable t) {
         [handle the raised exception]
       }
     }

The client's life-cycle should be managed like the following:

     NMClientAsync asyncClient =
         NMClientAsync.createNMClientAsync(new MyCallbackhandler());
     asyncClient.init(conf);
     asyncClient.start();
     asyncClient.startContainer(container, containerLaunchContext);
     [... wait for container being started]
     asyncClient.getContainerStatus(container.getId(), container.getNodeId(),
         container.getContainerToken());
     [... handle the status in the callback instance]
     asyncClient.stopContainer(container.getId(), container.getNodeId(),
         container.getContainerToken());
     [... wait for container being stopped]
     asyncClient.stop();

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `static class`
  `org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler`
  The callback abstract class.
  `static interface`
  `org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler`
  Deprecated.
  Use `NMClientAsync.AbstractCallbackHandler` instead.

  ## Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

  `Service.STATE`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `protected org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler`
  `callbackHandler`
  `protected NMClient`
  `client`

- ## Constructor Summary

  Constructors
  Modifier
  Constructor
  Description
  `protected`
  `NMClientAsync(String name, org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)`
  `protected`
  `NMClientAsync(String name, org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)`
  Deprecated.
  Use `NMClientAsync(String, AbstractCallbackHandler)` instead.
  `protected`
  `NMClientAsync(String name, NMClient client, org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)`
  `protected`
  `NMClientAsync(String name, NMClient client, org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)`
  Deprecated.
  `protected`
  `NMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)`
  `protected`
  `NMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)`
  Deprecated.
  Use `NMClientAsync(AbstractCallbackHandler)` instead.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `abstract void`
  `commitLastReInitializationAsync(ContainerId containerId)`
  Commit last reInitialization of the specified container.
  `static NMClientAsync`
  `createNMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)`
  `static NMClientAsync`
  `createNMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)`
  Deprecated.
  Use `createNMClientAsync(AbstractCallbackHandler)` instead.
  `org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler`
  `getCallbackHandler()`
  `NMClient`
  `getClient()`
  `abstract void`
  `getContainerStatusAsync(ContainerId containerId, NodeId nodeId)`
  `abstract void`
  `increaseContainerResourceAsync(Container container)`
  Deprecated.
  `abstract void`
  `reInitializeContainerAsync(ContainerId containerId, ContainerLaunchContext containerLaunchContex, boolean autoCommit)`
  Re-Initialize the Container.
  `abstract void`
  `restartContainerAsync(ContainerId containerId)`
  Restart the specified container.
  `abstract void`
  `rollbackLastReInitializationAsync(ContainerId containerId)`
  Rollback last reInitialization of the specified container.
  `void`
  `setCallbackHandler(org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)`
  `void`
  `setClient(NMClient client)`
  `abstract void`
  `startContainerAsync(Container container, ContainerLaunchContext containerLaunchContext)`
  `abstract void`
  `stopContainerAsync(ContainerId containerId, NodeId nodeId)`
  `abstract void`
  `updateContainerResourceAsync(Container container)`
  Update the resources of a container.

  ### Methods inherited from class org.apache.hadoop.service.AbstractService

  `close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, serviceStart, serviceStop, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- ## Field Details

  - ### client

    protected NMClient client

  - ### callbackHandler

    protected org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler

- ## Constructor Details

  - ### NMClientAsync

    protected NMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)

  - ### NMClientAsync

    protected NMClientAsync(String name, org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)

  - ### NMClientAsync

    protected NMClientAsync(String name, NMClient client, org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)

  - ### NMClientAsync

    @Deprecated protected NMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)
    Deprecated.
    Use `NMClientAsync(AbstractCallbackHandler)` instead.

  - ### NMClientAsync

    @Deprecated protected NMClientAsync(String name, org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)
    Deprecated.
    Use `NMClientAsync(String, AbstractCallbackHandler)` instead.

  - ### NMClientAsync

    @Private @VisibleForTesting @Deprecated protected NMClientAsync(String name, NMClient client, org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)
    Deprecated.

- ## Method Details

  - ### createNMClientAsync

    public static NMClientAsync createNMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.AbstractCallbackHandler callbackHandler)

  - ### createNMClientAsync

    @Deprecated public static NMClientAsync createNMClientAsync(org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)
    Deprecated.
    Use `createNMClientAsync(AbstractCallbackHandler)` instead.

  - ### startContainerAsync

    public abstract void startContainerAsync(Container container, ContainerLaunchContext containerLaunchContext)

  - ### increaseContainerResourceAsync

    @Deprecated public abstract void increaseContainerResourceAsync(Container container)
    Deprecated.

  - ### updateContainerResourceAsync

    public abstract void updateContainerResourceAsync(Container container)

    Update the resources of a container.

    The `ApplicationMaster` or other applications that use the client must provide the details of the container, including the Id and the target resource encapsulated in the updated container token via `Container`.

    Parameters:
      `container` - the container with updated token.

  - ### reInitializeContainerAsync

    public abstract void reInitializeContainerAsync(ContainerId containerId, ContainerLaunchContext containerLaunchContex, boolean autoCommit)

    Re-Initialize the Container.

    Parameters:
      `containerId` - the Id of the container to Re-Initialize.
      `containerLaunchContex` - the updated ContainerLaunchContext.
      `autoCommit` - commit re-initialization automatically ?

  - ### restartContainerAsync

    public abstract void restartContainerAsync(ContainerId containerId)

    Restart the specified container.

    Parameters:
      `containerId` - the Id of the container to restart.

  - ### rollbackLastReInitializationAsync

    public abstract void rollbackLastReInitializationAsync(ContainerId containerId)

    Rollback last reInitialization of the specified container.

    Parameters:
      `containerId` - the Id of the container to restart.

  - ### commitLastReInitializationAsync

    public abstract void commitLastReInitializationAsync(ContainerId containerId)

    Commit last reInitialization of the specified container.

    Parameters:
      `containerId` - the Id of the container to commit reInitialize.

  - ### stopContainerAsync

    public abstract void stopContainerAsync(ContainerId containerId, NodeId nodeId)

  - ### getContainerStatusAsync

    public abstract void getContainerStatusAsync(ContainerId containerId, NodeId nodeId)

  - ### getClient

    public NMClient getClient()

  - ### setClient

    public void setClient(NMClient client)

  - ### getCallbackHandler

    public org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler getCallbackHandler()

  - ### setCallbackHandler

    public void setCallbackHandler(org.apache.hadoop.yarn.client.api.async.NMClientAsync.CallbackHandler callbackHandler)