CheckpointAMPreemptionPolicy (Apache Hadoop MapReduce App 3.5.0 API)
Description: declaration: package: org.apache.hadoop.mapreduce.v2.app.rm.preemption, class: CheckpointAMPreemptionPolicy

Package org.apache.hadoop.mapreduce.v2.app.rm.preemption

# Class CheckpointAMPreemptionPolicy

java.lang.Object
org.apache.hadoop.mapreduce.v2.app.rm.preemption.CheckpointAMPreemptionPolicy

All Implemented Interfaces:
  `AMPreemptionPolicy`

public class CheckpointAMPreemptionPolicy extends Object implements AMPreemptionPolicy
This policy works in combination with an implementation of task checkpointing. It computes the tasks to be preempted in response to the RM request for preemption. For strict requests, it maps containers to corresponding tasks; for fungible requests, it attempts to pick the best containers to preempt (reducers in reverse allocation order). The TaskAttemptListener will interrogate this policy when handling a task heartbeat to check whether the task should be preempted or not. When handling fungible requests, the policy discount the RM ask by the amount of currently in-flight preemptions (i.e., tasks that are checkpointing). This class it is also used to maintain the list of checkpoints for existing tasks. Centralizing this functionality here, allows us to have visibility on preemption and checkpoints in a single location, thus coordinating preemption and checkpoint management decisions in a single policy.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from interface org.apache.hadoop.mapreduce.v2.app.rm.preemption.AMPreemptionPolicy

  `AMPreemptionPolicy.Context`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `CheckpointAMPreemptionPolicy()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID`
  `getCheckpointID(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskId)`
  Method to retrieve the latest checkpoint for a given `TaskId`
  `void`
  `handleCompletedContainer(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)`
  Callback informing the policy of containers exiting cleanly.
  `void`
  `handleFailedContainer(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)`
  Callback informing the policy of containers exiting with a failure.
  `void`
  `init(AppContext context)`
  `boolean`
  `isPreempted(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId yarnAttemptID)`
  This method is invoked by components interested to learn whether a certain task is being preempted.
  `void`
  `preempt(AMPreemptionPolicy.Context ctxt, org.apache.hadoop.yarn.api.records.PreemptionMessage preemptionRequests)`
  Callback informing the policy of ResourceManager. requests for resources to return to the cluster.
  `void`
  `reportSuccessfulPreemption(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId taskAttemptID)`
  This method is used to report to the policy that a certain task has been successfully preempted (for bookeeping, counters, etc..)
  `void`
  `setCheckpointID(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskId, org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID cid)`
  Method to store the latest `CheckpointID` for a given `TaskId`.

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### CheckpointAMPreemptionPolicy

    public CheckpointAMPreemptionPolicy()

- ## Method Details

  - ### init

    public void init(AppContext context)
    Specified by:
      `init` in interface `AMPreemptionPolicy`

  - ### preempt

    public void preempt(AMPreemptionPolicy.Context ctxt, org.apache.hadoop.yarn.api.records.PreemptionMessage preemptionRequests)
    Description copied from interface: `AMPreemptionPolicy`
    Callback informing the policy of ResourceManager. requests for resources to return to the cluster. The policy may take arbitrary action to satisfy requests by checkpointing task state, returning containers, or ignoring requests. The RM may elect to enforce these requests by forcibly killing containers not returned after some duration.
    Specified by:
      `preempt` in interface `AMPreemptionPolicy`
    Parameters:
      `ctxt` - Handle to the current state of running containers
      `preemptionRequests` - Request from RM for resources to return.

  - ### handleFailedContainer

    public void handleFailedContainer(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
    Description copied from interface: `AMPreemptionPolicy`
    Callback informing the policy of containers exiting with a failure. This allows the policy to implemnt cleanup/compensating actions.
    Specified by:
      `handleFailedContainer` in interface `AMPreemptionPolicy`
    Parameters:
      `attemptID` - Task attempt that failed

  - ### handleCompletedContainer

    public void handleCompletedContainer(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
    Description copied from interface: `AMPreemptionPolicy`
    Callback informing the policy of containers exiting cleanly. This is reported to the policy for bookeeping purposes.
    Specified by:
      `handleCompletedContainer` in interface `AMPreemptionPolicy`
    Parameters:
      `attemptID` - Task attempt that completed

  - ### isPreempted

    public boolean isPreempted(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId yarnAttemptID)
    Description copied from interface: `AMPreemptionPolicy`
    This method is invoked by components interested to learn whether a certain task is being preempted.
    Specified by:
      `isPreempted` in interface `AMPreemptionPolicy`
    Parameters:
      `yarnAttemptID` - Task attempt to query
    Returns:
      true if this attempt is being preempted

  - ### reportSuccessfulPreemption

    public void reportSuccessfulPreemption(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId taskAttemptID)
    Description copied from interface: `AMPreemptionPolicy`
    This method is used to report to the policy that a certain task has been successfully preempted (for bookeeping, counters, etc..)
    Specified by:
      `reportSuccessfulPreemption` in interface `AMPreemptionPolicy`
    Parameters:
      `taskAttemptID` - Task attempt that preempted

  - ### getCheckpointID

    public org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID getCheckpointID(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskId)
    Description copied from interface: `AMPreemptionPolicy`
    Method to retrieve the latest checkpoint for a given `TaskId`
    Specified by:
      `getCheckpointID` in interface `AMPreemptionPolicy`
    Parameters:
      `taskId` - TaskID
    Returns:
      CheckpointID associated with this task or null

  - ### setCheckpointID

    public void setCheckpointID(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskId, org.apache.hadoop.mapreduce.checkpoint.TaskCheckpointID cid)
    Description copied from interface: `AMPreemptionPolicy`
    Method to store the latest `CheckpointID` for a given `TaskId`. Assigning a null is akin to remove all previous checkpoints for this task.
    Specified by:
      `setCheckpointID` in interface `AMPreemptionPolicy`
    Parameters:
      `taskId` - TaskID
      `cid` - Checkpoint to assign or `null` to remove it.