BlockPlacementPolicyWithUpgradeDomain (Apache Hadoop HDFS 3.5.0 API)
Description: declaration: package: org.apache.hadoop.hdfs.server.blockmanagement, class: BlockPlacementPolicyWithUpgradeDomain

Package org.apache.hadoop.hdfs.server.blockmanagement

# Class BlockPlacementPolicyWithUpgradeDomain

java.lang.Object
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault
org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyWithUpgradeDomain

@Private @Evolving public class BlockPlacementPolicyWithUpgradeDomain extends BlockPlacementPolicyDefault
The class is responsible for choosing the desired number of targets for placing block replicas that honors upgrade domain policy. Here is the replica placement strategy. If the writer is on a datanode, the 1st replica is placed on the local machine, otherwise a random datanode. The 2nd replica is placed on a datanode that is on a different rack. The 3rd replica is placed on a datanode which is on a different node of the rack as the second replica. All 3 replicas have unique upgrade domains.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault

  `BlockPlacementPolicyDefault.NodeNotChosenReason`

  ## Nested classes/interfaces inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

  `BlockPlacementPolicy.NotEnoughReplicasException`

- ## Field Summary

  ### Fields inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault

  `clusterMap, considerLoad, considerLoadFactor, heartbeatInterval, host2datanodeMap, tolerateHeartbeatMultiplier`

  ### Fields inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

  `LOG`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `BlockPlacementPolicyWithUpgradeDomain()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `String`
  `getUpgradeDomainWithDefaultValue(org.apache.hadoop.hdfs.protocol.DatanodeInfo datanodeInfo)`
  `void`
  `initialize(org.apache.hadoop.conf.Configuration conf, FSClusterStats stats, org.apache.hadoop.net.NetworkTopology clusterMap, org.apache.hadoop.hdfs.server.blockmanagement.Host2NodesMap host2datanodeMap)`
  Used to setup a BlockPlacementPolicy object.
  `protected boolean`
  `isGoodDatanode(DatanodeDescriptor node, int maxTargetPerRack, boolean considerLoad, List<DatanodeStorageInfo> results, boolean avoidStaleNodes)`
  `boolean`
  `isMovable(Collection<org.apache.hadoop.hdfs.protocol.DatanodeInfo> locs, org.apache.hadoop.hdfs.protocol.DatanodeInfo source, org.apache.hadoop.hdfs.protocol.DatanodeInfo target)`
  Check if the move is allowed.
  `protected Collection<DatanodeStorageInfo>`
  `pickupReplicaSet(Collection<DatanodeStorageInfo> moreThanOne, Collection<DatanodeStorageInfo> exactlyOne, Map<String,List<DatanodeStorageInfo>> rackMap)`
  Pick up replica node set for deleting replica as over-replicated.
  `BlockPlacementStatus`
  `verifyBlockPlacement(org.apache.hadoop.hdfs.protocol.DatanodeInfo[] locs, int numberOfReplicas)`
  Verify if the block's placement meets requirement of placement policy, i.e. replicas are placed on no less than minRacks racks in the system.

  ### Methods inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicyDefault

  `addToExcludedNodes, chooseDataNode, chooseDataNode, chooseFavouredNodes, chooseLocalOrFavoredStorage, chooseLocalRack, chooseLocalStorage, chooseLocalStorage, chooseRandom, chooseRandom, chooseRemoteRack, chooseReplicasToDelete, chooseReplicaToDelete, chooseTarget, chooseTarget, chooseTargetInOrder, getExcludeSlowNodesEnabled, getMaxNodesPerRack, getMinBlocksForWrite, logNodeIsNotChosen, setExcludeSlowNodesEnabled, setMinBlocksForWrite`

  ### Methods inherited from class org.apache.hadoop.hdfs.server.blockmanagement.BlockPlacementPolicy

  `adjustSetsWithChosenReplica, getDatanodeInfo, getRack, splitNodesWithRack`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### BlockPlacementPolicyWithUpgradeDomain

    public BlockPlacementPolicyWithUpgradeDomain()

- ## Method Details

  - ### initialize

    public void initialize(org.apache.hadoop.conf.Configuration conf, FSClusterStats stats, org.apache.hadoop.net.NetworkTopology clusterMap, org.apache.hadoop.hdfs.server.blockmanagement.Host2NodesMap host2datanodeMap)
    Description copied from class: `BlockPlacementPolicy`
    Used to setup a BlockPlacementPolicy object. This should be defined by all implementations of a BlockPlacementPolicy.
    Overrides:
      `initialize` in class `BlockPlacementPolicyDefault`
    Parameters:
      `conf` - the configuration object
      `stats` - retrieve cluster status from here
      `clusterMap` - cluster topology

  - ### isGoodDatanode

    protected boolean isGoodDatanode(DatanodeDescriptor node, int maxTargetPerRack, boolean considerLoad, List<DatanodeStorageInfo> results, boolean avoidStaleNodes)

  - ### getUpgradeDomainWithDefaultValue

    public String getUpgradeDomainWithDefaultValue(org.apache.hadoop.hdfs.protocol.DatanodeInfo datanodeInfo)

  - ### verifyBlockPlacement

    public BlockPlacementStatus verifyBlockPlacement(org.apache.hadoop.hdfs.protocol.DatanodeInfo[] locs, int numberOfReplicas)
    Description copied from class: `BlockPlacementPolicy`
    Verify if the block's placement meets requirement of placement policy, i.e. replicas are placed on no less than minRacks racks in the system.
    Overrides:
      `verifyBlockPlacement` in class `BlockPlacementPolicyDefault`
    Parameters:
      `locs` - block with locations
      `numberOfReplicas` - replica number of file to be verified
    Returns:
      the result of verification

  - ### pickupReplicaSet

    protected Collection<DatanodeStorageInfo> pickupReplicaSet(Collection<DatanodeStorageInfo> moreThanOne, Collection<DatanodeStorageInfo> exactlyOne, Map<String,List<DatanodeStorageInfo>> rackMap)
    Description copied from class: `BlockPlacementPolicyDefault`
    Pick up replica node set for deleting replica as over-replicated. First set contains replica nodes on rack with more than one replica while second set contains remaining replica nodes. If only 1 rack, pick all. If 2 racks, pick all that have more than 1 replicas on the same rack; if no such replicas, pick all. If 3 or more racks, pick all.
    Overrides:
      `pickupReplicaSet` in class `BlockPlacementPolicyDefault`

  - ### isMovable

    public boolean isMovable(Collection<org.apache.hadoop.hdfs.protocol.DatanodeInfo> locs, org.apache.hadoop.hdfs.protocol.DatanodeInfo source, org.apache.hadoop.hdfs.protocol.DatanodeInfo target)
    Description copied from class: `BlockPlacementPolicy`
    Check if the move is allowed. Used by balancer and other tools.
    Overrides:
      `isMovable` in class `BlockPlacementPolicyDefault`
    Parameters:
      `locs` - all replicas including source and target
      `source` - source replica of the move
      `target` - target replica of the move