DataNode (Apache Hadoop HDFS 3.5.0 API)
Description: declaration: package: org.apache.hadoop.hdfs.server.datanode, class: DataNode

Package org.apache.hadoop.hdfs.server.datanode

# Class DataNode

java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.conf.ReconfigurableBase
org.apache.hadoop.hdfs.server.datanode.DataNode

All Implemented Interfaces:
  `org.apache.hadoop.conf.Configurable`, `org.apache.hadoop.conf.Reconfigurable`, `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`, `org.apache.hadoop.hdfs.protocol.ReconfigurationProtocol`, `DataNodeMXBean`, `InterDatanodeProtocol`

@Private public class DataNode extends org.apache.hadoop.conf.ReconfigurableBase implements InterDatanodeProtocol, org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol, DataNodeMXBean, org.apache.hadoop.hdfs.protocol.ReconfigurationProtocol
DataNode is a class (and program) that stores a set of blocks for a DFS deployment. A single deployment can have one or many DataNodes. Each DataNode communicates regularly with a single NameNode. It also communicates with client code and other DataNodes from time to time. DataNodes store a series of named blocks. The DataNode allows client code to read these blocks, or to write new block data. The DataNode may also, in response to instructions from its NameNode, delete blocks or copy blocks to/from other DataNodes. The DataNode maintains just one critical table: block-> stream of bytes (of BLOCK_SIZE or less) This info is stored on a local disk. The DataNode reports the table's contents to the NameNode upon startup and every so often afterwards. DataNodes spend their lives in an endless loop of asking the NameNode for something to do. A NameNode cannot connect to a DataNode directly; a NameNode simply returns values from functions invoked by a DataNode. DataNodes maintain an open server socket so that client code or other DataNodes can read/write data. The host/port for this server is reported to the NameNode, which then sends that information to clients or other DataNodes that might be interested.

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `static class`
  `DataNode.ShortCircuitFdsUnsupportedException`
  `static class`
  `DataNode.ShortCircuitFdsVersionException`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final String`
  `DN_CLIENTTRACE_FORMAT`
  `org.apache.hadoop.ipc.RPC.Server`
  `ipcServer`
  `static final org.slf4j.Logger`
  `LOG`
  `static final int`
  `MAX_VOLUME_FAILURE_TOLERATED_LIMIT`
  `static final String`
  `MAX_VOLUME_FAILURES_TOLERATED_MSG`
  `static final String`
  `METRICS_LOG_NAME`

  ### Fields inherited from interface org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol

  `versionID`

  ### Fields inherited from interface org.apache.hadoop.hdfs.server.protocol.InterDatanodeProtocol

  `versionID`

  ### Fields inherited from interface org.apache.hadoop.hdfs.protocol.ReconfigurationProtocol

  `VERSIONID`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `cancelDiskBalancePlan(String planID)`
  Cancels a running plan.
  `void`
  `checkDiskError()`
  Check the disk error synchronously.
  `void`
  `checkDiskErrorAsync(FsVolumeSpi volume)`
  Check if there is a disk failure asynchronously and if so, handle the error.
  `void`
  `clearAllBlockSecretKeys()`
  `static DataNode`
  `createDataNode(String[] args, org.apache.hadoop.conf.Configuration conf)`
  Instantiate & Start a single datanode daemon and wait for it to finish.
  `static DataNode`
  `createDataNode(String[] args, org.apache.hadoop.conf.Configuration conf, SecureDataNodeStarter.SecureResources resources)`
  Instantiate & Start a single datanode daemon and wait for it to finish.
  `static InterDatanodeProtocol`
  `createInterDataNodeProtocolProxy(org.apache.hadoop.hdfs.protocol.DatanodeID datanodeid, org.apache.hadoop.conf.Configuration conf, int socketTimeout, boolean connectToDnViaHostname)`
  `static InetSocketAddress`
  `createSocketAddr(String target)`
  Deprecated.
  `void`
  `decrementXmitsInProgress()`
  Decrements the xmitsInProgress count
  `void`
  `decrementXmitsInProgress(int delta)`
  Decrements the xmitsInProgress count by given value.
  `void`
  `deleteBlockPool(String blockPoolId, boolean force)`
  `void`
  `evictWriters()`
  `static String`
  `generateUuid()`
  `int`
  `getActiveTransferThreadCount()`
  Returns the number of Datanode threads actively transferring blocks.
  `long`
  `getBalancerBandwidth()`
  `org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier>`
  `getBlockAccessToken(org.apache.hadoop.hdfs.protocol.ExtendedBlock b, EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> mode, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds)`
  Use BlockTokenSecretManager to generate block token for current user.
  `org.apache.hadoop.hdfs.protocol.BlockLocalPathInfo`
  `getBlockLocalPathInfo(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token)`
  `org.apache.hadoop.hdfs.server.datanode.BlockPoolManager`
  `getBlockPoolManager()`
  `BlockPoolTokenSecretManager`
  `getBlockPoolTokenSecretManager()`
  `BlockRecoveryWorker`
  `getBlockRecoveryWorker()`
  `BlockScanner`
  `getBlockScanner()`
  `int`
  `getBpOsCount()`
  `String`
  `getBPServiceActorInfo()`
  Returned information is a JSON representation of an array, each element of the array is a map contains the information about a block pool service actor.
  `List<Map<String,String>>`
  `getBPServiceActorInfoMap()`
  `String`
  `getClusterId()`
  Gets the cluster id.
  `org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataEncryptionKeyFactory`
  `getDataEncryptionKeyFactoryForBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock block)`
  Returns a new DataEncryptionKeyFactory that generates a key from the BlockPoolTokenSecretManager, using the block pool ID of the given block.
  `String`
  `getDatanodeHostname()`
  Return hostname of the datanode.
  `org.apache.hadoop.hdfs.protocol.DatanodeID`
  `getDatanodeId()`
  `org.apache.hadoop.hdfs.protocol.DatanodeLocalInfo`
  `getDatanodeInfo()`
  `Map<String,Map<String,Long>>`
  `getDatanodeNetworkCounts()`
  Gets the network error counts on a per-Datanode basis.
  `String`
  `getDatanodeUuid()`
  `String`
  `getDataPort()`
  Gets the data port.
  `DataSetLockManager`
  `getDataSetLockManager()`
  `DiskBalancer`
  `getDiskBalancer()`
  `String`
  `getDiskBalancerSetting(String key)`
  Gets a runtime configuration value from diskbalancer instance.
  `String`
  `getDiskBalancerStatus()`
  Gets the diskBalancer Status.
  `DataNodeDiskMetrics`
  `getDiskMetrics()`
  `String`
  `getDisplayName()`
  `DNConf`
  `getDnConf()`
  `DatanodeRegistration`
  `getDNRegistrationForBP(String bpid)`
  get BP registration by blockPool id
  `long`
  `getDNStartedTimeInMillis()`
  Get the start time of the DataNode.
  `org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.ECN`
  `getECN()`
  The ECN bit for the DataNode.
  `DataTransferThrottler`
  `getEcReconstuctReadThrottler()`
  `DataTransferThrottler`
  `getEcReconstuctWriteThrottler()`
  `ErasureCodingWorker`
  `getErasureCodingWorker()`
  `FileIoProvider`
  `getFileIoProvider()`
  `FsDatasetSpi<?>`
  `getFSDataset()`
  Examples are adding and deleting blocks directly.
  `String`
  `getHttpPort()`
  Gets the http port.
  `static InetSocketAddress`
  `getInfoAddr(org.apache.hadoop.conf.Configuration conf)`
  Determine the http server's effective addr
  `int`
  `getInfoPort()`
  `int`
  `getInfoSecurePort()`
  `int`
  `getIpcPort()`
  `long`
  `getLastDiskErrorCheck()`
  `long`
  `getMaxNumberOfBlocksToLog()`
  `DataNodeMetrics`
  `getMetrics()`
  `String`
  `getNamenodeAddresses()`
  Returned information is a JSON representation of a map with name node host name as the key and block pool Id as the value.
  `protected org.apache.hadoop.conf.Configuration`
  `getNewConf()`
  `long`
  `getOOBTimeout(org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.Status status)`
  Get the timeout to be used for transmitting the OOB type
  `DataNodePeerMetrics`
  `getPeerMetrics()`
  `Collection<String>`
  `getReconfigurableProperties()`
  Get a list of the keys of the re-configurable properties in configuration.
  `org.apache.hadoop.conf.ReconfigurationTaskStatus`
  `getReconfigurationStatus()`
  `long`
  `getReplicaVisibleLength(org.apache.hadoop.hdfs.protocol.ExtendedBlock block)`
  `String`
  `getRevision()`
  `String`
  `getRpcPort()`
  Gets the rpc port.
  `org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient`
  `getSaslClient()`
  `SaslDataTransferServer`
  `getSaslServer()`
  `String`
  `getSendPacketDownstreamAvgInfo()`
  Gets the average info (e.g. time) of SendPacketDownstream when the DataNode acts as the penultimate (2nd to the last) node in pipeline.
  `ShortCircuitRegistry`
  `getShortCircuitRegistry()`
  `org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.SLOW`
  `getSLOWByBlockPoolId(String bpId)`
  The SLOW bit for the DataNode of the specific BlockPool.
  `String`
  `getSlowDisks()`
  Gets the slow disks in the Datanode.
  `String`
  `getSoftwareVersion()`
  Get the version of software running on the DataNode
  `static List<StorageLocation>`
  `getStorageLocations(org.apache.hadoop.conf.Configuration conf)`
  `org.apache.hadoop.tracing.Tracer`
  `getTracer()`
  `String`
  `getVersion()`
  Gets the version of Hadoop.
  `String`
  `getVolumeInfo()`
  Returned information is a JSON representation of a map with volume name as the key and value is a map of volume attribute keys to its values
  `List<org.apache.hadoop.hdfs.protocol.DatanodeVolumeInfo>`
  `getVolumeReport()`
  `int`
  `getXceiverCount()`
  Number of concurrent xceivers per node.
  `InetSocketAddress`
  `getXferAddress()`
  NB: The datanode can perform data transfer on the streaming address however clients are given the IPC IP address for data transfer, and that may be a different address.
  `int`
  `getXferPort()`
  `org.apache.hadoop.hdfs.server.datanode.DataXceiverServer`
  `getXferServer()`
  `int`
  `getXmitsInProgress()`
  Returns an estimate of the number of data replication/reconstruction tasks running currently.
  `void`
  `handleVolumeFailures(Set<FsVolumeSpi> unhealthyVolumes)`
  `void`
  `incrementXmitsInProcess(int delta)`
  Increments the xmitInProgress count by given value.
  `void`
  `incrementXmitsInProgress()`
  Increments the xmitsInProgress count. xmitsInProgress count represents the number of data replication/reconstruction tasks running currently.
  `ReplicaRecoveryInfo`
  `initReplicaRecovery(BlockRecoveryCommand.RecoveringBlock rBlock)`
  Initialize a replica recovery.
  `static DataNode`
  `instantiateDataNode(String[] args, org.apache.hadoop.conf.Configuration conf)`
  Instantiate a single datanode object.
  `static DataNode`
  `instantiateDataNode(String[] args, org.apache.hadoop.conf.Configuration conf, SecureDataNodeStarter.SecureResources resources)`
  Instantiate a single datanode object, along with its secure resources.
  `boolean`
  `isBPServiceAlive(String bpid)`
  `boolean`
  `isConnectedToNN(InetSocketAddress addr)`
  `boolean`
  `isDatanodeFullyStarted()`
  A datanode is considered to be fully started if all the BP threads are alive and all the block pools are initialized.
  `boolean`
  `isDatanodeFullyStarted(boolean checkConnectionToActiveNamenode)`
  A datanode is considered to be fully started if all the BP threads are alive and all the block pools are initialized.
  `boolean`
  `isDatanodeUp()`
  A data node is considered to be up if one of the bp services is up
  `boolean`
  `isSecurityEnabled()`
  Gets if security is enabled.
  `List<String>`
  `listReconfigurableProperties()`
  `static void`
  `main(String[] args)`
  `Socket`
  `newSocket()`
  Creates either NIO or regular depending on socketWriteTimeout.
  `void`
  `notifyNamenodeDeletedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, String storageUuid)`
  Notify the corresponding namenode to delete the block.
  `void`
  `notifyNamenodeReceivedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, String delHint, String storageUuid, boolean isOnTransientStorage)`
  `protected void`
  `notifyNamenodeReceivingBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, String storageUuid)`
  `org.apache.hadoop.hdfs.server.datanode.DiskBalancerWorkStatus`
  `queryDiskBalancerPlan()`
  Returns the status of current or last executed work plan.
  `String`
  `reconfigurePropertyImpl(String property, String newVal)`
  .
  `void`
  `refreshNamenodes()`
  `void`
  `refreshNamenodes(org.apache.hadoop.conf.Configuration conf)`
  `void`
  `reportBadBlocks(org.apache.hadoop.hdfs.protocol.ExtendedBlock block)`
  Report a bad block which is hosted on the local DN.
  `void`
  `reportBadBlocks(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, FsVolumeSpi volume)`
  Report a bad block which is hosted on the local DN.
  `void`
  `reportCorruptedBlocks(org.apache.hadoop.hdfs.DFSUtilClient.CorruptedBlocks corruptedBlocks)`
  `void`
  `reportRemoteBadBlock(org.apache.hadoop.hdfs.protocol.DatanodeInfo srcDataNode, org.apache.hadoop.hdfs.protocol.ExtendedBlock block)`
  Report a bad block on another DN (eg if we received a corrupt replica from a remote host).
  `void`
  `runDatanodeDaemon()`
  Start a single datanode daemon and wait for it to finish.
  `void`
  `scheduleAllBlockReport(long delay)`
  This methods arranges for the data node to send the block report at the next heartbeat.
  `static void`
  `secureMain(String[] args, SecureDataNodeStarter.SecureResources resources)`
  `void`
  `setHeartbeatsDisabledForTests(boolean heartbeatsDisabledForTests)`
  `void`
  `shutdown()`
  Shut down this instance of the datanode.
  `void`
  `shutdownDatanode(boolean forUpgrade)`
  `protected void`
  `startMetricsLogger()`
  Start a timer to periodically write DataNode metrics to the log file.
  `void`
  `startReconfiguration()`
  `protected void`
  `stopMetricsLogger()`
  `void`
  `submitDiskBalancerPlan(String planID, long planVersion, String planFile, String planData, boolean skipDateCheck)`
  Allows submission of a disk balancer Job.
  `String`
  `toString()`
  `void`
  `triggerBlockReport(org.apache.hadoop.hdfs.client.BlockReportOptions options)`
  `String`
  `updateReplicaUnderRecovery(org.apache.hadoop.hdfs.protocol.ExtendedBlock oldBlock, long recoveryId, long newBlockId, long newLength)`
  Update replica with the new generation stamp and length.

  ### Methods inherited from class org.apache.hadoop.conf.ReconfigurableBase

  `getChangedProperties, getReconfigurationTaskStatus, isPropertyReconfigurable, reconfigureProperty, setReconfigurationUtil, shutdownReconfigurationTask, startReconfigurationTask`

  ### Methods inherited from class org.apache.hadoop.conf.Configured

  `getConf, setConf`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

  ### Methods inherited from interface org.apache.hadoop.conf.Configurable

  `getConf, setConf`

- ## Field Details

  - ### LOG

    public static final org.slf4j.Logger LOG

  - ### DN_CLIENTTRACE_FORMAT

    public static final String DN_CLIENTTRACE_FORMAT
    See Also:
      - Constant Field Values

  - ### MAX_VOLUME_FAILURE_TOLERATED_LIMIT

    public static final int MAX_VOLUME_FAILURE_TOLERATED_LIMIT
    See Also:
      - Constant Field Values

  - ### MAX_VOLUME_FAILURES_TOLERATED_MSG

    public static final String MAX_VOLUME_FAILURES_TOLERATED_MSG
    See Also:
      - Constant Field Values

  - ### METRICS_LOG_NAME

    public static final String METRICS_LOG_NAME
    See Also:
      - Constant Field Values

  - ### ipcServer

    public org.apache.hadoop.ipc.RPC.Server ipcServer

- ## Method Details

  - ### createSocketAddr

    @Deprecated public static InetSocketAddress createSocketAddr(String target)
    Deprecated.
    Use `NetUtils.createSocketAddr(String)` instead.

  - ### getNewConf

    protected org.apache.hadoop.conf.Configuration getNewConf()
    Specified by:
      `getNewConf` in class `org.apache.hadoop.conf.ReconfigurableBase`

  - ### reconfigurePropertyImpl

    public String reconfigurePropertyImpl(String property, String newVal) throws org.apache.hadoop.conf.ReconfigurationException
    .
    Specified by:
      `reconfigurePropertyImpl` in class `org.apache.hadoop.conf.ReconfigurableBase`
    Throws:
      `org.apache.hadoop.conf.ReconfigurationException`

  - ### getReconfigurableProperties

    public Collection<String> getReconfigurableProperties()
    Get a list of the keys of the re-configurable properties in configuration.
    Specified by:
      `getReconfigurableProperties` in interface `org.apache.hadoop.conf.Reconfigurable`
    Specified by:
      `getReconfigurableProperties` in class `org.apache.hadoop.conf.ReconfigurableBase`

  - ### getECN

    public org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.ECN getECN()
    The ECN bit for the DataNode. The DataNode should return:
    - ECN.DISABLED when ECN is disabled.
    - ECN.SUPPORTED when ECN is enabled but the DN still has capacity.
    - ECN.CONGESTED when ECN is enabled and the DN is congested.

  - ### getSLOWByBlockPoolId

    public org.apache.hadoop.hdfs.protocol.datatransfer.PipelineAck.SLOW getSLOWByBlockPoolId(String bpId)
    The SLOW bit for the DataNode of the specific BlockPool. The DataNode should return:
    - SLOW.DISABLED when SLOW is disabled
    - SLOW.NORMAL when SLOW is enabled and DN is not slownode.
    - SLOW.SLOW when SLOW is enabled and DN is slownode.

  - ### getFileIoProvider

    public FileIoProvider getFileIoProvider()

  - ### notifyNamenodeReceivedBlock

    public void notifyNamenodeReceivedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, String delHint, String storageUuid, boolean isOnTransientStorage)

  - ### notifyNamenodeReceivingBlock

    protected void notifyNamenodeReceivingBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, String storageUuid)

  - ### notifyNamenodeDeletedBlock

    public void notifyNamenodeDeletedBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, String storageUuid)
    Notify the corresponding namenode to delete the block.

  - ### reportBadBlocks

    public void reportBadBlocks(org.apache.hadoop.hdfs.protocol.ExtendedBlock block) throws IOException
    Report a bad block which is hosted on the local DN.
    Throws:
      `IOException`

  - ### reportBadBlocks

    public void reportBadBlocks(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, FsVolumeSpi volume) throws IOException
    Report a bad block which is hosted on the local DN.
    Parameters:
      `block` - the bad block which is hosted on the local DN
      `volume` - the volume that block is stored in and the volume must not be null
    Throws:
      `IOException`

  - ### reportRemoteBadBlock

    public void reportRemoteBadBlock(org.apache.hadoop.hdfs.protocol.DatanodeInfo srcDataNode, org.apache.hadoop.hdfs.protocol.ExtendedBlock block) throws IOException
    Report a bad block on another DN (eg if we received a corrupt replica from a remote host).
    Parameters:
      `srcDataNode` - the DN hosting the bad block
      `block` - the block itself
    Throws:
      `IOException`

  - ### reportCorruptedBlocks

    public void reportCorruptedBlocks(org.apache.hadoop.hdfs.DFSUtilClient.CorruptedBlocks corruptedBlocks) throws IOException
    Throws:
      `IOException`

  - ### setHeartbeatsDisabledForTests

    @VisibleForTesting public void setHeartbeatsDisabledForTests(boolean heartbeatsDisabledForTests)

  - ### generateUuid

    public static String generateUuid()

  - ### getSaslClient

    public org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient getSaslClient()

  - ### getBpOsCount

    public int getBpOsCount()

  - ### getInfoAddr

    public static InetSocketAddress getInfoAddr(org.apache.hadoop.conf.Configuration conf)
    Determine the http server's effective addr

  - ### getXferServer

    @VisibleForTesting public org.apache.hadoop.hdfs.server.datanode.DataXceiverServer getXferServer()

  - ### getXferPort

    @VisibleForTesting public int getXferPort()

  - ### getSaslServer

    @VisibleForTesting public SaslDataTransferServer getSaslServer()

  - ### getDisplayName

    public String getDisplayName()
    Returns:
      name useful for logging or display

  - ### getXferAddress

    public InetSocketAddress getXferAddress()
    NB: The datanode can perform data transfer on the streaming address however clients are given the IPC IP address for data transfer, and that may be a different address.
    Returns:
      socket address for data transfer

  - ### getIpcPort

    public int getIpcPort()
    Returns:
      the datanode's IPC port

  - ### getDNRegistrationForBP

    @VisibleForTesting public DatanodeRegistration getDNRegistrationForBP(String bpid) throws IOException
    get BP registration by blockPool id
    Returns:
      BP registration object
    Throws:
      `IOException` - on error

  - ### newSocket

    public Socket newSocket() throws IOException
    Creates either NIO or regular depending on socketWriteTimeout.
    Throws:
      `IOException`

  - ### createInterDataNodeProtocolProxy

    public static InterDatanodeProtocol createInterDataNodeProtocolProxy(org.apache.hadoop.hdfs.protocol.DatanodeID datanodeid, org.apache.hadoop.conf.Configuration conf, int socketTimeout, boolean connectToDnViaHostname) throws IOException
    Throws:
      `IOException`

  - ### getMetrics

    public DataNodeMetrics getMetrics()

  - ### getDiskMetrics

    public DataNodeDiskMetrics getDiskMetrics()

  - ### getPeerMetrics

    public DataNodePeerMetrics getPeerMetrics()

  - ### getMaxNumberOfBlocksToLog

    public long getMaxNumberOfBlocksToLog()

  - ### getBlockLocalPathInfo

    public org.apache.hadoop.hdfs.protocol.BlockLocalPathInfo getBlockLocalPathInfo(org.apache.hadoop.hdfs.protocol.ExtendedBlock block, org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> token) throws IOException
    Specified by:
      `getBlockLocalPathInfo` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Throws:
      `IOException`

  - ### shutdown

    public void shutdown()
    Shut down this instance of the datanode. Returns only after shutdown is complete. This method can only be called by the offerService thread. Otherwise, deadlock might occur.

  - ### checkDiskErrorAsync

    public void checkDiskErrorAsync(FsVolumeSpi volume)
    Check if there is a disk failure asynchronously and if so, handle the error.

  - ### getXceiverCount

    public int getXceiverCount()
    Number of concurrent xceivers per node.
    Specified by:
      `getXceiverCount` in interface `DataNodeMXBean`

  - ### getActiveTransferThreadCount

    public int getActiveTransferThreadCount()
    Description copied from interface: `DataNodeMXBean`
    Returns the number of Datanode threads actively transferring blocks.
    Specified by:
      `getActiveTransferThreadCount` in interface `DataNodeMXBean`

  - ### getDatanodeNetworkCounts

    public Map<String,Map<String,Long>> getDatanodeNetworkCounts()
    Description copied from interface: `DataNodeMXBean`
    Gets the network error counts on a per-Datanode basis.
    Specified by:
      `getDatanodeNetworkCounts` in interface `DataNodeMXBean`

  - ### getXmitsInProgress

    public int getXmitsInProgress()
    Description copied from interface: `DataNodeMXBean`
    Returns an estimate of the number of data replication/reconstruction tasks running currently.
    Specified by:
      `getXmitsInProgress` in interface `DataNodeMXBean`

  - ### incrementXmitsInProgress

    public void incrementXmitsInProgress()
    Increments the xmitsInProgress count. xmitsInProgress count represents the number of data replication/reconstruction tasks running currently.

  - ### incrementXmitsInProcess

    public void incrementXmitsInProcess(int delta)
    Increments the xmitInProgress count by given value.
    Parameters:
      `delta` - the amount of xmitsInProgress to increase.
    See Also:
      - `incrementXmitsInProgress()`

  - ### decrementXmitsInProgress

    public void decrementXmitsInProgress()
    Decrements the xmitsInProgress count

  - ### decrementXmitsInProgress

    public void decrementXmitsInProgress(int delta)
    Decrements the xmitsInProgress count by given value.
    See Also:
      - `decrementXmitsInProgress()`

  - ### getBlockAccessToken

    public org.apache.hadoop.security.token.Token<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier> getBlockAccessToken(org.apache.hadoop.hdfs.protocol.ExtendedBlock b, EnumSet<org.apache.hadoop.hdfs.security.token.block.BlockTokenIdentifier.AccessMode> mode, org.apache.hadoop.fs.StorageType[] storageTypes, String[] storageIds) throws IOException
    Use BlockTokenSecretManager to generate block token for current user.
    Throws:
      `IOException`

  - ### getDataEncryptionKeyFactoryForBlock

    public org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataEncryptionKeyFactory getDataEncryptionKeyFactoryForBlock(org.apache.hadoop.hdfs.protocol.ExtendedBlock block)
    Returns a new DataEncryptionKeyFactory that generates a key from the BlockPoolTokenSecretManager, using the block pool ID of the given block.
    Parameters:
      `block` - for which the factory needs to create a key
    Returns:
      DataEncryptionKeyFactory for block's block pool ID

  - ### runDatanodeDaemon

    public void runDatanodeDaemon() throws IOException
    Start a single datanode daemon and wait for it to finish. If this thread is specifically interrupted, it will stop waiting.
    Throws:
      `IOException`

  - ### isDatanodeUp

    public boolean isDatanodeUp()
    A data node is considered to be up if one of the bp services is up

  - ### instantiateDataNode

    public static DataNode instantiateDataNode(String[] args, org.apache.hadoop.conf.Configuration conf) throws IOException
    Instantiate a single datanode object. This must be run by invoking `runDatanodeDaemon()` subsequently.
    Throws:
      `IOException`

  - ### instantiateDataNode

    public static DataNode instantiateDataNode(String[] args, org.apache.hadoop.conf.Configuration conf, SecureDataNodeStarter.SecureResources resources) throws IOException
    Instantiate a single datanode object, along with its secure resources. This must be run by invoking`runDatanodeDaemon()` subsequently.
    Throws:
      `IOException`

  - ### getStorageLocations

    public static List<StorageLocation> getStorageLocations(org.apache.hadoop.conf.Configuration conf)

  - ### createDataNode

    @VisibleForTesting public static DataNode createDataNode(String[] args, org.apache.hadoop.conf.Configuration conf) throws IOException
    Instantiate & Start a single datanode daemon and wait for it to finish. If this thread is specifically interrupted, it will stop waiting.
    Throws:
      `IOException`

  - ### createDataNode

    @VisibleForTesting @Private public static DataNode createDataNode(String[] args, org.apache.hadoop.conf.Configuration conf, SecureDataNodeStarter.SecureResources resources) throws IOException
    Instantiate & Start a single datanode daemon and wait for it to finish. If this thread is specifically interrupted, it will stop waiting.
    Throws:
      `IOException`

  - ### toString

    public String toString()
    Overrides:
      `toString` in class `Object`

  - ### scheduleAllBlockReport

    public void scheduleAllBlockReport(long delay)
    This methods arranges for the data node to send the block report at the next heartbeat.

  - ### getFSDataset

    @VisibleForTesting public FsDatasetSpi<?> getFSDataset()
    Examples are adding and deleting blocks directly. The most common usage will be when the data node's storage is simulated.
    Returns:
      the fsdataset that stores the blocks

  - ### getBlockScanner

    @VisibleForTesting public BlockScanner getBlockScanner()

  - ### getBlockPoolTokenSecretManager

    @VisibleForTesting public BlockPoolTokenSecretManager getBlockPoolTokenSecretManager()

  - ### secureMain

    public static void secureMain(String[] args, SecureDataNodeStarter.SecureResources resources)

  - ### main

    public static void main(String[] args)

  - ### initReplicaRecovery

    public ReplicaRecoveryInfo initReplicaRecovery(BlockRecoveryCommand.RecoveringBlock rBlock) throws IOException
    Description copied from interface: `InterDatanodeProtocol`
    Initialize a replica recovery.
    Specified by:
      `initReplicaRecovery` in interface `InterDatanodeProtocol`
    Returns:
      actual state of the replica on this data-node or null if data-node does not have the replica.
    Throws:
      `IOException`

  - ### updateReplicaUnderRecovery

    public String updateReplicaUnderRecovery(org.apache.hadoop.hdfs.protocol.ExtendedBlock oldBlock, long recoveryId, long newBlockId, long newLength) throws IOException
    Update replica with the new generation stamp and length.
    Specified by:
      `updateReplicaUnderRecovery` in interface `InterDatanodeProtocol`
    Throws:
      `IOException`

  - ### getReplicaVisibleLength

    public long getReplicaVisibleLength(org.apache.hadoop.hdfs.protocol.ExtendedBlock block) throws IOException
    Specified by:
      `getReplicaVisibleLength` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Throws:
      `IOException`

  - ### getSoftwareVersion

    public String getSoftwareVersion()
    Description copied from interface: `DataNodeMXBean`
    Get the version of software running on the DataNode
    Specified by:
      `getSoftwareVersion` in interface `DataNodeMXBean`
    Returns:
      a string representing the version

  - ### getVersion

    public String getVersion()
    Description copied from interface: `DataNodeMXBean`
    Gets the version of Hadoop.
    Specified by:
      `getVersion` in interface `DataNodeMXBean`
    Returns:
      the version of Hadoop

  - ### getRpcPort

    public String getRpcPort()
    Description copied from interface: `DataNodeMXBean`
    Gets the rpc port.
    Specified by:
      `getRpcPort` in interface `DataNodeMXBean`
    Returns:
      the rpc port

  - ### getDataPort

    public String getDataPort()
    Description copied from interface: `DataNodeMXBean`
    Gets the data port.
    Specified by:
      `getDataPort` in interface `DataNodeMXBean`
    Returns:
      the data port

  - ### getHttpPort

    public String getHttpPort()
    Description copied from interface: `DataNodeMXBean`
    Gets the http port.
    Specified by:
      `getHttpPort` in interface `DataNodeMXBean`
    Returns:
      the http port

  - ### getDNStartedTimeInMillis

    public long getDNStartedTimeInMillis()
    Description copied from interface: `DataNodeMXBean`
    Get the start time of the DataNode.
    Specified by:
      `getDNStartedTimeInMillis` in interface `DataNodeMXBean`
    Returns:
      Start time of the DataNode.

  - ### getRevision

    public String getRevision()

  - ### getInfoPort

    public int getInfoPort()
    Returns:
      the datanode's http port

  - ### getInfoSecurePort

    public int getInfoSecurePort()
    Returns:
      the datanode's https port

  - ### getNamenodeAddresses

    public String getNamenodeAddresses()
    Returned information is a JSON representation of a map with name node host name as the key and block pool Id as the value. Note that, if there are multiple NNs in an NA nameservice, a given block pool may be represented twice.
    Specified by:
      `getNamenodeAddresses` in interface `DataNodeMXBean`
    Returns:
      the namenode IP addresses that the datanode is talking to

  - ### getDatanodeHostname

    public String getDatanodeHostname()
    Return hostname of the datanode.
    Specified by:
      `getDatanodeHostname` in interface `DataNodeMXBean`
    Returns:
      the datanode hostname for the datanode.

  - ### getBPServiceActorInfo

    public String getBPServiceActorInfo()
    Returned information is a JSON representation of an array, each element of the array is a map contains the information about a block pool service actor.
    Specified by:
      `getBPServiceActorInfo` in interface `DataNodeMXBean`
    Returns:
      block pool service actors info

  - ### getBPServiceActorInfoMap

    @VisibleForTesting public List<Map<String,String>> getBPServiceActorInfoMap()

  - ### getVolumeInfo

    public String getVolumeInfo()
    Returned information is a JSON representation of a map with volume name as the key and value is a map of volume attribute keys to its values
    Specified by:
      `getVolumeInfo` in interface `DataNodeMXBean`
    Returns:
      the volume info

  - ### getClusterId

    public String getClusterId()
    Description copied from interface: `DataNodeMXBean`
    Gets the cluster id.
    Specified by:
      `getClusterId` in interface `DataNodeMXBean`
    Returns:
      the cluster id

  - ### getDiskBalancerStatus

    public String getDiskBalancerStatus()
    Description copied from interface: `DataNodeMXBean`
    Gets the diskBalancer Status. Please see implementation for the format of the returned information.
    Specified by:
      `getDiskBalancerStatus` in interface `DataNodeMXBean`
    Returns:
      DiskBalancer Status

  - ### isSecurityEnabled

    public boolean isSecurityEnabled()
    Description copied from interface: `DataNodeMXBean`
    Gets if security is enabled.
    Specified by:
      `isSecurityEnabled` in interface `DataNodeMXBean`
    Returns:
      true, if security is enabled.

  - ### refreshNamenodes

    public void refreshNamenodes(org.apache.hadoop.conf.Configuration conf) throws IOException
    Throws:
      `IOException`

  - ### refreshNamenodes

    public void refreshNamenodes() throws IOException
    Specified by:
      `refreshNamenodes` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Throws:
      `IOException`

  - ### deleteBlockPool

    public void deleteBlockPool(String blockPoolId, boolean force) throws IOException
    Specified by:
      `deleteBlockPool` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Throws:
      `IOException`

  - ### shutdownDatanode

    public void shutdownDatanode(boolean forUpgrade) throws IOException
    Specified by:
      `shutdownDatanode` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Throws:
      `IOException`

  - ### evictWriters

    public void evictWriters() throws IOException
    Specified by:
      `evictWriters` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Throws:
      `IOException`

  - ### getDatanodeInfo

    public org.apache.hadoop.hdfs.protocol.DatanodeLocalInfo getDatanodeInfo()
    Specified by:
      `getDatanodeInfo` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`

  - ### startReconfiguration

    public void startReconfiguration() throws IOException
    Specified by:
      `startReconfiguration` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Specified by:
      `startReconfiguration` in interface `org.apache.hadoop.hdfs.protocol.ReconfigurationProtocol`
    Throws:
      `IOException`

  - ### getReconfigurationStatus

    public org.apache.hadoop.conf.ReconfigurationTaskStatus getReconfigurationStatus() throws IOException
    Specified by:
      `getReconfigurationStatus` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Specified by:
      `getReconfigurationStatus` in interface `org.apache.hadoop.hdfs.protocol.ReconfigurationProtocol`
    Throws:
      `IOException`

  - ### listReconfigurableProperties

    public List<String> listReconfigurableProperties() throws IOException
    Specified by:
      `listReconfigurableProperties` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Specified by:
      `listReconfigurableProperties` in interface `org.apache.hadoop.hdfs.protocol.ReconfigurationProtocol`
    Throws:
      `IOException`

  - ### triggerBlockReport

    public void triggerBlockReport(org.apache.hadoop.hdfs.client.BlockReportOptions options) throws IOException
    Specified by:
      `triggerBlockReport` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Throws:
      `IOException`

  - ### isConnectedToNN

    public boolean isConnectedToNN(InetSocketAddress addr)
    Parameters:
      `addr` - rpc address of the namenode
    Returns:
      true if the datanode is connected to a NameNode at the given address

  - ### isBPServiceAlive

    public boolean isBPServiceAlive(String bpid)
    Parameters:
      `bpid` - block pool Id
    Returns:
      true - if BPOfferService thread is alive

  - ### isDatanodeFullyStarted

    public boolean isDatanodeFullyStarted()
    A datanode is considered to be fully started if all the BP threads are alive and all the block pools are initialized.
    Returns:
      true - if the data node is fully started

  - ### isDatanodeFullyStarted

    public boolean isDatanodeFullyStarted(boolean checkConnectionToActiveNamenode)
    A datanode is considered to be fully started if all the BP threads are alive and all the block pools are initialized. If checkConnectionToActiveNamenode is true, the datanode is considered to be fully started if it is also heartbeating to active namenode in addition to the above-mentioned conditions.
    Parameters:
      `checkConnectionToActiveNamenode` - if true, performs additional check of whether datanode is heartbeating to active namenode.
    Returns:
      true if the datanode is fully started and also conditionally connected to active namenode, false otherwise.

  - ### getDatanodeId

    @VisibleForTesting public org.apache.hadoop.hdfs.protocol.DatanodeID getDatanodeId()

  - ### clearAllBlockSecretKeys

    @VisibleForTesting public void clearAllBlockSecretKeys()

  - ### getBalancerBandwidth

    public long getBalancerBandwidth()
    Specified by:
      `getBalancerBandwidth` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`

  - ### getDnConf

    public DNConf getDnConf()

  - ### getDatanodeUuid

    public String getDatanodeUuid()

  - ### getShortCircuitRegistry

    public ShortCircuitRegistry getShortCircuitRegistry()

  - ### getEcReconstuctReadThrottler

    public DataTransferThrottler getEcReconstuctReadThrottler()

  - ### getEcReconstuctWriteThrottler

    public DataTransferThrottler getEcReconstuctWriteThrottler()

  - ### checkDiskError

    @VisibleForTesting public void checkDiskError() throws IOException
    Check the disk error synchronously.
    Throws:
      `IOException`

  - ### handleVolumeFailures

    @VisibleForTesting public void handleVolumeFailures(Set<FsVolumeSpi> unhealthyVolumes)

  - ### getLastDiskErrorCheck

    @VisibleForTesting public long getLastDiskErrorCheck()

  - ### getBlockRecoveryWorker

    public BlockRecoveryWorker getBlockRecoveryWorker()

  - ### getErasureCodingWorker

    public ErasureCodingWorker getErasureCodingWorker()

  - ### getOOBTimeout

    public long getOOBTimeout(org.apache.hadoop.hdfs.protocol.proto.DataTransferProtos.Status status) throws IOException
    Get the timeout to be used for transmitting the OOB type
    Returns:
      the timeout in milliseconds
    Throws:
      `IOException`

  - ### startMetricsLogger

    protected void startMetricsLogger()
    Start a timer to periodically write DataNode metrics to the log file. This behavior can be disabled by configuration.

  - ### stopMetricsLogger

    protected void stopMetricsLogger()

  - ### getTracer

    public org.apache.hadoop.tracing.Tracer getTracer()

  - ### submitDiskBalancerPlan

    public void submitDiskBalancerPlan(String planID, long planVersion, String planFile, String planData, boolean skipDateCheck) throws IOException
    Allows submission of a disk balancer Job.
    Specified by:
      `submitDiskBalancerPlan` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Parameters:
      `planID` - - Hash value of the plan.
      `planVersion` - - Plan version, reserved for future use. We have only version 1 now.
      `planFile` - - Plan file name
      `planData` - - Actual plan data in json format
    Throws:
      `IOException`

  - ### cancelDiskBalancePlan

    public void cancelDiskBalancePlan(String planID) throws IOException
    Cancels a running plan.
    Specified by:
      `cancelDiskBalancePlan` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Parameters:
      `planID` - - Hash string that identifies a plan
    Throws:
      `IOException`

  - ### queryDiskBalancerPlan

    public org.apache.hadoop.hdfs.server.datanode.DiskBalancerWorkStatus queryDiskBalancerPlan() throws IOException
    Returns the status of current or last executed work plan.
    Specified by:
      `queryDiskBalancerPlan` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Returns:
      DiskBalancerWorkStatus.
    Throws:
      `IOException`

  - ### getDiskBalancerSetting

    public String getDiskBalancerSetting(String key) throws IOException
    Gets a runtime configuration value from diskbalancer instance. For example : DiskBalancer bandwidth.
    Specified by:
      `getDiskBalancerSetting` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Parameters:
      `key` - - String that represents the run time key value.
    Returns:
      value of the key as a string.
    Throws:
      `IOException` - - Throws if there is no such key

  - ### getSendPacketDownstreamAvgInfo

    public String getSendPacketDownstreamAvgInfo()
    Description copied from interface: `DataNodeMXBean`
    Gets the average info (e.g. time) of SendPacketDownstream when the DataNode acts as the penultimate (2nd to the last) node in pipeline.

    Example Json: {"[185.164.159.81:9801]RollingAvgTime":504.867, "[49.236.149.246:9801]RollingAvgTime":504.463, "[84.125.113.65:9801]RollingAvgTime":497.954}

    Specified by:
      `getSendPacketDownstreamAvgInfo` in interface `DataNodeMXBean`

  - ### getSlowDisks

    public String getSlowDisks()
    Description copied from interface: `DataNodeMXBean`
    Gets the slow disks in the Datanode.
    Specified by:
      `getSlowDisks` in interface `DataNodeMXBean`
    Returns:
      list of slow disks

  - ### getVolumeReport

    public List<org.apache.hadoop.hdfs.protocol.DatanodeVolumeInfo> getVolumeReport() throws IOException
    Specified by:
      `getVolumeReport` in interface `org.apache.hadoop.hdfs.protocol.ClientDatanodeProtocol`
    Throws:
      `IOException`

  - ### getDiskBalancer

    @VisibleForTesting public DiskBalancer getDiskBalancer() throws IOException
    Throws:
      `IOException`

  - ### getDataSetLockManager

    public DataSetLockManager getDataSetLockManager()

  - ### getBlockPoolManager

    @VisibleForTesting public org.apache.hadoop.hdfs.server.datanode.BlockPoolManager getBlockPoolManager()