DirectoryScanner (Apache Hadoop HDFS 3.5.0 API)
Description: declaration: package: org.apache.hadoop.hdfs.server.datanode, class: DirectoryScanner

Package org.apache.hadoop.hdfs.server.datanode

# Class DirectoryScanner

java.lang.Object
org.apache.hadoop.hdfs.server.datanode.DirectoryScanner

All Implemented Interfaces:
  `Runnable`

@Private public class DirectoryScanner extends Object implements Runnable
Periodically scans the data directories for block and block metadata files. Reconciles the differences with block information maintained in the dataset.

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `static class`
  `DirectoryScanner.BlockPoolReport`
  Helper class for compiling block info reports per block pool.
  `class`
  `DirectoryScanner.ReportCompiler`
  The ReportCompiler class encapsulates the process of searching a datanode's disks for block information.
  `static class`
  `DirectoryScanner.ScanInfoVolumeReport`
  Helper class for compiling block info reports from report compiler threads.

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `DirectoryScanner(FsDatasetSpi<?> dataset, org.apache.hadoop.conf.Configuration conf)`
  Create a new directory scanner, but don't cycle it running yet.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `Collection<DirectoryScanner.ScanInfoVolumeReport>`
  `getVolumeReports()`
  Get the lists of blocks on the disks in the data set.
  `void`
  `reconcile()`
  Reconcile differences between disk and in-memory blocks
  `void`
  `run()`
  Main program loop for DirectoryScanner.
  `void`
  `setRetainDiffs(boolean b)`
  Allow retaining diffs for unit test and analysis.
  `void`
  `start()`
  Start the scanner.

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### DirectoryScanner

    public DirectoryScanner(FsDatasetSpi<?> dataset, org.apache.hadoop.conf.Configuration conf)
    Create a new directory scanner, but don't cycle it running yet.
    Parameters:
      `dataset` - the dataset to scan
      `conf` - the Configuration object

- ## Method Details

  - ### setRetainDiffs

    @VisibleForTesting public void setRetainDiffs(boolean b)
    Allow retaining diffs for unit test and analysis. Defaults to false (off).
    Parameters:
      `b` - whether to retain diffs

  - ### start

    @VisibleForTesting public void start()
    Start the scanner. The scanner will run every `DFSConfigKeys.DFS_DATANODE_DIRECTORYSCAN_INTERVAL_KEY` seconds.

  - ### run

    public void run()
    Main program loop for DirectoryScanner. Runs `reconcile()` and handles any exceptions.
    Specified by:
      `run` in interface `Runnable`

  - ### reconcile

    @VisibleForTesting public void reconcile() throws IOException
    Reconcile differences between disk and in-memory blocks
    Throws:
      `IOException`

  - ### getVolumeReports

    @VisibleForTesting public Collection<DirectoryScanner.ScanInfoVolumeReport> getVolumeReports()
    Get the lists of blocks on the disks in the data set.