HistoryFileManager (Apache Hadoop MapReduce HistoryServer 3.5.0 API)
Description: declaration: package: org.apache.hadoop.mapreduce.v2.hs, class: HistoryFileManager

Package org.apache.hadoop.mapreduce.v2.hs

# Class HistoryFileManager

java.lang.Object
org.apache.hadoop.service.AbstractService
org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `org.apache.hadoop.service.Service`

@Public @Unstable public class HistoryFileManager extends org.apache.hadoop.service.AbstractService
This class provides a way to interact with history files in a thread safe manor.

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `class`
  `HistoryFileManager.HistoryFileInfo`

  ## Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

  `org.apache.hadoop.service.Service.STATE`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `protected org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.JobListCache`
  `jobListCache`
  `protected ThreadPoolExecutor`
  `moveToDoneExecutor`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `HistoryFileManager()`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `protected HistoryFileManager.HistoryFileInfo`
  `createHistoryFileInfo(org.apache.hadoop.fs.Path historyFile, org.apache.hadoop.fs.Path confFile, org.apache.hadoop.fs.Path summaryFile, org.apache.hadoop.mapreduce.v2.jobhistory.JobIndexInfo jobIndexInfo, boolean isInDone)`
  `protected org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.JobListCache`
  `createJobListCache()`
  `protected ThreadPoolExecutor`
  `createMoveToDoneThreadPool(int numMoveThreads)`
  `protected boolean`
  `deleteDir(org.apache.hadoop.fs.FileStatus serialDir)`
  `protected List<org.apache.hadoop.fs.FileStatus>`
  `findTimestampedDirectories()`
  Finds all history directories with a timestamp component by scanning the filesystem.
  `Collection<HistoryFileManager.HistoryFileInfo>`
  `getAllFileInfo()`
  `HistoryFileManager.HistoryFileInfo`
  `getFileInfo(org.apache.hadoop.mapreduce.v2.api.records.JobId jobId)`
  `protected static List<org.apache.hadoop.fs.FileStatus>`
  `scanDirectory(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileContext fc, org.apache.hadoop.fs.PathFilter pathFilter)`
  `protected List<org.apache.hadoop.fs.FileStatus>`
  `scanDirectoryForHistoryFiles(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileContext fc)`
  `protected void`
  `serviceInit(org.apache.hadoop.conf.Configuration conf)`
  `void`
  `serviceStop()`

  ### Methods inherited from class org.apache.hadoop.service.AbstractService

  `close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceStart, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- ## Field Details

  - ### jobListCache

    protected org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.JobListCache jobListCache

  - ### moveToDoneExecutor

    @VisibleForTesting protected ThreadPoolExecutor moveToDoneExecutor

- ## Constructor Details

  - ### HistoryFileManager

    public HistoryFileManager()

- ## Method Details

  - ### serviceInit

    protected void serviceInit(org.apache.hadoop.conf.Configuration conf) throws Exception
    Overrides:
      `serviceInit` in class `org.apache.hadoop.service.AbstractService`
    Throws:
      `Exception`

  - ### createMoveToDoneThreadPool

    protected ThreadPoolExecutor createMoveToDoneThreadPool(int numMoveThreads)

  - ### serviceStop

    public void serviceStop() throws Exception
    Overrides:
      `serviceStop` in class `org.apache.hadoop.service.AbstractService`
    Throws:
      `Exception`

  - ### createJobListCache

    protected org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.JobListCache createJobListCache()

  - ### createHistoryFileInfo

    protected HistoryFileManager.HistoryFileInfo createHistoryFileInfo(org.apache.hadoop.fs.Path historyFile, org.apache.hadoop.fs.Path confFile, org.apache.hadoop.fs.Path summaryFile, org.apache.hadoop.mapreduce.v2.jobhistory.JobIndexInfo jobIndexInfo, boolean isInDone)

  - ### scanDirectory

    @VisibleForTesting protected static List<org.apache.hadoop.fs.FileStatus> scanDirectory(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileContext fc, org.apache.hadoop.fs.PathFilter pathFilter) throws IOException
    Throws:
      `IOException`

  - ### scanDirectoryForHistoryFiles

    protected List<org.apache.hadoop.fs.FileStatus> scanDirectoryForHistoryFiles(org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FileContext fc) throws IOException
    Throws:
      `IOException`

  - ### findTimestampedDirectories

    protected List<org.apache.hadoop.fs.FileStatus> findTimestampedDirectories() throws IOException
    Finds all history directories with a timestamp component by scanning the filesystem. Used when the JobHistory server is started.
    Returns:
      list of history directories
    Throws:
      `IOException`

  - ### getAllFileInfo

    public Collection<HistoryFileManager.HistoryFileInfo> getAllFileInfo() throws IOException
    Throws:
      `IOException`

  - ### getFileInfo

    public HistoryFileManager.HistoryFileInfo getFileInfo(org.apache.hadoop.mapreduce.v2.api.records.JobId jobId) throws IOException
    Throws:
      `IOException`

  - ### deleteDir

    protected boolean deleteDir(org.apache.hadoop.fs.FileStatus serialDir) throws org.apache.hadoop.security.AccessControlException, FileNotFoundException, org.apache.hadoop.fs.UnsupportedFileSystemException, IOException
    Throws:
      `org.apache.hadoop.security.AccessControlException`
      `FileNotFoundException`
      `org.apache.hadoop.fs.UnsupportedFileSystemException`
      `IOException`