FSImageFormat (Apache Hadoop HDFS 3.5.0 API)
Description: declaration: package: org.apache.hadoop.hdfs.server.namenode, class: FSImageFormat

Package org.apache.hadoop.hdfs.server.namenode

# Class FSImageFormat

java.lang.Object
org.apache.hadoop.hdfs.server.namenode.FSImageFormat

@Private @Evolving public class FSImageFormat extends Object
Contains inner classes for reading or writing the on-disk format for FSImages. In particular, the format of the FSImage looks like:

     FSImage {
       layoutVersion: int, namespaceID: int, numberItemsInFSDirectoryTree: long,
       namesystemGenerationStampV1: long, namesystemGenerationStampV2: long,
       generationStampAtBlockIdSwitch:long, lastAllocatedBlockId:
       long transactionID: long, snapshotCounter: int, numberOfSnapshots: int,
       numOfSnapshottableDirs: int,
       {FSDirectoryTree, FilesUnderConstruction, SecretManagerState} (can be compressed)
     }

     FSDirectoryTree (if LayoutVersion.Feature.FSIMAGE_NAME_OPTIMIZATION is supported) {
       INodeInfo of root, numberOfChildren of root: int
       [list of INodeInfo of root's children],
       [list of INodeDirectoryInfo of root's directory children]
     }

     FSDirectoryTree (if LayoutVersion.Feature.FSIMAGE_NAME_OPTIMIZATION not supported){
       [list of INodeInfo of INodes in topological order]
     }

     INodeInfo {
       {
         localName: short + byte[]
       } when LayoutVersion.Feature.FSIMAGE_NAME_OPTIMIZATION is supported
       or
       {
         fullPath: byte[]
       } when LayoutVersion.Feature.FSIMAGE_NAME_OPTIMIZATION is not supported
       replicationFactor: short, modificationTime: long,
       accessTime: long, preferredBlockSize: long,
       numberOfBlocks: int (-1 for INodeDirectory, -2 for INodeSymLink),
       {
         nsQuota: long, dsQuota: long,
         {
           isINodeSnapshottable: byte,
           isINodeWithSnapshot: byte (if isINodeSnapshottable is false)
         } (when LayoutVersion.Feature.SNAPSHOT is supported),
         fsPermission: short, PermissionStatus
       } for INodeDirectory
       or
       {
         symlinkString, fsPermission: short, PermissionStatus
       } for INodeSymlink
       or
       {
         [list of BlockInfo]
         [list of FileDiff]
         {
           isINodeFileUnderConstructionSnapshot: byte,
           {clientName: short + byte[], clientMachine: short + byte[]} (when
           isINodeFileUnderConstructionSnapshot is true),
         } (when LayoutVersion.Feature.SNAPSHOT is supported and writing snapshotINode),
         fsPermission: short, PermissionStatus
       } for INodeFile
     }

     INodeDirectoryInfo {
       fullPath of the directory: short + byte[],
       numberOfChildren: int, [list of INodeInfo of children INode],
       {
         numberOfSnapshots: int,
         [list of Snapshot] (when NumberOfSnapshots is positive),
         numberOfDirectoryDiffs: int,
         [list of DirectoryDiff] (NumberOfDirectoryDiffs is positive),
         number of children that are directories,
         [list of INodeDirectoryInfo of the directory children] (includes
         snapshot copies of deleted sub-directories)
       } (when LayoutVersion.Feature.SNAPSHOT is supported),
     }

     Snapshot {
       snapshotID: int, root of Snapshot: INodeDirectoryInfo (its local name is
       the name of the snapshot)
     }

     DirectoryDiff {
       full path of the root of the associated Snapshot: short + byte[],
       childrenSize: int,
       isSnapshotRoot: byte,
       snapshotINodeIsNotNull: byte (when isSnapshotRoot is false),
       snapshotINode: INodeDirectory (when SnapshotINodeIsNotNull is true), Diff
     }

     Diff {
       createdListSize: int, [Local name of INode in created list],
       deletedListSize: int, [INode in deleted list: INodeInfo]
     }

     FileDiff {
       full path of the root of the associated Snapshot: short + byte[],
       fileSize: long,
       snapshotINodeIsNotNull: byte,
       snapshotINode: INodeFile (when SnapshotINodeIsNotNull is true), Diff
     }

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `static class`
  `FSImageFormat.Loader`
  A one-shot class responsible for loading an image.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final TreeMap<String,String>`
  `renameReservedMap`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `static org.apache.hadoop.hdfs.server.namenode.FSImageFormat.LoaderDelegator`
  `newLoader(org.apache.hadoop.conf.Configuration conf, FSNamesystem fsn)`
  Construct a loader class to load the image.
  `static void`
  `setRenameReservedPairs(String renameReserved)`
  Set the key-value pairs that will be used to determine how to rename reserved paths on upgrade.
  `static void`
  `useDefaultRenameReservedPairs()`
  Use the default key-value pairs that will be used to determine how to rename reserved paths on upgrade.

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### renameReservedMap

    @VisibleForTesting public static final TreeMap<String,String> renameReservedMap

- ## Method Details

  - ### newLoader

    public static org.apache.hadoop.hdfs.server.namenode.FSImageFormat.LoaderDelegator newLoader(org.apache.hadoop.conf.Configuration conf, FSNamesystem fsn)
    Construct a loader class to load the image. It chooses the loader based on the layout version.

  - ### useDefaultRenameReservedPairs

    @VisibleForTesting public static void useDefaultRenameReservedPairs()
    Use the default key-value pairs that will be used to determine how to rename reserved paths on upgrade.

  - ### setRenameReservedPairs

    @VisibleForTesting public static void setRenameReservedPairs(String renameReserved)
    Set the key-value pairs that will be used to determine how to rename reserved paths on upgrade.