ChecksumFs (Apache Hadoop Common 3.5.0 API)
Description: declaration: package: org.apache.hadoop.fs, class: ChecksumFs

Package org.apache.hadoop.fs

# Class ChecksumFs

java.lang.Object
org.apache.hadoop.fs.AbstractFileSystem
org.apache.hadoop.fs.FilterFs
org.apache.hadoop.fs.ChecksumFs

All Implemented Interfaces:
  `PathCapabilities`

Direct Known Subclasses:
  `LocalFs`

@Private @Evolving public abstract class ChecksumFs extends FilterFs
Abstract Checksumed Fs. It provide a basic implementation of a Checksumed Fs, which creates a checksum file for each raw file. It generates & verifies checksums at the client side.

- ## Field Summary

  ### Fields inherited from class org.apache.hadoop.fs.AbstractFileSystem

  `statistics`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `ChecksumFs(AbstractFileSystem theFs)`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `FSDataOutputStream`
  `createInternal(Path f, EnumSet<CreateFlag> createFlag, FsPermission absolutePermission, int bufferSize, short replication, long blockSize, Progressable progress, Options.ChecksumOpt checksumOpt, boolean createParent)`
  The specification of this method matches that of `AbstractFileSystem.create(Path, EnumSet, Options.CreateOpts...)` except that the opts have been declared explicitly.
  `boolean`
  `delete(Path f, boolean recursive)`
  Implement the delete(Path, boolean) in checksum file system.
  `static double`
  `getApproxChkSumLength(long size)`
  `int`
  `getBytesPerSum()`
  Return the bytes Per Checksum.
  `Path`
  `getChecksumFile(Path file)`
  Return the name of the checksum file associated with a file.
  `long`
  `getChecksumFileLength(Path file, long fileSize)`
  Return the length of the checksum file given the size of the actual file.
  `static long`
  `getChecksumLength(long size, int bytesPerSum)`
  Calculated the length of the checksum file in bytes.
  `AbstractFileSystem`
  `getRawFs()`
  get the raw file system.
  `static boolean`
  `isChecksumFile(Path file)`
  Return true iff file is a checksum file name.
  `RemoteIterator<LocatedFileStatus>`
  `listLocatedStatus(Path f)`
  The specification of this method matches that of `FileContext.listLocatedStatus(Path)` except that Path f must be for this file system.
  `FileStatus[]`
  `listStatus(Path f)`
  The specification of this method matches that of `FileContext.Util.listStatus(Path)` except that Path f must be for this file system.
  `FSDataInputStream`
  `open(Path f, int bufferSize)`
  Opens an FSDataInputStream at the indicated Path.
  `void`
  `renameInternal(Path src, Path dst)`
  Rename files/dirs.
  `void`
  `renameInternal(Path src, Path dst, boolean overwrite)`
  The specification of this method matches that of `FileContext.rename(Path, Path, Options.Rename...)` except that Path f must be for this file system.
  `boolean`
  `reportChecksumFailure(Path f, FSDataInputStream in, long inPos, FSDataInputStream sums, long sumsPos)`
  Report a checksum error to the file system.
  `boolean`
  `setReplication(Path src, short replication)`
  Set replication for an existing file.
  `void`
  `setVerifyChecksum(boolean inVerifyChecksum)`
  Set whether to verify checksum.
  `boolean`
  `truncate(Path f, long newLength)`
  The specification of this method matches that of `FileContext.truncate(Path, long)` except that Path f must be for this file system.

  ### Methods inherited from class org.apache.hadoop.fs.FilterFs

  `access, checkPath, createMultipartUploader, createSnapshot, createSymlink, deleteSnapshot, getAclStatus, getAllStoragePolicies, getCanonicalServiceName, getDelegationTokens, getEnclosingRoot, getFileBlockLocations, getFileChecksum, getFileLinkStatus, getFileStatus, getFsStatus, getFsStatus, getHomeDirectory, getInitialWorkingDirectory, getLinkTarget, getMyFs, getServerDefaults, getServerDefaults, getStatistics, getStoragePolicy, getUri, getUriDefaultPort, getUriPath, getXAttr, getXAttrs, getXAttrs, hasPathCapability, isValidName, listCorruptFileBlocks, listXAttrs, makeQualified, mkdir, modifyAclEntries, msync, open, openFileWithOptions, removeAcl, removeAclEntries, removeDefaultAcl, removeXAttr, renameSnapshot, resolvePath, satisfyStoragePolicy, setAcl, setOwner, setPermission, setStoragePolicy, setTimes, setXAttr, setXAttr, supportsSymlinks, unsetStoragePolicy`

  ### Methods inherited from class org.apache.hadoop.fs.AbstractFileSystem

  `checkScheme, clearStatistics, create, createFileSystem, equals, get, getAllStatistics, getStatistics, hashCode, listStatusIterator, methodNotSupported, printStatistics, rename`

  ### Methods inherited from class java.lang.Object

  `clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait`

- ## Constructor Details

  - ### ChecksumFs

    public ChecksumFs(AbstractFileSystem theFs) throws IOException, URISyntaxException
    Throws:
      `IOException`
      `URISyntaxException`

- ## Method Details

  - ### getApproxChkSumLength

    public static double getApproxChkSumLength(long size)

  - ### setVerifyChecksum

    public void setVerifyChecksum(boolean inVerifyChecksum)
    Set whether to verify checksum.
    Overrides:
      `setVerifyChecksum` in class `FilterFs`
    Parameters:
      `inVerifyChecksum` - verify check sum flag.

  - ### getRawFs

    public AbstractFileSystem getRawFs()
    get the raw file system.
    Returns:
      abstract file system.

  - ### getChecksumFile

    public Path getChecksumFile(Path file)
    Return the name of the checksum file associated with a file.
    Parameters:
      `file` - the file path.
    Returns:
      the checksum file associated with a file.

  - ### isChecksumFile

    public static boolean isChecksumFile(Path file)
    Return true iff file is a checksum file name.
    Parameters:
      `file` - the file path.
    Returns:
      if is checksum file true,not false.

  - ### getChecksumFileLength

    public long getChecksumFileLength(Path file, long fileSize)
    Return the length of the checksum file given the size of the actual file.
    Parameters:
      `file` - the file path.
      `fileSize` - file size.
    Returns:
      check sum file length.

  - ### getBytesPerSum

    public int getBytesPerSum()
    Return the bytes Per Checksum.
    Returns:
      bytes per sum.

  - ### truncate

    public boolean truncate(Path f, long newLength) throws IOException
    Description copied from class: `AbstractFileSystem`
    The specification of this method matches that of `FileContext.truncate(Path, long)` except that Path f must be for this file system.
    Overrides:
      `truncate` in class `FilterFs`
    Parameters:
      `f` - the path.
      `newLength` - new length.
    Returns:
      if successfully truncate success true, not false.
    Throws:
      `AccessControlException` - access control exception.
      `FileNotFoundException` - file not found exception.
      `UnresolvedLinkException` - unresolved link exception.
      `IOException` - raised on errors performing I/O.

  - ### open

    public FSDataInputStream open(Path f, int bufferSize) throws IOException, UnresolvedLinkException
    Opens an FSDataInputStream at the indicated Path.
    Overrides:
      `open` in class `FilterFs`
    Parameters:
      `f` - the file name to open
      `bufferSize` - the size of the buffer to be used.
    Returns:
      if successfully open success true, not false.
    Throws:
      `AccessControlException` - access control exception.
      `FileNotFoundException` - file not found exception.
      `UnresolvedLinkException` - unresolved link exception.
      `IOException` - raised on errors performing I/O.

  - ### getChecksumLength

    public static long getChecksumLength(long size, int bytesPerSum)
    Calculated the length of the checksum file in bytes.
    Parameters:
      `size` - the length of the data file in bytes
      `bytesPerSum` - the number of bytes in a checksum block
    Returns:
      the number of bytes in the checksum file

  - ### createInternal

    public FSDataOutputStream createInternal(Path f, EnumSet<CreateFlag> createFlag, FsPermission absolutePermission, int bufferSize, short replication, long blockSize, Progressable progress, Options.ChecksumOpt checksumOpt, boolean createParent) throws IOException
    Description copied from class: `AbstractFileSystem`
    The specification of this method matches that of `AbstractFileSystem.create(Path, EnumSet, Options.CreateOpts...)` except that the opts have been declared explicitly.
    Overrides:
      `createInternal` in class `FilterFs`
    Parameters:
      `f` - the path.
      `createFlag` - create flag.
      `absolutePermission` - absolute permission.
      `bufferSize` - buffer size.
      `replication` - replications.
      `blockSize` - block size.
      `progress` - progress.
      `checksumOpt` - check sum opt.
      `createParent` - create parent.
    Returns:
      output stream.
    Throws:
      `AccessControlException` - access control exception.
      `FileAlreadyExistsException` - file already exists exception.
      `FileNotFoundException` - file not found exception.
      `ParentNotDirectoryException` - parent not directory exception.
      `UnsupportedFileSystemException` - unsupported filesystem exception.
      `UnresolvedLinkException` - unresolved link exception.
      `IOException` - raised on errors performing I/O.

  - ### setReplication

    public boolean setReplication(Path src, short replication) throws IOException, UnresolvedLinkException
    Set replication for an existing file. Implement the abstract `setReplication` of `FileSystem`
    Overrides:
      `setReplication` in class `FilterFs`
    Parameters:
      `src` - file name
      `replication` - new replication
    Returns:
      true if successful; false if file does not exist or is a directory
    Throws:
      `IOException` - if an I/O error occurs.
      `UnresolvedLinkException` - unresolved link exception.

  - ### renameInternal

    public void renameInternal(Path src, Path dst) throws IOException, UnresolvedLinkException
    Rename files/dirs.
    Overrides:
      `renameInternal` in class `FilterFs`
    Parameters:
      `src` - src.
      `dst` - dst.
    Throws:
      `AccessControlException` - access control exception.
      `FileAlreadyExistsException` - file already exists exception.
      `FileNotFoundException` - file not found exception.
      `ParentNotDirectoryException` - parent not directory exception.
      `UnresolvedLinkException` - unresolved link exception.
      `IOException` - raised on errors performing I/O.

  - ### renameInternal

    public void renameInternal(Path src, Path dst, boolean overwrite) throws AccessControlException, FileAlreadyExistsException, FileNotFoundException, ParentNotDirectoryException, UnresolvedLinkException, IOException
    Description copied from class: `AbstractFileSystem`
    The specification of this method matches that of `FileContext.rename(Path, Path, Options.Rename...)` except that Path f must be for this file system.
    Overrides:
      `renameInternal` in class `FilterFs`
    Parameters:
      `src` - src.
      `dst` - dst.
      `overwrite` - overwrite flag.
    Throws:
      `AccessControlException` - access control exception.
      `FileAlreadyExistsException` - file already exists exception.
      `FileNotFoundException` - file not found exception.
      `ParentNotDirectoryException` - parent not directory exception.
      `UnresolvedLinkException` - unresolved link exception.
      `IOException` - raised on errors performing I/O.

  - ### delete

    public boolean delete(Path f, boolean recursive) throws IOException, UnresolvedLinkException
    Implement the delete(Path, boolean) in checksum file system.
    Overrides:
      `delete` in class `FilterFs`
    Parameters:
      `f` - the path.
      `recursive` - recursive flag.
    Returns:
      if successfully deleted success true, not false.
    Throws:
      `AccessControlException` - access control exception.
      `FileNotFoundException` - file not found exception.
      `UnresolvedLinkException` - unresolved link exception.
      `IOException` - raised on errors performing I/O.

  - ### reportChecksumFailure

    public boolean reportChecksumFailure(Path f, FSDataInputStream in, long inPos, FSDataInputStream sums, long sumsPos)
    Report a checksum error to the file system.
    Parameters:
      `f` - the file name containing the error
      `in` - the stream open on the file
      `inPos` - the position of the beginning of the bad data in the file
      `sums` - the stream open on the checksum file
      `sumsPos` - the position of the beginning of the bad data in the checksum file
    Returns:
      if retry is necessary

  - ### listStatus

    public FileStatus[] listStatus(Path f) throws IOException, UnresolvedLinkException
    Description copied from class: `AbstractFileSystem`
    The specification of this method matches that of `FileContext.Util.listStatus(Path)` except that Path f must be for this file system.
    Overrides:
      `listStatus` in class `FilterFs`
    Parameters:
      `f` - the path.
    Returns:
      FileStatus Iterator.
    Throws:
      `AccessControlException` - access control exception.
      `FileNotFoundException` - file not found exception.
      `UnresolvedLinkException` - unresolved link exception.
      `IOException` - raised on errors performing I/O.

  - ### listLocatedStatus

    public RemoteIterator<LocatedFileStatus> listLocatedStatus(Path f) throws AccessControlException, FileNotFoundException, UnresolvedLinkException, IOException
    Description copied from class: `AbstractFileSystem`
    The specification of this method matches that of `FileContext.listLocatedStatus(Path)` except that Path f must be for this file system. In HDFS implementation, the BlockLocation of returned LocatedFileStatus will have different formats for replicated and erasure coded file. Please refer to `FileSystem.getFileBlockLocations(FileStatus, long, long)` for more details.
    Overrides:
      `listLocatedStatus` in class `FilterFs`
    Parameters:
      `f` - the path.
    Returns:
      FileStatus Iterator.
    Throws:
      `AccessControlException` - access control exception.
      `FileNotFoundException` - file not found exception.
      `UnresolvedLinkException` - unresolved link exception.
      `IOException` - raised on errors performing I/O.