DataChecksum (Apache Hadoop Common 3.5.0 API)
Description: declaration: package: org.apache.hadoop.util, class: DataChecksum

Package org.apache.hadoop.util

# Class DataChecksum

java.lang.Object
org.apache.hadoop.util.DataChecksum

All Implemented Interfaces:
  `Checksum`

@LimitedPrivate({"HDFS","MapReduce"}) @Evolving public class DataChecksum extends Object implements Checksum
This class provides interface and utilities for processing checksums for DFS data transfers.

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `static enum`
  `DataChecksum.Type`
  The checksum types

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final int`
  `CHECKSUM_CRC32`
  `static final int`
  `CHECKSUM_CRC32C`
  `static final int`
  `CHECKSUM_DEFAULT`
  `static final int`
  `CHECKSUM_MIXED`
  `static final int`
  `CHECKSUM_NULL`
  `static final int`
  `SIZE_OF_INTEGER`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `calculateChunkedSums(byte[] data, int dataOffset, int dataLength, byte[] sums, int sumsOffset)`
  Implementation of chunked calculation specifically on byte arrays.
  `void`
  `calculateChunkedSums(ByteBuffer data, ByteBuffer checksums)`
  Calculate checksums for the given data.
  `boolean`
  `compare(byte[] buf, int offset)`
  Compares the checksum located at buf[offset] with the current checksum.
  `boolean`
  `equals(Object other)`
  `int`
  `getBytesPerChecksum()`
  `static int`
  `getChecksumHeaderSize()`
  `int`
  `getChecksumSize()`
  the size for a checksum.
  `int`
  `getChecksumSize(int dataSize)`
  the required checksum size given the data length.
  `DataChecksum.Type`
  `getChecksumType()`
  `byte[]`
  `getHeader()`
  `int`
  `getNumBytesInSum()`
  `long`
  `getValue()`
  `int`
  `hashCode()`
  `static Checksum`
  `newCrc32()`
  Create a Crc32 Checksum object.
  `static DataChecksum`
  `newDataChecksum(byte[] bytes, int offset)`
  Creates a DataChecksum from HEADER_LEN bytes from arr[offset].
  `static DataChecksum`
  `newDataChecksum(DataInputStream in)`
  This constructs a DataChecksum by reading HEADER_LEN bytes from input stream in.
  `static DataChecksum`
  `newDataChecksum(DataChecksum.Type type, int bytesPerChecksum)`
  `void`
  `reset()`
  `String`
  `toString()`
  `void`
  `update(byte[] b, int off, int len)`
  `void`
  `update(int b)`
  `void`
  `verifyChunkedSums(ByteBuffer data, ByteBuffer checksums, String fileName, long basePos)`
  Verify that the given checksums match the given data.
  `void`
  `writeHeader(DataOutputStream out)`
  Writes the checksum header to the output stream out.
  `int`
  `writeValue(byte[] buf, int offset, boolean reset)`
  Writes the current checksum to a buffer.
  `int`
  `writeValue(DataOutputStream out, boolean reset)`
  Writes the current checksum to the stream.

  ### Methods inherited from class java.lang.Object

  `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

  ### Methods inherited from interface java.util.zip.Checksum

  `update, update`

- ## Field Details

  - ### CHECKSUM_NULL

    public static final int CHECKSUM_NULL
    See Also:
      - Constant Field Values

  - ### CHECKSUM_CRC32

    public static final int CHECKSUM_CRC32
    See Also:
      - Constant Field Values

  - ### CHECKSUM_CRC32C

    public static final int CHECKSUM_CRC32C
    See Also:
      - Constant Field Values

  - ### CHECKSUM_DEFAULT

    public static final int CHECKSUM_DEFAULT
    See Also:
      - Constant Field Values

  - ### CHECKSUM_MIXED

    public static final int CHECKSUM_MIXED
    See Also:
      - Constant Field Values

  - ### SIZE_OF_INTEGER

    public static final int SIZE_OF_INTEGER
    See Also:
      - Constant Field Values

- ## Method Details

  - ### newCrc32

    public static Checksum newCrc32()
    Create a Crc32 Checksum object. The implementation of the Crc32 algorithm is chosen depending on the platform.
    Returns:
      Checksum.

  - ### newDataChecksum

    public static DataChecksum newDataChecksum(DataChecksum.Type type, int bytesPerChecksum)

  - ### newDataChecksum

    public static DataChecksum newDataChecksum(byte[] bytes, int offset) throws InvalidChecksumSizeException
    Creates a DataChecksum from HEADER_LEN bytes from arr[offset].
    Parameters:
      `bytes` - bytes.
      `offset` - offset.
    Returns:
      DataChecksum of the type in the array or null in case of an error.
    Throws:
      `InvalidChecksumSizeException` - when the stored checksum is invalid.

  - ### newDataChecksum

    public static DataChecksum newDataChecksum(DataInputStream in) throws IOException
    This constructs a DataChecksum by reading HEADER_LEN bytes from input stream in.
    Parameters:
      `in` - data input stream.
    Returns:
      DataChecksum by reading HEADER_LEN bytes from input stream.
    Throws:
      `IOException` - raised on errors performing I/O.

  - ### writeHeader

    public void writeHeader(DataOutputStream out) throws IOException
    Writes the checksum header to the output stream out.
    Parameters:
      `out` - output stream.
    Throws:
      `IOException` - raised on errors performing I/O.

  - ### getHeader

    public byte[] getHeader()

  - ### writeValue

    public int writeValue(DataOutputStream out, boolean reset) throws IOException
    Writes the current checksum to the stream. If reset is true, then resets the checksum.
    Parameters:
      `out` - out.
      `reset` - reset.
    Returns:
      number of bytes written. Will be equal to getChecksumSize();
    Throws:
      `IOException` - raised on errors performing I/O.

  - ### writeValue

    public int writeValue(byte[] buf, int offset, boolean reset) throws IOException
    Writes the current checksum to a buffer. If reset is true, then resets the checksum.
    Parameters:
      `buf` - buf.
      `offset` - offset.
      `reset` - reset.
    Returns:
      number of bytes written. Will be equal to getChecksumSize();
    Throws:
      `IOException` - raised on errors performing I/O.

  - ### compare

    public boolean compare(byte[] buf, int offset)
    Compares the checksum located at buf[offset] with the current checksum.
    Parameters:
      `buf` - buf.
      `offset` - offset.
    Returns:
      true if the checksum matches and false otherwise.

  - ### getChecksumType

    public DataChecksum.Type getChecksumType()
    Returns:
      the checksum algorithm type.

  - ### getChecksumSize

    public int getChecksumSize()
    the size for a checksum.
    Returns:
      the size for a checksum.

  - ### getChecksumSize

    public int getChecksumSize(int dataSize)
    the required checksum size given the data length.
    Parameters:
      `dataSize` - data size.
    Returns:
      the required checksum size given the data length.

  - ### getBytesPerChecksum

    public int getBytesPerChecksum()

  - ### getNumBytesInSum

    public int getNumBytesInSum()

  - ### getChecksumHeaderSize

    public static int getChecksumHeaderSize()

  - ### getValue

    public long getValue()
    Specified by:
      `getValue` in interface `Checksum`

  - ### reset

    public void reset()
    Specified by:
      `reset` in interface `Checksum`

  - ### update

    public void update(byte[] b, int off, int len)
    Specified by:
      `update` in interface `Checksum`

  - ### update

    public void update(int b)
    Specified by:
      `update` in interface `Checksum`

  - ### verifyChunkedSums

    public void verifyChunkedSums(ByteBuffer data, ByteBuffer checksums, String fileName, long basePos) throws ChecksumException
    Verify that the given checksums match the given data. The 'mark' of the ByteBuffer parameters may be modified by this function,. but the position is maintained.
    Parameters:
      `data` - the DirectByteBuffer pointing to the data to verify.
      `checksums` - the DirectByteBuffer pointing to a series of stored checksums
      `fileName` - the name of the file being read, for error-reporting
      `basePos` - the file position to which the start of 'data' corresponds
    Throws:
      `ChecksumException` - if the checksums do not match

  - ### calculateChunkedSums

    public void calculateChunkedSums(ByteBuffer data, ByteBuffer checksums)
    Calculate checksums for the given data. The 'mark' of the ByteBuffer parameters may be modified by this function, but the position is maintained.
    Parameters:
      `data` - the DirectByteBuffer pointing to the data to checksum.
      `checksums` - the DirectByteBuffer into which checksums will be stored. Enough space must be available in this buffer to put the checksums.

  - ### calculateChunkedSums

    public void calculateChunkedSums(byte[] data, int dataOffset, int dataLength, byte[] sums, int sumsOffset)
    Implementation of chunked calculation specifically on byte arrays. This is to avoid the copy when dealing with ByteBuffers that have array backing.
    Parameters:
      `data` - data.
      `dataOffset` - dataOffset.
      `dataLength` - dataLength.
      `sums` - sums.
      `sumsOffset` - sumsOffset.

  - ### equals

    public boolean equals(Object other)
    Overrides:
      `equals` in class `Object`

  - ### hashCode

    public int hashCode()
    Overrides:
      `hashCode` in class `Object`

  - ### toString

    public String toString()
    Overrides:
      `toString` in class `Object`