DFSStripedInputStream (Apache Hadoop HDFS Client 3.5.0 API)
Description: declaration: package: org.apache.hadoop.hdfs, class: DFSStripedInputStream

Package org.apache.hadoop.hdfs

# Class DFSStripedInputStream

java.lang.Object
java.io.InputStream
org.apache.hadoop.fs.FSInputStream
org.apache.hadoop.hdfs.DFSInputStream
org.apache.hadoop.hdfs.DFSStripedInputStream

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `org.apache.hadoop.fs.ByteBufferPositionedReadable`, `org.apache.hadoop.fs.ByteBufferReadable`, `org.apache.hadoop.fs.CanSetDropBehind`, `org.apache.hadoop.fs.CanSetReadahead`, `org.apache.hadoop.fs.CanUnbuffer`, `org.apache.hadoop.fs.HasEnhancedByteBufferAccess`, `org.apache.hadoop.fs.PositionedReadable`, `org.apache.hadoop.fs.Seekable`, `org.apache.hadoop.fs.StreamCapabilities`

@Private public class DFSStripedInputStream extends DFSInputStream
DFSStripedInputStream reads from striped block groups.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from interface org.apache.hadoop.fs.StreamCapabilities

  `org.apache.hadoop.fs.StreamCapabilities.StreamCapability`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `protected ByteBuffer`
  `parityBuf`

  ### Fields inherited from class org.apache.hadoop.hdfs.DFSInputStream

  `blockEnd, cachingStrategy, closed, currentLocatedBlock, dfsClient, failures, infoLock, locatedBlocks, pos, readStatistics, src, tcpReadsDisabledForTesting, verifyChecksum`

  ### Fields inherited from interface org.apache.hadoop.fs.StreamCapabilities

  `ABORTABLE_STREAM, DROPBEHIND, HFLUSH, HSYNC, IOSTATISTICS, IOSTATISTICS_CONTEXT, PREADBYTEBUFFER, READAHEAD, READBYTEBUFFER, UNBUFFER, VECTOREDIO, VECTOREDIO_BUFFERS_SLICED`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `close()`
  Close it down!
  `protected void`
  `closeCurrentBlockReaders()`
  Extend the super method with the logic of switching between cells.
  `protected void`
  `closeReader(org.apache.hadoop.hdfs.StripeReader.BlockReaderInfo readerInfo)`
  `protected void`
  `fetchBlockByteRange(LocatedBlock block, long start, long end, ByteBuffer buf, DFSUtilClient.CorruptedBlocks corruptedBlocks, Map<InetSocketAddress,List<IOException>> exceptionMap)`
  Real implementation of pread.
  `protected org.apache.hadoop.io.ByteBufferPool`
  `getBufferPool()`
  `protected ByteBuffer`
  `getCurStripeBuf()`
  `protected ByteBuffer`
  `getParityBuffer()`
  `protected ThreadPoolExecutor`
  `getStripedReadsThreadPool()`
  `ByteBuffer`
  `read(org.apache.hadoop.io.ByteBufferPool bufferPool, int maxLength, EnumSet<org.apache.hadoop.fs.ReadOption> opts)`
  May need online read recovery, zero-copy read doesn't make sense, so don't support it.
  `protected int`
  `readWithStrategy(org.apache.hadoop.hdfs.ReaderStrategy strategy)`
  `protected LocatedBlock`
  `refreshLocatedBlock(LocatedBlock block)`
  The super method `DFSInputStream.refreshLocatedBlock(org.apache.hadoop.hdfs.protocol.LocatedBlock)` refreshes cached LocatedBlock by executing `DFSInputStream.getBlockAt(long)` again.
  `void`
  `releaseBuffer(ByteBuffer buffer)`
  `protected void`
  `reportLostBlock(LocatedBlock lostBlock, Collection<DatanodeInfo> ignoredNodes)`
  Warn the user of a lost block
  `void`
  `seek(long targetPos)`
  Seek to a new arbitrary location.
  `boolean`
  `seekToNewSource(long targetPos)`
  Seek to given position on a node other than the current node.
  `void`
  `unbuffer()`

  ### Methods inherited from class org.apache.hadoop.hdfs.DFSInputStream

  `addToLocalDeadNodes, available, clearReadStatistics, fetchBlockAt, getAllBlocks, getBestNodeDNAddrPair, getBlockAt, getBlockReader, getCurrentBlock, getCurrentBlockLocationsLength, getCurrentDatanode, getDFSClient, getFileEncryptionInfo, getFileLength, getHedgedReadOpsLoopNumForTesting, getLocalDeadNodes, getLocatedBlocks, getPos, getReadStatistics, getSrc, hasCapability, mark, markSupported, maybeRegisterBlockRefresh, read, read, read, read, read, readFully, removeFromLocalDeadNodes, reportCheckSumFailure, reset, setDropBehind, setReadahead, skip, tokenRefetchNeeded`

  ### Methods inherited from class org.apache.hadoop.fs.FSInputStream

  `readFully, readFully, toString, validatePositionedReadArgs`

  ### Methods inherited from class java.io.InputStream

  `nullInputStream, read, readAllBytes, readNBytes, readNBytes, skipNBytes, transferTo`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

  ### Methods inherited from interface org.apache.hadoop.fs.PositionedReadable

  `maxReadSizeForVectorReads, minSeekForVectorReads, readVectored, readVectored`

- ## Field Details

  - ### parityBuf

    @VisibleForTesting protected ByteBuffer parityBuf

- ## Method Details

  - ### getParityBuffer

    protected ByteBuffer getParityBuffer()

  - ### getCurStripeBuf

    protected ByteBuffer getCurStripeBuf()

  - ### getBufferPool

    protected org.apache.hadoop.io.ByteBufferPool getBufferPool()

  - ### getStripedReadsThreadPool

    protected ThreadPoolExecutor getStripedReadsThreadPool()

  - ### close

    public void close() throws IOException
    Description copied from class: `DFSInputStream`
    Close it down!
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `Closeable`
    Overrides:
      `close` in class `DFSInputStream`
    Throws:
      `IOException`

  - ### closeCurrentBlockReaders

    protected void closeCurrentBlockReaders()
    Extend the super method with the logic of switching between cells. When reaching the end of a cell, proceed to the next cell and read it with the next blockReader.
    Overrides:
      `closeCurrentBlockReaders` in class `DFSInputStream`

  - ### closeReader

    protected void closeReader(org.apache.hadoop.hdfs.StripeReader.BlockReaderInfo readerInfo)

  - ### seek

    public void seek(long targetPos) throws IOException
    Seek to a new arbitrary location.
    Specified by:
      `seek` in interface `org.apache.hadoop.fs.Seekable`
    Overrides:
      `seek` in class `DFSInputStream`
    Throws:
      `IOException`

  - ### seekToNewSource

    public boolean seekToNewSource(long targetPos) throws IOException
    Description copied from class: `DFSInputStream`
    Seek to given position on a node other than the current node. If a node other than the current node is found, then returns true. If another node could not be found, then returns false.
    Specified by:
      `seekToNewSource` in interface `org.apache.hadoop.fs.Seekable`
    Overrides:
      `seekToNewSource` in class `DFSInputStream`
    Throws:
      `IOException`

  - ### readWithStrategy

    protected int readWithStrategy(org.apache.hadoop.hdfs.ReaderStrategy strategy) throws IOException
    Overrides:
      `readWithStrategy` in class `DFSInputStream`
    Throws:
      `IOException`

  - ### refreshLocatedBlock

    protected LocatedBlock refreshLocatedBlock(LocatedBlock block) throws IOException
    The super method `DFSInputStream.refreshLocatedBlock(org.apache.hadoop.hdfs.protocol.LocatedBlock)` refreshes cached LocatedBlock by executing `DFSInputStream.getBlockAt(long)` again. This method extends the logic by first remembering the index of the internal block, and re-parsing the refreshed block group with the same index.
    Overrides:
      `refreshLocatedBlock` in class `DFSInputStream`
    Parameters:
      `block` - The currently cached block locations
    Returns:
      Refreshed block locations
    Throws:
      `IOException`

  - ### fetchBlockByteRange

    protected void fetchBlockByteRange(LocatedBlock block, long start, long end, ByteBuffer buf, DFSUtilClient.CorruptedBlocks corruptedBlocks, Map<InetSocketAddress,List<IOException>> exceptionMap) throws IOException
    Real implementation of pread.

    Note: exceptionMap is not populated with ioExceptions as what we added for DFSInputStream. If you need this function, please implement it.

    Overrides:
      `fetchBlockByteRange` in class `DFSInputStream`
    Throws:
      `IOException`

  - ### reportLostBlock

    protected void reportLostBlock(LocatedBlock lostBlock, Collection<DatanodeInfo> ignoredNodes)
    Description copied from class: `DFSInputStream`
    Warn the user of a lost block
    Overrides:
      `reportLostBlock` in class `DFSInputStream`

  - ### read

    public ByteBuffer read(org.apache.hadoop.io.ByteBufferPool bufferPool, int maxLength, EnumSet<org.apache.hadoop.fs.ReadOption> opts) throws IOException, UnsupportedOperationException
    May need online read recovery, zero-copy read doesn't make sense, so don't support it.
    Specified by:
      `read` in interface `org.apache.hadoop.fs.HasEnhancedByteBufferAccess`
    Overrides:
      `read` in class `DFSInputStream`
    Throws:
      `IOException`
      `UnsupportedOperationException`

  - ### releaseBuffer

    public void releaseBuffer(ByteBuffer buffer)
    Specified by:
      `releaseBuffer` in interface `org.apache.hadoop.fs.HasEnhancedByteBufferAccess`
    Overrides:
      `releaseBuffer` in class `DFSInputStream`

  - ### unbuffer

    public void unbuffer()
    Specified by:
      `unbuffer` in interface `org.apache.hadoop.fs.CanUnbuffer`
    Overrides:
      `unbuffer` in class `DFSInputStream`