FSDataInputStream (Apache Hadoop Main 3.5.0 API)
Description: declaration: package: org.apache.hadoop.fs, class: FSDataInputStream

Package org.apache.hadoop.fs

# Class FSDataInputStream

java.lang.Object
java.io.InputStream
java.io.FilterInputStream
java.io.DataInputStream
org.apache.hadoop.fs.FSDataInputStream

All Implemented Interfaces:
  `Closeable`, `DataInput`, `AutoCloseable`, `ByteBufferPositionedReadable`, `ByteBufferReadable`, `CanSetDropBehind`, `CanSetReadahead`, `CanUnbuffer`, `org.apache.hadoop.fs.HasEnhancedByteBufferAccess`, `org.apache.hadoop.fs.HasFileDescriptor`, `PositionedReadable`, `Seekable`, `org.apache.hadoop.fs.statistics.IOStatisticsSource`, `StreamCapabilities`

Direct Known Subclasses:
  `HdfsDataInputStream`

@Public @Stable public class FSDataInputStream extends DataInputStream implements Seekable, PositionedReadable, ByteBufferReadable, org.apache.hadoop.fs.HasFileDescriptor, CanSetDropBehind, CanSetReadahead, org.apache.hadoop.fs.HasEnhancedByteBufferAccess, CanUnbuffer, StreamCapabilities, ByteBufferPositionedReadable, org.apache.hadoop.fs.statistics.IOStatisticsSource
Utility that wraps a `FSInputStream` in a `DataInputStream` and buffers input through a `BufferedInputStream`.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from interface org.apache.hadoop.fs.StreamCapabilities

  `StreamCapabilities.StreamCapability`

- ## Field Summary

  ### Fields inherited from class java.io.FilterInputStream

  `in`

  ### Fields inherited from interface org.apache.hadoop.fs.StreamCapabilities

  `ABORTABLE_STREAM, DROPBEHIND, HFLUSH, HSYNC, IOSTATISTICS, IOSTATISTICS_CONTEXT, PREADBYTEBUFFER, READAHEAD, READBYTEBUFFER, UNBUFFER, VECTOREDIO, VECTOREDIO_BUFFERS_SLICED`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `FSDataInputStream(InputStream in)`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `FileDescriptor`
  `getFileDescriptor()`
  `IOStatistics`
  `getIOStatistics()`
  Get the IO Statistics of the nested stream, falling back to null if the stream does not implement the interface `IOStatisticsSource`.
  `long`
  `getPos()`
  Get the current position in the input stream.
  `InputStream`
  `getWrappedStream()`
  Get a reference to the wrapped input stream.
  `boolean`
  `hasCapability(String capability)`
  Query the stream for a specific capability.
  `int`
  `maxReadSizeForVectorReads()`
  What is the largest size that we should group ranges together as?
  `int`
  `minSeekForVectorReads()`
  What is the smallest reasonable seek?
  `int`
  `read(long position, byte[] buffer, int offset, int length)`
  Read bytes from the given position in the stream to the given buffer.
  `int`
  `read(long position, ByteBuffer buf)`
  Reads up to `buf.remaining()` bytes into buf from a given position in the file and returns the number of bytes read.
  `int`
  `read(ByteBuffer buf)`
  Reads up to buf.remaining() bytes into buf.
  `final ByteBuffer`
  `read(ByteBufferPool bufferPool, int maxLength)`
  `ByteBuffer`
  `read(ByteBufferPool bufferPool, int maxLength, EnumSet<ReadOption> opts)`
  Get a ByteBuffer containing file data.
  `void`
  `readFully(long position, byte[] buffer)`
  See `readFully(long, byte[], int, int)`.
  `void`
  `readFully(long position, byte[] buffer, int offset, int length)`
  Read bytes from the given position in the stream to the given buffer.
  `void`
  `readFully(long position, ByteBuffer buf)`
  Delegate to the underlying stream.
  `void`
  `readVectored(List<? extends org.apache.hadoop.fs.FileRange> ranges, IntFunction<ByteBuffer> allocate)`
  Read fully a list of file ranges asynchronously from this file.
  `void`
  `readVectored(List<? extends org.apache.hadoop.fs.FileRange> ranges, IntFunction<ByteBuffer> allocate, Consumer<ByteBuffer> release)`
  Extension of `PositionedReadable.readVectored(List, IntFunction)` where a `release(buffer)` operation may be invoked if problems surface during reads.
  `void`
  `releaseBuffer(ByteBuffer buffer)`
  Release a ByteBuffer which was created by the enhanced ByteBuffer read function.
  `void`
  `seek(long desired)`
  Seek to the given offset.
  `boolean`
  `seekToNewSource(long targetPos)`
  Seek to the given position on an alternate copy of the data.
  `void`
  `setDropBehind(Boolean dropBehind)`
  Configure whether the stream should drop the cache.
  `void`
  `setReadahead(Long readahead)`
  Set the readahead on this stream.
  `String`
  `toString()`
  String value.
  `void`
  `unbuffer()`
  Reduce the buffering.

  ### Methods inherited from class java.io.DataInputStream

  `read, read, readBoolean, readByte, readChar, readDouble, readFloat, readFully, readFully, readInt, readLine, readLong, readShort, readUnsignedByte, readUnsignedShort, readUTF, readUTF, skipBytes`

  ### Methods inherited from class java.io.FilterInputStream

  `available, close, mark, markSupported, read, reset, skip`

  ### Methods inherited from class java.io.InputStream

  `nullInputStream, readAllBytes, readNBytes, readNBytes, skipNBytes, transferTo`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- ## Constructor Details

  - ### FSDataInputStream

    public FSDataInputStream(InputStream in)

- ## Method Details

  - ### seek

    public void seek(long desired) throws IOException
    Seek to the given offset.
    Specified by:
      `seek` in interface `Seekable`
    Parameters:
      `desired` - offset to seek to
    Throws:
      `IOException` - raised on errors performing I/O.

  - ### getPos

    public long getPos() throws IOException
    Get the current position in the input stream.
    Specified by:
      `getPos` in interface `Seekable`
    Returns:
      current position in the input stream
    Throws:
      `IOException` - raised on errors performing I/O.

  - ### read

    public int read(long position, byte[] buffer, int offset, int length) throws IOException
    Read bytes from the given position in the stream to the given buffer.
    Specified by:
      `read` in interface `PositionedReadable`
    Parameters:
      `position` - position in the input stream to seek
      `buffer` - buffer into which data is read
      `offset` - offset into the buffer in which data is written
      `length` - maximum number of bytes to read
    Returns:
      total number of bytes read into the buffer, or `-1` if there is no more data because the end of the stream has been reached
    Throws:
      `IOException` - IO problems.

  - ### readFully

    public void readFully(long position, byte[] buffer, int offset, int length) throws IOException
    Read bytes from the given position in the stream to the given buffer. Continues to read until `length` bytes have been read.
    Specified by:
      `readFully` in interface `PositionedReadable`
    Parameters:
      `position` - position in the input stream to seek
      `buffer` - buffer into which data is read
      `offset` - offset into the buffer in which data is written
      `length` - the number of bytes to read
    Throws:
      `IOException` - IO problems
      `EOFException` - If the end of stream is reached while reading. If an exception is thrown an undetermined number of bytes in the buffer may have been written.

  - ### readFully

    public void readFully(long position, byte[] buffer) throws IOException
    See `readFully(long, byte[], int, int)`.
    Specified by:
      `readFully` in interface `PositionedReadable`
    Parameters:
      `position` - position within file
      `buffer` - destination buffer
    Throws:
      `IOException` - IO problems.
      `EOFException` - the end of the data was reached before the read operation completed

  - ### seekToNewSource

    public boolean seekToNewSource(long targetPos) throws IOException
    Seek to the given position on an alternate copy of the data.
    Specified by:
      `seekToNewSource` in interface `Seekable`
    Parameters:
      `targetPos` - position to seek to
    Returns:
      true if a new source is found, false otherwise
    Throws:
      `IOException` - raised on errors performing I/O.

  - ### getWrappedStream

    @Public @Stable public InputStream getWrappedStream()
    Get a reference to the wrapped input stream. Used by unit tests.
    Returns:
      the underlying input stream

  - ### read

    public int read(ByteBuffer buf) throws IOException
    Description copied from interface: `ByteBufferReadable`
    Reads up to buf.remaining() bytes into buf. Callers should use buf.limit(..) to control the size of the desired read.

    After a successful call, `buf.position()` will be advanced by the number of bytes read and `buf.limit()` will be unchanged.

    In the case of an exception, the state of the buffer (the contents of the buffer, the `buf.position()`, the `buf.limit()`, etc.) is undefined, and callers should be prepared to recover from this eventuality.

    Callers should use `StreamCapabilities.hasCapability(String)` with `StreamCapabilities.READBYTEBUFFER` to check if the underlying stream supports this interface, otherwise they might get a `UnsupportedOperationException`.

    Implementations should treat 0-length requests as legitimate, and must not signal an error upon their receipt.

    Specified by:
      `read` in interface `ByteBufferReadable`
    Parameters:
      `buf` - the ByteBuffer to receive the results of the read operation.
    Returns:
      the number of bytes read, possibly zero, or -1 if reach end-of-stream
    Throws:
      `IOException` - if there is some error performing the read

  - ### getFileDescriptor

    public FileDescriptor getFileDescriptor() throws IOException
    Specified by:
      `getFileDescriptor` in interface `org.apache.hadoop.fs.HasFileDescriptor`
    Returns:
      the FileDescriptor
    Throws:
      `IOException` - raised on errors performing I/O.

  - ### setReadahead

    public void setReadahead(Long readahead) throws IOException, UnsupportedOperationException
    Description copied from interface: `CanSetReadahead`
    Set the readahead on this stream.
    Specified by:
      `setReadahead` in interface `CanSetReadahead`
    Parameters:
      `readahead` - The readahead to use. null means to use the default.
    Throws:
      `IOException` - If there was an error changing the dropBehind setting. UnsupportedOperationException If this stream doesn't support setting readahead.
      `UnsupportedOperationException`

  - ### setDropBehind

    public void setDropBehind(Boolean dropBehind) throws IOException, UnsupportedOperationException
    Description copied from interface: `CanSetDropBehind`
    Configure whether the stream should drop the cache.
    Specified by:
      `setDropBehind` in interface `CanSetDropBehind`
    Parameters:
      `dropBehind` - Whether to drop the cache. null means to use the default value.
    Throws:
      `IOException` - If there was an error changing the dropBehind setting. UnsupportedOperationException If this stream doesn't support setting the drop-behind.
      `UnsupportedOperationException`

  - ### read

    public ByteBuffer read(ByteBufferPool bufferPool, int maxLength, EnumSet<ReadOption> opts) throws IOException, UnsupportedOperationException
    Description copied from interface: `org.apache.hadoop.fs.HasEnhancedByteBufferAccess`
    Get a ByteBuffer containing file data. This ByteBuffer may come from the stream itself, via a call like mmap, or it may come from the ByteBufferFactory which is passed in as an argument.
    Specified by:
      `read` in interface `org.apache.hadoop.fs.HasEnhancedByteBufferAccess`
    Parameters:
      `bufferPool` - If this is non-null, it will be used to create a fallback ByteBuffer when the stream itself cannot create one.
      `maxLength` - The maximum length of buffer to return. We may return a buffer which is shorter than this.
      `opts` - Options to use when reading.
    Returns:
      We will always return an empty buffer if maxLength was 0, whether or not we are at EOF. If maxLength > 0, we will return null if the stream has reached EOF. Otherwise, we will return a ByteBuffer containing at least one byte. You must free this ByteBuffer when you are done with it by calling releaseBuffer on it. The buffer will continue to be readable until it is released in this manner. However, the input stream's close method may warn about unclosed buffers.
    Throws:
      `IOException` - if there was an error reading.
      `UnsupportedOperationException` - if factory was null, and we needed an external byte buffer.

  - ### read

    public final ByteBuffer read(ByteBufferPool bufferPool, int maxLength) throws IOException, UnsupportedOperationException
    Throws:
      `IOException`
      `UnsupportedOperationException`

  - ### releaseBuffer

    public void releaseBuffer(ByteBuffer buffer)
    Description copied from interface: `org.apache.hadoop.fs.HasEnhancedByteBufferAccess`
    Release a ByteBuffer which was created by the enhanced ByteBuffer read function. You must not continue using the ByteBuffer after calling this function.
    Specified by:
      `releaseBuffer` in interface `org.apache.hadoop.fs.HasEnhancedByteBufferAccess`
    Parameters:
      `buffer` - The ByteBuffer to release.

  - ### unbuffer

    public void unbuffer()
    Description copied from interface: `CanUnbuffer`
    Reduce the buffering. This will also free sockets and file descriptors held by the stream, if possible.
    Specified by:
      `unbuffer` in interface `CanUnbuffer`

  - ### hasCapability

    public boolean hasCapability(String capability)
    Description copied from interface: `StreamCapabilities`
    Query the stream for a specific capability.
    Specified by:
      `hasCapability` in interface `StreamCapabilities`
    Parameters:
      `capability` - string to query the stream support for.
    Returns:
      True if the stream supports capability.

  - ### toString

    public String toString()
    String value. Includes the string value of the inner stream
    Overrides:
      `toString` in class `Object`
    Returns:
      the stream

  - ### read

    public int read(long position, ByteBuffer buf) throws IOException
    Description copied from interface: `ByteBufferPositionedReadable`
    Reads up to `buf.remaining()` bytes into buf from a given position in the file and returns the number of bytes read. Callers should use `buf.limit(...)` to control the size of the desired read and `buf.position(...)` to control the offset into the buffer the data should be written to.

    After a successful call, `buf.position()` will be advanced by the number of bytes read and `buf.limit()` will be unchanged.

    In the case of an exception, the state of the buffer (the contents of the buffer, the `buf.position()`, the `buf.limit()`, etc.) is undefined, and callers should be prepared to recover from this eventuality.

    Callers should use `StreamCapabilities.hasCapability(String)` with `StreamCapabilities.PREADBYTEBUFFER` to check if the underlying stream supports this interface, otherwise they might get a `UnsupportedOperationException`.

    Implementations should treat 0-length requests as legitimate, and must not signal an error upon their receipt.

    This does not change the current offset of a file, and is thread-safe.

    Specified by:
      `read` in interface `ByteBufferPositionedReadable`
    Parameters:
      `position` - position within file
      `buf` - the ByteBuffer to receive the results of the read operation.
    Returns:
      the number of bytes read, possibly zero, or -1 if reached end-of-stream
    Throws:
      `IOException` - if there is some error performing the read

  - ### readFully

    public void readFully(long position, ByteBuffer buf) throws IOException
    Delegate to the underlying stream.
    Specified by:
      `readFully` in interface `ByteBufferPositionedReadable`
    Parameters:
      `position` - position within file
      `buf` - the ByteBuffer to receive the results of the read operation.
    Throws:
      `IOException` - on a failure from the nested stream.
      `UnsupportedOperationException` - if the inner stream does not support this operation.
    See Also:
      - `ByteBufferPositionedReadable.read(long, ByteBuffer)`

  - ### getIOStatistics

    public IOStatistics getIOStatistics()
    Get the IO Statistics of the nested stream, falling back to null if the stream does not implement the interface `IOStatisticsSource`.
    Specified by:
      `getIOStatistics` in interface `org.apache.hadoop.fs.statistics.IOStatisticsSource`
    Returns:
      an IOStatistics instance or null

  - ### minSeekForVectorReads

    public int minSeekForVectorReads()
    Description copied from interface: `PositionedReadable`
    What is the smallest reasonable seek?
    Specified by:
      `minSeekForVectorReads` in interface `PositionedReadable`
    Returns:
      the minimum number of bytes

  - ### maxReadSizeForVectorReads

    public int maxReadSizeForVectorReads()
    Description copied from interface: `PositionedReadable`
    What is the largest size that we should group ranges together as?
    Specified by:
      `maxReadSizeForVectorReads` in interface `PositionedReadable`
    Returns:
      the number of bytes to read at once

  - ### readVectored

    public void readVectored(List<? extends org.apache.hadoop.fs.FileRange> ranges, IntFunction<ByteBuffer> allocate) throws IOException
    Description copied from interface: `PositionedReadable`
    Read fully a list of file ranges asynchronously from this file. The default iterates through the ranges to read each synchronously, but the intent is that FSDataInputStream subclasses can make more efficient readers. As a result of the call, each range will have FileRange.setData(CompletableFuture) called with a future that when complete will have a ByteBuffer with the data from the file's range.

    The position returned by getPos() after readVectored() is undefined.

    If a file is changed while the readVectored() operation is in progress, the output is undefined. Some ranges may have old data, some may have new and some may have both.

    While a readVectored() operation is in progress, normal read api calls may block.

    Specified by:
      `readVectored` in interface `PositionedReadable`
    Parameters:
      `ranges` - the byte ranges to read
      `allocate` - the function to allocate ByteBuffer
    Throws:
      `IOException` - any IOE.

  - ### readVectored

    public void readVectored(List<? extends org.apache.hadoop.fs.FileRange> ranges, IntFunction<ByteBuffer> allocate, Consumer<ByteBuffer> release) throws IOException
    Description copied from interface: `PositionedReadable`
    Extension of `PositionedReadable.readVectored(List, IntFunction)` where a `release(buffer)` operation may be invoked if problems surface during reads.

    The `release` operation is invoked after an IOException to return the actively buffer to a pool before reporting a failure in the future.

    The default implementation calls `PositionedReadable.readVectored(List, IntFunction)`.p

    Implementations SHOULD override this method if they can release buffers as part of their error handling.

    Specified by:
      `readVectored` in interface `PositionedReadable`
    Parameters:
      `ranges` - the byte ranges to read
      `allocate` - function to allocate ByteBuffer
      `release` - callable to release a ByteBuffer.
    Throws:
      `IOException` - any IOE.