ShortCircuitCache (Apache Hadoop HDFS Client 3.5.0 API)
Description: declaration: package: org.apache.hadoop.hdfs.shortcircuit, class: ShortCircuitCache

Package org.apache.hadoop.hdfs.shortcircuit

# Class ShortCircuitCache

java.lang.Object
org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`

@Private public class ShortCircuitCache extends Object implements Closeable
The ShortCircuitCache tracks things which the client needs to access HDFS block files via short-circuit. These things include: memory-mapped regions, file descriptors, and shared memory areas for communicating with the DataNode.

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `static interface`
  `ShortCircuitCache.CacheVisitor`
  `static interface`
  `ShortCircuitCache.ShortCircuitReplicaCreator`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final org.slf4j.Logger`
  `LOG`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `ShortCircuitCache(int maxTotalSize, long maxNonMmappedEvictableLifespanMs, int maxEvictableMmapedSize, long maxEvictableMmapedLifespanMs, long mmapRetryTimeoutMs, long staleThresholdMs, int shmInterruptCheckMs)`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `accept(ShortCircuitCache.CacheVisitor visitor)`
  `ShortCircuitShm.Slot`
  `allocShmSlot(DatanodeInfo datanode, DomainPeer peer, org.apache.commons.lang3.mutable.MutableBoolean usedPeer, ExtendedBlockId blockId, String clientName)`
  Allocate a new shared memory slot.
  `void`
  `close()`
  Close the cache and free all associated resources.
  `protected ShortCircuitReplicaInfo`
  `fetch(ExtendedBlockId key, org.apache.hadoop.util.Waitable<ShortCircuitReplicaInfo> waitable)`
  Fetch an existing ReplicaInfo object.
  `ShortCircuitReplicaInfo`
  `fetchOrCreate(ExtendedBlockId key, ShortCircuitCache.ShortCircuitReplicaCreator creator)`
  Fetch or create a replica.
  `void`
  `freeSlot(ShortCircuitShm.Slot slot)`
  Free a slot immediately.
  `static ShortCircuitCache`
  `fromConf(DfsClientConf.ShortCircuitConf conf)`
  `DfsClientShmManager`
  `getDfsClientShmManager()`
  `int`
  `getReplicaInfoMapSize()`
  Can be used in testing to verify whether a read went through SCR, after the read is done and before the stream is closed.
  `long`
  `getStaleThresholdMs()`
  `void`
  `scheduleSlotReleaser(ShortCircuitShm.Slot slot)`
  Schedule a shared memory slot to be released.
  `void`
  `setMaxTotalSize(int maxTotalSize)`
  `String`
  `toString()`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- ## Field Details

  - ### LOG

    public static final org.slf4j.Logger LOG

- ## Constructor Details

  - ### ShortCircuitCache

    public ShortCircuitCache(int maxTotalSize, long maxNonMmappedEvictableLifespanMs, int maxEvictableMmapedSize, long maxEvictableMmapedLifespanMs, long mmapRetryTimeoutMs, long staleThresholdMs, int shmInterruptCheckMs)

- ## Method Details

  - ### fromConf

    public static ShortCircuitCache fromConf(DfsClientConf.ShortCircuitConf conf)

  - ### getStaleThresholdMs

    public long getStaleThresholdMs()

  - ### setMaxTotalSize

    @VisibleForTesting public void setMaxTotalSize(int maxTotalSize)

  - ### fetchOrCreate

    public ShortCircuitReplicaInfo fetchOrCreate(ExtendedBlockId key, ShortCircuitCache.ShortCircuitReplicaCreator creator)
    Fetch or create a replica. You must hold the cache lock while calling this function.
    Parameters:
      `key` - Key to use for lookup.
      `creator` - Replica creator callback. Will be called without the cache lock being held.
    Returns:
      Null if no replica could be found or created. The replica, otherwise.

  - ### fetch

    @VisibleForTesting protected ShortCircuitReplicaInfo fetch(ExtendedBlockId key, org.apache.hadoop.util.Waitable<ShortCircuitReplicaInfo> waitable) throws org.apache.hadoop.ipc.RetriableException
    Fetch an existing ReplicaInfo object.
    Parameters:
      `key` - The key that we're using.
      `waitable` - The waitable object to wait on.
    Returns:
      The existing ReplicaInfo object, or null if there is none.
    Throws:
      `org.apache.hadoop.ipc.RetriableException` - If the caller needs to retry.

  - ### close

    public void close()
    Close the cache and free all associated resources.
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `Closeable`

  - ### accept

    @VisibleForTesting public void accept(ShortCircuitCache.CacheVisitor visitor)

  - ### toString

    public String toString()
    Overrides:
      `toString` in class `Object`

  - ### allocShmSlot

    public ShortCircuitShm.Slot allocShmSlot(DatanodeInfo datanode, DomainPeer peer, org.apache.commons.lang3.mutable.MutableBoolean usedPeer, ExtendedBlockId blockId, String clientName) throws IOException
    Allocate a new shared memory slot.
    Parameters:
      `datanode` - The datanode to allocate a shm slot with.
      `peer` - A peer connected to the datanode.
      `usedPeer` - Will be set to true if we use up the provided peer.
      `blockId` - The block id and block pool id of the block we're allocating this slot for.
      `clientName` - The name of the DFSClient allocating the shared memory.
    Returns:
      Null if short-circuit shared memory is disabled; a short-circuit memory slot otherwise.
    Throws:
      `IOException` - An exception if there was an error talking to the datanode.

  - ### freeSlot

    public void freeSlot(ShortCircuitShm.Slot slot)
    Free a slot immediately. ONLY use this if the DataNode is not yet aware of the slot.
    Parameters:
      `slot` - The slot to free.

  - ### scheduleSlotReleaser

    public void scheduleSlotReleaser(ShortCircuitShm.Slot slot)
    Schedule a shared memory slot to be released.
    Parameters:
      `slot` - The slot to release.

  - ### getDfsClientShmManager

    @VisibleForTesting public DfsClientShmManager getDfsClientShmManager()

  - ### getReplicaInfoMapSize

    @VisibleForTesting public int getReplicaInfoMapSize()
    Can be used in testing to verify whether a read went through SCR, after the read is done and before the stream is closed.