JournalSet (Apache Hadoop HDFS 3.5.0 API)
Description: declaration: package: org.apache.hadoop.hdfs.server.namenode, class: JournalSet

Package org.apache.hadoop.hdfs.server.namenode

# Class JournalSet

java.lang.Object
org.apache.hadoop.hdfs.server.namenode.JournalSet

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `Storage.FormatConfirmable`, `JournalManager`

@Private public class JournalSet extends Object implements JournalManager
Manages a collection of Journals. None of the methods are synchronized, it is assumed that FSEditLog methods, that use this class, use proper synchronization.

- ## Nested Class Summary

  ## Nested classes/interfaces inherited from interface org.apache.hadoop.hdfs.server.namenode.JournalManager

  `JournalManager.CorruptionException`

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `static final Comparator<EditLogInputStream>`
  `EDIT_LOG_INPUT_STREAM_COMPARATOR`

- ## Method Summary

  Modifier and Type
  Method
  Description
  `boolean`
  `canRollBack(StorageInfo storage, StorageInfo prevStorage, int targetLayoutVersion)`
  Return true if this JM can roll back to the previous storage state, false otherwise.
  `static void`
  `chainAndMakeRedundantStreams(Collection<EditLogInputStream> outStreams, PriorityQueue<EditLogInputStream> allStreams, long fromTxId)`
  `void`
  `close()`
  Close the journal manager, freeing any resources it may hold.
  `void`
  `discardSegments(long startTxId)`
  Discard the segments whose first txid is >= the given txid.
  `void`
  `doFinalize()`
  Finalize the upgrade.
  `void`
  `doPreUpgrade()`
  Perform any steps that must succeed across all JournalManagers involved in an upgrade before proceeding onto the actual upgrade stage.
  `void`
  `doRollback()`
  Perform the rollback to the previous FS state.
  `void`
  `doUpgrade(Storage storage)`
  Perform the actual upgrade of the JM.
  `void`
  `finalizeLogSegment(long firstTxId, long lastTxId)`
  Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.
  `void`
  `format(NamespaceInfo nsInfo, boolean force)`
  Format the underlying storage, removing any previously stored data.
  `RemoteEditLogManifest`
  `getEditLogManifest(long fromTxId)`
  Return a manifest of what finalized edit logs are available.
  `long`
  `getJournalCTime()`
  `boolean`
  `hasSomeData()`
  `boolean`
  `isEmpty()`
  Returns true if there are no journals, all redundant journals are disabled, or any required journals are disabled.
  `boolean`
  `isOpen()`
  `void`
  `purgeLogsOlderThan(long minTxIdToKeep)`
  `void`
  `recoverUnfinalizedSegments()`
  Recover segments which have not been finalized.
  `void`
  `selectInputStreams(Collection<EditLogInputStream> streams, long fromTxId, boolean inProgressOk, boolean onlyDurableTxns)`
  In this function, we get a bunch of streams from all of our JournalManager objects.
  `void`
  `setOutputBufferCapacity(int size)`
  Set the amount of memory that this stream should use to buffer edits
  `EditLogOutputStream`
  `startLogSegment(long txId, int layoutVersion)`
  Begin writing to a new segment of the log stream, which starts at the given transaction ID.

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

  ### Methods inherited from interface org.apache.hadoop.hdfs.server.common.Storage.FormatConfirmable

  `toString`

- ## Field Details

  - ### EDIT_LOG_INPUT_STREAM_COMPARATOR

    public static final Comparator<EditLogInputStream> EDIT_LOG_INPUT_STREAM_COMPARATOR

- ## Method Details

  - ### format

    public void format(NamespaceInfo nsInfo, boolean force) throws IOException
    Description copied from interface: `JournalManager`
    Format the underlying storage, removing any previously stored data.
    Specified by:
      `format` in interface `JournalManager`
    Throws:
      `IOException`

  - ### hasSomeData

    public boolean hasSomeData() throws IOException
    Specified by:
      `hasSomeData` in interface `Storage.FormatConfirmable`
    Returns:
      true if the storage seems to have some valid data in it, and the user should be required to confirm the format. Otherwise, false.
    Throws:
      `IOException` - if the storage cannot be accessed at all.

  - ### startLogSegment

    public EditLogOutputStream startLogSegment(long txId, int layoutVersion) throws IOException
    Description copied from interface: `JournalManager`
    Begin writing to a new segment of the log stream, which starts at the given transaction ID.
    Specified by:
      `startLogSegment` in interface `JournalManager`
    Throws:
      `IOException`

  - ### finalizeLogSegment

    public void finalizeLogSegment(long firstTxId, long lastTxId) throws IOException
    Description copied from interface: `JournalManager`
    Mark the log segment that spans from firstTxId to lastTxId as finalized and complete.
    Specified by:
      `finalizeLogSegment` in interface `JournalManager`
    Throws:
      `IOException`

  - ### close

    public void close() throws IOException
    Description copied from interface: `JournalManager`
    Close the journal manager, freeing any resources it may hold.
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `Closeable`
    Specified by:
      `close` in interface `JournalManager`
    Throws:
      `IOException`

  - ### isOpen

    public boolean isOpen()

  - ### selectInputStreams

    public void selectInputStreams(Collection<EditLogInputStream> streams, long fromTxId, boolean inProgressOk, boolean onlyDurableTxns)
    In this function, we get a bunch of streams from all of our JournalManager objects. Then we add these to the collection one by one.
    Parameters:
      `streams` - The collection to add the streams to. It may or may not be sorted-- this is up to the caller.
      `fromTxId` - The transaction ID to start looking for streams at
      `inProgressOk` - Should we consider unfinalized streams?
      `onlyDurableTxns` - Set to true if streams are bounded by the durable TxId. A durable TxId is the committed txid in QJM or the largest txid written into file in FJM

  - ### chainAndMakeRedundantStreams

    public static void chainAndMakeRedundantStreams(Collection<EditLogInputStream> outStreams, PriorityQueue<EditLogInputStream> allStreams, long fromTxId)

  - ### isEmpty

    public boolean isEmpty()
    Returns true if there are no journals, all redundant journals are disabled, or any required journals are disabled.
    Returns:
      True if there no journals, all redundant journals are disabled, or any required journals are disabled.

  - ### setOutputBufferCapacity

    public void setOutputBufferCapacity(int size)
    Description copied from interface: `JournalManager`
    Set the amount of memory that this stream should use to buffer edits
    Specified by:
      `setOutputBufferCapacity` in interface `JournalManager`

  - ### purgeLogsOlderThan

    public void purgeLogsOlderThan(long minTxIdToKeep) throws IOException
    Throws:
      `IOException`

  - ### recoverUnfinalizedSegments

    public void recoverUnfinalizedSegments() throws IOException
    Description copied from interface: `JournalManager`
    Recover segments which have not been finalized.
    Specified by:
      `recoverUnfinalizedSegments` in interface `JournalManager`
    Throws:
      `IOException`

  - ### getEditLogManifest

    public RemoteEditLogManifest getEditLogManifest(long fromTxId)
    Return a manifest of what finalized edit logs are available. All available edit logs are returned starting from the transaction id passed. If 'fromTxId' falls in the middle of a log, that log is returned as well.
    Parameters:
      `fromTxId` - Starting transaction id to read the logs.
    Returns:
      RemoteEditLogManifest object.

  - ### doPreUpgrade

    public void doPreUpgrade() throws IOException
    Description copied from interface: `JournalManager`
    Perform any steps that must succeed across all JournalManagers involved in an upgrade before proceeding onto the actual upgrade stage. If a call to any JM's doPreUpgrade method fails, then doUpgrade will not be called for any JM.
    Specified by:
      `doPreUpgrade` in interface `JournalManager`
    Throws:
      `IOException`

  - ### doUpgrade

    public void doUpgrade(Storage storage) throws IOException
    Description copied from interface: `JournalManager`
    Perform the actual upgrade of the JM. After this is completed, the NN can begin to use the new upgraded metadata. This metadata may later be either finalized or rolled back to the previous state.
    Specified by:
      `doUpgrade` in interface `JournalManager`
    Parameters:
      `storage` - info about the new upgraded versions.
    Throws:
      `IOException`

  - ### doFinalize

    public void doFinalize() throws IOException
    Description copied from interface: `JournalManager`
    Finalize the upgrade. JMs should purge any state that they had been keeping around during the upgrade process. After this is completed, rollback is no longer allowed.
    Specified by:
      `doFinalize` in interface `JournalManager`
    Throws:
      `IOException`

  - ### canRollBack

    public boolean canRollBack(StorageInfo storage, StorageInfo prevStorage, int targetLayoutVersion) throws IOException
    Description copied from interface: `JournalManager`
    Return true if this JM can roll back to the previous storage state, false otherwise. The NN will refuse to run the rollback operation unless at least one JM or fsimage storage directory can roll back.
    Specified by:
      `canRollBack` in interface `JournalManager`
    Parameters:
      `storage` - the storage info for the current state
      `prevStorage` - the storage info for the previous (unupgraded) state
      `targetLayoutVersion` - the layout version we intend to roll back to
    Returns:
      true if this JM can roll back, false otherwise.
    Throws:
      `IOException`

  - ### doRollback

    public void doRollback() throws IOException
    Description copied from interface: `JournalManager`
    Perform the rollback to the previous FS state. JMs which do not need to roll back their state should just return without error.
    Specified by:
      `doRollback` in interface `JournalManager`
    Throws:
      `IOException`

  - ### discardSegments

    public void discardSegments(long startTxId) throws IOException
    Description copied from interface: `JournalManager`
    Discard the segments whose first txid is >= the given txid.
    Specified by:
      `discardSegments` in interface `JournalManager`
    Parameters:
      `startTxId` - The given txid should be right at the segment boundary, i.e., it should be the first txid of some segment, if segment corresponding to the txid exists.
    Throws:
      `IOException`

  - ### getJournalCTime

    public long getJournalCTime() throws IOException
    Specified by:
      `getJournalCTime` in interface `JournalManager`
    Returns:
      the CTime of the journal manager.
    Throws:
      `IOException`