CompositeRecordReader (Apache Hadoop Main 3.5.0 API)
Description: declaration: package: org.apache.hadoop.mapreduce.lib.join, class: CompositeRecordReader

Package org.apache.hadoop.mapreduce.lib.join

# Class CompositeRecordReader<K extends WritableComparable<?>,V extends Writable,X extends Writable>

java.lang.Object
org.apache.hadoop.mapreduce.RecordReader<K,V>
org.apache.hadoop.mapreduce.lib.join.ComposableRecordReader<K,X>
org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader<K,V,X>

All Implemented Interfaces:
  `Closeable`, `AutoCloseable`, `Comparable<ComposableRecordReader<K,?>>`, `Configurable`

Direct Known Subclasses:
  `JoinRecordReader`, `MultiFilterRecordReader`

@Public @Stable public abstract class CompositeRecordReader<K extends WritableComparable<?>,V extends Writable,X extends Writable> extends ComposableRecordReader<K,X> implements Configurable
A RecordReader that can effect joins of RecordReaders sharing a common key type and partitioning.

- ## Nested Class Summary

  Nested Classes
  Modifier and Type
  Class
  Description
  `class`
  `org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader.JoinCollector`
  Collector for join values.

- ## Field Summary

  Fields
  Modifier and Type
  Field
  Description
  `protected Configuration`
  `conf`
  `protected final CompositeRecordReader<K,V,X>.org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader.JoinCollector`
  `jc`
  `protected K`
  `key`
  `protected Class<? extends WritableComparable>`
  `keyclass`
  `protected final ComposableRecordReader<K,? extends V>[]`
  `kids`
  `protected X`
  `value`

- ## Constructor Summary

  Constructors
  Constructor
  Description
  `CompositeRecordReader(int id, int capacity, Class<? extends WritableComparator> cmpcl)`
  Create a RecordReader with `capacity` children to position `id` in the parent reader.

- ## Method Summary

  Modifier and Type
  Method
  Description
  `void`
  `accept(org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader.JoinCollector jc, K key)`
  If key provided matches that of this Composite, give JoinCollector iterator over values it may emit.
  `void`
  `add(ComposableRecordReader<K,? extends V> rr)`
  Add a RecordReader to this collection.
  `void`
  `close()`
  Close all child RRs.
  `protected abstract boolean`
  `combine(Object[] srcs, TupleWritable value)`
  `int`
  `compareTo(ComposableRecordReader<K,?> other)`
  Implement Comparable contract (compare key of join or head of heap with that of another).
  `protected K`
  `createKey()`
  Create a new key common to all child RRs.
  `protected TupleWritable`
  `createTupleWritable()`
  Create a value to be used internally for joins.
  `protected void`
  `fillJoinCollector(K iterkey)`
  For all child RRs offering the key provided, obtain an iterator at that position in the JoinCollector.
  `protected WritableComparator`
  `getComparator()`
  Return comparator defining the ordering for RecordReaders in this composite.
  `Configuration`
  `getConf()`
  Return the configuration used by this object.
  `K`
  `getCurrentKey()`
  Get the current key
  `X`
  `getCurrentValue()`
  Get the current value.
  `protected abstract ResetableIterator<X>`
  `getDelegate()`
  Obtain an iterator over the child RRs apropos of the value type ultimately emitted from this join.
  `float`
  `getProgress()`
  Report progress as the minimum of all child RR progress.
  `protected PriorityQueue<ComposableRecordReader<K,?>>`
  `getRecordReaderQueue()`
  Return sorted list of RecordReaders for this composite.
  `boolean`
  `hasNext()`
  Return true if it is possible that this could emit more values.
  `int`
  `id()`
  Return the position in the collector this class occupies.
  `void`
  `initialize(InputSplit split, TaskAttemptContext context)`
  Called once at initialization.
  `K`
  `key()`
  Return the key for the current join or the value at the top of the RecordReader heap.
  `void`
  `key(K key)`
  Clone the key at the top of this RR into the given object.
  `void`
  `setConf(Configuration conf)`
  Set the configuration to be used by this object.
  `void`
  `skip(K key)`
  Pass skip key to child RRs.

  ### Methods inherited from class org.apache.hadoop.mapreduce.RecordReader

  `nextKeyValue`

  ### Methods inherited from class java.lang.Object

  `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- ## Field Details

  - ### conf

    protected Configuration conf

  - ### keyclass

    protected Class<? extends WritableComparable> keyclass

  - ### jc

    protected final CompositeRecordReader<K extends WritableComparable<?>,V extends Writable,X extends Writable>.org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader.JoinCollector jc

  - ### kids

    protected final ComposableRecordReader<K extends WritableComparable<?>,? extends V extends Writable>[] kids

  - ### key

    protected K extends WritableComparable<?> key

  - ### value

    protected X extends Writable value

- ## Constructor Details

  - ### CompositeRecordReader

    public CompositeRecordReader(int id, int capacity, Class<? extends WritableComparator> cmpcl) throws IOException
    Create a RecordReader with `capacity` children to position `id` in the parent reader. The id of a root CompositeRecordReader is -1 by convention, but relying on this is not recommended.
    Throws:
      `IOException`

- ## Method Details

  - ### combine

    protected abstract boolean combine(Object[] srcs, TupleWritable value)

  - ### initialize

    public void initialize(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException
    Description copied from class: `RecordReader`
    Called once at initialization.
    Specified by:
      `initialize` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
    Parameters:
      `split` - the split that defines the range of records to read
      `context` - the information about the task
    Throws:
      `IOException`
      `InterruptedException`

  - ### id

    public int id()
    Return the position in the collector this class occupies.

  - ### setConf

    public void setConf(Configuration conf)
    Set the configuration to be used by this object.
    Specified by:
      `setConf` in interface `Configurable`
    Parameters:
      `conf` - configuration to be used

  - ### getConf

    public Configuration getConf()
    Return the configuration used by this object.
    Specified by:
      `getConf` in interface `Configurable`
    Returns:
      Configuration

  - ### getRecordReaderQueue

    protected PriorityQueue<ComposableRecordReader<K,?>> getRecordReaderQueue()
    Return sorted list of RecordReaders for this composite.

  - ### getComparator

    protected WritableComparator getComparator()
    Return comparator defining the ordering for RecordReaders in this composite.

  - ### add

    public void add(ComposableRecordReader<K,? extends V> rr) throws IOException, InterruptedException
    Add a RecordReader to this collection. The id() of a RecordReader determines where in the Tuple its entry will appear. Adding RecordReaders with the same id has undefined behavior.
    Throws:
      `IOException`
      `InterruptedException`

  - ### key

    public K key()
    Return the key for the current join or the value at the top of the RecordReader heap.

  - ### key

    public void key(K key) throws IOException
    Clone the key at the top of this RR into the given object.
    Throws:
      `IOException`

  - ### getCurrentKey

    public K getCurrentKey()
    Description copied from class: `RecordReader`
    Get the current key
    Specified by:
      `getCurrentKey` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
    Returns:
      the current key or null if there is no current key

  - ### hasNext

    public boolean hasNext()
    Return true if it is possible that this could emit more values.

  - ### skip

    public void skip(K key) throws IOException, InterruptedException
    Pass skip key to child RRs.
    Throws:
      `IOException`
      `InterruptedException`

  - ### getDelegate

    protected abstract ResetableIterator<X> getDelegate()
    Obtain an iterator over the child RRs apropos of the value type ultimately emitted from this join.

  - ### accept

    public void accept(org.apache.hadoop.mapreduce.lib.join.CompositeRecordReader.JoinCollector jc, K key) throws IOException, InterruptedException
    If key provided matches that of this Composite, give JoinCollector iterator over values it may emit.
    Throws:
      `IOException`
      `InterruptedException`

  - ### fillJoinCollector

    protected void fillJoinCollector(K iterkey) throws IOException, InterruptedException
    For all child RRs offering the key provided, obtain an iterator at that position in the JoinCollector.
    Throws:
      `IOException`
      `InterruptedException`

  - ### compareTo

    public int compareTo(ComposableRecordReader<K,?> other)
    Implement Comparable contract (compare key of join or head of heap with that of another).
    Specified by:
      `compareTo` in interface `Comparable<K extends WritableComparable<?>>`

  - ### createKey

    protected K createKey()
    Create a new key common to all child RRs.
    Throws:
      `ClassCastException` - if key classes differ.

  - ### createTupleWritable

    protected TupleWritable createTupleWritable()
    Create a value to be used internally for joins.

  - ### getCurrentValue

    public X getCurrentValue() throws IOException, InterruptedException
    Get the current value.
    Specified by:
      `getCurrentValue` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
    Returns:
      the object that was read
    Throws:
      `IOException`
      `InterruptedException`

  - ### close

    public void close() throws IOException
    Close all child RRs.
    Specified by:
      `close` in interface `AutoCloseable`
    Specified by:
      `close` in interface `Closeable`
    Specified by:
      `close` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
    Throws:
      `IOException`

  - ### getProgress

    public float getProgress() throws IOException, InterruptedException
    Report progress as the minimum of all child RR progress.
    Specified by:
      `getProgress` in class `RecordReader<K extends WritableComparable<?>,X extends Writable>`
    Returns:
      a number between 0.0 and 1.0 that is the fraction of the data read
    Throws:
      `IOException`
      `InterruptedException`