Apache Hadoop 3.5.0 – Hadoop: CLI MiniCluster.

##### General

- Overview
- Single Node Setup
- Cluster Setup
- Commands Reference
- FileSystem Shell
- Compatibility Specification
- Downstream Developer's Guide
- Admin Compatibility Guide
- Interface Classification
- FileSystem Specification

##### Common

- CLI Mini Cluster
- Fair Call Queue
- Native Libraries
- Proxy User
- Rack Awareness
- Secure Mode
- Service Level Authorization
- HTTP Authentication
- Credential Provider API
- Hadoop KMS
- Tracing
- Unix Shell Guide
- Registry
- Async Profiler

##### HDFS

- Architecture
- User Guide
- Commands Reference
- NameNode HA With QJM
- NameNode HA With NFS
- Observer NameNode
- Federation
- ViewFs
- ViewFsOverloadScheme
- Snapshots
- Edits Viewer
- Image Viewer
- Permissions and HDFS
- Quotas and HDFS
- libhdfs (C API)
- WebHDFS (REST API)
- HttpFS
- Short Circuit Local Reads
- Centralized Cache Management
- NFS Gateway
- Rolling Upgrade
- Extended Attributes
- Transparent Encryption
- Multihoming
- Storage Policies
- Memory Storage Support
- Synthetic Load Generator
- Erasure Coding
- Disk Balancer
- Upgrade Domain
- DataNode Admin
- Router Federation
- Provided Storage

##### MapReduce

- Tutorial
- Commands Reference
- Encrypted Shuffle
- Pluggable Shuffle/Sort
- Distributed Cache Deploy
- Support for YARN Shared Cache

##### MapReduce REST APIs

- MR Application Master
- MR History Server

##### YARN

- Architecture
- Commands Reference
- Capacity Scheduler
- Fair Scheduler
- ResourceManager Restart
- ResourceManager HA
- Resource Model
- Node Labels
- Node Attributes
- Web Application Proxy
- Timeline Server
- Timeline Service V.2
- Writing YARN Applications
- YARN Application Security
- NodeManager
- Running Applications in Docker Containers
- Running Applications in runC Containers
- Using CGroups
- Secure Containers
- Reservation System
- Graceful Decommission
- Opportunistic Containers
- YARN Federation
- Shared Cache
- Using GPU
- Using FPGA
- Placement Constraints
- YARN UI2

##### YARN REST APIs

- Introduction
- Resource Manager
- Node Manager
- Timeline Server
- Timeline Service V.2

##### YARN Service

- Overview
- QuickStart
- Concepts
- Yarn Service API
- Service Discovery
- System Services

##### Hadoop Compatible File Systems

- Aliyun OSS
- Amazon S3
- Azure Blob Storage
- Azure Data Lake Storage
- Tencent COS
- Huaweicloud OBS
- VolcanoEngine TOS

##### Auth

- Overview
- Examples
- Configuration
- Building

##### Tools

- Hadoop Streaming
- Hadoop Archives
- Hadoop Archive Logs
- DistCp
- HDFS Federation Balance
- GridMix
- Rumen
- Resource Estimator Service
- Scheduler Load Simulator
- Hadoop Benchmarking
- Dynamometer

##### Reference

- Changelog and Release Notes
- Java API docs
- Unix Shell API
- Metrics

##### Configuration

- core-default.xml
- hdfs-default.xml
- hdfs-rbf-default.xml
- mapred-default.xml
- yarn-default.xml
- kms-default.xml
- httpfs-default.xml
- Deprecated Properties

# Hadoop: CLI MiniCluster.

- Purpose
- Hadoop Tarball
- Running the MiniCluster

## Purpose

Using the CLI MiniCluster, users can simply start and stop a single-node Hadoop cluster with a single command, and without the need to set any environment variables or manage configuration files. The CLI MiniCluster starts both a `YARN`/`MapReduce` & `HDFS` clusters.

This is useful for cases where users want to quickly experiment with a real Hadoop cluster or test non-Java programs that rely on significant Hadoop functionality.

## Hadoop Tarball

You should be able to obtain the Hadoop tarball from the release. Also, you can directly create a tarball from the source:

    $ mvn clean install -DskipTests
    $ mvn package -Pdist -Dtar -DskipTests -Dmaven.javadoc.skip

The tarball should be available in `hadoop-dist/target/` directory.

## Running the MiniCluster

From inside the root directory of the extracted tarball, you can start the CLI MiniCluster using the following command:

    $ bin/mapred minicluster -format

The format option is required when running the minicluster for the first time, from next time -format option isn’t required.

There are a number of command line arguments that the users can use to control which services to start, and to pass other configuration properties. The available command line arguments:

    $ -D <property=value>    Options to pass into configuration object
    $ -datanodes <arg>       How many datanodes to start (default 1)
    $ -format                Format the DFS (default false)
    $ -help                  Prints option help.
    $ -jhsport <arg>         JobHistoryServer port (default 0--we choose)
    $ -namenode <arg>        URL of the namenode (default is either the DFS
    $                        cluster or a temporary dir)
    $ -nnport <arg>          NameNode port (default 0--we choose)
    $ -nnhttpport <arg>      NameNode HTTP port (default 0--we choose)
    $ -nodemanagers <arg>    How many nodemanagers to start (default 1)
    $ -nodfs                 Don't start a mini DFS cluster
    $ -nomr                  Don't start a mini MR cluster
    $ -rmport <arg>          ResourceManager port (default 0--we choose)
    $ -writeConfig <path>    Save configuration to this XML file.
    $ -writeDetails <path>   Write basic information to this JSON file.

To display this full list of available arguments, the user can pass the `-help` argument to the above command.