Apache Hadoop 3.5.0 – Apache Hadoop Changelog

##### General

- Overview
- Single Node Setup
- Cluster Setup
- Commands Reference
- FileSystem Shell
- Compatibility Specification
- Downstream Developer's Guide
- Admin Compatibility Guide
- Interface Classification
- FileSystem Specification

##### Common

- CLI Mini Cluster
- Fair Call Queue
- Native Libraries
- Proxy User
- Rack Awareness
- Secure Mode
- Service Level Authorization
- HTTP Authentication
- Credential Provider API
- Hadoop KMS
- Tracing
- Unix Shell Guide
- Registry
- Async Profiler

##### HDFS

- Architecture
- User Guide
- Commands Reference
- NameNode HA With QJM
- NameNode HA With NFS
- Observer NameNode
- Federation
- ViewFs
- ViewFsOverloadScheme
- Snapshots
- Edits Viewer
- Image Viewer
- Permissions and HDFS
- Quotas and HDFS
- libhdfs (C API)
- WebHDFS (REST API)
- HttpFS
- Short Circuit Local Reads
- Centralized Cache Management
- NFS Gateway
- Rolling Upgrade
- Extended Attributes
- Transparent Encryption
- Multihoming
- Storage Policies
- Memory Storage Support
- Synthetic Load Generator
- Erasure Coding
- Disk Balancer
- Upgrade Domain
- DataNode Admin
- Router Federation
- Provided Storage

##### MapReduce

- Tutorial
- Commands Reference
- Encrypted Shuffle
- Pluggable Shuffle/Sort
- Distributed Cache Deploy
- Support for YARN Shared Cache

##### MapReduce REST APIs

- MR Application Master
- MR History Server

##### YARN

- Architecture
- Commands Reference
- Capacity Scheduler
- Fair Scheduler
- ResourceManager Restart
- ResourceManager HA
- Resource Model
- Node Labels
- Node Attributes
- Web Application Proxy
- Timeline Server
- Timeline Service V.2
- Writing YARN Applications
- YARN Application Security
- NodeManager
- Running Applications in Docker Containers
- Running Applications in runC Containers
- Using CGroups
- Secure Containers
- Reservation System
- Graceful Decommission
- Opportunistic Containers
- YARN Federation
- Shared Cache
- Using GPU
- Using FPGA
- Placement Constraints
- YARN UI2

##### YARN REST APIs

- Introduction
- Resource Manager
- Node Manager
- Timeline Server
- Timeline Service V.2

##### YARN Service

- Overview
- QuickStart
- Concepts
- Yarn Service API
- Service Discovery
- System Services

##### Hadoop Compatible File Systems

- Aliyun OSS
- Amazon S3
- Azure Blob Storage
- Azure Data Lake Storage
- Tencent COS
- Huaweicloud OBS
- VolcanoEngine TOS

##### Auth

- Overview
- Examples
- Configuration
- Building

##### Tools

- Hadoop Streaming
- Hadoop Archives
- Hadoop Archive Logs
- DistCp
- HDFS Federation Balance
- GridMix
- Rumen
- Resource Estimator Service
- Scheduler Load Simulator
- Hadoop Benchmarking
- Dynamometer

##### Reference

- Changelog and Release Notes
- Java API docs
- Unix Shell API
- Metrics

##### Configuration

- core-default.xml
- hdfs-default.xml
- hdfs-rbf-default.xml
- mapred-default.xml
- yarn-default.xml
- kms-default.xml
- httpfs-default.xml
- Deprecated Properties

# Apache Hadoop Changelog

## Release 0.9.0 - 2006-12-01

### NEW FEATURES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-611 | SequenceFile.Sorter should have a merge method that returns an iterator | Major | io | Owen O’Malley | Devaraj Das

### IMPROVEMENTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-655 | remove deprecations | Minor | . | Doug Cutting | Doug Cutting
HADOOP-565 | Upgrade Jetty to 6.x | Major | . | Owen O’Malley | Sanjay Dahiya
HADOOP-688 | move dfs administrative interfaces to a separate command | Major | . | dhruba borthakur | dhruba borthakur
HADOOP-613 | The final merge on the reduces should feed the reduce directly | Major | . | Owen O’Malley | Devaraj Das
HADOOP-661 | JobConf for a job should be viewable from the web/ui | Major | . | Owen O’Malley | Arun C Murthy
HADOOP-489 | Seperating user logs from system logs in map reduce | Minor | . | Mahadev konar | Arun C Murthy
HADOOP-668 | improvement to DFS browsing WI | Minor | . | Yoram Arnon | Hairong Kuang
HADOOP-538 | Implement a nio’s ‘direct buffer’ based wrapper over zlib to improve performance of java.util.zip.{De|In}flater as a ‘custom codec’ | Major | io | Arun C Murthy | Arun C Murthy
HADOOP-721 | jobconf.jsp shouldn’t find the jobconf.xsl via http | Major | . | Owen O’Malley | Arun C Murthy
HADOOP-725 | chooseTargets method in FSNamesystem is very inefficient | Major | . | Milind Bhandarkar | Milind Bhandarkar
HADOOP-677 | RPC should send a fixed header and version at the start of connection | Major | ipc | Owen O’Malley | Owen O’Malley
HADOOP-76 | Implement speculative re-execution of reduces | Minor | . | Doug Cutting | Sanjay Dahiya
HADOOP-689 | hadoop should provide a common way to wrap instances with different types into one type | Major | io | Feng Jiang |

### BUG FIXES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-682 | hadoop namenode -format doesnt work anymore if target directory doesnt exist | Major | . | Sanjay Dahiya |
HADOOP-687 | Upgrade to Jetty 6 does not patch bin/hadoop | Critical | scripts | Sameer Paranjpye | Sameer Paranjpye
HADOOP-683 | bin/hadoop.sh doesn’t work for /bin/dash (eg ubuntu 6.10b) | Trivial | scripts | James Todd |
HADOOP-604 | DataNodes get NullPointerException and become unresponsive on 50010 | Major | . | Christian Kunz | Raghu Angadi
HADOOP-459 | libhdfs leaks memory when writing to files | Major | . | Christian Kunz | Sameer Paranjpye
HADOOP-694 | jobtracker expireluanching tasks throws out Nullpointer exceptions | Minor | . | Mahadev konar | Mahadev konar
HADOOP-637 | ipc.Server has memory leak – serious issue for namenode server | Major | ipc | Christian Kunz | Raghu Angadi
HADOOP-696 | TestTextInputFormat fails on some platforms due to non-determinism in format.getSplits() | Minor | test | Sameer Paranjpye | Sameer Paranjpye
HADOOP-708 | test-libhdfs.sh does not properly capture and return error status | Major | test | Nigel Daley | Nigel Daley
HADOOP-646 | name node server does not load large (> 2^31 bytes) edits file | Critical | . | Christian Kunz | Milind Bhandarkar
HADOOP-705 | IOException: job.xml already exists | Major | . | Nigel Daley | Mahadev konar
HADOOP-712 | Record-IO XML serialization is broken for control characters | Major | record | Milind Bhandarkar | Milind Bhandarkar
HADOOP-716 | Javadoc warning in SequenceFile.java | Minor | io | Devaraj Das | Devaraj Das
HADOOP-715 | build.xml sets up wrong ‘hadoop.log.dir’ property for ‘ant test’ | Minor | test | Arun C Murthy | Arun C Murthy
HADOOP-722 | native-hadoop deficiencies | Critical | . | Arun C Murthy | Arun C Murthy
HADOOP-710 | block size isn’t honored | Major | . | Nigel Daley | Milind Bhandarkar
HADOOP-695 | Unexpected NPE from the next method of StreamLineRecordReader fails map/reduce jobs | Major | . | Hairong Kuang | Hairong Kuang
HADOOP-645 | Map-reduce task does not finish correctly when -reducer NONE is specified | Major | . | dhruba borthakur | dhruba borthakur
HADOOP-729 | packageNativeHadoop.sh has non-standard sh code | Critical | scripts | Arun C Murthy | Arun C Murthy
HADOOP-652 | Not all Datastructures are updated when a block is deleted | Major | . | Raghu Angadi |
HADOOP-723 | Race condition exists in the method MapOutputLocation.getFile | Major | . | Devaraj Das | Owen O’Malley
HADOOP-447 | DistributedFileSystem.getBlockSize(Path) does not resolve absolute path | Major | . | Benjamin Reed | Raghu Angadi
HADOOP-724 | bin/hadoop:111 uses java directly, it should use JAVA_HOME | Minor | scripts | Sanjay Dahiya | Arun C Murthy
HADOOP-733 | dfs shell has inconsistent exit codes | Major | . | Christian Kunz | dhruba borthakur
HADOOP-734 | Link to the FAQ in the “Documentation” section of the hadoop website | Major | . | Konstantin Shvachko | Konstantin Shvachko
HADOOP-709 | streaming job with Control characters in the command causes runtime exception in the job tracker | Major | . | dhruba borthakur | dhruba borthakur
HADOOP-699 | “Browse the filesystem” link on Name Node redirects to wrong port on DataNode | Major | . | Raghu Angadi | Raghu Angadi
HADOOP-736 | Roll back Jetty6.0.1 to Jetty5.1.4 | Major | . | Devaraj Das | Devaraj Das
HADOOP-739 | TestIPC occassionally fails with BindException | Minor | test | Nigel Daley | Nigel Daley
HADOOP-741 | Fix average progress calculation in speculative execution of maps + fix pending issues in speculative execution of reduces | Major | . | Sanjay Dahiya | Sanjay Dahiya
HADOOP-749 | The jobfailures.jsp gets a NullPointerException after a task tracker has been lost | Major | . | Owen O’Malley | Owen O’Malley
HADOOP-747 | RecordIO compiler does not produce correct Java code when buffer is used as key or value in map | Major | record | Milind Bhandarkar | Milind Bhandarkar
HADOOP-698 | When DFS client fails to read from a datanode, the failed datanode is not excluded from target reselection | Major | . | Hairong Kuang | Milind Bhandarkar
HADOOP-430 | http server should die if the Datanode fails. | Major | . | Konstantin Shvachko | Wendy Chien
HADOOP-750 | race condition on stalled map output fetches | Major | . | Owen O’Malley | Owen O’Malley
HADOOP-728 | Map-reduce task does not produce correct results when -reducer NONE is specified through streaming | Major | . | dhruba borthakur | Sanjay Dahiya

### TESTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-382 | add a unit test for multiple datanodes in a machine | Major | . | Yoram Arnon | Milind Bhandarkar