Apache Hadoop 3.5.0 – Apache Hadoop Changelog

##### General

- Overview
- Single Node Setup
- Cluster Setup
- Commands Reference
- FileSystem Shell
- Compatibility Specification
- Downstream Developer's Guide
- Admin Compatibility Guide
- Interface Classification
- FileSystem Specification

##### Common

- CLI Mini Cluster
- Fair Call Queue
- Native Libraries
- Proxy User
- Rack Awareness
- Secure Mode
- Service Level Authorization
- HTTP Authentication
- Credential Provider API
- Hadoop KMS
- Tracing
- Unix Shell Guide
- Registry
- Async Profiler

##### HDFS

- Architecture
- User Guide
- Commands Reference
- NameNode HA With QJM
- NameNode HA With NFS
- Observer NameNode
- Federation
- ViewFs
- ViewFsOverloadScheme
- Snapshots
- Edits Viewer
- Image Viewer
- Permissions and HDFS
- Quotas and HDFS
- libhdfs (C API)
- WebHDFS (REST API)
- HttpFS
- Short Circuit Local Reads
- Centralized Cache Management
- NFS Gateway
- Rolling Upgrade
- Extended Attributes
- Transparent Encryption
- Multihoming
- Storage Policies
- Memory Storage Support
- Synthetic Load Generator
- Erasure Coding
- Disk Balancer
- Upgrade Domain
- DataNode Admin
- Router Federation
- Provided Storage

##### MapReduce

- Tutorial
- Commands Reference
- Encrypted Shuffle
- Pluggable Shuffle/Sort
- Distributed Cache Deploy
- Support for YARN Shared Cache

##### MapReduce REST APIs

- MR Application Master
- MR History Server

##### YARN

- Architecture
- Commands Reference
- Capacity Scheduler
- Fair Scheduler
- ResourceManager Restart
- ResourceManager HA
- Resource Model
- Node Labels
- Node Attributes
- Web Application Proxy
- Timeline Server
- Timeline Service V.2
- Writing YARN Applications
- YARN Application Security
- NodeManager
- Running Applications in Docker Containers
- Running Applications in runC Containers
- Using CGroups
- Secure Containers
- Reservation System
- Graceful Decommission
- Opportunistic Containers
- YARN Federation
- Shared Cache
- Using GPU
- Using FPGA
- Placement Constraints
- YARN UI2

##### YARN REST APIs

- Introduction
- Resource Manager
- Node Manager
- Timeline Server
- Timeline Service V.2

##### YARN Service

- Overview
- QuickStart
- Concepts
- Yarn Service API
- Service Discovery
- System Services

##### Hadoop Compatible File Systems

- Aliyun OSS
- Amazon S3
- Azure Blob Storage
- Azure Data Lake Storage
- Tencent COS
- Huaweicloud OBS
- VolcanoEngine TOS

##### Auth

- Overview
- Examples
- Configuration
- Building

##### Tools

- Hadoop Streaming
- Hadoop Archives
- Hadoop Archive Logs
- DistCp
- HDFS Federation Balance
- GridMix
- Rumen
- Resource Estimator Service
- Scheduler Load Simulator
- Hadoop Benchmarking
- Dynamometer

##### Reference

- Changelog and Release Notes
- Java API docs
- Unix Shell API
- Metrics

##### Configuration

- core-default.xml
- hdfs-default.xml
- hdfs-rbf-default.xml
- mapred-default.xml
- yarn-default.xml
- kms-default.xml
- httpfs-default.xml
- Deprecated Properties

# Apache Hadoop Changelog

## Release 2.6.3 - 2015-12-17

### IMPROVEMENTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-12413 | AccessControlList should avoid calling getGroupNames in isUserInList with empty groups. | Major | security | zhihai xu | zhihai xu
HDFS-9434 | Recommission a datanode with 500k blocks may pause NN for 30 seconds | Major | namenode | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze

### BUG FIXES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-9242 | Duplicate surefire plugin config in hadoop-common | Major | test | Andrey Klochkov | Andrey Klochkov
HADOOP-11267 | TestSecurityUtil fails when run with JDK8 because of empty principal names | Minor | security, test | Stephen Chu | Stephen Chu
HADOOP-10134 | [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments | Minor | . | Andrew Purtell | Andrew Purtell
MAPREDUCE-6273 | HistoryFileManager should check whether summaryFile exists to avoid FileNotFoundException causing HistoryFileInfo into MOVE_FAILED state | Minor | jobhistoryserver | zhihai xu | zhihai xu
MAPREDUCE-6377 | JHS sorting on state column not working in webUi | Minor | jobhistoryserver | Bibin A Chundatt | zhihai xu
HDFS-8615 | Correct HTTP method in WebHDFS document | Major | documentation | Akira Ajisaka | Brahma Reddy Battula
MAPREDUCE-5948 | org.apache.hadoop.mapred.LineRecordReader does not handle multibyte record delimiters well | Critical | . | Kris Geusebroek | Akira Ajisaka
YARN-3878 | AsyncDispatcher can hang while stopping if it is configured for draining events on stop | Critical | . | Varun Saxena | Varun Saxena
YARN-3925 | ContainerLogsUtils#getContainerLogFile fails to read container log files from full disks. | Critical | nodemanager | zhihai xu | zhihai xu
MAPREDUCE-6481 | LineRecordReader may give incomplete record and wrong position/key information for uncompressed input sometimes. | Critical | mrv2 | zhihai xu | zhihai xu
HADOOP-12230 | hadoop-project declares duplicate, conflicting curator dependencies | Minor | build | Steve Loughran | Rakesh R
HDFS-9273 | ACLs on root directory may be lost after NN restart | Critical | namenode | Xiao Chen | Xiao Chen
HDFS-9083 | Replication violates block placement policy. | Blocker | namenode | Rushabh S Shah | Rushabh S Shah
YARN-4312 | TestSubmitApplicationWithRMHA fails on branch-2.7 and branch-2.6 as some of the test cases time out | Major | . | Varun Saxena | Varun Saxena
YARN-4320 | TestJobHistoryEventHandler fails as AHS in MiniYarnCluster no longer binds to default port 8188 | Major | . | Varun Saxena | Varun Saxena
MAPREDUCE-6528 | Memory leak for HistoryFileManager.getJobSummary() | Critical | jobhistoryserver | Junping Du | Junping Du
YARN-4326 | Fix TestDistributedShell timeout as AHS in MiniYarnCluster no longer binds to default port 8188 | Major | . | MENG DING | MENG DING
HDFS-9289 | Make DataStreamer#block thread safe and verify genStamp in commitBlock | Critical | . | Chang Li | Chang Li
HADOOP-12526 | [Branch-2] there are duplicate dependency definitions in pom’s | Major | build | Sangjin Lee | Sangjin Lee
YARN-4241 | Fix typo of property name in yarn-default.xml | Major | documentation | Anthony Rojas | Anthony Rojas
MAPREDUCE-6540 | TestMRTimelineEventHandling fails | Major | test | Sangjin Lee | Sangjin Lee
HDFS-9431 | DistributedFileSystem#concat fails if the target path is relative. | Major | hdfs-client | Kazuho Fujii | Kazuho Fujii
YARN-2859 | ApplicationHistoryServer binds to default port 8188 in MiniYARNCluster | Critical | timelineserver | Hitesh Shah | Vinod Kumar Vavilapalli
HADOOP-12577 | Bump up commons-collections version to 3.2.2 to address a security flaw | Blocker | build, security | Wei-Chiu Chuang | Wei-Chiu Chuang
YARN-4344 | NMs reconnecting with changed capabilities can lead to wrong cluster resource calculations | Critical | resourcemanager | Varun Vasudev | Varun Vasudev
MAPREDUCE-5883 | “Total megabyte-seconds” in job counters is slightly misleading | Minor | . | Nathan Roberts | Nathan Roberts
YARN-4365 | FileSystemNodeLabelStore should check for root dir existence on startup | Major | resourcemanager | Jason Lowe | Kuhu Shukla
MAPREDUCE-6549 | multibyte delimiters with LineRecordReader cause duplicate records | Major | mrv1, mrv2 | Dustin Cote | Wilfred Spiegelenburg
HDFS-9470 | Encryption zone on root not loaded from fsimage after NN restart | Critical | . | Xiao Chen | Xiao Chen
YARN-4348 | ZKRMStateStore.syncInternal shouldn’t wait for sync completion for avoiding blocking ZK’s event thread | Blocker | . | Tsuyoshi Ozawa | Tsuyoshi Ozawa
YARN-4424 | Fix deadlock in RMAppImpl | Blocker | . | Yesha Vora | Jian He
YARN-4434 | NodeManager Disk Checker parameter documentation is not correct | Minor | documentation, nodemanager | Takashi Ohnishi | Weiwei Yang

### TESTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-10668 | TestZKFailoverControllerStress#testExpireBackAndForth occasionally fails | Major | test | Ted Yu | Ming Ma