Apache Hadoop 3.5.0 – Apache Hadoop Changelog

##### General

- Overview
- Single Node Setup
- Cluster Setup
- Commands Reference
- FileSystem Shell
- Compatibility Specification
- Downstream Developer's Guide
- Admin Compatibility Guide
- Interface Classification
- FileSystem Specification

##### Common

- CLI Mini Cluster
- Fair Call Queue
- Native Libraries
- Proxy User
- Rack Awareness
- Secure Mode
- Service Level Authorization
- HTTP Authentication
- Credential Provider API
- Hadoop KMS
- Tracing
- Unix Shell Guide
- Registry
- Async Profiler

##### HDFS

- Architecture
- User Guide
- Commands Reference
- NameNode HA With QJM
- NameNode HA With NFS
- Observer NameNode
- Federation
- ViewFs
- ViewFsOverloadScheme
- Snapshots
- Edits Viewer
- Image Viewer
- Permissions and HDFS
- Quotas and HDFS
- libhdfs (C API)
- WebHDFS (REST API)
- HttpFS
- Short Circuit Local Reads
- Centralized Cache Management
- NFS Gateway
- Rolling Upgrade
- Extended Attributes
- Transparent Encryption
- Multihoming
- Storage Policies
- Memory Storage Support
- Synthetic Load Generator
- Erasure Coding
- Disk Balancer
- Upgrade Domain
- DataNode Admin
- Router Federation
- Provided Storage

##### MapReduce

- Tutorial
- Commands Reference
- Encrypted Shuffle
- Pluggable Shuffle/Sort
- Distributed Cache Deploy
- Support for YARN Shared Cache

##### MapReduce REST APIs

- MR Application Master
- MR History Server

##### YARN

- Architecture
- Commands Reference
- Capacity Scheduler
- Fair Scheduler
- ResourceManager Restart
- ResourceManager HA
- Resource Model
- Node Labels
- Node Attributes
- Web Application Proxy
- Timeline Server
- Timeline Service V.2
- Writing YARN Applications
- YARN Application Security
- NodeManager
- Running Applications in Docker Containers
- Running Applications in runC Containers
- Using CGroups
- Secure Containers
- Reservation System
- Graceful Decommission
- Opportunistic Containers
- YARN Federation
- Shared Cache
- Using GPU
- Using FPGA
- Placement Constraints
- YARN UI2

##### YARN REST APIs

- Introduction
- Resource Manager
- Node Manager
- Timeline Server
- Timeline Service V.2

##### YARN Service

- Overview
- QuickStart
- Concepts
- Yarn Service API
- Service Discovery
- System Services

##### Hadoop Compatible File Systems

- Aliyun OSS
- Amazon S3
- Azure Blob Storage
- Azure Data Lake Storage
- Tencent COS
- Huaweicloud OBS
- VolcanoEngine TOS

##### Auth

- Overview
- Examples
- Configuration
- Building

##### Tools

- Hadoop Streaming
- Hadoop Archives
- Hadoop Archive Logs
- DistCp
- HDFS Federation Balance
- GridMix
- Rumen
- Resource Estimator Service
- Scheduler Load Simulator
- Hadoop Benchmarking
- Dynamometer

##### Reference

- Changelog and Release Notes
- Java API docs
- Unix Shell API
- Metrics

##### Configuration

- core-default.xml
- hdfs-default.xml
- hdfs-rbf-default.xml
- mapred-default.xml
- yarn-default.xml
- kms-default.xml
- httpfs-default.xml
- Deprecated Properties

# Apache Hadoop Changelog

## Release 2.9.1 - 2018-05-03

### INCOMPATIBLE CHANGES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-12883 | RBF: Document Router and State Store metrics | Major | documentation | Yiqun Lin | Yiqun Lin
HDFS-12895 | RBF: Add ACL support for mount table | Major | . | Yiqun Lin | Yiqun Lin
YARN-7190 | Ensure only NM classpath in 2.x gets TSv2 related hbase jars, not the user classpath | Major | timelineclient, timelinereader, timelineserver | Vrushali C | Varun Saxena
HDFS-13099 | RBF: Use the ZooKeeper as the default State Store | Minor | documentation | Yiqun Lin | Yiqun Lin

### IMPORTANT ISSUES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-13083 | RBF: Fix doc error setting up client | Major | federation | tartarus | tartarus

### NEW FEATURES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-12756 | Incorporate Aliyun OSS file system implementation | Major | fs, fs/oss | shimingfei | mingfei.shi

### IMPROVEMENTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-14872 | CryptoInputStream should implement unbuffer | Major | fs, security | John Zhuge | John Zhuge
HADOOP-14964 | AliyunOSS: backport Aliyun OSS module to branch-2 | Major | fs/oss | Genmao Yu | Sammi Chen
YARN-6851 | Capacity Scheduler: document configs for controlling # containers allowed to be allocated per node heartbeat | Minor | . | Wei Yan | Wei Yan
YARN-7495 | Improve robustness of the AggregatedLogDeletionService | Major | log-aggregation | Jonathan Eagles | Jonathan Eagles
YARN-7611 | Node manager web UI should display container type in containers page | Major | nodemanager, webapp | Weiwei Yang | Weiwei Yang
HADOOP-15056 | Fix TestUnbuffer#testUnbufferException failure | Minor | test | Jack Bearden | Jack Bearden
HADOOP-15012 | Add readahead, dropbehind, and unbuffer to StreamCapabilities | Major | fs | John Zhuge | John Zhuge
HADOOP-15104 | AliyunOSS: change the default value of max error retry | Major | fs/oss | wujinhu | wujinhu
YARN-7642 | Add test case to verify context update after container promotion or demotion with or without auto update | Minor | nodemanager | Weiwei Yang | Weiwei Yang
HADOOP-15111 | AliyunOSS: backport HADOOP-14993 to branch-2 | Major | fs/oss | Genmao Yu | Genmao Yu
HDFS-9023 | When NN is not able to identify DN for replication, reason behind it can be logged | Critical | hdfs-client, namenode | Surendra Singh Lilhore | Xiao Chen
YARN-7678 | Ability to enable logging of container memory stats | Major | nodemanager | Jim Brennan | Jim Brennan
HDFS-12945 | Switch to ClientProtocol instead of NamenodeProtocols in NamenodeWebHdfsMethods | Minor | . | Wei Yan | Wei Yan
YARN-7590 | Improve container-executor validation check | Major | security, yarn | Eric Yang | Eric Yang
HADOOP-15189 | backport HADOOP-15039 to branch-2 and branch-3 | Blocker | . | Genmao Yu | Genmao Yu
HADOOP-15212 | Add independent secret manager method for logging expired tokens | Major | security | Daryn Sharp | Daryn Sharp
YARN-7728 | Expose container preemptions related information in Capacity Scheduler queue metrics | Major | . | Eric Payne | Eric Payne
MAPREDUCE-7048 | Uber AM can crash due to unknown task in statusUpdate | Major | mr-am | Peter Bacsko | Peter Bacsko
HADOOP-13972 | ADLS to support per-store configuration | Major | fs/adl | John Zhuge | Sharad Sonker
YARN-7813 | Capacity Scheduler Intra-queue Preemption should be configurable for each queue | Major | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne
HDFS-11187 | Optimize disk access for last partial chunk checksum of Finalized replica | Major | datanode | Wei-Chiu Chuang | Gabor Bota
HADOOP-15279 | increase maven heap size recommendations | Minor | build, documentation, test | Allen Wittenauer | Allen Wittenauer
HDFS-12884 | BlockUnderConstructionFeature.truncateBlock should be of type BlockInfo | Major | namenode | Konstantin Shvachko | chencan
HADOOP-15334 | Upgrade Maven surefire plugin | Major | build | Arpit Agarwal | Arpit Agarwal
YARN-7623 | Fix the CapacityScheduler Queue configuration documentation | Major | . | Arun Suresh | Jonathan Hung
HDFS-13314 | NameNode should optionally exit if it detects FsImage corruption | Major | namenode | Arpit Agarwal | Arpit Agarwal

### BUG FIXES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-13723 | AliyunOSSInputStream#read() should update read bytes stat correctly | Major | tools | Mingliang Liu | Mingliang Liu
HADOOP-14045 | Aliyun OSS documentation missing from website | Major | documentation, fs/oss | Andrew Wang | Yiqun Lin
HADOOP-14458 | Add missing imports to TestAliyunOSSFileSystemContract.java | Trivial | fs/oss, test | Mingliang Liu | Mingliang Liu
HADOOP-14466 | Remove useless document from TestAliyunOSSFileSystemContract.java | Minor | documentation | Akira Ajisaka | Chen Liang
HDFS-12318 | Fix IOException condition for openInfo in DFSInputStream | Major | . | legend | legend
HDFS-12614 | FSPermissionChecker#getINodeAttrs() throws NPE when INodeAttributesProvider configured | Major | . | Manoj Govindassamy | Manoj Govindassamy
HDFS-12788 | Reset the upload button when file upload fails | Critical | ui, webhdfs | Brahma Reddy Battula | Brahma Reddy Battula
YARN-7388 | TestAMRestart should be scheduler agnostic | Major | . | Haibo Chen | Haibo Chen
HDFS-12705 | WebHdfsFileSystem exceptions should retain the caused by exception | Major | hdfs | Daryn Sharp | Hanisha Koneru
YARN-7361 | Improve the docker container runtime documentation | Major | . | Shane Kumpf | Shane Kumpf
YARN-7469 | Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit | Major | capacity scheduler, yarn | Eric Payne | Eric Payne
YARN-7489 | ConcurrentModificationException in RMAppImpl#getRMAppMetrics | Major | capacityscheduler | Tao Yang | Tao Yang
YARN-7525 | Incorrect query parameters in cluster nodes REST API document | Minor | documentation | Tao Yang | Tao Yang
HADOOP-15045 | ISA-L build options are documented in branch-2 | Major | build, documentation | Akira Ajisaka | Akira Ajisaka
YARN-7390 | All reservation related test cases failed when TestYarnClient runs against Fair Scheduler. | Major | fairscheduler, reservation system | Yufei Gu | Yufei Gu
HDFS-12754 | Lease renewal can hit a deadlock | Major | . | Kuhu Shukla | Kuhu Shukla
HDFS-12832 | INode.getFullPathName may throw ArrayIndexOutOfBoundsException lead to NameNode exit | Critical | namenode | DENG FEI | Konstantin Shvachko
HDFS-11754 | Make FsServerDefaults cache configurable. | Minor | . | Rushabh S Shah | Mikhail Erofeev
YARN-7509 | AsyncScheduleThread and ResourceCommitterService are still running after RM is transitioned to standby | Critical | . | Tao Yang | Tao Yang
YARN-7558 | “yarn logs” command fails to get logs for running containers if UI authentication is enabled. | Critical | . | Namit Maheshwari | Xuan Gong
HDFS-12638 | Delete copy-on-truncate block along with the original block, when deleting a file being truncated | Blocker | hdfs | Jiandan Yang | Konstantin Shvachko
MAPREDUCE-5124 | AM lacks flow control for task events | Major | mr-am | Jason Lowe | Peter Bacsko
YARN-7455 | quote_and_append_arg can overflow buffer | Major | nodemanager | Jason Lowe | Jim Brennan
HADOOP-14985 | Remove subversion related code from VersionInfoMojo.java | Minor | build | Akira Ajisaka | Ajay Kumar
HDFS-12889 | Router UI is missing robots.txt file | Major | . | Bharat Viswanadham | Bharat Viswanadham
HDFS-11576 | Block recovery will fail indefinitely if recovery time > heartbeat interval | Critical | datanode, hdfs, namenode | Lukas Majercak | Lukas Majercak
YARN-7607 | Remove the trailing duplicated timestamp in container diagnostics message | Minor | nodemanager | Weiwei Yang | Weiwei Yang
HADOOP-15080 | Aliyun OSS: update oss sdk from 2.8.1 to 2.8.3 to remove its dependency on Cat-x “json-lib” | Blocker | fs/oss | Chris Douglas | Sammi Chen
YARN-7591 | NPE in async-scheduling mode of CapacityScheduler | Critical | capacityscheduler | Tao Yang | Tao Yang
YARN-7608 | Incorrect sTarget column causing DataTable warning on RM application and scheduler web page | Major | resourcemanager, webapp | Weiwei Yang | Gergely Novák
HDFS-12833 | Distcp : Update the usage of delete option for dependency with update and overwrite option | Minor | distcp, hdfs | Harshakiran Reddy | usharani
YARN-7647 | NM print inappropriate error log when node-labels is enabled | Minor | . | Yang Wang | Yang Wang
HDFS-12907 | Allow read-only access to reserved raw for non-superusers | Major | namenode | Daryn Sharp | Rushabh S Shah
HDFS-12881 | Output streams closed with IOUtils suppressing write errors | Major | . | Jason Lowe | Ajay Kumar
YARN-7595 | Container launching code suppresses close exceptions after writes | Major | nodemanager | Jason Lowe | Jim Brennan
HADOOP-15085 | Output streams closed with IOUtils suppressing write errors | Major | . | Jason Lowe | Jim Brennan
YARN-7661 | NodeManager metrics return wrong value after update node resource | Major | . | Yang Wang | Yang Wang
HDFS-12347 | TestBalancerRPCDelay#testBalancerRPCDelay fails very frequently | Critical | test | Xiao Chen | Bharat Viswanadham
YARN-7542 | Fix issue that causes some Running Opportunistic Containers to be recovered as PAUSED | Major | . | Arun Suresh | Sampada Dehankar
HADOOP-15143 | NPE due to Invalid KerberosTicket in UGI | Major | . | Jitendra Nath Pandey | Mukul Kumar Singh
YARN-7692 | Skip validating priority acls while recovering applications | Blocker | resourcemanager | Charan Hebri | Sunil Govindan
MAPREDUCE-7028 | Concurrent task progress updates causing NPE in Application Master | Blocker | mr-am | Gergo Repas | Gergo Repas
YARN-7619 | Max AM Resource value in Capacity Scheduler UI has to be refreshed for every user | Major | capacity scheduler, yarn | Eric Payne | Eric Payne
YARN-7699 | queueUsagePercentage is coming as INF for getApp REST api call | Major | webapp | Sunil Govindan | Sunil Govindan
YARN-7508 | NPE in FiCaSchedulerApp when debug log enabled in async-scheduling mode | Major | capacityscheduler | Tao Yang | Tao Yang
YARN-7663 | RMAppImpl:Invalid event: START at KILLED | Major | resourcemanager | lujie | lujie
YARN-6948 | Invalid event: ATTEMPT_ADDED at FINAL_SAVING | Major | yarn | lujie | lujie
YARN-7735 | Fix typo in YARN documentation | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma
YARN-7727 | Incorrect log levels in few logs with QueuePriorityContainerCandidateSelector | Minor | yarn | Prabhu Joseph | Prabhu Joseph
HDFS-11915 | Sync rbw dir on the first hsync() to avoid file lost on power failure | Critical | . | Kanaka Kumar Avvaru | Vinayakumar B
HDFS-9049 | Make Datanode Netty reverse proxy port to be configurable | Major | datanode | Vinayakumar B | Vinayakumar B
HADOOP-15150 | in FsShell, UGI params should be overidden through env vars(-D arg) | Major | . | Brahma Reddy Battula | Brahma Reddy Battula
HADOOP-15181 | Typo in SecureMode.md | Trivial | documentation | Masahiro Tanaka | Masahiro Tanaka
YARN-7737 | prelaunch.err file not found exception on container failure | Major | . | Jonathan Hung | Keqiu Hu
HDFS-13063 | Fix the incorrect spelling in HDFSHighAvailabilityWithQJM.md | Trivial | documentation | Jianfei Jiang | Jianfei Jiang
YARN-7102 | NM heartbeat stuck when responseId overflows MAX_INT | Critical | . | Botong Huang | Botong Huang
HADOOP-15151 | MapFile.fix creates a wrong index file in case of block-compressed data file. | Major | common | Grigori Rybkine | Grigori Rybkine
MAPREDUCE-7020 | Task timeout in uber mode can crash AM | Major | mr-am | Akira Ajisaka | Peter Bacsko
YARN-7698 | A misleading variable’s name in ApplicationAttemptEventDispatcher | Minor | resourcemanager | Jinjiang Ling | Jinjiang Ling
HDFS-13100 | Handle IllegalArgumentException when GETSERVERDEFAULTS is not implemented in webhdfs. | Critical | hdfs, webhdfs | Yongjun Zhang | Yongjun Zhang
YARN-6868 | Add test scope to certain entries in hadoop-yarn-server-resourcemanager pom.xml | Major | yarn | Ray Chiang | Ray Chiang
YARN-7849 | TestMiniYarnClusterNodeUtilization#testUpdateNodeUtilization fails due to heartbeat sync error | Major | test | Jason Lowe | Botong Huang
YARN-7801 | AmFilterInitializer should addFilter after fill all parameters | Critical | . | Sumana Sathish | Wangda Tan
YARN-7890 | NPE during container relaunch | Major | . | Billie Rinaldi | Jason Lowe
HDFS-12935 | Get ambiguous result for DFSAdmin command in HA mode when only one namenode is up | Major | tools | Jianfei Jiang | Jianfei Jiang
HDFS-13120 | Snapshot diff could be corrupted after concat | Major | namenode, snapshots | Xiaoyu Yao | Xiaoyu Yao
HDFS-10453 | ReplicationMonitor thread could stuck for long time due to the race between replication and delete of same file in a large cluster. | Major | namenode | He Xiaoqiao | He Xiaoqiao
HDFS-8693 | refreshNamenodes does not support adding a new standby to a running DN | Critical | datanode, ha | Jian Fang | Ajith S
MAPREDUCE-7052 | TestFixedLengthInputFormat#testFormatCompressedIn is flaky | Major | client, test | Peter Bacsko | Peter Bacsko
HDFS-13112 | Token expiration edits may cause log corruption or deadlock | Critical | namenode | Daryn Sharp | Daryn Sharp
MAPREDUCE-7053 | Timed out tasks can fail to produce thread dump | Major | . | Jason Lowe | Jason Lowe
HADOOP-15206 | BZip2 drops and duplicates records when input split size is small | Major | . | Aki Tanaka | Aki Tanaka
YARN-7947 | Capacity Scheduler intra-queue preemption can NPE for non-schedulable apps | Major | capacity scheduler, scheduler preemption | Eric Payne | Eric Payne
YARN-7945 | Java Doc error in UnmanagedAMPoolManager for branch-2 | Major | . | Rohith Sharma K S | Botong Huang
HADOOP-14903 | Add json-smart explicitly to pom.xml | Major | common | Ray Chiang | Ray Chiang
HDFS-12781 | After Datanode down, In Namenode UI Datanode tab is throwing warning message. | Major | datanode | Harshakiran Reddy | Brahma Reddy Battula
HDFS-12070 | Failed block recovery leaves files open indefinitely and at risk for data loss | Major | . | Daryn Sharp | Kihwal Lee
HADOOP-15251 | Backport HADOOP-13514 (surefire upgrade) to branch-2 | Major | test | Chris Douglas | Chris Douglas
HADOOP-15275 | Incorrect javadoc for return type of RetryPolicy#shouldRetry | Minor | documentation | Nanda kumar | Nanda kumar
YARN-7511 | NPE in ContainerLocalizer when localization failed for running container | Major | nodemanager | Tao Yang | Tao Yang
MAPREDUCE-7023 | TestHadoopArchiveLogs.testCheckFilesAndSeedApps fails on rerun | Minor | test | Gergely Novák | Gergely Novák
HADOOP-15283 | Upgrade from findbugs 3.0.1 to spotbugs 3.1.2 in branch-2 to fix docker image build | Major | . | Xiao Chen | Akira Ajisaka
YARN-7736 | Fix itemization in YARN federation document | Minor | documentation | Akira Ajisaka | Sen Zhao
HDFS-13164 | File not closed if streamer fail with DSQuotaExceededException | Major | hdfs-client | Xiao Chen | Xiao Chen
HDFS-13109 | Support fully qualified hdfs path in EZ commands | Major | hdfs | Hanisha Koneru | Hanisha Koneru
MAPREDUCE-6930 | mapreduce.map.cpu.vcores and mapreduce.reduce.cpu.vcores are both present twice in mapred-default.xml | Major | mrv2 | Daniel Templeton | Sen Zhao
HDFS-12156 | TestFSImage fails without -Pnative | Major | test | Akira Ajisaka | Akira Ajisaka
YARN-7636 | Re-reservation count may overflow when cluster resource exhausted for a long time | Major | capacityscheduler | Tao Yang | Tao Yang
HDFS-12886 | Ignore minReplication for block recovery | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak
HDFS-13296 | GenericTestUtils generates paths with drive letter in Windows and fail webhdfs related test cases | Major | . | Xiao Liang | Xiao Liang
HDFS-13268 | TestWebHdfsFileContextMainOperations fails on Windows | Major | . | Íñigo Goiri | Xiao Liang
YARN-8054 | Improve robustness of the LocalDirsHandlerService MonitoringTimerTask thread | Major | . | Jonathan Eagles | Jonathan Eagles
YARN-7873 | Revert YARN-6078 | Blocker | . | Billie Rinaldi | Billie Rinaldi
HDFS-13195 | DataNode conf page cannot display the current value after reconfig | Minor | datanode | maobaolong | maobaolong
HADOOP-15320 | Remove customized getFileBlockLocations for hadoop-azure and hadoop-azure-datalake | Major | fs/adl, fs/azure | shanyu zhao | shanyu zhao
HADOOP-12862 | LDAP Group Mapping over SSL can not specify trust store | Major | . | Wei-Chiu Chuang | Wei-Chiu Chuang
HDFS-13427 | Fix the section titles of transparent encryption document | Minor | documentation | Akira Ajisaka | Akira Ajisaka
MAPREDUCE-7073 | Optimize TokenCache#obtainTokensForNamenodesInternal | Major | . | Bibin A Chundatt | Bibin A Chundatt

### TESTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-14696 | parallel tests don’t work for Windows | Minor | test | Allen Wittenauer | Allen Wittenauer

### SUB-TASKS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-13481 | User end documents for Aliyun OSS FileSystem | Minor | fs, fs/oss | Genmao Yu | Genmao Yu
HADOOP-13591 | Unit test failure in TestOSSContractGetFileStatus and TestOSSContractRootDir | Major | fs, fs/oss | Genmao Yu | Genmao Yu
HADOOP-13624 | Rename TestAliyunOSSContractDispCp | Major | fs, fs/oss | Kai Zheng | Genmao Yu
HADOOP-14065 | AliyunOSS: oss directory filestatus should use meta time | Major | fs/oss | Fei Hui | Fei Hui
HADOOP-13768 | AliyunOSS: handle the failure in the batch delete operation `deleteDirs`. | Major | fs | Genmao Yu | Genmao Yu
HADOOP-14069 | AliyunOSS: listStatus returns wrong file info | Major | fs/oss | Fei Hui | Fei Hui
HADOOP-13769 | AliyunOSS: update oss sdk version | Major | fs, fs/oss | Genmao Yu | Genmao Yu
HADOOP-14072 | AliyunOSS: Failed to read from stream when seek beyond the download size | Major | fs/oss | Genmao Yu | Genmao Yu
HADOOP-14192 | Aliyun OSS FileSystem contract test should implement getTestBaseDir() | Major | fs/oss | Mingliang Liu | Mingliang Liu
HADOOP-14194 | Aliyun OSS should not use empty endpoint as default | Major | fs/oss | Mingliang Liu | Genmao Yu
HADOOP-14787 | AliyunOSS: Implement the `createNonRecursive` operator | Major | fs, fs/oss | Genmao Yu | Genmao Yu
HADOOP-14649 | Update aliyun-sdk-oss version to 2.8.1 | Major | fs/oss | Ray Chiang | Genmao Yu
HADOOP-14799 | Update nimbus-jose-jwt to 4.41.1 | Major | . | Ray Chiang | Ray Chiang
HADOOP-14997 | Add hadoop-aliyun as dependency of hadoop-cloud-storage | Minor | fs/oss | Genmao Yu | Genmao Yu
HDFS-12801 | RBF: Set MountTableResolver as default file resolver | Minor | . | Íñigo Goiri | Íñigo Goiri
YARN-7430 | Enable user re-mapping for Docker containers by default | Blocker | security, yarn | Eric Yang | Eric Yang
YARN-6128 | Add support for AMRMProxy HA | Major | amrmproxy, nodemanager | Subru Krishnan | Botong Huang
HADOOP-15024 | AliyunOSS: support user agent configuration and include that & Hadoop version information to oss server | Major | fs, fs/oss | Sammi Chen | Sammi Chen
HDFS-12858 | RBF: Add router admin commands usage in HDFS commands reference doc | Minor | documentation | Yiqun Lin | Yiqun Lin
HDFS-12835 | RBF: Fix Javadoc parameter errors | Minor | . | Wei Yan | Wei Yan
YARN-7587 | Skip dispatching opportunistic containers to nodes whose queue is already full | Major | . | Weiwei Yang | Weiwei Yang
HDFS-12396 | Webhdfs file system should get delegation token from kms provider. | Major | encryption, kms, webhdfs | Rushabh S Shah | Rushabh S Shah
YARN-6704 | Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService | Major | . | Botong Huang | Botong Huang
HDFS-12875 | RBF: Complete logic for -readonly option of dfsrouteradmin add command | Major | . | Yiqun Lin | Íñigo Goiri
YARN-7630 | Fix AMRMToken rollover handling in AMRMProxy | Minor | . | Botong Huang | Botong Huang
HDFS-12937 | RBF: Add more unit tests for router admin commands | Major | test | Yiqun Lin | Yiqun Lin
HDFS-12988 | RBF: Mount table entries not properly updated in the local cache | Major | . | Íñigo Goiri | Íñigo Goiri
HADOOP-15156 | backport HADOOP-15086 rename fix to branch-2 | Major | fs/azure | Thomas Marquardt | Thomas Marquardt
YARN-7716 | metricsTimeStart and metricsTimeEnd should be all lower case in the doc | Major | timelinereader | Haibo Chen | Haibo Chen
HDFS-12802 | RBF: Control MountTableResolver cache size | Major | . | Íñigo Goiri | Íñigo Goiri
HADOOP-15027 | AliyunOSS: Support multi-thread pre-read to improve sequential read from Hadoop to Aliyun OSS performance | Major | fs/oss | wujinhu | wujinhu
HDFS-13028 | RBF: Fix spurious TestRouterRpc#testProxyGetStats | Minor | . | Íñigo Goiri | Íñigo Goiri
YARN-5094 | some YARN container events have timestamp of -1 | Critical | . | Sangjin Lee | Haibo Chen
YARN-7782 | Enable user re-mapping for Docker containers in yarn-default.xml | Blocker | security, yarn | Eric Yang | Eric Yang
HDFS-12772 | RBF: Federation Router State State Store internal API | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13042 | RBF: Heartbeat Router State | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13049 | RBF: Inconsistent Router OPTS config in branch-2 and branch-3 | Minor | . | Wei Yan | Wei Yan
HDFS-12574 | Add CryptoInputStream to WebHdfsFileSystem read call. | Major | encryption, kms, webhdfs | Rushabh S Shah | Rushabh S Shah
HDFS-13044 | RBF: Add a safe mode for the Router | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13043 | RBF: Expose the state of the Routers in the federation | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13068 | RBF: Add router admin option to manage safe mode | Major | . | Íñigo Goiri | Yiqun Lin
HDFS-13119 | RBF: Manage unavailable clusters | Major | . | Íñigo Goiri | Yiqun Lin
HDFS-13187 | RBF: Fix Routers information shown in the web UI | Minor | . | Wei Yan | Wei Yan
HDFS-13184 | RBF: Improve the unit test TestRouterRPCClientRetries | Minor | test | Yiqun Lin | Yiqun Lin
HDFS-13199 | RBF: Fix the hdfs router page missing label icon issue | Major | federation, hdfs | maobaolong | maobaolong
HDFS-13214 | RBF: Complete document of Router configuration | Major | . | Tao Jie | Yiqun Lin
HDFS-13230 | RBF: ConnectionManager’s cleanup task will compare each pool’s own active conns with its total conns | Minor | . | Wei Yan | Chao Sun
HDFS-13233 | RBF: MountTableResolver doesn’t return the correct mount point of the given path | Major | hdfs | wangzhiyuan | wangzhiyuan
HDFS-13212 | RBF: Fix router location cache issue | Major | federation, hdfs | Weiwei Wu | Weiwei Wu
HDFS-13232 | RBF: ConnectionPool should return first usable connection | Minor | . | Wei Yan | Ekanth Sethuramalingam
HDFS-13240 | RBF: Update some inaccurate document descriptions | Minor | . | Yiqun Lin | Yiqun Lin
HDFS-11399 | Many tests fails in Windows due to injecting disk failures | Major | . | Yiqun Lin | Yiqun Lin
HDFS-13241 | RBF: TestRouterSafemode failed if the port 8888 is in use | Major | hdfs, test | maobaolong | maobaolong
HDFS-13253 | RBF: Quota management incorrect parent-child relationship judgement | Major | . | Yiqun Lin | Yiqun Lin
HDFS-13226 | RBF: Throw the exception if mount table entry validated failed | Major | hdfs | maobaolong | maobaolong
HADOOP-15308 | TestConfiguration fails on Windows because of paths | Major | test | Íñigo Goiri | Xiao Liang
HDFS-12773 | RBF: Improve State Store FS implementation | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13198 | RBF: RouterHeartbeatService throws out CachedStateStore related exceptions when starting router | Minor | . | Wei Yan | Wei Yan
HDFS-13224 | RBF: Resolvers to support mount points across multiple subclusters | Major | . | Íñigo Goiri | Íñigo Goiri
HADOOP-15262 | AliyunOSS: move files under a directory in parallel when rename a directory | Major | fs/oss | wujinhu | wujinhu
HDFS-13215 | RBF: Move Router to its own module | Major | . | Íñigo Goiri | Wei Yan
HDFS-13250 | RBF: Router to manage requests across multiple subclusters | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13318 | RBF: Fix FindBugs in hadoop-hdfs-rbf | Minor | . | Íñigo Goiri | Ekanth Sethuramalingam
HDFS-12792 | RBF: Test Router-based federation using HDFSContract | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-12512 | RBF: Add WebHDFS | Major | fs | Íñigo Goiri | Wei Yan
HDFS-13291 | RBF: Implement available space based OrderResolver | Major | . | Yiqun Lin | Yiqun Lin
HDFS-13204 | RBF: Optimize name service safe mode icon | Minor | . | liuhongtong | liuhongtong
HDFS-13352 | RBF: Add xsl stylesheet for hdfs-rbf-default.xml | Major | documentation | Takanobu Asanuma | Takanobu Asanuma
YARN-8010 | Add config in FederationRMFailoverProxy to not bypass facade cache when failing over | Minor | . | Botong Huang | Botong Huang
HDFS-13347 | RBF: Cache datanode reports | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-13289 | RBF: TestConnectionManager#testCleanup() test case need correction | Minor | . | Dibyendu Karmakar | Dibyendu Karmakar
HDFS-13364 | RBF: Support NamenodeProtocol in the Router | Major | . | Íñigo Goiri | Íñigo Goiri
HADOOP-14651 | Update okhttp version to 2.7.5 | Major | fs/adl | Ray Chiang | Ray Chiang
HADOOP-14999 | AliyunOSS: provide one asynchronous multi-part based uploading mechanism | Major | fs/oss | Genmao Yu | Genmao Yu
YARN-7810 | TestDockerContainerRuntime test failures due to UID lookup of a non-existent user | Major | . | Shane Kumpf | Shane Kumpf

### OTHER:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-15149 | CryptoOutputStream should implement StreamCapabilities | Major | fs | Mike Drob | Xiao Chen
YARN-7691 | Add Unit Tests for ContainersLauncher | Major | . | Sampada Dehankar | Sampada Dehankar
HADOOP-15177 | Update the release year to 2018 | Blocker | build | Akira Ajisaka | Bharat Viswanadham