Apache Hadoop 3.5.0 – Apache Hadoop Changelog

##### General

- Overview
- Single Node Setup
- Cluster Setup
- Commands Reference
- FileSystem Shell
- Compatibility Specification
- Downstream Developer's Guide
- Admin Compatibility Guide
- Interface Classification
- FileSystem Specification

##### Common

- CLI Mini Cluster
- Fair Call Queue
- Native Libraries
- Proxy User
- Rack Awareness
- Secure Mode
- Service Level Authorization
- HTTP Authentication
- Credential Provider API
- Hadoop KMS
- Tracing
- Unix Shell Guide
- Registry
- Async Profiler

##### HDFS

- Architecture
- User Guide
- Commands Reference
- NameNode HA With QJM
- NameNode HA With NFS
- Observer NameNode
- Federation
- ViewFs
- ViewFsOverloadScheme
- Snapshots
- Edits Viewer
- Image Viewer
- Permissions and HDFS
- Quotas and HDFS
- libhdfs (C API)
- WebHDFS (REST API)
- HttpFS
- Short Circuit Local Reads
- Centralized Cache Management
- NFS Gateway
- Rolling Upgrade
- Extended Attributes
- Transparent Encryption
- Multihoming
- Storage Policies
- Memory Storage Support
- Synthetic Load Generator
- Erasure Coding
- Disk Balancer
- Upgrade Domain
- DataNode Admin
- Router Federation
- Provided Storage

##### MapReduce

- Tutorial
- Commands Reference
- Encrypted Shuffle
- Pluggable Shuffle/Sort
- Distributed Cache Deploy
- Support for YARN Shared Cache

##### MapReduce REST APIs

- MR Application Master
- MR History Server

##### YARN

- Architecture
- Commands Reference
- Capacity Scheduler
- Fair Scheduler
- ResourceManager Restart
- ResourceManager HA
- Resource Model
- Node Labels
- Node Attributes
- Web Application Proxy
- Timeline Server
- Timeline Service V.2
- Writing YARN Applications
- YARN Application Security
- NodeManager
- Running Applications in Docker Containers
- Running Applications in runC Containers
- Using CGroups
- Secure Containers
- Reservation System
- Graceful Decommission
- Opportunistic Containers
- YARN Federation
- Shared Cache
- Using GPU
- Using FPGA
- Placement Constraints
- YARN UI2

##### YARN REST APIs

- Introduction
- Resource Manager
- Node Manager
- Timeline Server
- Timeline Service V.2

##### YARN Service

- Overview
- QuickStart
- Concepts
- Yarn Service API
- Service Discovery
- System Services

##### Hadoop Compatible File Systems

- Aliyun OSS
- Amazon S3
- Azure Blob Storage
- Azure Data Lake Storage
- Tencent COS
- Huaweicloud OBS
- VolcanoEngine TOS

##### Auth

- Overview
- Examples
- Configuration
- Building

##### Tools

- Hadoop Streaming
- Hadoop Archives
- Hadoop Archive Logs
- DistCp
- HDFS Federation Balance
- GridMix
- Rumen
- Resource Estimator Service
- Scheduler Load Simulator
- Hadoop Benchmarking
- Dynamometer

##### Reference

- Changelog and Release Notes
- Java API docs
- Unix Shell API
- Metrics

##### Configuration

- core-default.xml
- hdfs-default.xml
- hdfs-rbf-default.xml
- mapred-default.xml
- yarn-default.xml
- kms-default.xml
- httpfs-default.xml
- Deprecated Properties

# “Apache Hadoop” Changelog

## Release 2.9.2 - 2018-11-19

### INCOMPATIBLE CHANGES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---

### IMPORTANT ISSUES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---

### NEW FEATURES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-13283 | Percentage based Reserved Space Calculation for DataNode | Major | datanode, hdfs | Lukas Majercak | Lukas Majercak

### IMPROVEMENTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-14987 | Improve KMSClientProvider log around delegation token checking | Major | . | Xiaoyu Yao | Xiaoyu Yao
YARN-7274 | Ability to disable elasticity at leaf queue level | Major | capacityscheduler | Scott Brokaw | Zian Chen
HADOOP-15394 | Backport PowerShell NodeFencer HADOOP-14309 to branch-2 | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-13462 | Add BIND_HOST configuration for JournalNode’s HTTP and RPC Servers | Major | hdfs, journal-node | Lukas Majercak | Lukas Majercak
HADOOP-14841 | Kms client should disconnect if unable to get output stream from connection. | Major | kms | Xiao Chen | Rushabh S Shah
HDFS-13272 | DataNodeHttpServer to have configurable HttpServer2 threads | Major | datanode | Erik Krogen | Erik Krogen
HADOOP-15441 | Log kms url and token service at debug level. | Minor | . | Wei-Chiu Chuang | Gabor Bota
HADOOP-15486 | Make NetworkTopology#netLock fair | Major | net | Nanda kumar | Nanda kumar
HADOOP-15449 | Increase default timeout of ZK session to avoid frequent NameNode failover | Critical | common | Karthik Palanisamy | Karthik Palanisamy
HDFS-13602 | Add checkOperation(WRITE) checks in FSNamesystem | Major | ha, namenode | Erik Krogen | Chao Sun
HDFS-13653 | Make dfs.client.failover.random.order a per nameservice configuration | Major | federation | Ekanth Sethuramalingam | Ekanth Sethuramalingam
HDFS-13686 | Add overall metrics for FSNamesystemLock | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak
HDFS-13714 | Fix TestNameNodePrunesMissingStorages test failures on Windows | Major | hdfs, namenode, test | Lukas Majercak | Lukas Majercak
HDFS-13719 | Docs around dfs.image.transfer.timeout are misleading | Major | documentation | Kitti Nanasi | Kitti Nanasi
HDFS-11060 | make DEFAULT_MAX_CORRUPT_FILEBLOCKS_RETURNED configurable | Minor | hdfs | Lantao Jin | Lantao Jin
HDFS-13813 | Exit NameNode if dangling child inode is detected when saving FsImage | Major | hdfs, namenode | Siyao Meng | Siyao Meng
HDFS-13821 | RBF: Add dfs.federation.router.mount-table.cache.enable so that users can disable cache | Major | hdfs | Fei Hui | Fei Hui
HADOOP-15689 | Add “*.patch” into .gitignore file of branch-2 | Major | . | Rui Gao | Rui Gao
HDFS-13854 | RBF: The ProcessingAvgTime and ProxyAvgTime should display by JMX with ms unit. | Major | federation, hdfs | yanghuafeng | yanghuafeng
YARN-8051 | TestRMEmbeddedElector#testCallbackSynchronization is flakey | Major | test | Robert Kanter | Robert Kanter
HDFS-13857 | RBF: Choose to enable the default nameservice to read/write files | Major | federation, hdfs | yanghuafeng | yanghuafeng
HDFS-13812 | Fix the inconsistent default refresh interval on Caching documentation | Trivial | documentation | BELUGA BEHR | Hrishikesh Gadre
HDFS-13902 | Add JMX, conf and stacks menus to the datanode page | Minor | datanode | fengchuang | fengchuang

### BUG FIXES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-15121 | Encounter NullPointerException when using DecayRpcScheduler | Major | . | Tao Jie | Tao Jie
YARN-7765 | [Atsv2] GSSException: No valid credentials provided - Failed to find any Kerberos tgt thrown by Timelinev2Client & HBaseClient in NM | Blocker | . | Sumana Sathish | Rohith Sharma K S
MAPREDUCE-7027 | HadoopArchiveLogs shouldn’t delete the original logs if the HAR creation fails | Critical | harchive | Gergely Novák | Gergely Novák
HDFS-10803 | TestBalancerWithMultipleNameNodes#testBalancing2OutOf3Blockpools fails intermittently due to no free space available | Major | . | Yiqun Lin | Yiqun Lin
YARN-8068 | Application Priority field causes NPE in app timeline publish when Hadoop 2.7 based clients to 2.8+ | Blocker | yarn | Sunil Govindan | Sunil Govindan
HADOOP-15317 | Improve NetworkTopology chooseRandom’s loop | Major | . | Xiao Chen | Xiao Chen
HADOOP-15375 | Branch-2 pre-commit failed to build docker image | Major | . | Xiao Chen | Xiao Chen
HADOOP-15357 | Configuration.getPropsWithPrefix no longer does variable substitution | Major | . | Jim Brennan | Jim Brennan
HDFS-7101 | Potential null dereference in DFSck#doWork() | Minor | . | Ted Yu | skrho
YARN-8120 | JVM can crash with SIGSEGV when exiting due to custom leveldb logger | Major | nodemanager, resourcemanager | Jason Lowe | Jason Lowe
YARN-8147 | TestClientRMService#testGetApplications sporadically fails | Major | test | Jason Lowe | Jason Lowe
HADOOP-14970 | MiniHadoopClusterManager doesn’t respect lack of format option | Minor | . | Erik Krogen | Erik Krogen
HDFS-12828 | OIV ReverseXML Processor fails with escaped characters | Critical | hdfs | Erik Krogen | Erik Krogen
HADOOP-15180 | branch-2 : daemon processes’ sysout overwrites ‘ulimit -a’ in daemon’s out file | Minor | scripts | Ranith Sardar | Ranith Sardar
HADOOP-15396 | Some java source files are executable | Minor | . | Akira Ajisaka | Shashikant Banerjee
YARN-7786 | NullPointerException while launching ApplicationMaster | Major | . | lujie | lujie
HDFS-10183 | Prevent race condition during class initialization | Minor | fs | Pavel Avgustinov | Pavel Avgustinov
HDFS-13408 | MiniDFSCluster to support being built on randomized base directory | Major | test | Xiao Liang | Xiao Liang
HADOOP-15390 | Yarn RM logs flooded by DelegationTokenRenewer trying to renew KMS tokens | Critical | . | Xiao Chen | Xiao Chen
HDFS-13336 | Test cases of TestWriteToReplica failed in windows | Major | . | Xiao Liang | Xiao Liang
HADOOP-15385 | Many tests are failing in hadoop-distcp project in branch-2 | Critical | tools/distcp | Rushabh S Shah | Jason Lowe
HDFS-13509 | Bug fix for breakHardlinks() of ReplicaInfo/LocalReplica, and fix TestFileAppend failures on Windows | Major | . | Xiao Liang | Xiao Liang
MAPREDUCE-7073 | Optimize TokenCache#obtainTokensForNamenodesInternal | Major | . | Bibin A Chundatt | Bibin A Chundatt
YARN-8232 | RMContainer lost queue name when RM HA happens | Major | resourcemanager | Hu Ziqian | Hu Ziqian
HDFS-13537 | TestHdfsHelper does not generate jceks path properly for relative path in Windows | Major | . | Xiao Liang | Xiao Liang
HADOOP-15446 | WASB: PageBlobInputStream.skip breaks HBASE replication | Major | fs/azure | Thomas Marquardt | Thomas Marquardt
YARN-7003 | DRAINING state of queues is not recovered after RM restart | Major | capacityscheduler | Tao Yang | Tao Yang
YARN-8244 | TestContainerSchedulerQueuing.testStartMultipleContainers failed | Major | . | Miklos Szegedi | Jim Brennan
HDFS-13581 | DN UI logs link is broken when https is enabled | Minor | datanode | Namit Maheshwari | Shashikant Banerjee
HDFS-13586 | Fsync fails on directories on Windows | Critical | datanode, hdfs | Lukas Majercak | Lukas Majercak
HDFS-13590 | Backport HDFS-12378 to branch-2 | Major | datanode, hdfs, test | Lukas Majercak | Lukas Majercak
HADOOP-15450 | Avoid fsync storm triggered by DiskChecker and handle disk full situation | Blocker | . | Kihwal Lee | Arpit Agarwal
HDFS-13588 | Fix TestFsDatasetImpl test failures on Windows | Major | . | Xiao Liang | Xiao Liang
YARN-8344 | Missing nm.stop() in TestNodeManagerResync to fix testKillContainersOnResync | Major | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
YARN-8327 | Fix TestAggregatedLogFormat#testReadAcontainerLogs1 on Windows | Major | log-aggregation | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
YARN-8346 | Upgrading to 3.1 kills running containers with error “Opportunistic container queue is full” | Blocker | . | Rohith Sharma K S | Jason Lowe
HDFS-13618 | Fix TestDataNodeFaultInjector test failures on Windows | Major | test | Xiao Liang | Xiao Liang
HADOOP-15473 | Configure serialFilter in KeyProvider to avoid UnrecoverableKeyException caused by JDK-8189997 | Critical | kms | Gabor Bota | Gabor Bota
MAPREDUCE-7103 | Fix TestHistoryViewerPrinter on windows due to a mismatch line separator | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
YARN-8359 | Exclude containermanager.linux test classes on Windows | Major | . | Giovanni Matteo Fumarola | Jason Lowe
HDFS-13664 | Refactor ConfiguredFailoverProxyProvider to make inheritance easier | Minor | hdfs-client | Chao Sun | Chao Sun
HDFS-13667 | Typo: Marking all “datandoes” as stale | Trivial | namenode | Wei-Chiu Chuang | Nanda kumar
YARN-8405 | RM zk-state-store.parent-path ACLs has been changed since HADOOP-14773 | Major | . | Rohith Sharma K S | Íñigo Goiri
MAPREDUCE-7108 | TestFileOutputCommitter fails on Windows | Minor | test | Zuoming Zhang | Zuoming Zhang
YARN-8404 | Timeline event publish need to be async to avoid Dispatcher thread leak in case ATS is down | Blocker | . | Rohith Sharma K S | Rohith Sharma K S
HDFS-13675 | Speed up TestDFSAdminWithHA | Major | hdfs, namenode | Lukas Majercak | Lukas Majercak
HDFS-13673 | TestNameNodeMetrics fails on Windows | Minor | test | Zuoming Zhang | Zuoming Zhang
HDFS-13676 | TestEditLogRace fails on Windows | Minor | test | Zuoming Zhang | Zuoming Zhang
HADOOP-15523 | Shell command timeout given is in seconds whereas it is taken as millisec while scheduling | Major | . | Bilwa S T | Bilwa S T
YARN-8444 | NodeResourceMonitor crashes on bad swapFree value | Major | . | Jim Brennan | Jim Brennan
YARN-8443 | Total #VCores in cluster metrics is wrong when CapacityScheduler reserved some containers | Major | webapp | Tao Yang | Tao Yang
YARN-8457 | Compilation is broken with -Pyarn-ui | Major | webapp | Sunil Govindan | Sunil Govindan
YARN-8401 | [UI2] new ui is not accessible with out internet connection | Blocker | . | Bibin A Chundatt | Bibin A Chundatt
YARN-8451 | Multiple NM heartbeat thread created when a slow NM resync with RM | Major | nodemanager | Botong Huang | Botong Huang
HADOOP-15548 | Randomize local dirs | Minor | . | Jim Brennan | Jim Brennan
YARN-8473 | Containers being launched as app tears down can leave containers in NEW state | Major | nodemanager | Jason Lowe | Jason Lowe
HDFS-13729 | Fix broken links to RBF documentation | Minor | documentation | jwhitter | Gabor Bota
YARN-8518 | test-container-executor test_is_empty() is broken | Major | . | Jim Brennan | Jim Brennan
YARN-8515 | container-executor can crash with SIGPIPE after nodemanager restart | Major | . | Jim Brennan | Jim Brennan
YARN-8421 | when moving app, activeUsers is increased, even though app does not have outstanding request | Major | . | kyungwan nam |
HADOOP-15614 | TestGroupsCaching.testExceptionOnBackgroundRefreshHandled reliably fails | Major | . | Kihwal Lee | Weiwei Yang
YARN-8577 | Fix the broken anchor in SLS site-doc | Minor | documentation | Weiwei Yang | Weiwei Yang
YARN-4606 | CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps | Critical | capacity scheduler, capacityscheduler | Karam Singh | Manikandan R
HADOOP-15637 | LocalFs#listLocatedStatus does not filter out hidden .crc files | Minor | fs | Erik Krogen | Erik Krogen
HADOOP-15644 | Hadoop Docker Image Pip Install Fails on branch-2 | Critical | build | Haibo Chen | Haibo Chen
YARN-8331 | Race condition in NM container launched after done | Major | . | Yang Wang | Pradeep Ambati
HDFS-13758 | DatanodeManager should throw exception if it has BlockRecoveryCommand but the block is not under construction | Major | namenode | Wei-Chiu Chuang | chencan
YARN-8612 | Fix NM Collector Service Port issue in YarnConfiguration | Major | ATSv2 | Prabha Manepalli | Prabha Manepalli
HADOOP-15674 | Test failure TestSSLHttpServer.testExcludedCiphers with TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 cipher suite | Major | common | Gabor Bota | Szilard Nemeth
YARN-8640 | Restore previous state in container-executor after failure | Major | . | Jim Brennan | Jim Brennan
HADOOP-14314 | The OpenSolaris taxonomy link is dead in InterfaceClassification.md | Major | documentation | Daniel Templeton | Rui Gao
YARN-8649 | NPE in localizer hearbeat processing if a container is killed while localizing | Major | . | lujie | lujie
HADOOP-10219 | ipc.Client.setupIOstreams() needs to check for ClientCache.stopClient requested shutdowns | Major | ipc | Steve Loughran | Kihwal Lee
MAPREDUCE-7131 | Job History Server has race condition where it moves files from intermediate to finished but thinks file is in intermediate | Major | . | Anthony Hsu | Anthony Hsu
HDFS-13836 | RBF: Handle mount table znode with null value | Major | federation, hdfs | yanghuafeng | yanghuafeng
YARN-8709 | CS preemption monitor always fails since one under-served queue was deleted | Major | capacityscheduler, scheduler preemption | Tao Yang | Tao Yang
HDFS-13051 | Fix dead lock during async editlog rolling if edit queue is full | Major | namenode | zhangwei | Daryn Sharp
YARN-8729 | Node status updater thread could be lost after it is restarted | Critical | nodemanager | Tao Yang | Tao Yang
HDFS-13914 | Fix DN UI logs link broken when https is enabled after HDFS-13902 | Minor | datanode | Jianfei Jiang | Jianfei Jiang
MAPREDUCE-7133 | History Server task attempts REST API returns invalid data | Major | jobhistoryserver | Oleksandr Shevchenko | Oleksandr Shevchenko
YARN-8720 | CapacityScheduler does not enforce max resource allocation check at queue level | Major | capacity scheduler, capacityscheduler, resourcemanager | Tarun Parimi | Tarun Parimi
HDFS-13844 | Fix the fmt_bytes function in the dfs-dust.js | Minor | hdfs, ui | yanghuafeng | yanghuafeng
HADOOP-15755 | StringUtils#createStartupShutdownMessage throws NPE when args is null | Major | . | Lokesh Jain | Dinesh Chitlangia
MAPREDUCE-3801 | org.apache.hadoop.mapreduce.v2.app.TestRuntimeEstimators.testExponentialEstimator fails intermittently | Major | mrv2 | Robert Joseph Evans | Jason Lowe
MAPREDUCE-7137 | MRAppBenchmark.benchmark1() fails with NullPointerException | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko
MAPREDUCE-7138 | ThrottledContainerAllocator in MRAppBenchmark should implement RMHeartbeatHandler | Minor | test | Oleksandr Shevchenko | Oleksandr Shevchenko
HDFS-13908 | TestDataNodeMultipleRegistrations is flaky | Major | . | Íñigo Goiri | Ayush Saxena
YARN-8804 | resourceLimits may be wrongly calculated when leaf-queue is blocked in cluster with 3+ level queues | Critical | capacityscheduler | Tao Yang | Tao Yang
YARN-8774 | Memory leak when CapacityScheduler allocates from reserved container with non-default label | Critical | capacityscheduler | Tao Yang | Tao Yang
HADOOP-15817 | Reuse Object Mapper in KMSJSONReader | Major | kms | Jonathan Eagles | Jonathan Eagles
HADOOP-15820 | ZStandardDecompressor native code sets an integer field as a long | Blocker | . | Jason Lowe | Jason Lowe
HDFS-13964 | RBF: TestRouterWebHDFSContractAppend fails with No Active Namenode under nameservice | Major | . | Ayush Saxena | Ayush Saxena
HADOOP-15835 | Reuse Object Mapper in KMSJSONWriter | Major | . | Jonathan Eagles | Jonathan Eagles
HDFS-13976 | Backport HDFS-12813 to branch-2.9 | Major | hdfs, hdfs-client | Lukas Majercak | Lukas Majercak
HADOOP-15679 | ShutdownHookManager shutdown time needs to be configurable & extended | Major | util | Steve Loughran | Steve Loughran
HDFS-13802 | RBF: Remove FSCK from Router Web UI | Major | . | Fei Hui | Fei Hui
HADOOP-15859 | ZStandardDecompressor.c mistakes a class for an instance | Blocker | . | Ben Lau | Jason Lowe
HADOOP-15850 | CopyCommitter#concatFileChunks should check that the blocks per chunk is not 0 | Critical | tools/distcp | Ted Yu | Ted Yu
YARN-7502 | Nodemanager restart docs should describe nodemanager supervised property | Major | documentation | Jason Lowe | Suma Shivaprasad
HADOOP-15866 | Renamed HADOOP_SECURITY_GROUP_SHELL_COMMAND_TIMEOUT keys break compatibility | Blocker | . | Wei-Chiu Chuang | Wei-Chiu Chuang
HADOOP-15822 | zstd compressor can fail with a small output buffer | Major | . | Jason Lowe | Jason Lowe
HADOOP-15899 | Update AWS Java SDK versions in NOTICE.txt | Major | . | Akira Ajisaka | Akira Ajisaka
HADOOP-15900 | Update JSch versions in LICENSE.txt | Major | . | Akira Ajisaka | Akira Ajisaka
YARN-8858 | CapacityScheduler should respect maximum node resource when per-queue maximum-allocation is being used. | Major | . | Sumana Sathish | Wangda Tan
YARN-8233 | NPE in CapacityScheduler#tryCommit when handling allocate/reserve proposal whose allocatedOrReservedContainer is null | Critical | capacityscheduler | Tao Yang | Tao Yang
HADOOP-15923 | create-release script should set max-cache-ttl as well as default-cache-ttl for gpg-agent | Blocker | build | Akira Ajisaka | Akira Ajisaka

### TESTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-13337 | Backport HDFS-4275 to branch-2.9 | Minor | . | Íñigo Goiri | Xiao Liang
HDFS-13503 | Fix TestFsck test failures on Windows | Major | hdfs | Xiao Liang | Xiao Liang
HDFS-13542 | TestBlockManager#testNeededReplicationWhileAppending fails due to improper cluster shutdown in TestBlockManager#testBlockManagerMachinesArray on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13551 | TestMiniDFSCluster#testClusterSetStorageCapacity does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu
HDFS-11700 | TestHDFSServerPorts#testBackupNodePorts doesn’t pass on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13548 | TestResolveHdfsSymlink#testFcResolveAfs fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13567 | TestNameNodeMetrics#testGenerateEDEKTime,TestNameNodeMetrics#testResourceCheck should use a different cluster basedir | Minor | . | Anbang Hu | Anbang Hu
HDFS-13557 | TestDFSAdmin#testListOpenFiles fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13550 | TestDebugAdmin#testComputeMetaCommand fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13559 | TestBlockScanner does not close TestContext properly | Minor | . | Anbang Hu | Anbang Hu
HDFS-13570 | TestQuotaByStorageType,TestQuota,TestDFSOutputStream fail on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13558 | TestDatanodeHttpXFrame does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu
HDFS-13554 | TestDatanodeRegistration#testForcedRegistration does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu
HDFS-13556 | TestNestedEncryptionZones does not shut down cluster | Minor | . | Anbang Hu | Anbang Hu
HDFS-13560 | Insufficient system resources exist to complete the requested service for some tests on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13592 | TestNameNodePrunesMissingStorages#testNameNodePrunesUnreportedStorages does not shut down cluster properly | Minor | . | Anbang Hu | Anbang Hu
HDFS-13593 | TestBlockReaderLocalLegacy#testBlockReaderLocalLegacyWithAppend fails on Windows | Minor | test | Anbang Hu | Anbang Hu
HDFS-13587 | TestQuorumJournalManager fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13620 | Randomize the test directory path for TestHDFSFileSystemContract | Minor | . | Anbang Hu | Anbang Hu
HDFS-13591 | TestDFSShell#testSetrepLow fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13632 | Randomize baseDir for MiniJournalCluster in MiniQJMHACluster for TestDFSAdminWithHA | Minor | . | Anbang Hu | Anbang Hu
MAPREDUCE-7102 | Fix TestJavaSerialization for Windows due a mismatch line separator | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
HDFS-13652 | Randomize baseDir for MiniDFSCluster in TestBlockScanner | Minor | . | Anbang Hu | Anbang Hu
YARN-8370 | Some Node Manager tests fail on Windows due to improper path/file separator | Minor | . | Anbang Hu | Anbang Hu
YARN-8422 | TestAMSimulator failing with NPE | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
HADOOP-15532 | TestBasicDiskValidator fails with NoSuchFileException | Minor | . | Íñigo Goiri | Giovanni Matteo Fumarola
HDFS-13563 | TestDFSAdminWithHA times out on Windows | Minor | . | Anbang Hu | Lukas Majercak
HDFS-13681 | Fix TestStartup.testNNFailToStartOnReadOnlyNNDir test failure on Windows | Major | test | Xiao Liang | Xiao Liang

### SUB-TASKS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HDFS-13299 | RBF : Fix compilation error in branch-2 (TestMultipleDestinationResolver) | Blocker | . | Brahma Reddy Battula | Brahma Reddy Battula
HDFS-13353 | RBF: TestRouterWebHDFSContractCreate failed | Major | test | Takanobu Asanuma | Takanobu Asanuma
YARN-8110 | AMRMProxy recover should catch for all throwable to avoid premature exit | Major | . | Botong Huang | Botong Huang
HDFS-13402 | RBF: Fix java doc for StateStoreFileSystemImpl | Minor | hdfs | Yiran Wu | Yiran Wu
HDFS-13410 | RBF: Support federation with no subclusters | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-13384 | RBF: Improve timeout RPC call mechanism | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-13045 | RBF: Improve error message returned from subcluster | Minor | . | Wei Yan | Íñigo Goiri
HDFS-13428 | RBF: Remove LinkedList From StateStoreFileImpl.java | Trivial | federation | BELUGA BEHR | BELUGA BEHR
HDFS-13386 | RBF: Wrong date information in list file(-ls) result | Minor | . | Dibyendu Karmakar | Dibyendu Karmakar
YARN-7810 | TestDockerContainerRuntime test failures due to UID lookup of a non-existent user | Major | . | Shane Kumpf | Shane Kumpf
HDFS-13435 | RBF: Improve the error loggings for printing the stack trace | Major | . | Yiqun Lin | Yiqun Lin
YARN-7189 | Container-executor doesn’t remove Docker containers that error out early | Major | yarn | Eric Badger | Eric Badger
HDFS-13466 | RBF: Add more router-related information to the UI | Minor | . | Wei Yan | Wei Yan
HDFS-13453 | RBF: getMountPointDates should fetch latest subdir time/date when parent dir is not present but /parent/child dirs are present in mount table | Major | . | Dibyendu Karmakar | Dibyendu Karmakar
HDFS-13478 | RBF: Disabled Nameservice store API | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13490 | RBF: Fix setSafeMode in the Router | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13484 | RBF: Disable Nameservices from the federation | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13326 | RBF: Improve the interfaces to modify and view mount tables | Minor | . | Wei Yan | Gang Li
HDFS-13499 | RBF: Show disabled name services in the UI | Minor | . | Íñigo Goiri | Íñigo Goiri
HDFS-13508 | RBF: Normalize paths (automatically) when adding, updating, removing or listing mount table entries | Minor | . | Ekanth Sethuramalingam | Ekanth Sethuramalingam
HDFS-13434 | RBF: Fix dead links in RBF document | Major | documentation | Akira Ajisaka | Chetna Chaudhari
HDFS-13488 | RBF: Reject requests when a Router is overloaded | Major | . | Íñigo Goiri | Íñigo Goiri
HDFS-13525 | RBF: Add unit test TestStateStoreDisabledNameservice | Major | . | Yiqun Lin | Yiqun Lin
YARN-8253 | HTTPS Ats v2 api call fails with “bad HTTP parsed” | Critical | ATSv2 | Yesha Vora | Charan Hebri
HADOOP-15454 | TestRollingFileSystemSinkWithLocal fails on Windows | Major | test | Xiao Liang | Xiao Liang
HADOOP-15498 | TestHadoopArchiveLogs (#testGenerateScript, #testPrepareWorkingDir) fails on Windows | Minor | . | Anbang Hu | Anbang Hu
HADOOP-15497 | TestTrash should use proper test path to avoid failing on Windows | Minor | . | Anbang Hu | Anbang Hu
HDFS-13637 | RBF: Router fails when threadIndex (in ConnectionPool) wraps around Integer.MIN_VALUE | Critical | federation | CR Hota | CR Hota
YARN-4781 | Support intra-queue preemption for fairness ordering policy. | Major | scheduler | Wangda Tan | Eric Payne
HDFS-13281 | Namenode#createFile should be /.reserved/raw/ aware. | Critical | encryption | Rushabh S Shah | Rushabh S Shah
YARN-4677 | RMNodeResourceUpdateEvent update from scheduler can lead to race condition | Major | graceful, resourcemanager, scheduler | Brook Zhou | Wilfred Spiegelenburg
HADOOP-15529 | ContainerLaunch#testInvalidEnvVariableSubstitutionType is not supported in Windows | Minor | . | Giovanni Matteo Fumarola | Giovanni Matteo Fumarola
HADOOP-15458 | TestLocalFileSystem#testFSOutputStreamBuilder fails on Windows | Minor | test | Xiao Liang | Xiao Liang
YARN-8481 | AMRMProxyPolicies should accept heartbeat response from new/unknown subclusters | Minor | amrmproxy, federation | Botong Huang | Botong Huang
HDFS-13475 | RBF: Admin cannot enforce Router enter SafeMode | Major | . | Wei Yan | Chao Sun
HDFS-13733 | RBF: Add Web UI configurations and descriptions to RBF document | Minor | documentation | Takanobu Asanuma | Takanobu Asanuma
HDFS-13743 | RBF: Router throws NullPointerException due to the invalid initialization of MountTableResolver | Major | . | Takanobu Asanuma | Takanobu Asanuma
HDFS-13750 | RBF: Router ID in RouterRpcClient is always null | Major | . | Takanobu Asanuma | Takanobu Asanuma
HDFS-13848 | Refactor NameNode failover proxy providers | Major | ha, hdfs-client | Konstantin Shvachko | Konstantin Shvachko
HADOOP-15699 | Fix some of testContainerManager failures in Windows | Major | . | Botong Huang | Botong Huang
HADOOP-15731 | TestDistributedShell fails on Windows | Major | . | Botong Huang | Botong Huang
HADOOP-15759 | AliyunOSS: update oss-sdk version to 3.0.0 | Major | fs/oss | wujinhu | wujinhu
HADOOP-15671 | AliyunOSS: Support Assume Roles in AliyunOSS | Major | fs/oss | wujinhu | wujinhu
HDFS-13790 | RBF: Move ClientProtocol APIs to its own module | Major | . | Íñigo Goiri | Chao Sun
HADOOP-15607 | AliyunOSS: fix duplicated partNumber issue in AliyunOSSBlockOutputStream | Critical | . | wujinhu | wujinhu
HADOOP-15868 | AliyunOSS: update document for properties of multiple part download, multiple part upload and directory copy | Major | fs/oss | wujinhu | wujinhu

### OTHER:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---