Apache Hadoop 3.5.0 – Apache Hadoop Changelog

##### General

- Overview
- Single Node Setup
- Cluster Setup
- Commands Reference
- FileSystem Shell
- Compatibility Specification
- Downstream Developer's Guide
- Admin Compatibility Guide
- Interface Classification
- FileSystem Specification

##### Common

- CLI Mini Cluster
- Fair Call Queue
- Native Libraries
- Proxy User
- Rack Awareness
- Secure Mode
- Service Level Authorization
- HTTP Authentication
- Credential Provider API
- Hadoop KMS
- Tracing
- Unix Shell Guide
- Registry
- Async Profiler

##### HDFS

- Architecture
- User Guide
- Commands Reference
- NameNode HA With QJM
- NameNode HA With NFS
- Observer NameNode
- Federation
- ViewFs
- ViewFsOverloadScheme
- Snapshots
- Edits Viewer
- Image Viewer
- Permissions and HDFS
- Quotas and HDFS
- libhdfs (C API)
- WebHDFS (REST API)
- HttpFS
- Short Circuit Local Reads
- Centralized Cache Management
- NFS Gateway
- Rolling Upgrade
- Extended Attributes
- Transparent Encryption
- Multihoming
- Storage Policies
- Memory Storage Support
- Synthetic Load Generator
- Erasure Coding
- Disk Balancer
- Upgrade Domain
- DataNode Admin
- Router Federation
- Provided Storage

##### MapReduce

- Tutorial
- Commands Reference
- Encrypted Shuffle
- Pluggable Shuffle/Sort
- Distributed Cache Deploy
- Support for YARN Shared Cache

##### MapReduce REST APIs

- MR Application Master
- MR History Server

##### YARN

- Architecture
- Commands Reference
- Capacity Scheduler
- Fair Scheduler
- ResourceManager Restart
- ResourceManager HA
- Resource Model
- Node Labels
- Node Attributes
- Web Application Proxy
- Timeline Server
- Timeline Service V.2
- Writing YARN Applications
- YARN Application Security
- NodeManager
- Running Applications in Docker Containers
- Running Applications in runC Containers
- Using CGroups
- Secure Containers
- Reservation System
- Graceful Decommission
- Opportunistic Containers
- YARN Federation
- Shared Cache
- Using GPU
- Using FPGA
- Placement Constraints
- YARN UI2

##### YARN REST APIs

- Introduction
- Resource Manager
- Node Manager
- Timeline Server
- Timeline Service V.2

##### YARN Service

- Overview
- QuickStart
- Concepts
- Yarn Service API
- Service Discovery
- System Services

##### Hadoop Compatible File Systems

- Aliyun OSS
- Amazon S3
- Azure Blob Storage
- Azure Data Lake Storage
- Tencent COS
- Huaweicloud OBS
- VolcanoEngine TOS

##### Auth

- Overview
- Examples
- Configuration
- Building

##### Tools

- Hadoop Streaming
- Hadoop Archives
- Hadoop Archive Logs
- DistCp
- HDFS Federation Balance
- GridMix
- Rumen
- Resource Estimator Service
- Scheduler Load Simulator
- Hadoop Benchmarking
- Dynamometer

##### Reference

- Changelog and Release Notes
- Java API docs
- Unix Shell API
- Metrics

##### Configuration

- core-default.xml
- hdfs-default.xml
- hdfs-rbf-default.xml
- mapred-default.xml
- yarn-default.xml
- kms-default.xml
- httpfs-default.xml
- Deprecated Properties

# Apache Hadoop Changelog

## Release 0.20.0 - 2009-04-22

### INCOMPATIBLE CHANGES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-4210 | Findbugs warnings are printed related to equals implementation of several classes | Major | . | Suresh Srinivas | Suresh Srinivas
HADOOP-4253 | Fix warnings generated by FindBugs | Major | conf, fs, record | Suresh Srinivas | Suresh Srinivas
HADOOP-4572 | INode and its sub-classes should be package private | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-4618 | Move http server from FSNamesystem into NameNode. | Major | . | Konstantin Shvachko | Konstantin Shvachko
HADOOP-4567 | GetFileBlockLocations should return the NetworkTopology information of the machines that hosts those blocks | Major | . | dhruba borthakur | dhruba borthakur
HADOOP-4435 | The JobTracker should display the amount of heap memory used | Minor | . | dhruba borthakur | dhruba borthakur
HADOOP-3923 | Deprecate org.apache.hadoop.mapred.StatusHttpServer | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-4188 | Remove Task’s dependency on concrete file systems | Major | . | Tom White | Sharad Agarwal
HADOOP-1650 | Upgrade Jetty to 6.x | Major | . | Devaraj Das | Chris Douglas
HADOOP-3986 | JobClient should not have a static configuration | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-4422 | S3 file systems should not create bucket | Major | fs/s3 | David Phillips | David Phillips
HADOOP-3422 | Ganglia counter metrics are all reported with the metric name “value”, so the counter values can not be seen | Major | metrics | Jason | stack
HADOOP-4035 | Modify the capacity scheduler (HADOOP-3445) to schedule tasks based on memory requirements and task trackers free memory | Blocker | . | Hemanth Yamijala | Vinod Kumar Vavilapalli
HADOOP-3750 | Fix and enforce module dependencies | Major | . | Tom White | Sharad Agarwal
HADOOP-3497 | File globbing with a PathFilter is too restrictive | Major | fs | Tom White | Tom White
HADOOP-4576 | Modify pending tasks count in the UI to pending jobs count in the UI | Major | . | Hemanth Yamijala | Sreekanth Ramakrishnan
HADOOP-4445 | Wrong number of running map/reduce tasks are displayed in queue information. | Major | . | Karam Singh | Sreekanth Ramakrishnan
HADOOP-4029 | NameNode should report status and performance for each replica of image and log | Major | . | Robert Chansler | Boris Shkolnik
HADOOP-4826 | Admin command saveNamespace. | Major | . | Konstantin Shvachko | Konstantin Shvachko
HADOOP-4631 | Split the default configurations into 3 parts | Major | conf | Owen O’Malley | Sharad Agarwal
HADOOP-3344 | libhdfs: always builds 32bit, even when x86_64 Java used | Major | build | Craig Macdonald | Giridharan Kesavan
HADOOP-4789 | Change fair scheduler to share between pools by default, not between invidual jobs | Minor | . | Matei Zaharia | Matei Zaharia
HADOOP-4970 | Use the full path when move files to .Trash/Current | Major | . | Prasad Chakka | Prasad Chakka
HADOOP-4103 | Alert for missing blocks | Major | . | Christian Kunz | Raghu Angadi
HADOOP-4783 | History files are given world readable permissions. | Blocker | . | Hemanth Yamijala | Amareshwari Sriramadasu
HADOOP-5531 | Remove Chukwa on branch-0.20 | Blocker | . | Nigel Daley | Nigel Daley

### NEW FEATURES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-4284 | Support for user configurable global filters on HttpServer | Major | security | Kan Zhang | Kan Zhang
HADOOP-4575 | An independent HTTPS proxy for HDFS | Major | security | Kan Zhang | Kan Zhang
HADOOP-4622 | Explicitly specify interpretor for non-native binaries | Minor | . | Fredrik Hedberg | Fredrik Hedberg
HADOOP-4661 | distch: a tool for distributed ch{mod,own} | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-4179 | Hadoop-Vaidya : Rule based performance diagnostic tool for Map/Reduce jobs | Major | . | Suhas Gogate | Suhas Gogate
HADOOP-4348 | Adding service-level authorization to Hadoop | Major | security | Kan Zhang | Arun C Murthy
HADOOP-4937 | [HOD] Include ringmaster RPC port information in the notes attribute | Major | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi
HADOOP-4804 | Create Forrest documentation for the fair scheduler | Minor | . | Matei Zaharia | Sreekanth Ramakrishnan

### IMPROVEMENTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-4234 | KFS: Allow KFS layer to interface with multiple KFS namenodes | Minor | fs | Sriram Rao | Sriram Rao
HADOOP-4244 | In “ant test-patch”, runContribTestOnEclipseFiles should not be run on Hudson only | Major | build, test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-3180 | add name of faulty class to WritableName.getClass IOException upon ClassNotFoundException | Trivial | io | Pete Wyckoff | Pete Wyckoff
HADOOP-4178 | The capacity scheduler’s defaults for queues should be configurable. | Blocker | . | Owen O’Malley | Sreekanth Ramakrishnan
HADOOP-4226 | LineReader::readLine cleanup | Minor | . | Yuri Pradkin | Yuri Pradkin
HADOOP-4440 | TestJobInProgressListener should also test for jobs killed in queued state | Minor | test | Amar Kamat | Amar Kamat
HADOOP-4276 | The mapred.*ID classes are inefficient for hashCode and serialization | Major | . | Owen O’Malley | Owen O’Malley
HADOOP-4485 | ant compile-native shorthand | Trivial | build | Enis Soztutar | Enis Soztutar
HADOOP-4454 | Support comments in ‘slaves’ file | Minor | conf, scripts | Rama, Ramasamy | Rama, Ramasamy
HADOOP-3461 | Remove dfs.StringBytesWritbale | Major | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-4437 | Use qMC sequence to improve the accuracy of PiEstimator | Minor | . | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-4599 | BlocksMap and BlockInfo should be package private. | Major | . | Konstantin Shvachko | Konstantin Shvachko
HADOOP-4453 | Improve ssl handling for distcp | Major | . | Chris Douglas | Kan Zhang
HADOOP-4126 | Allow access to HDFS web UI on EC2 | Major | contrib/cloud | Tom White | Tom White
HADOOP-4523 | Enhance how memory-intensive user tasks are handled | Major | . | Vivek Ratan | Vinod Kumar Vavilapalli
HADOOP-4668 | Improve JavaDoc on JobConf.setCombinerClass to better document restrictions on combiners | Major | documentation | Owen O’Malley | Owen O’Malley
HADOOP-4640 | Add ability to split text files compressed with lzo | Trivial | io | Johan Oskarsson | Johan Oskarsson
HADOOP-4666 | Launch reduces only after a few maps have run in the Fair Scheduler | Major | . | Matei Zaharia | Matei Zaharia
HADOOP-4339 | Improve FsShell -du/-dus and FileSystem.getContentSummary efficiency | Major | fs | David Phillips | David Phillips
HADOOP-2774 | Add counters to show number of key/values that have been sorted and merged in the maps and reduces | Major | . | Owen O’Malley | Ravi Gummadi
HADOOP-4705 | [HOD] Grant read permissions for files/directories created by hod as part of provisioning hadoop clusters | Major | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi
HADOOP-4649 | Improve abstraction for spill indices | Minor | . | Chris Douglas | Chris Douglas
HADOOP-3770 | improve composition, submission and result collection of gridmix | Major | benchmarks | Lingyun Yang | Runping Qi
HADOOP-4708 | Add support for dfsadmin commands for test TestCLI unit test | Major | test | Boris Shkolnik | Boris Shkolnik
HADOOP-4758 | Add a splitter for metrics contexts | Minor | metrics | Chris Douglas | Chris Douglas
HADOOP-4690 | fuse-dfs - create source file/function + utils + config + main source files | Minor | . | Pete Wyckoff | Pete Wyckoff
HADOOP-4747 | Reuse FileStatus in FsShell where possible | Minor | fs | David Phillips | David Phillips
HADOOP-4305 | repeatedly blacklisted tasktrackers should get declared dead | Major | . | Christian Kunz | Amareshwari Sriramadasu
HADOOP-4628 | Move Hive out of Core | Major | . | Owen O’Malley |
HADOOP-4749 | reducer should output input data size when shuffling is done | Major | . | Zheng Shao | He Yongqiang
HADOOP-4837 | Move HADOOP_CONF_DIR to chukwa-env.sh | Major | . | Jerome Boulon | Jerome Boulon
HADOOP-4699 | Change TaskTracker.MapOutputServlet to send only the IFile segment, validate checksum in Reduce | Major | . | Chris Douglas | Chris Douglas
HADOOP-3063 | BloomMapFile - fail-fast version of MapFile for sparsely populated key space | Major | io | Andrzej Bialecki | Andrzej Bialecki
HADOOP-1230 | Replace parameters with context objects in Mapper, Reducer, Partitioner, InputFormat, and OutputFormat classes | Major | . | Owen O’Malley | Owen O’Malley
HADOOP-4838 | Cleanup Metrics and the MBeans | Major | . | Sanjay Radia | Sanjay Radia
HADOOP-3136 | Assign multiple tasks per TaskTracker heartbeat | Major | . | Devaraj Das | Arun C Murthy
HADOOP-4545 | Add an example of a secondary sort | Major | . | Owen O’Malley | Owen O’Malley
HADOOP-4753 | gridmix2 code can be condensed | Minor | benchmarks | Chris Douglas | Chris Douglas
HADOOP-4827 | Improve data aggregation in database | Major | . | Eric Yang | Eric Yang
HADOOP-4880 | Improvements to TestJobTrackerRestart | Major | test | Arun C Murthy | Amar Kamat
HADOOP-4909 | Clean up map/reduce api to take JobContext in the static methods | Major | . | Owen O’Malley | Owen O’Malley
HADOOP-4920 | do not keep forrest output in subversion | Major | documentation | Doug Cutting | Doug Cutting
HADOOP-4980 | Cleanup the Capacity Scheduler code | Major | . | Vivek Ratan | Vivek Ratan
HADOOP-4916 | adjust build.xml to reference external properties when building the “init.d” for the rpm | Major | . | Eric Yang | Eric Yang
HADOOP-4950 | CompressorStream and BlockCompressorStream should be public | Major | io | Owen O’Malley | Owen O’Malley
HADOOP-4843 | Collect Job History log file and Job Conf file into Chukwa | Major | . | Eric Yang | Eric Yang
HADOOP-5030 | Chukwa RPM build improvements | Major | . | Eric Yang | Eric Yang
HADOOP-4828 | Update documentation for default configuration | Major | documentation | Sharad Agarwal | Sharad Agarwal
HADOOP-5122 | libhdfs test conf uses deprecated fs.default.name value | Trivial | test | Craig Macdonald | Craig Macdonald
HADOOP-4565 | MultiFileInputSplit can use data locality information to create splits | Major | . | dhruba borthakur | dhruba borthakur
HADOOP-5275 | ivy directory should be there in hadoop tar ball | Major | build | Suman Sehgal | Giridharan Kesavan
HADOOP-5379 | Throw exception instead of writing to System.err when there is a CRC error on CBZip2InputStream | Minor | io | Rodrigo Schmidt | Rodrigo Schmidt
HADOOP-5395 | When queue ACLs are enabled, the error message shown when a job is submitted to a non-existent queue is misleading | Major | . | Hemanth Yamijala | rahul k singh
HADOOP-4664 | Parallelize job initialization | Blocker | . | Matei Zaharia | Jothi Padmanabhan
HADOOP-5466 | CSS Style Changes for Hadoop Doc Headers and Code | Major | documentation | Corinne Chandel | Corinne Chandel
HADOOP-5514 | Add waiting/failed tasks to JobTracker metrics | Major | metrics | Chris Douglas | Chris Douglas
HADOOP-5521 | Remove dependency of testcases on RESTART_COUNT | Major | test | Amar Kamat | Ravi Gummadi
HADOOP-5468 | Change Hadoop doc menu to sub-menus | Major | documentation | Corinne Chandel | Corinne Chandel
HADOOP-5520 | Typo in diskQuota help documentation | Trivial | documentation | Ravi Phulari | Ravi Phulari
HADOOP-5519 | Remove claims from mapred-default.xml that prime numbers of tasks are helpful. | Major | . | Owen O’Malley | Owen O’Malley
HADOOP-4944 | Allow Xinclude in hadoop config file | Minor | conf | Rama, Ramasamy | Rama, Ramasamy
HADOOP-6150 | Need to be able to instantiate a comparator instance from a comparator string without creating a TFile.Reader object | Minor | io | Hong Tang | Hong Tang
HADOOP-466 | Startup scripts will not start instances of Hadoop daemons w/different configs w/o setting separate PID directories | Major | conf | Vetle Roeim |

### BUG FIXES:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-4245 | KFS: Update the kfs jar file | Major | fs | Sriram Rao | Sriram Rao
HADOOP-4262 | message generated when the client exception has a null message is not useful | Minor | ipc | Steve Loughran |
HADOOP-4204 | Fix warnings generated by FindBugs | Blocker | . | Suresh Srinivas | Suresh Srinivas
HADOOP-4207 | update derby.jar to eliminate the one time wait of 2-3 minutes while running junit tests | Minor | . | Prasad Chakka | Prasad Chakka
HADOOP-4325 | Hadoop SocketInputStream.read() should return -1 in case of EOF. | Minor | io | Raghu Angadi | Raghu Angadi
HADOOP-4408 | FsAction operations shouldn’t create new Objects | Minor | fs | Chris Douglas | Chris Douglas
HADOOP-4238 | [mapred] Unavailable schedulingInfo in “hadoop list [all]” output should be marked “N/A” | Major | . | Vinod Kumar Vavilapalli | Sreekanth Ramakrishnan
HADOOP-3842 | There is a window where the JobTracker is in the RUNNING state (i.e ready to accept jobs) and never executes them. | Major | . | Amar Kamat | Hemanth Yamijala
HADOOP-4346 | Hadoop triggers a “soft” fd leak. | Major | io | Raghu Angadi | Raghu Angadi
HADOOP-4388 | Bug in Datanode transferBlocks code | Major | . | Suresh Srinivas | Suresh Srinivas
HADOOP-4587 | typo in javadoc for map.input.file | Trivial | documentation | Koji Noguchi | Koji Noguchi
HADOOP-4530 | In fsck, HttpServletResponse sendError fails with IllegalStateException | Major | . | Koji Noguchi | Hairong Kuang
HADOOP-4377 | Race condition creating S3 buffer directory for NativeS3FileSystem | Major | fs/s3 | David Phillips | David Phillips
HADOOP-4583 | Code optimization/cleanup in HDFS | Major | . | Suresh Srinivas | Suresh Srinivas
HADOOP-4621 | javadoc: warning - Multiple sources of package comments found for some packages | Major | documentation | Tsz Wo Nicholas Sze | Kan Zhang
HADOOP-4571 | chukwa conf files are not cleaned up | Minor | . | Tsz Wo Nicholas Sze | Eric Yang
HADOOP-4589 | Fix the PiEstimator output messages and code comments | Minor | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-4650 | local.cache.size is set to 10 GB, while DEFAULT_CACHE_SIZE is set to 1 MB | Trivial | filecache | Jeff Hammerbacher | Jeff Hammerbacher
HADOOP-4608 | Examples -Driver does not check first argument. | Major | . | Edward J. Yoon | Edward J. Yoon
HADOOP-4645 | hdfs proxy doesn’t package itself right. | Major | . | Owen O’Malley | Kan Zhang
HADOOP-4606 | There is a cygpath error if log directory does not exist | Minor | scripts | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-4141 | ScriptBasedMapping has infinite loop with invalid jobconf parameter | Minor | . | Aaron Kimball | Aaron Kimball
HADOOP-3293 | When an input split spans cross block boundary, the split location should be the host having most of bytes on it. | Major | . | Runping Qi | Jothi Padmanabhan
HADOOP-4691 | In the javadoc of IndexedSortable.compare(…), the link is wrong. | Minor | documentation, util | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-4598 | ‘-setrep’ command skips under-replicated blocks | Minor | . | Koji Noguchi | Hairong Kuang
HADOOP-4429 | Misconfigured UNIX Groups Break Hadoop | Minor | . | Alex Loddengaard | Alex Loddengaard
HADOOP-4676 | Go back to Job has a wrong URL on blacklisted trackers page | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-4704 | javadoc: “the the” => “the” | Trivial | documentation | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-4513 | Capacity scheduler should initialize tasks asynchronously | Major | . | Hemanth Yamijala | Sreekanth Ramakrishnan
HADOOP-4677 | FileSystem.getFileBlockLocations() (aka default implementation for Local FileSystem) incorrect. | Major | fs | Hong Tang | Hong Tang
HADOOP-4669 | A file-open sometimes may not set the access time of a file | Major | . | dhruba borthakur | dhruba borthakur
HADOOP-4732 | connectionTimeout and readTimeout are passed in wrong order in ReduceTask.java | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-2771 | changing the number of reduces dramatically changes the time of the map time | Major | . | Owen O’Malley |
HADOOP-4558 | Scheduler fails to reclaim capacity if Jobs are submitted to queue one after the other | Major | . | Karam Singh | Amar Kamat
HADOOP-4770 | gridmix2 run script doesn’t work on trunk | Trivial | benchmarks | Chris Douglas | Chris Douglas
HADOOP-4738 | saveVersion.sh could write a package-info.java that cannot be compiled when used with git | Trivial | build | Hemanth Yamijala | Chris Douglas
HADOOP-4623 | Running tasks are not maintained by JobInProgress if speculation is off | Major | . | Amar Kamat | Amar Kamat
HADOOP-4786 | TestTrackerBlacklistAcrossJobs compilation is broken | Blocker | . | Hemanth Yamijala | Hemanth Yamijala
HADOOP-4785 | avoid two calls to System.currentTimeMillis() in heartbeat | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-4792 | Chukwa build process generates files not tracked by svn | Major | build | Chris Douglas | Chris Douglas
HADOOP-4796 | Test target for chukwa build.xml needs to comply to hadoop build.xml test suites | Major | build | Eric Yang | Eric Yang
HADOOP-4579 | StorageDirectory is not removed from the storage directories list when writing to an edits fails | Major | . | Boris Shkolnik | Boris Shkolnik
HADOOP-4811 | Import style sheet images for Chukwa | Trivial | . | Eric Yang | Eric Yang
HADOOP-4805 | Remove black list feature from Chukwa Agent to Chukwa Collector communication | Major | . | Eric Yang | Eric Yang
HADOOP-4737 | Job end notification should give status as “KILLED” when job gets killed. | Major | . | Ruchir Shah | Amareshwari Sriramadasu
HADOOP-4825 | Chukwa - change jps to ps for process status detection | Major | . | Eric Yang | Eric Yang
HADOOP-4844 | Document deprecation of o.a.h.fs.permission.AccessControlException better | Minor | documentation, fs | Arun C Murthy | Arun C Murthy
HADOOP-4706 | IFileOutputStream.close should call close of the underlying stream | Minor | . | Jothi Padmanabhan | Jothi Padmanabhan
HADOOP-4683 | Move the call to getMapCompletionEvents in ReduceTask.ReduceCopier.fetchOutputs to a separate thread | Major | . | Devaraj Das | Jothi Padmanabhan
HADOOP-4855 | Fix help message in MRAdmin | Minor | fs, security | Arun C Murthy | Arun C Murthy
HADOOP-4820 | Remove unused method FSNamesystem.deleteInSafeMode | Minor | . | Suresh Srinivas | Suresh Srinivas
HADOOP-4698 | TestMapRed fails with 64bit JDK | Major | test | Nigel Daley | Nigel Daley
HADOOP-4860 | File Tailing Adaptor test cases causes random failure | Major | . | Eric Yang | Eric Yang
HADOOP-3921 | Clover breaks nightly builds | Major | build | Tom White | Tom White
HADOOP-4845 | Shuffle counter issues | Major | . | Chris Douglas | He Yongqiang
HADOOP-4879 | TestJobTrackerRestart fails on trunk | Blocker | test | Arun C Murthy | Owen O’Malley
HADOOP-4791 | Improve rpm build process for Chukwa | Major | . | Eric Yang | Eric Yang
HADOOP-4771 | FSImage saveFSImage() will have problem. | Major | . | Ruyue Ma | Ruyue Ma
HADOOP-4878 | After introduction of ivy ant test-patch always returns -1 score | Blocker | build | Sreekanth Ramakrishnan | Giridharan Kesavan
HADOOP-4874 | Remove bindings to lzo | Blocker | io | Owen O’Malley | Owen O’Malley
HADOOP-4774 | CapacityScheduler should work even if capacity-scheduler.xml is not in the classpath i.e with the code defaults | Major | . | Amar Kamat | Sreekanth Ramakrishnan
HADOOP-4876 | Capacity reclaimation for queue would not work as intended | Blocker | . | Sreekanth Ramakrishnan | Sreekanth Ramakrishnan
HADOOP-4782 | [HOD] HOD opens up directory permissions more than required | Blocker | contrib/hod | Hemanth Yamijala | Peeyush Bishnoi
HADOOP-4849 | Document service level authorization - HADOOP-4348 | Major | documentation, scripts | Arun C Murthy | Arun C Murthy
HADOOP-4894 | fix Chukwa’s jettyCollector.sh | Minor | . | Ari Rabkin | Ari Rabkin
HADOOP-4892 | File descriptor leak in Chukwa’s ExecPlugin | Major | . | Ari Rabkin | Ari Rabkin
HADOOP-4889 | Chukwa RPM needs to chown user/group in build stage instead of post install stage. | Trivial | . | Eric Yang | Eric Yang
HADOOP-4869 | Lost Trackers may not be able to join back | Blocker | . | Devaraj Das | Amareshwari Sriramadasu
HADOOP-4716 | testRestartWithLostTracker frequently times out | Blocker | . | Johan Oskarsson | Amar Kamat
HADOOP-4854 | Capacity Scheduler should read the reclaim-capacity-interval config value from its own config file | Minor | . | Vivek Ratan | Sreekanth Ramakrishnan
HADOOP-4896 | hdfs fsck does not load hdfs configuration. | Blocker | . | Raghu Angadi | Raghu Angadi
HADOOP-4956 | NPE when jobdetails.jsp | Blocker | . | Devaraj Das | Devaraj Das
HADOOP-4979 | Capacity Scheduler does not always return no task to a TT if a job’s memry requirements are not met | Major | . | Vivek Ratan | Vivek Ratan
HADOOP-4830 | Have end to end tests based on MiniMRCluster to verify that queue capacities are honoured. | Major | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli
HADOOP-4949 | Native compilation is broken | Blocker | . | Chris Douglas | Chris Douglas
HADOOP-4787 | TestTrackerBlacklistAcrossJobs fails randomly | Blocker | test | Hemanth Yamijala | Amareshwari Sriramadasu
HADOOP-4914 | Support chkconfig for chukwa start up scripts | Trivial | . | Eric Yang | Eric Yang
HADOOP-4884 | Change Date format pattern for Time Series graph | Trivial | . | Eric Yang | Eric Yang
HADOOP-4925 | Chukwa HTTP connection policies are too hard to configure | Major | . | Ari Rabkin | Ari Rabkin
HADOOP-4947 | use regex to parse chukwa commands | Minor | . | Ari Rabkin | Ari Rabkin
HADOOP-5026 | Startup scripts should be svn-executable | Minor | . | Andy Konwinski | Andy Konwinski
HADOOP-4977 | Deadlock between reclaimCapacity and assignTasks | Blocker | . | Matei Zaharia | Vivek Ratan
HADOOP-4988 | An earlier fix, for HADOOP-4373, results in a problem with reclaiming capacity when one or more queues have a capacity equal to zero | Blocker | . | Vivek Ratan | Vivek Ratan
HADOOP-5065 | setOutputFormatClass in mapreduce.Job fails for SequenceFileOutputFormat | Major | . | Chris Douglas | Chris Douglas
HADOOP-4818 | Enable JobTracker Instrumentation API with user job configuration | Major | . | Eric Yang | Eric Yang
HADOOP-4993 | chukwa agent startup should be more modular | Major | . | Ari Rabkin | Ari Rabkin
HADOOP-5048 | Sometimes job is still displayed in jobqueue_details page for long time after job was killed. | Major | . | Karam Singh | Sreekanth Ramakrishnan
HADOOP-5075 | Potential infinite loop in updateMinSlots | Blocker | . | Matei Zaharia | Matei Zaharia
HADOOP-4671 | Make some “flag” booleans volatile because they are set/read in different threads | Minor | . | Owen O’Malley | Chris Douglas
HADOOP-4999 | IndexOutOfBoundsException in FSEditLog | Major | . | Boris Shkolnik | Boris Shkolnik
HADOOP-5139 | RPC call throws IllegalArgumentException complaining duplicate metrics registration | Major | ipc | Hairong Kuang | Hairong Kuang
HADOOP-5085 | Copying a file to local with Crc throws an exception | Major | fs | Hairong Kuang | Hairong Kuang
HADOOP-5079 | HashFunction inadvertently destroys some randomness | Major | util | Jonathan Ellis | Jonathan Ellis
HADOOP-5211 | TestSetupAndCleanupFailure fails with timeout | Major | . | Enis Soztutar | Enis Soztutar
HADOOP-4873 | display minMaps/Reduces on advanced scheduler page | Major | . | Joydeep Sen Sarma | Joydeep Sen Sarma
HADOOP-4692 | Namenode in infinite loop for replicating/deleting corrupted block | Major | . | Hairong Kuang | Hairong Kuang
HADOOP-5255 | Fix for HADOOP-5079 HashFunction inadvertently destroys some randomness | Minor | io | stack | Jonathan Ellis
HADOOP-5269 | TaskTracker.runningTasks holding FAILED_UNCLEAN and KILLED_UNCLEAN taskStatuses forever in some cases. | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-5214 | ConcurrentModificationException in FairScheduler.getTotalSlots | Blocker | . | Vinod Kumar Vavilapalli | rahul k singh
HADOOP-5233 | Reducer not Succeded after 100% | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-5247 | NPEs in JobTracker and JobClient when mapred.jobtracker.completeuserjobs.maximum is set to zero. | Blocker | . | Vinod Kumar Vavilapalli | Amar Kamat
HADOOP-5282 | Running tasks on lost trackers are incorrectly logging the attempt and task failures | Blocker | . | Amar Kamat | Amar Kamat
HADOOP-4963 | Logs saying org.apache.hadoop.util.DiskChecker$DiskErrorException in TaskTracker are not relevant | Minor | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-5292 | KFS: calling getFileBlockLocations() on 0-length file causes a NPE | Major | . | Sriram Rao | Sriram Rao
HADOOP-5285 | JobTracker hangs for long periods of time | Blocker | . | Vinod Kumar Vavilapalli | Devaraj Das
HADOOP-5241 | Reduce tasks get stuck because of over-estimated task size (regression from 0.18) | Blocker | . | Andy Pavlo | Sharad Agarwal
HADOOP-5142 | MapWritable#putAll does not store classes | Minor | io | Doğacan Güney |
HADOOP-5280 | When expiring a lost launched task, JT doesn’t remove the attempt from the taskidToTIPMap. | Blocker | . | Vinod Kumar Vavilapalli | Devaraj Das
HADOOP-5272 | JobTracker does not log TIP start information after restart | Critical | . | Amar Kamat | Amar Kamat
HADOOP-5154 | 4-way deadlock in FairShare scheduler | Blocker | . | Vinod Kumar Vavilapalli | Matei Zaharia
HADOOP-5306 | Job History file can have empty string as http port after JobTracker Restart in case of lost TT, which can result in NumberFormatException when JT is restarted 2nd time | Blocker | . | Ravi Gummadi | Amar Kamat
HADOOP-5248 | Job directories could remain undeleted in some scenarios after job completes. | Blocker | . | Hemanth Yamijala | Devaraj Das
HADOOP-5111 | Generic mapreduce classes cannot be used with Job::set* methods | Major | . | Chris Douglas | Chris Douglas
HADOOP-5146 | LocalDirAllocator misses files on the local filesystem | Blocker | . | Arun C Murthy | Devaraj Das
HADOOP-5326 | bzip2 codec (CBZip2OutputStream) creates corrupted output file for some inputs | Major | io | Rodrigo Schmidt | Rodrigo Schmidt
HADOOP-4744 | Wrong resolution of hostname and port | Blocker | . | Amareshwari Sriramadasu | Jothi Padmanabhan
HADOOP-4638 | Exception thrown in/from RecoveryManager.recover() should be caught and handled | Blocker | . | Amar Kamat | Amar Kamat
HADOOP-5274 | gridmix2 is not getting compiled to generate gridmix.jar | Critical | benchmarks | Suman Sehgal | Chris Douglas
HADOOP-5066 | ant binary should not compile docs | Major | build | Amareshwari Sriramadasu | Giridharan Kesavan
HADOOP-5384 | DataNodeCluster should not create blocks with generationStamp == 1 | Blocker | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-5145 | Balancer sometimes runs out of memory after days or weeks running | Major | . | Hairong Kuang | Hairong Kuang
HADOOP-5338 | Reduce tasks are stuck waiting for map outputs when none are in progress | Blocker | . | Ramya Sunil | Amar Kamat
HADOOP-5376 | JobInProgress.obtainTaskCleanupTask() throws an ArrayIndexOutOfBoundsException | Blocker | . | Vinod Kumar Vavilapalli | Amareshwari Sriramadasu
HADOOP-4695 | TestGlobalFilter.testServletFilter fails | Blocker | test | Tsz Wo Nicholas Sze | Kan Zhang
HADOOP-5298 | Unit test fails out on trunk org.apache.hadoop.http.TestServletFilter.testServletFilter | Major | test | Lee Tucker | Tsz Wo Nicholas Sze
HADOOP-5432 | TestHdfsProxy fails on 0.20 | Major | . | Chris Douglas | Chris Douglas
HADOOP-5416 | Wrong description of " hadoop fs -test " in FS Shell guide . | Trivial | documentation | Ravi Phulari | Ravi Phulari
HADOOP-5327 | Job files for a job failing because of ACLs are not clean from the system directory | Blocker | . | Karam Singh | Amar Kamat
HADOOP-5440 | Successful taskid are not removed from TaskMemoryManager | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-5465 | Blocks remain under-replicated | Blocker | . | Hairong Kuang | Hairong Kuang
HADOOP-5490 | TestParallelInitialization failed on NoSuchElementException | Blocker | . | Hairong Kuang | Jothi Padmanabhan
HADOOP-5493 | Shuffle copiers do not return Codecs back to the pool at the end of shuffling | Blocker | . | Devaraj Das | Jothi Padmanabhan
HADOOP-5505 | TestMissingBlocksAlert fails on 0.20. | Major | . | Raghu Angadi | Raghu Angadi
HADOOP-5414 | IO exception while executing hadoop fs -touchz fileName | Major | fs | Ravi Phulari | Hairong Kuang
HADOOP-5483 | Directory/file cleanup thread throws IllegalStateException | Major | . | Hairong Kuang |
HADOOP-5281 | GzipCodec fails second time it is used in a process | Blocker | . | Ben Maurer | Chris Douglas
HADOOP-5463 | Balancer throws “Not a host:port pair” unless port is specified in fs.default.name | Minor | . | Stuart White | Stuart White
HADOOP-5479 | NameNode should not send empty block replication request to DataNode | Critical | . | Hairong Kuang | Hairong Kuang
HADOOP-5259 | Job with output hdfs:/user/<username>/outputpath (no authority) fails with Wrong FS | Major | fs | Koji Noguchi | Doug Cutting
HADOOP-5499 | JobTracker metrics do not match job counters | Major | metrics | Chris Douglas |
HADOOP-5516 | TaskMemoryManagerThread crashes in a corner case | Blocker | . | Vinod Kumar Vavilapalli | Vinod Kumar Vavilapalli
HADOOP-5382 | The new map/reduce api doesn’t support combiners | Blocker | . | Owen O’Malley | Owen O’Malley
HADOOP-5471 | SyncLogs thread in Child.java would update wrong file for a cleanup attempt, in some cases. | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-5522 | Document job setup/cleaup tasks and task cleanup tasks in mapred tutorial | Blocker | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-5534 | Deadlock triggered by FairScheduler scheduler’s servlet due to changes from HADOOP-5214. | Blocker | . | Vinod Kumar Vavilapalli | rahul k singh
HADOOP-5328 | Renaming of Job histroy file is incorrect if Jobtracker is restarted multimple times | Major | . | Karam Singh | Amar Kamat
HADOOP-5417 | IPC client drops interrupted exceptions | Blocker | ipc | Owen O’Malley | Owen O’Malley
HADOOP-5320 | TestMapReduceLocal is missing a close() that is causing it to fail while running the test on NFS | Minor | test | Jothi Padmanabhan | Jothi Padmanabhan
HADOOP-5549 | ReplicationMonitor should schedule both replication and deletion work in one iteration | Major | . | Hairong Kuang | Hairong Kuang
HADOOP-5554 | DataNodeCluster should create blocks with the same generation stamp as the blocks created in CreateEditsLog | Major | test | Hairong Kuang | Hairong Kuang
HADOOP-5484 | TestRecoveryManager fails wtih FileAlreadyExistsException | Major | test | Hairong Kuang | Amar Kamat
HADOOP-5557 | Two minor problems in TestOverReplicatedBlocks | Minor | test | Tsz Wo Nicholas Sze | Tsz Wo Nicholas Sze
HADOOP-5564 | hadoop command uses large JVM heap size | Blocker | build | Suresh Srinivas | Suresh Srinivas
HADOOP-5565 | The job instrumentation API needs to have a method for finalizeJob, | Major | . | Jerome Boulon | Jerome Boulon
HADOOP-5390 | hdfsproxy includes duplicate jars in tarball, source in binary tarball | Major | build | Chris Douglas | zhiyong zhang
HADOOP-5459 | CRC errors not detected reading intermediate output into memory with problematic length | Major | . | Chris Douglas | Chris Douglas
HADOOP-5571 | TupleWritable can return incorrect results if it contains more than 32 values | Major | . | Jingkei Ly | Jingkei Ly
HADOOP-5588 | hadoop commands seem extremely slow in 0.20 branch | Blocker | fs | Koji Noguchi | Hairong Kuang
HADOOP-5473 | Race condition in command-line kill for a task | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-5551 | Namenode permits directory destruction on overwrite | Critical | . | Brian Bockelman | Brian Bockelman
HADOOP-5576 | LocalJobRunner does not run jobs using new MapReduce API | Blocker | . | Tom White | Tom White
HADOOP-4374 | JVM should not be killed but given an opportunity to exit gracefully | Major | . | Devaraj Das | Ravi Gummadi
HADOOP-5577 | The Job.waitForCompletion doesn’t print status as the job runs | Blocker | . | Owen O’Malley | Owen O’Malley
HADOOP-5607 | TestCapacityScheduler fails with NPE | Major | test | Chris Douglas | Chris Douglas
HADOOP-5605 | All the replicas incorrectly got marked as corrupt. | Blocker | . | Raghu Angadi | Hairong Kuang
HADOOP-5337 | JobTracker greedily schedules tasks without running tasks to join | Major | . | Karam Singh | Amar Kamat
HADOOP-5227 | distcp -delete option deletes all files from the destination directory | Major | . | Suhas Gogate | Tsz Wo Nicholas Sze
HADOOP-5305 | Unit test org.apache.hadoop.fs.TestCopyFiles.testMapCount fails on trunk | Minor | test | Ramya Sunil | Tsz Wo Nicholas Sze
HADOOP-5437 | Unit test for jvm -reuse | Blocker | . | Amareshwari Sriramadasu | Owen O’Malley
HADOOP-5548 | Observed negative running maps on the job tracker | Blocker | . | Owen O’Malley | Amareshwari Sriramadasu
HADOOP-3810 | NameNode seems unstable on a cluster with little space left | Major | . | Raghu Angadi | Hairong Kuang
HADOOP-5068 | testClusterBlockingForLackOfMemory in TestCapacityScheduler fails randomly | Major | . | Sreekanth Ramakrishnan | Vinod Kumar Vavilapalli
HADOOP-5585 | FileSystem statistic counters are too high when JVM reuse is enabled. | Blocker | . | Owen O’Malley | Owen O’Malley
HADOOP-5394 | JobTracker might schedule 2 attempts of the same task with the same attempt id across restarts | Critical | . | Amar Kamat | Amar Kamat
HADOOP-5645 | After HADOOP-4920 we need a place to checkin releasenotes.html | Major | build | Nigel Daley | Nigel Daley
HADOOP-5644 | Namnode is stuck in safe mode | Major | . | Suresh Srinivas | Suresh Srinivas

### TESTS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-4505 | Add a unit test to test faulty setup task and cleanup task killing the job | Major | . | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-4722 | adding tests for quotas command line error messages | Minor | test | Boris Shkolnik | Boris Shkolnik
HADOOP-4688 | multiple spills/reducers test case for mergeParts() needed | Major | . | Yuri Pradkin | Chris Douglas
HADOOP-4728 | Tests for NameNode -checkpoint option with different configurations | Minor | . | Boris Shkolnik | Boris Shkolnik
HADOOP-4857 | TestUlimit is failing after Hadoop-4620 | Blocker | test | Sharad Agarwal | Ravi Gummadi
HADOOP-4458 | Add a unit test for applications creating symlinks in wokring directory | Major | test | Amareshwari Sriramadasu | Amareshwari Sriramadasu
HADOOP-3192 | Test behavior of very large (32GB) name nodes | Major | . | Robert Chansler |

### SUB-TASKS:

JIRA | Summary | Priority | Component | Reporter | Contributor
--- | --- | --- | --- | --- | ---
HADOOP-4187 | Create a MapReduce-specific ReflectionUtils that handles JobConf and JobConfigurable | Major | . | Tom White | Sharad Agarwal
HADOOP-4612 | Remove RunJar’s dependency on JobClient | Major | . | Sharad Agarwal | Sharad Agarwal
HADOOP-4185 | Add setVerifyChecksum() method to FileSystem | Major | fs | Tom White | Sharad Agarwal