hadoop-2.4.1.zip

Hadoop MapReduce Change Log Release 2.4.1 - 2014-06-23 INCOMPATIBLE CHANGES NEW FEATURES IMPROVEMENTS MAPREDUCE-5830. Added back the private API HostUtil.getTaskLogUrl(..) for binary compatibility with older clients like Hive 0.13. (Akira Ajisaka via vinodkv) OPTIMIZATIONS BUG FIXES MAPREDUCE-5714. Removed forceful JVM exit in shutDownJob. (Jinghui Wang via Eric Yang) MAPREDUCE-5818. Added "hsadmin" command into mapred.cmd. (Jian He via zjshen) MAPREDUCE-5824. Fixed test-failure of TestPipesNonJavaInputFormat in Windows. (Xuan Gong via vinodkv) MAPREDUCE-5815. Fixed test-failure of TestMRAppMaster by making MRAppMaster gracefully handle empty-queue names. (Akira Ajisaka via vinodkv) MAPREDUCE-5826. Fixed HistoryServerFileSystemStore to use right permissions on Windows for temporary files and thus also fix the test-issue with TestHistoryServerFileSystemStateStoreService. (Varun Vasudev via vinodkv) MAPREDUCE-5828. Fixed a test issue with TestMapReduceJobControl that was causing it to fail on Windows. (vinodkv) MAPREDUCE-5827. TestSpeculativeExecutionWithMRApp fails. (Zhijie Shen via cnauroth) MAPREDUCE-5833. TestRMContainerAllocator fails ocassionally. (Zhijie Shen via cnauroth) MAPREDUCE-5832. Fixed TestJobClient to not fail on JDK7 or on Windows. (Jian He and Vinod Kumar Vavilapalli via vinodkv) MAPREDUCE-5841. uber job doesn't terminate on getting mapred job kill (Sangjin Lee via jlowe) MAPREDUCE-5843. Fixed TestMRKeyValueTextInputFormat to not leak files and thus avoid failing on Windows. (Varun Vasudev via vinodkv) MAPREDUCE-5835. Killing Task might cause the job to go to ERROR state (Ming Ma via jlowe) MAPREDUCE-5821. Avoid unintentional reallocation of byte arrays in segments during merge. (Todd Lipcon via cdouglas) Release 2.4.0 - 2014-04-07 INCOMPATIBLE CHANGES NEW FEATURES MAPREDUCE-5787. Added the ability to keep alive shuffle connections in the MapReduce shuffle-handler. (Rajesh Balamohan via vinodkv) IMPROVEMENTS MAPREDUCE-5464. Add analogs of the SLOTS_MILLIS counters that jive with the YARN resource model (Sandy Ryza) MAPREDUCE-5732. Report proper queue when job has been automatically placed (Sandy Ryza) MAPREDUCE-5699. Allow setting tags on MR jobs (kasha) MAPREDUCE-5761. Added a simple log message to denote when encrypted shuffle is on in the shuffle-handler. (Jian He via vinodkv) MAPREDUCE-5754. Preserve Job diagnostics in history (Gera Shegalov via jlowe) MAPREDUCE-5773. Provide dedicated MRAppMaster syslog length limit (Gera Shegalov via jlowe) MAPREDUCE-5553. Allow users to easily access completed/pending/successful/failed tasks on MR AM web-ui. (Paul Han via acmurthy) MAPREDUCE-4052. Improved MapReduce clients to use NodeManagers' ability to handle cross platform application submissions. (Jian He via vinodkv) MAPREDUCE-2349. Modified FileInputFormat to be able to issue file and block location calls in parallel. (Siddharth Seth via vinodkv) OPTIMIZATIONS BUG FIXES MAPREDUCE-5746. Job diagnostics can implicate wrong task for a failed job. (Jason Lowe via kasha) MAPREDUCE-5670. CombineFileRecordReader should report progress when moving to the next file (Chen He via jlowe) MAPREDUCE-5757. ConcurrentModificationException in JobControl.toList (jlowe) MAPREDUCE-5770. Fixed MapReduce ApplicationMaster to correctly redirect to the YARN's web-app proxy with the correct scheme prefix. (Jian He via vinodkv) MAPREDUCE-5768. TestMRJobs.testContainerRollingLog fails on trunk (Gera Shegalov via jlowe) MAPREDUCE-5780. SliveTest should use the specified path to get the particular FileSystem instead of using the default FileSystem. (szetszwo) MAPREDUCE-5028. Fixed a bug in MapTask that was causing mappers to fail when a large value of io.sort.mb is set. (Karthik Kambatla via vinodkv) MAPREDUCE-5778. JobSummary does not escape newlines in the job name (Akira AJISAKA via jlowe) MAPREDUCE-5789. Average Reduce time is incorrect on Job Overview page (Rushabh S Shah via jlowe) MAPREDUCE-5794. SliveMapper always uses default FileSystem. (szetszwo via Arpit Agarwal) MAPREDUCE-5751. MR app master fails to start in some cases if mapreduce.job.classloader is true (Sangjin Lee via jlowe) MAPREDUCE-5688. TestStagingCleanup fails intermittently with JDK7 (Mit Desai via jeagles) MAPREDUCE-5769. Unregistration to RM should not be called if AM is crashed before registering with RM (Rohith via jlowe) MAPREDUCE-5570. Map task attempt with fetch failure has incorrect attempt finish time (Rushabh S Shah via jlowe) MAPREDUCE-5806. Fixed a bug in MRAppMaster so as to enable users to properly override HADOOP_ROOT_LOGGER or HADOOP_CLIENT_OPTS. (Varun Vasudev via vinodkv) MAPREDUCE-5791. Shuffle phase is slow in Windows - FadviseFileRegion::transferTo does not read disks efficiently. (Nikola Vujic via cnauroth) MAPREDUCE-5795. Fixed MRAppMaster to record the correct job-state after it recovers from a commit during a previous attempt. (Xuan Gong via vinodkv) MAPREDUCE-5805. Fixed MapReduce JobHistory encoding of queue-name to escape hyphens and thus avoid parsing errors. (Akira AJISAKA via vinodkv) MAPREDUCE-5810. Removed the faulty and failing streaming test TestStreamingTaskLog. (Akira Ajisaka via vinodkv) MAPREDUCE-5813. Fix YarnChild to explicitly load job.xml from the local-filesystem, rather than rely on the classpath. (Gera Shegalov via acmurthy) Release 2.3.1 - UNRELEASED INCOMPATIBLE CHANGES NEW FEATURES IMPROVEMENTS OPTIMIZATIONS BUG FIXES Release 2.3.0 - 2014-02-18 INCOMPATIBLE CHANGES NEW FEATURES MAPREDUCE-5265. History server admin service to refresh user and superuser group mappings (Ashwin Shankar via jlowe) MAPREDUCE-5356. Ability to refresh aggregated log retention period and check interval (Ashwin Shankar via jlowe) MAPREDUCE-5386. Ability to refresh history server job retention and job cleaner settings (Ashwin Shankar via jlowe) MAPREDUCE-5411. Refresh size of loaded job cache on history server (Ashwin Shankar via jlowe) MAPREDUCE-5332. Support token-preserving restart of history server (jlowe) IMPROVEMENTS MAPREDUCE-5329. Allow MR applications to use additional AuxServices, which are compatible with the default MapReduce shuffle. (Avner BenHanoch via sseth) MAPREDUCE-5463. Deprecate SLOTS_MILLIS counters (Tzuyoshi Ozawa via Sandy Ryza) MAPREDUCE-5457. Add a KeyOnlyTextOutputReader to enable streaming to write out text files without separators (Sandy Ryza) MAPREDUCE-5596. Allow configuring the number of threads used to serve shuffle connections (Sandy Ryza via jlowe) MAPREDUCE-434. LocalJobRunner limited to single reducer (Sandy Ryza and Aaron Kimball via Sandy Ryza) MAPREDUCE-4421. Run MapReduce framework via the distributed cache (jlowe) MAPREDUCE-1176. FixedLengthInputFormat and FixedLengthRecordReader (Mariappan Asokan and BitsOfInfo via Sandy Ryza) MAPREDUCE-5613. DefaultSpeculator holds and checks hashmap that is always empty (Gera Shegalov via Sandy Ryza) MAPREDUCE-5431. Missing pom dependency in MR-client (Timothy St. Clair via stevel) MAPREDUCE-5624 Move grizzly-test and junit dependencies to test scope (Ted Yu via stevel) MAPREDUCE-5481. Enable uber jobs to have multiple reducers (Sandy Ryza) MAPREDUCE-5052. Job History UI and web services confusing job start time and job submit time (Chen He via jeagles) MAPREDUCE-5692. Add explicit diagnostics when a task attempt is killed due to speculative exec