Hadoop无法找到joda依赖关系 [英] Hadoop can't find joda dependency

问题描述

我试图编写一个mapreduce应用程序，该应用程序从CSV文件获取输入并汇总每个帐户的滚动利息.我已经通过了MRunit测试，一切似乎都已签出.当我尝试在计算机上运行jar文件(我在本地运行Hadoop)时，出现以下错误

I've tried to write a mapreduce app that takes input from a CSV file and aggregates rollover interest per account. I've passed my MRunit tests and everything seems to check out. When I try to run the jar file on my machine (I'm running Hadoop locally), the following error comes up

..........>>
14/07/16 09:09:05 INFO mapred.MapTask: Starting flush of map output
14/07/16 09:09:05 INFO mapred.LocalJobRunner: map task executor complete.
14/07/16 09:09:05 WARN mapred.LocalJobRunner: job_local1566570594_0001
java.lang.Exception: java.lang.NoClassDefFoundError: org/joda/time/ReadableInstant
    at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.lang.NoClassDefFoundError: org/joda/time/ReadableInstant
    at org.mrIntCalc.CalcInterestMapper.map(CalcInterestMapper.java:38)
    at org.mrIntCalc.CalcInterestMapper.map(CalcInterestMapper.java:12)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
    at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: org.joda.time.ReadableInstant
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    ... 11 more
14/07/16 09:09:05 INFO mapreduce.Job: Job job_local1566570594_0001 running in uber mode : false
14/07/16 09:09:05 INFO mapreduce.Job:  map 0% reduce 0%
14/07/16 09:09:05 INFO mapreduce.Job: Job job_local1566570594_0001 failed with state FAILED due to: NA
14/07/16 09:09:05 INFO mapreduce.Job: Counters: 0

我使用maven构建了应用程序，因此在pom.xml中将joda(v2.3)列为依赖项.就像我说的那样，我的单元测试非常清楚.查阅这个问题，我发现有人提到这个问题的根源是类路径问题，尽管我认为并非如此.有什么想法吗?

I built the app using maven, so joda(v2.3) is listed as a dependency in the pom.xml. Like I said, my unit tests came out all clear. Looking up this issue, I see that some people mention there's a classpath issue at the root of this, although I don't think that's quite it. Any thoughts?

推荐答案

将joda-time.jar复制到hadoop公共库中.可能不是最好的解决方案，但至少可以奏效.

Copied the joda-time.jar to a hadoop common library. Probably not the best solution, but at least it works.

这篇关于Hadoop无法找到joda依赖关系的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！