引起原因:java.lang.ClassNotFoundException:org.apache.hadoop.mapreduce.InputFormat [英] Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.InputFormat

问题描述

我正在MAC OSX 10.9.4中进行sqoop导入，并收到以下错误消息:

I am doing sqoop import in MAC OSX 10.9.4 and getting error as:

14/10/24 11:51:41 INFO sqoop.Sqoop: Running Sqoop version: 1.4.5
14/10/24 11:51:41 INFO tool.BaseSqoopTool: Using Hive-specific delimiters for output. You can override
14/10/24 11:51:41 INFO tool.BaseSqoopTool: delimiters with --fields-terminated-by, etc.
14/10/24 11:51:41 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset.
14/10/24 11:51:41 INFO tool.CodeGenTool: Beginning code generation
14/10/24 11:51:42 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test_log` AS t LIMIT 1
14/10/24 11:51:42 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `test_log` AS t LIMIT 1
14/10/24 11:51:42 INFO orm.CompilationManager: HADOOP_MAPRED_HOME is /Users/hadoop/hadoop-2.5.1/share/hadoop/mapreduce
Note: /tmp/sqoop-hadoop/compile/2398ed5be0489f1c76d6fb556f3e0e9d/test_log.java uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
14/10/24 11:51:45 INFO orm.CompilationManager: Writing jar file: /tmp/sqoop-hadoop/compile/2398ed5be0489f1c76d6fb556f3e0e9d/test_log.jar
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/mapreduce/InputFormat
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:760)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:455)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:367)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at org.apache.sqoop.manager.ImportJobContext.<init>(ImportJobContext.java:51)
    at com.cloudera.sqoop.manager.ImportJobContext.<init>(ImportJobContext.java:33)
    at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:483)
    at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:601)
    at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
    at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
    at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.mapreduce.InputFormat
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 58 more

Hadoop版本:2.5.1Sqoop版本:1.4.5操作系统:OSX10.9.4

Hadoop version : 2.5.1 Sqoop version :1.4.5 os : OSX10.9.4

设置环境变量

导出HADOOP_COMMON_HOME =/Users/hadoop/hadoop-2.5.1/

export HADOOP_COMMON_HOME=/Users/hadoop/hadoop-2.5.1/

导出HADOOP_MAPRED_HOME =/Users/hadoop/hadoop-2.5.1/share/hadoop/mapreduce/

export HADOOP_MAPRED_HOME=/Users/hadoop/hadoop-2.5.1/share/hadoop/mapreduce/

Sqoop jar(sqoop-1.4.5.jar)放在/Users/hadoop/hadoop-2.5.1/lib//Users/hadoop/sqoop/lib中的mysql连接器

Sqoop jar (sqoop-1.4.5.jar) is placed in /Users/hadoop/hadoop-2.5.1/lib/ and mysql connector in /Users/hadoop/sqoop/lib

core.jar也存在于

also the core.jar exist in

/Users/hadoop/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.1.jar

/Users/hadoop/hadoop-2.5.1/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.1.jar

推荐答案

以某种方式无法访问Jars.将Sqoop罐子放入HDFS后，Sqoop导入可以正常进行.该链接帮助​​找不到Sqoop jar文件

Somehow the Jars were not accessible. After puting the Sqoop jars in HDFS, Sqoop import went fine. This link helped Sqoop jar files not found

这篇关于引起原因:java.lang.ClassNotFoundException:org.apache.hadoop.mapreduce.InputFormat的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！