与ooopop一起使用hadoop与Oracle数据导入的Sqoop集成 [英] Sqoop integration with hadoop for oracle data import with oraoop
问题描述
我安装了CDH sqoop，试图将已经运行的apache hadoop。
我发现oraoop使用正确，但我在导入时遇到以下问题。我也尝试用apache hadoop的apache sqoop，但仍面临以下问题。
网页搜索建议使用CDH hadoop，而不是Apache Hadoop。
$ b
**
线程main中的异常java.lang.IncompatibleClassChangeError：
找到类org.apache.hadoop.mapreduce.JobContext，但接口为
期望
at com .a.OrOopDataDrivenDBInputFormat.getSplits（OraOopDataDrivenDBInputFormat.java:51）
$ b $ p总之，
CDH sqoop + Apache Hadoop - 数据导入失败，出现上述情况
异常Apache Sqoop + Apache hadoop - 数据导入失败，出现上述异常情况
CDH Sqoop + CDH Hadoop - 这是正确的组合吗？
有什么建议吗？我不确定是否正确的做法。 Hadoop已经经历了从Hadoop 1.0到Hadoop 2.0（相应的从CDH3到CDH4）的巨大代码重构。一个副作用是，针对Hadoop 1.0（CDH3）编译的代码与Hadoop 2.0（CDH4）不兼容，反之亦然。但是源代码是兼容的，因此只需要用目标Hadoop发行版重新编译代码即可。
例外情况Found class X，but interface was expected atis very common when您正在运行在Hadoop 2.0（CDH4）上为Hadoop 1.0（CDH3）编译的代码，反之亦然。
解决方案很简单，您需要同步版本。使用CDH3 Hadoop + CDH3 Sqoop或CDH4 Hadoop + CDH4 Sqoop是最简单的方法。如果您更喜欢使用上游Sqoop版本，那么您必须确保您使用的是针对您的Hadoop分发版编译的二进制工件。由于目标hadoop分布是在工件名称中编码的，因此Sqoop使得这一点变得简单 - 例如，sqoop-1.4.2.bin__hadoop-1.0.0.tar.gz旨在用于Hadoop 1.0 [1]。
完全相同的约束适用于连接器。您必须下载您正在运行的Hadoop版本的连接器。在OraOop的情况下，CDH3和CDH4也有单独的工件。[b]
1： http：/ /www.apache.org/dist/sqoop/1.4.2/
<2> https://ccp.cloudera.com/display/con/Quest+Data+Connectors
I have been trying to import data from oracle Express edition 11g R2 to hadoop using scoop with oraoop.
I installed CDH sqoop and tried to integrate the already running apache hadoop.
I found that oraoop is used correctly but i face the following issue on import. I also tried with apache sqoop with apache hadoop but still faced the following issue. The web search suggested to use CDH hadoop as well instead of apache Hadoop.
**
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class org.apache.hadoop.mapreduce.JobContext, but interface was expected at com.quest.oraoop.OraOopDataDrivenDBInputFormat.getDesiredNumberOfMappers(OraOopDataDrivenDBInputFormat.java:201) at com.quest.oraoop.OraOopDataDrivenDBInputFormat.getSplits(OraOopDataDrivenDBInputFormat.java:51)
**
To summarize,
CDH sqoop + Apache Hadoop - Data import failed with the above
exception Apache Sqoop + Apache hadoop - Data import failed with the above exception
CDH Sqoop + CDH Hadoop - Is this the right combination?
Any suggestions? I am not sure if am going the right way. Please help.
Hadoop has gone through a huge code refactoring from Hadoop 1.0 to Hadoop 2.0 (correspondingly from CDH3 to CDH4). One side effect is that code compiled against Hadoop 1.0 (CDH3) is not compatible with Hadoop 2.0 (CDH4) and vice-versa. However source code is compatible and thus one just need to recompile code with target Hadoop distribution.
The exception "Found class X, but interface was expected at" is very common when you're running code that is compiled for Hadoop 1.0 (CDH3) on Hadoop 2.0 (CDH4) or vice-versa.
Solution is simple, you need to synchronize the versions. Using CDH3 Hadoop + CDH3 Sqoop or CDH4 Hadoop + CDH4 Sqoop is the simplest way to go. If you prefer to use upstream Sqoop release, then you have to make sure that you're using binary artifact compiled for your Hadoop distribution. Sqoop makes this easy as the target hadoop distribution is encoded in the artifact name - for example sqoop-1.4.2.bin__hadoop-1.0.0.tar.gz is meant to be used on Hadoop 1.0 [1].
Exactly the same constraint applies to connectors. You must download connector for Hadoop version that you're running on. In case of OraOop there are also separate artifacts for CDH3 and CDH4 [2].
Jarcec
Links:
1: http://www.apache.org/dist/sqoop/1.4.2/
2: https://ccp.cloudera.com/display/con/Quest+Data+Connectors
这篇关于与ooopop一起使用hadoop与Oracle数据导入的Sqoop集成的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
