OrcNewInputformat作为hadoop streaming的输入格式 [英] OrcNewInputformat as a inputformat for hadoop streaming

问题描述

我正在使用hadoop streaming，我想给输入格式作为OrcNewFormat ..
我正在执行命令： - $ /

hadoop jar hadoop-streaming .jar -libjars /usr/hdp/2.2.4.2-2/hive/lib/hive-exec.jar -input / user / orcfiles -output / streamf -mapper'cat'-inputformat org.apache.hadoop.hive.ql .io.orc.OrcNewInputFormat -outputformat org.apache.hadoop.hive.ql.io.orc.OrcNewOutputFormat

但是我得到了以下异常：

 线程main中的异常java.lang.RuntimeException：class org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat not org。 apache.hadoop.mapred.InputFormat 
 at org.apache.hadoop.conf.Configuration.setClass（Configuration.java:2150）
 at org.apache.hadoop.mapred.JobConf.setInputFormat（JobConf.java ：702）
 at org.apache.hadoop.streaming.StreamJob.setJobConf（StreamJob.java:796）
 at org.apache.hadoop.streaming.StreamJob.run（StreamJob.java:128）
在org.apache.hado op.util.ToolRunner.run（ToolRunner.java:70）
位于org.apache.hadoop.util.ToolRunner.run（ToolRunner.java:84）
位于org.apache.hadoop.streaming。 HadoopStreaming.main（HadoopStreaming.java:50）
 at sun.reflect.NativeMethodAccessorImpl.invoke0（Native Method）
 at sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:57）
 at sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:43）
 at java.lang.reflect.Method.invoke（Method.java:606）
 at org.apache.hadoop.util.RunJar。运行（RunJar.java:221）
 at org.apache.hadoop.util.RunJar.main（RunJar.java:136）
  

从这个链接

http://hive.apache.org/javadocs/r1.2.0/api/

我可以看到Class OrcNewInputFormat扩展org.apache.hadoop.mapreduce.InputFormat，但从异常我可以f找出那个类org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat而不是org.apache.hadoop.mapred.InputFormat。

我在这里错过了什么？

解决方案

现在我给了错误的类名。

I am using hadoop streaming and i want to give input format as a OrcNewFormat.. I am executing command:-

hadoop jar hadoop-streaming.jar -libjars /usr/hdp/2.2.4.2-2/hive/lib/hive-exec.jar -input /user/orcfiles -output /streamf -mapper 'cat' -inputformat org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat -outputformat org.apache.hadoop.hive.ql.io.orc.OrcNewOutputFormat

But I am getting below exception:

    Exception in thread "main" java.lang.RuntimeException: class org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat not org.apache.hadoop.mapred.InputFormat
        at org.apache.hadoop.conf.Configuration.setClass(Configuration.java:2150)
        at org.apache.hadoop.mapred.JobConf.setInputFormat(JobConf.java:702)
        at org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:796)
        at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:128)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
        at org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

from this link

http://hive.apache.org/javadocs/r1.2.0/api/

I could see that Class OrcNewInputFormat extends org.apache.hadoop.mapreduce.InputFormat, but from exception i could figure out that class org.apache.hadoop.hive.ql.io.orc.OrcNewInputFormat not org.apache.hadoop.mapred.InputFormat.

What am I missing here?

解决方案

IT is working fine now I was giving wrong classname.

这篇关于OrcNewInputformat作为hadoop streaming的输入格式的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！