Hadoop中的ClassNotFoundException [英] ClassNotFoundException in Hadoop

问题描述

使用Hadoop mapreduce我正在编写代码以获取不同长度的子字符串。
示例给定字符串ZYXCBA和长度3.我的代码必须返回所有可能的长度为3的字符串（ZYX，YXC，XCB，CBA），长度4（ZYXC YXCB，XCBA）最后5号（ZYXCB，YXCBA）。

Using Hadoop mapreduce I am writing code to get substrings of different lengths. Example given string "ZYXCBA" and length 3. My code has to return all possible strings of length 3 ("ZYX","YXC","XCB","CBA"), length 4("ZYXC","YXCB","XCBA") finally length 5("ZYXCB","YXCBA").

在地图阶段，我做了以下工作：

In map phase I did the following:

key =我想要的子字符串长度

key = length of substrings I want

value =ZYXCBA。

value = "ZYXCBA".

所以mapper输出是

So mapper output is

3,"ZYXCBA"
4,"ZYXCBA"
5,"ZYXCBA"

在减少我取字符串（ZYXCBA）和键3以获取所有子字符串长度3.同样发生在4,5。结果收集在一个ArrayList中。

In reduce I take string ("ZYXCBA") and key 3 to get all substrings of length 3. Same occurs for 4,5. Results are collected in an ArrayList.

我使用以下命令运行我的代码：

I am running my code using following command:

hduser@Ganesh:~/Documents$ hadoop jar Saishingles.jar hadoopshingles.Saishingles Behara/Shingles/input Behara/Shingles/output

我的代码如下所示::

My code is as shown below ::

package hadoopshingles;

import java.io.IOException;

import java.util.ArrayList;

import org.apache.hadoop.fs.Path; 

import org.apache.hadoop.io.IntWritable;

import org.apache.hadoop.io.Text;

import org.apache.hadoop.mapreduce.Mapper;

import org.apache.hadoop.mapreduce.Reducer;

import org.apache.hadoop.conf.Configuration;

import org.apache.hadoop.mapreduce.Job;

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;

import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;


public class Saishingles{

public static class shinglesmapper extends Mapper<Object, Text, IntWritable, Text>{

        public void map(Object key, Text value, Context context
                ) throws IOException, InterruptedException {

            String str = new String(value.toString());
            String[] list = str.split(" ");
            int index = Integer.parseInt(list[0]);
            String val = list[1];
            int length = val.length();
            for(int i = index; i <= length; i++)
            {
                context.write(new IntWritable(index),new Text(val));
            }       
        }

     }


public static class shinglesreducer extends Reducer<IntWritable,Text,IntWritable,ArrayList<String>> {
    private ArrayList<String> result = new ArrayList<String>();

    public void reduce(IntWritable key, Text value, Context context
            ) throws IOException, InterruptedException {
        String str = new String(value.toString());
        int newkey = key.get();
        int Tz = str.length() - newkey + 1;
        int position = 0;
        while (position <= Tz)
        {
            result.add(str.substring(position,position + newkey -1));
            position = position + 1;
        }   
        context.write(new IntWritable(newkey),result);
    }
}





public static void main(String[] args) throws Exception {

      Configuration conf = new Configuration();
      Job job = Job.getInstance(conf, "Saishingles");
      job.setJarByClass(hadoopshingles.Saishingles.class);
      job.setMapperClass(shinglesmapper.class);
      job.setCombinerClass(shinglesreducer.class);
      job.setReducerClass(shinglesreducer.class);
      job.setMapOutputKeyClass(IntWritable.class);
      job.setMapOutputValueClass(Text.class);
      job.setOutputKeyClass(IntWritable.class);
      job.setOutputValueClass(ArrayList.class);
      FileInputFormat.addInputPath(job, new Path(args[0]));
      FileOutputFormat.setOutputPath(job, new Path(args[1]));
      System.exit(job.waitForCompletion(true) ? 0 : 1);

}

}

以下错误：

Exception in thread "main" java.lang.ClassNotFoundException: hadoopshingles.Saishingles
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:278)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

请帮助我，提前谢谢你：）

please help me and thank you in advance :)

推荐答案

我相信你应该不要在类名中包含 .class 。

I believe you should not be including ".class" in the classname.

而不是

job.setJarByClass（hadoopshingles.Saishingles.class）;

job.setJarByClass(hadoopshingles.Saishingles.class);

应该是

job.setJarByClass（hadoopshingles.Saishingles）;

job.setJarByClass(hadoopshingles.Saishingles);

这篇关于Hadoop中的ClassNotFoundException的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！