如何在猪文件中使用.jar [英] How to use .jar in a pig file

问题描述

我有两个输入文件smt.txt和smo.txt。 jar文件读取文本文件并根据java文件中描述的某些规则拆分数据。

 注册'maprfs：/// user / username / fl.jar; 
 DEFINE FixedLoader fl（）; 
 
 mt =加​​载'maprfs：///user/username/smt.txt'使用FixedLoader（' -  30'，'30-33'，........... ）AS（.........）; 
 
 mo =加载'maprfs：///user/username/smo.txt*'使用FixedLoader（' -  30'，'30-33'，.....）AS（.. ....）; 
 
使用JsonStorage（）将mt存入'maprfs：/// user / username / mt_out'; 
使用JsonStorage（）将mo存储到'maprfs：/// user / username / mo_out'中; 
  

以及下面的一部分java代码。 （我相信方法的内容不是必需的）：

  package com.mapr.util; 
 
 import org.apache.hadoop.mapreduce.lib.input。*; 
 import org.apache.hadoop.mapreduce。*; 
 import org.apache.hadoop.io。*; 
 import org.apache.pig。*; 
 import org.apache.pig.data。*; 
 import org.apache.pig.backend.hadoop.executionengine.mapReduceLayer。*; 
 import java.util。*; 
 import java.io. *; 
 
 public class FixedLoader extends LoadFunc 
 {
 
 ............ 
 
} 
  

当我在命令pig -x mapreduce sample.pig的末尾运行这个猪程序时，我给出了错误消息：

ERROR org.apache.pig.tools.grunt.Grunt - 错误1070：无法解析com.mapr.util .FixedLoader使用导入：[，org.apache.pig.builtin。，org.apache.pig.impl.builtin。]

如何将这些文件导入到我的项目中，或者是否有任何建议/解决方案来运行此程序？

FixedLoader 及其完整软件包名称：

  register'maprfs：///用户/用户名/ fl.jar'; 
 DEFINE FixedLoader com.mapr.util.FixedLoader（）; 
 ... 
  

还要注册您的所有第三方依赖关系罐自定义UDF。

I have two input files smt.txt and smo.txt. The jar file reads the text files and split the data according to some rule which is described in java file. And the pig file takes these data put into output files with doing mapreduce.

register 'maprfs:///user/username/fl.jar';
DEFINE FixedLoader fl();

mt = load 'maprfs:///user/username/smt.txt' using FixedLoader('-30','30-33',...........) AS (.........);

mo = load 'maprfs:///user/username/smo.txt*' using FixedLoader('-30','30-33',.....) AS (......);

store mt into 'maprfs:///user/username/mt_out' using JsonStorage();
store mo into 'maprfs:///user/username/mo_out' using JsonStorage();

and a part of java code like in the following. (The content of methods are not neccessary I believe):

package com.mapr.util;

import org.apache.hadoop.mapreduce.lib.input.*;
import org.apache.hadoop.mapreduce.*;
import org.apache.hadoop.io.*;
import org.apache.pig.*;
import org.apache.pig.data.*;
import org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.*;
import java.util.*;
import java.io.*;

public class FixedLoader extends LoadFunc
{

............

}

When I run this pig program in a teminal with the command "pig -x mapreduce sample.pig", I gave an Error message:

ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1070: Could not resolve com.mapr.util.FixedLoader using imports: [, org.apache.pig.builtin., org.apache.pig.impl.builtin.]

How can I import these into my project or are there any suggestions/solutions to run this program?

解决方案

You need to define FixedLoader with its full package name:

register 'maprfs:///user/username/fl.jar';
DEFINE FixedLoader com.mapr.util.FixedLoader();
...

Also register all of the 3rd party dependency jars that are used in your custom UDF.

这篇关于如何在猪文件中使用.jar的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！