hadoop java.net.URISyntaxException：绝对URI中的相对路径：rsrc：hbase-common-0.98.1-hadoop2.jar [英] hadoop java.net.URISyntaxException: Relative path in absolute URI: rsrc:hbase-common-0.98.1-hadoop2.jar

问题描述

我有一个连接到HBASE的map reduce作业，我无法弄清楚我跑到这个错误的位置：

 异常在线程 主 java.lang.reflect.InvocationTargetException 
。在sun.reflect.NativeMethodAccessorImpl.invoke0（本机方法）
。在sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:57） 
 at sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:43）
 at java.lang.reflect.Method.invoke（Method.java:606）
 at org.eclipse.jdt .internal.jarinjarloader.JarRsrcLoader.main（JarRsrcLoader.java:58）
在sun.reflect.NativeMethodAccessorImpl.invoke0（本机方法）
在sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl.java:57） 
 at sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:43）
 at java.lang.reflect.Method.invoke（Method.java:606）
 at o rg.apache.hadoop.util.RunJar.main（RunJar.java:212）
导致：java.lang.IllegalArgumentException：java.net.URISyntaxException：绝对URI中的相对路径：rsrc：hbase-common-0.98 .1-hadoop2.jar 
 at org.apache.hadoop.fs.Path.initialize（Path.java:206）
 at org.apache.hadoop.fs.Path。< init>（Path的.java：172）在org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.findOrCreateJar（TableMapReduceUtil.java:703 
）
。在org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars（TableMapReduceUtil的.java：656）在org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addHBaseDependencyJars 
（TableMapReduceUtil.java:573）
在org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars（TableMapReduceUtil的.java：617）
。在在org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.con org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad（HFileOutputFormat2.java:398）
 figureIncrementalLoad（HFileOutputFormat2.java:356）
在com.ancestry.bigtree.hfile.JsonToHFileDriver.run（JsonToHFileDriver.java:117）
在org.apache.hadoop.util.ToolRunner.run（ToolRunner。 java：70）
 at com.ancestry.bigtree.hfile.JsonToHFileDriver.main（JsonToHFileDriver.java:69）
 ... 10 more 
导致：java.net.URISyntaxException：相对在绝对URI路径：rsrc：hbase-common-0.98.1-hadoop2.jar $ b $在java.net.URI.checkPath（URI.java:1804）
在java.net.URI。< init>（URI.java:752）
 at org.apache.hadoop.fs.Path.initialize（Path.java:203）
  

如果我没有Hbase库，作业运行良好。相对路径在哪里产生？我如何强制生成的路径是绝对的？

在我的代码中，我有以下两行：

TableMapReduceUtil.addHBaseDependencyJars（conf）;
HFileOutputFormat2.configureIncrementalLoad（job，htable）;

如果我删除它们，我很好，但是这项工作并没有做到我所需要的。我最终试图创建HFILE以与hbase bulkloader一起使用。

环境：
HBase 0.96.1.2.0.10.0-1-hadoop2
Hadoop 2.2.0.2.0.10.0-1

预先感谢您的帮助和指导。 有点误导;没有真正的相对路径被解析，这里的问题是HadoopPath不支持文件名中的：。在你的情况下，rsrc：hbase-common-0.98.1-hadoop2.jar被解释为rsrc是scheme，而我怀疑你真的打算添加资源文件：/// path / to /your/jarfile/rsrc:hbase-common-0.98.1-hadoop2.jar。这里有一个旧的JIRA讨论非法字符：

https://issues.apache.org/jira/browse/HADOOP-3257

请注意，您可能无法使用该绝对路径，因为文件名中仍然有'：'，您可以尝试转义文件名，如rsrc％3Ahbase-common-0.98.1-hadoop2。 jar，但在使用它的另一端可能无法正确找到它。

解决此问题的最佳方法是解决根本原因引入了rsrc：hbase-common-0.98.1-hadoop2.jar - 使用Eclipse构建可运行jar是可能导致此问题的原因之一。如果可能，请尝试使用Eclipse以外的其他工具构建jar，并参阅如果发生同样的问题;你也可以尝试在Eclipse中创建jar时选择将所需的库包装到生成的jar中。

如果uber-jar过大，您也可以尝试把像hbase-common-0.98.1-hadoop2.jar这样的原始依赖关系jar放入所有节点的类路径中，以及其他可能需要的依赖关系，然后跳过对TableMapReduceUtil.addHBaseDependencyJars（conf）;的调用。

以下是另一位用户的旧线程，其中会遇到类似于您所看到的问题：

< a href =http://lucene.472066.n3.nabble.com/Error-while-running-MapR-program-on-multinode-configuration-td4053610.html> http：//lucene.472066.n3.nabble .com /运行中的错误 - 运行MapR - 多节点配置-td4053610.html

I have a map reduce job that connects to HBASE and I can't figure out where I am running into this error:

Exception in thread "main" java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: rsrc:hbase-common-0.98.1-hadoop2.jar
        at org.apache.hadoop.fs.Path.initialize(Path.java:206)
        at org.apache.hadoop.fs.Path.<init>(Path.java:172)
        at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.findOrCreateJar(TableMapReduceUtil.java:703)
        at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:656)
        at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addHBaseDependencyJars(TableMapReduceUtil.java:573)
        at org.apache.hadoop.hbase.mapreduce.TableMapReduceUtil.addDependencyJars(TableMapReduceUtil.java:617)
        at org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:398)
        at org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:356)
        at com.ancestry.bigtree.hfile.JsonToHFileDriver.run(JsonToHFileDriver.java:117)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
        at com.ancestry.bigtree.hfile.JsonToHFileDriver.main(JsonToHFileDriver.java:69)
        ... 10 more
Caused by: java.net.URISyntaxException: Relative path in absolute URI: rsrc:hbase-common-0.98.1-hadoop2.jar
        at java.net.URI.checkPath(URI.java:1804)
        at java.net.URI.<init>(URI.java:752)
        at org.apache.hadoop.fs.Path.initialize(Path.java:203)

If I don't have Hbase libraries, the job runs fine. Where is the relative path being generated? How can I force the generated paths to be absolute?

In my code I have these two lines:

TableMapReduceUtil.addHBaseDependencyJars(conf); HFileOutputFormat2.configureIncrementalLoad(job, htable);

if I remove them I am ok but the job does not do what I need it to do. I am ultimately trying to create HFILE to use with hbase bulkloader.

Environment: HBase 0.96.1.2.0.10.0-1-hadoop2 Hadoop 2.2.0.2.0.10.0-1

Thank you in advance for any help or direction.

解决方案

The exception is a bit misleading; there's no real relative path being parsed, the issue here is that Hadoop "Path" doesn't support ':' in filenames. In your case, "rsrc:hbase-common-0.98.1-hadoop2.jar" is being interpreted as "rsrc" being the "scheme", whereas I suspect you really intended to add the resource file:///path/to/your/jarfile/rsrc:hbase-common-0.98.1-hadoop2.jar". Here's an old JIRA discussing the illegal character:

https://issues.apache.org/jira/browse/HADOOP-3257

Note that you probably won't be able use that absolute path either, since it still has ':' in the filename. You can try escaping the filename like "rsrc%3Ahbase-common-0.98.1-hadoop2.jar", but then it may not be found correctly on the other end where it is being used either.

The best way to fix this is to tackle the root cause of "rsrc:hbase-common-0.98.1-hadoop2.jar" being introduced--using Eclipse to build your runnable jar is one likely cause of the issue. If possible, try to build your jar using something other than Eclipse and see if the same problem occurs; you can also try to select "Package required libraries into generated jar" when creating your jar in Eclipse.

If the uber-jar ends up too large, you can also try to place the original dependency jars like hbase-common-0.98.1-hadoop2.jar into the classpath on all your nodes along with any other dependencies you may need, and then skip the call to "TableMapReduceUtil.addHBaseDependencyJars(conf);".

Here's an old thread of another user running into a similar problem as what you're seeing:

http://lucene.472066.n3.nabble.com/Error-while-running-MapR-program-on-multinode-configuration-td4053610.html

这篇关于hadoop java.net.URISyntaxException：绝对URI中的相对路径：rsrc：hbase-common-0.98.1-hadoop2.jar的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！