无法从命令行获取镶木地板工具 [英] Unable to get parquet-tools working from the command-line

问题描述

我正在尝试运行最新版本的 parquet-tools，但遇到了一些问题.出于某种原因，org.apache.hadoop.conf.Configuration 不在阴影 jar 中.(我对 v1.6.0 也有同样的问题).

I'm attempting to get the newest version of parquet-tools running, but I'm having some issues. For some reason org.apache.hadoop.conf.Configuration isn't in the shaded jar. (I have the same issue with v1.6.0 as well).

在 mvn package 或 mvn install 之外还有什么我应该做的吗?(我使用的实际 mvn 调用是 mvn install -DskipTests -pl \!parquet-thrift,\!parquet-cascading,\!parquet-pig-bundle,\!parquet-猪,\!parquet-scrooge,\!parquet-hive,\!parquet-protobuf).这工作得很好，如果我选择运行它们，测试就会通过.

Is there something beyond mvn package or mvn install that I should be doing? (The actual mvn invocation I'm using is mvn install -DskipTests -pl \!parquet-thrift,\!parquet-cascading,\!parquet-pig-bundle,\!parquet-pig,\!parquet-scrooge,\!parquet-hive,\!parquet-protobuf). This works just fine, and the tests pass if I choose to run them.

我得到的错误如下(你可以看到我试图从旧的镶木地板版本中粘贴 hadoop jar，该版本似乎将它捆绑到类路径中；无论有没有它，我都得到了相同的结果).

The error I get is below (You can see I've attempted to stick the hadoop jar from an old parquet version that seemed to bundle it into the classpath; I get the same results with or without it).

> java -classpath /path/to/hadoop-core-1.1.0.jar -jar parquet-tools-1.7.0-incubating-SNAPSHOT.jar meta --debug part-r-00000.gz.parquet

java.lang.NoClassDefFoundError: org/apache/hadoop/conf/Configuration
    at parquet.tools.command.ShowMetaCommand.execute(ShowMetaCommand.java:59)
    at parquet.tools.Main.main(Main.java:222)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 2 more
org/apache/hadoop/conf/Configuration

推荐答案

在使用自制软件的 MacOS 上，这是最简单的入门方法:

On MacOS using homebrew, this is the easiest way to get started:

$ brew install parquet-tools

这篇关于无法从命令行获取镶木地板工具的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！