如何在Java程序中使用Sqoop？ [英] How to use Sqoop in Java Program?

问题描述

我知道如何通过命令行使用sqoop。
但是不知道如何使用java程序调用sqoop命令。
任何人都可以提供一些代码视图吗？

解决方案

您可以从java代码中运行sqoop，方法是将sqoop jar在你的classpath中调用 Sqoop.runTool（）方法。您必须以编程方式创建所需的参数给sqoop，就好像它是命令行一样（例如 - connect 等）。

请注意以下几点：

• 确保sqoop工具名称（例如导入/导出等）为第一个参数。
  
• 注意类路径排序 - 执行可能失败，因为sqoop需要库的版本X并且使用不同的版本。确保sqoop所需的库不会被你自己的依赖所掩盖。我遇到了commons-io（sqoop需要v1.4）这样的问题，并且因为我使用了commons-io v1.2而出现NoSuchMethod异常。


• 每个参数都需要在一个单独的数组元素上。例如，--connect jdbc：mysql：...应该作为数组中的两个单独元素传递，而不是一个。


• sqoop解析器知道如何接受双引号参数，所以如果你需要使用双引号（我总是建议）。唯一的例外是fields-delimited-by参数需要单个字符，所以不要双引号。


• 我建议分割命令行参数创建逻辑和实际的执行，所以你的逻辑可以在没有真正运行该工具的情况下被正确地测试。
  
• 最好使用--hadoop-home参数，以防止依赖于环境。


• 与 Sqoop.Main（） Sqoop.runTool（） c $ c>是这样一个事实： runTool（）返回执行的错误代码。
  

希望有帮助。

  final int ret = Sqoop.runTool（new String [] {...} ）; 
 if（ret！= 0）{
 throw new RuntimeException（Sqoop failed  -  return code+ Integer.toString（ret））; 
} 
  

RL

I know how to use sqoop through command line. But dont know how to call sqoop command using java programs . Can anyone give some code view?

解决方案

You can run sqoop from inside your java code by including the sqoop jar in your classpath and calling the Sqoop.runTool() method. You would have to create the required parameters to sqoop programmatically as if it were the command line (e.g. --connect etc.).

Please pay attention to the following:

• Make sure that the sqoop tool name (e.g. import/export etc.) is the first parameter.
• Pay attention to classpath ordering - The execution might fail because sqoop requires version X of a library and you use a different version. Ensure that the libraries that sqoop requires are not overshadowed by your own dependencies. I've encountered such a problem with commons-io (sqoop requires v1.4) and had a NoSuchMethod exception since I was using commons-io v1.2.
• Each argument needs to be on a separate array element. For example, "--connect jdbc:mysql:..." should be passed as two separate elements in the array, not one.
• The sqoop parser knows how to accept double-quoted parameters, so use double quotes if you need to (I suggest always). The only exception is the fields-delimited-by parameter which expects a single char, so don't double-quote it.
• I'd suggest splitting the command-line-arguments creation logic and the actual execution so your logic can be tested properly without actually running the tool.
• It would be better to use the --hadoop-home parameter, in order to prevent dependency on the environment.
• The advantage of Sqoop.runTool() as opposed to Sqoop.Main() is the fact that runTool() return the error code of the execution.

Hope that helps.

final int ret = Sqoop.runTool(new String[] { ... });
if (ret != 0) {
  throw new RuntimeException("Sqoop failed - return code " + Integer.toString(ret));
}

RL

这篇关于如何在Java程序中使用Sqoop？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！