如何使用 Ruby 在 Hadoop HDFS 中写入和读取文件? [英] How to write and read files in/from Hadoop HDFS using Ruby?

问题描述

有没有办法使用 Ruby 来处理 HDFS Api?据我所知，没有多语言文件 Api，唯一的方法是使用本机 Java Api.我尝试使用 JRuby，但此解决方案不稳定且不是很原生.此外，我查看了 HDFS Thrift Api，但它并不完整，而且还缺少许多功能(例如写入索引文件).

Is there a way to work with HDFS Api using Ruby? As I can understand there is no multilanguage file Api and the only way is to use native Java Api. I tried using JRuby but this solution is to unstable and not very native. Also I looked at HDFS Thrift Api but it's not complete and also lacks many features (like writing to indexed files).

除了使用 JRuby 或 Thrift Api 之外，还有没有办法使用 Ruby 来处理 HDFS?

Is there a way to work with HDFS using Ruby besides from using JRuby or Thrift Api?

推荐答案

github 中有两个项目可以满足您的要求.ruby-hdfs 为 Ruby 提供到 HDFS 的本机 C 绑定.ganapati 与 Thrift 服务器接口.

There are two projects in github that fit what you're asking. ruby-hdfs provides native C bindings to HDFS for Ruby. ganapati interfaces with a Thrift server.

您还可以直接对文件系统外壳进行系统调用.例如:

You could also make system calls directly to the file system shell. For example:

cmd = "hadoop fs -mkdir #{hdfs_path}"
cmd += " 2> /dev/null"
system(cmd)
if $? == 0
  puts 'ok'
  exit(0)
else
  puts "Error: failed to create hdfs://#{hdfs_path}"
  exit(2)
end

这篇关于如何使用 Ruby 在 Hadoop HDFS 中写入和读取文件?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！