Hadoop Rest API用于上传/下载 [英] Hadoop Rest API for upload / download

问题描述

我试图使用C＃应用程序从Hadoop集群上传/下载文件，但是我无法找到用于上传的API并从文档中下载。

那么你能让我知道如何使用RestAPI上传和下载Hadoop中的文件吗？
$ b

谢谢

nofollow> http://hadoop.apache.org/docs/r1.0.4/webhdfs.html

编辑：

创建并写入文件

第1步：

提交一个HTTP PUT请求，但不会自动跟踪重定向，也不发送文件数据。
$ b

curl -i -X PUThttp：//：/ webhdfs / v1 /？op = CREATE
[& overwrite =] [& blocksize =] [& replication =]
[& permission =] [&安培;

请求被重定向到要写入文件数据的数据节点：$ b​​ $ b HTTP / 1.1 307 TEMPORARY_REDIRECT
位置：http：//：/ webhdfs / v1 /？op = CREATE ...
Content-Length：0

步骤2：使用位置标题中的网址提交另一个HTTP PUT请求，其中包含要写入的文件数据。

curl -i -X PUT -T
http：//：/ webhdfs / v1 /？op = CREATE ...

客户端收到一个201创建的响应，其内容长度为零，位置标题中的文件的WebHDFS URI为：
HTTP / 1.1 201创建
位置：webhdfs：//：/
内容长度：0

请注意，有两步创建/附加是为了防止客户端在重定向之前发送数据。 HTTP / 1.1中的Expect：100-continue标题解决了此问题;请参阅RFC 2616，第8.2.3节。不幸的是，有软件库错误（例如Jetty 6 HTTP服务器和Java 6 HTTP客户端），它们没有正确实现Expect：100-continue。两步创建/追加是软件库bug的临时解决方法。

I am trying to perform upload/download a file from Hadoop cluster, using a C# app, but I couldn't find the APIs for Upload and download from the documentation.

So can you please let me know how to upload and download files from Hadoop using RestAPIs?

Thanks

解决方案

You can use the WebHDFS REST API as described here http://hadoop.apache.org/docs/r1.0.4/webhdfs.html

Edit:

Create and Write to a File

Step 1:

Submit a HTTP PUT request without automatically following redirects and without sending the file data.

curl -i -X PUT "http://:/webhdfs/v1/?op=CREATE [&overwrite=][&blocksize=][&replication=] [&permission=][&buffersize=]"

The request is redirected to a datanode where the file data is to be written: HTTP/1.1 307 TEMPORARY_REDIRECT Location: http://:/webhdfs/v1/?op=CREATE... Content-Length: 0

Step 2:

Submit another HTTP PUT request using the URL in the Location header with the file data to be written.

curl -i -X PUT -T "http://:/webhdfs/v1/?op=CREATE..."

The client receives a 201 Created response with zero content length and the WebHDFS URI of the file in the Location header: HTTP/1.1 201 Created Location: webhdfs://:/ Content-Length: 0

Note that the reason of having two-step create/append is for preventing clients to send out data before the redirect. This issue is addressed by the "Expect: 100-continue" header in HTTP/1.1; see RFC 2616, Section 8.2.3. Unfortunately, there are software library bugs (e.g. Jetty 6 HTTP server and Java 6 HTTP client), which do not correctly implement "Expect: 100-continue". The two-step create/append is a temporary workaround for the software library bugs.

这篇关于Hadoop Rest API用于上传/下载的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！