Webhdfs返回错误的datanode地址 [英] Webhdfs returns wrong datanode address
问题描述
curl -i -X PUThttp:// SomeHostname:50070 / webhdfs / v1 / file1?op = CREATE
HTTP / 1.1 307 TEMPORARY_REDIRECT
Content-Type:application / octet-stream
位置:http:// sslave0:50075 / webhdfs / v1 / file1?op = CREATE& overwrite = false
Content-Length:0
Server :Jetty(6.1.26)
在这里它返回datanode的sslave0,看起来像是我的内部地址
在WebHDFS中,NameNode Web界面@port 50070接受put请求并分配关于文件的元数据信息被存储。然后它返回一个重定向到一个数据节点地址(ssalve0:50075地址),其中实际的Http PUT已经完成。
底线 - 名称节点不存储任何真实的数据,只有元数据。它是存储文件的实际信息的数据节点 - 因此是重定向如果你想拥有一个你不需要处理的代理节点数据节点以这种方式,那么你想配置 HttpFs 代替。
curl -i -X PUT "http://SomeHostname:50070/webhdfs/v1/file1?op=CREATE"
HTTP/1.1 307 TEMPORARY_REDIRECT
Content-Type: application/octet-stream
Location: http://sslave0:50075/webhdfs/v1/file1?op=CREATE&overwrite=false
Content-Length: 0
Server: Jetty(6.1.26)
here it return sslave0 for datanode, seem like an internal address to me
With WebHDFS, the NameNode web interface @port 50070 in your case accepts the put request and assigns the metadata information about the file to be stored. It then returns a redirect to a Data Node address (the ssalve0:50075 address) where the actual Http PUT is completed.
Bottom line - the Name Node doesn't store any real data, only metadata. It's the Data Nodes which store the actual information for a file - hence the redirect
If you want to have a single proxy node by which you don't have to deal with the data nodes in this way, then you want to configure HttpFs instead.
这篇关于Webhdfs返回错误的datanode地址的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!