HDFS是否提供异步写入? [英] Does HDFS provide asynchronous writes?

问题描述

通过浏览API和文档，我得到的印象是hdfs不提供完全异步的写入操作.在写操作返回之前，它必须等待该块的最小复制(默认为1)完成.有什么方法可以使用HDFS API完全异步编写吗? 谢谢.

From going through the API and the documentation, I get the impression that hdfs does not provide completely asynchronous write operations. It has to wait for minimum replication (by default 1) of the block to be completed before the write operation returns. Is there any way to write completely asynchronously using HDFS API? Thanks.

推荐答案

不，HDFS API没有提供异步写入的任何功能，因为它应该提供持久性和阻止写入.一个好主意是，如果性能是目标，则在写入HDFS之前，如果可能的话，将数据累积到最小块大小.但是，如果计算机出现故障，而您尚未将数据写入HDFS，则数据将丢失.

No, HDFS API does not have any function to provide for asynchronous writes, as it is supposed to provide persistence and blocking write. One good idea is to accumulate the data if possible to a minimum chunk size before writing to HDFS, if performance is the goal. However, if the machine fails and you haven't written data to HDFS the data will be lost.

这篇关于HDFS是否提供异步写入?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！