为什么Hbase需要WAL? [英] Why Hbase need WAL？

问题描述

我是Hbase的新手，我发现Hbase会将所有操作写入WAL和memstore.

Q1:我想知道为什么Hbase需要WAL?

Q2:每次放置或删除数据时，Hbase都必须写入WAL，为什么 不只是在其数据文件中对其进行操作?

解决方案

Q1)为什么Hbase需要WAL?

WAL用于恢复目的.通过 MapR文档 ，可以使您更加了解hbase架构. /p>

当客户端发出Put请求时，第一步是将数据写入预写日志WAL:

• 将编辑内容附加到存储在磁盘上的WAL文件的末尾.
• WAL用于在服务器崩溃时恢复尚未持久的数据.

将数据写入WAL之后，它将放置在MemStore中.然后，放置请求确认将返回给客户端.

Q2)每次我放置或删除数据时，Hbase都必须写入WAL，为什么不仅仅在其数据文件中对其进行操作?

如果启用了 .. 是

如果WAL被禁用 ，则可以通过消除写入WAL的额外开销来直接对文件进行操作.

注意:

一般情况下WAL将被禁用以进行突变(行级突变)/执行写入操作.如果这样做的话，根本的警告是，不要恢复……意味着数据丢失.另外，如果您正在使用SOLR，它将在WAL上运行，因此SOLR文档将不会更新.如果没有，您可以禁用WAL

进一步阅读，请参见我的回答

I'm new to Hbase, and I found that Hbase will write all the operations to WAL and memstore.

Q1: I wonder why Hbase need WAL?

Q2 : Hbase must write to WAL every time when I put or delete data, why don't operate it just in its data file?

解决方案

Q1) why Hbase need WAL?

WAL is for recovery purpose. lets understand hbase architecture in a close way by MapR docs.

When the client issues a Put request, the first step is to write the data to the write-ahead log, the WAL:

• Edits are appended to the end of the WAL file that is stored on disk.
• The WAL is used to recover not-yet-persisted data in case a server crashes.

Once the data is written to the WAL, it is placed in the MemStore. Then, the put request acknowledgement returns to the client.

Q2) Hbase must write to WAL every time when I put or delete data, why don't operate it just in its data file?

If WAL is enabled.. Yes

If WAL is disabled it can operate on the files directly by removing additional overhead of writing in to WAL.

NOTE:

General cases WAL will be disabled for mutation(Row-level mutations)/write performance purpose. Underlying caveat if you do so is, dont wont be recoverable... means data loss. Also if you are using SOLR, which will work on WAL and hence SOLR documents wont be updated. if dont have the case you can go ahead with disabling WAL

Further reading see my answer here

这篇关于为什么Hbase需要WAL?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！