HDFS复制因素发生变化 [英] HDFS replication factor change

问题描述

如果群集中的复制因子已更改（例如从5更改为3），并且群集重新启动，那么旧文件块会发生什么情况？他们是否会被视为过度复制并被删除或复制因素仅适用于新文件？这意味着旧文件块被复制5次，新文件块（重启后）被复制3次。
如果群集未重新启动，会发生什么情况？

If the replication factor is changed in the cluster,say, from 5 to 3 and the cluster is restarted, what happens to the old file blocks? Will they be considered as over replicated and get deleted or replication factor is applicable to only new files? Which means old file blocks are replicated 5 times and the new file blocks (after restart) are replicated 3 times. What happens if the cluster is not restarted?

推荐答案

如果复制因子在群集中发生更改，例如从5更改为3，集群重新启动，旧文件块会发生什么变化？

If the replication factor is changed in the cluster,say, from 5 to 3 and the cluster is restarted, what happens to the old file blocks?

现有/旧文件块没有任何反应。

Nothing happens to existing/old file blocks.

它们会被视为过度复制并被删除，或者复制因子仅适用于新文件？

Will they be considered as over replicated and get deleted or replication factor is applicable to only new files?

新复制因子只适用于新文件，因为复制因子不是HDFS范围设置，而是每个文件属性。

The new replication factor will only apply to new files, as replication factor is not a HDFS-wide setting but a per-file attribute.

这意味着旧的文件块被复制5次，新的文件块（重新启动后）被复制3次。

Which means old file blocks are replicated 5 times and the new file blocks (after restart) are replicated 3 times.

它是这个的倒置。复制因子设置为3的现有文件将继续携带3个块。使用更高默认复制因子创建的新文件将携带5个块。

Its the invert of this. Existing files with replication factor set to 3 will continue to carry 3 blocks. New files created with a higher default replication factor will carry 5 blocks.

如果群集未重新启动，会发生什么情况？

What happens if the cluster is not restarted?

如果重新启动或不重新启动群集，则不会发生任何情况。由于属性是每个文件，并且在创建文件时由客户端引导，所以不需要重新启动集群即可更改此配置。您只需更新您的客户端配置。

Nothing happens if you do restart or don't restart your cluster. Since the property is per-file and is guided by clients when creating a file, a cluster restart isn't required to change this config either. You only need to update your client configs.

如果您希望更改所有旧文件的复制因子，请考虑运行复制更改程序命令： hadoop fs -setrep -R 5 /

If you look to change all your old files' replication factor, consider running the replication changer command: hadoop fs -setrep -R 5 /

这篇关于HDFS复制因素发生变化的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！