如何单独改变HIVE的HDFS复制因子 [英] How to change HDFS replication factor for HIVE alone

问题描述

我们当前的HDFS集群具有复制因子1.但是为了提高性能和可靠性（节点故障），我们希望将Hive中间文件（hive.exec.scratchdir）复制因子单独增加到5.是否可以实现？

问候，
Selva

解决方案

strong> -setrep 可以帮助您。

>

用法：

  hadoop fs -setrep [ -R] [-w]< numReplicas> <路径> 
  

更改 复制因子 的文件。如果 path是一个目录，则递归命令会更改所有文件的 复制因子

选项：

  -w标志请求该命令等待复制完成。这可能需要很长时间。 
 
 -R标志被接受用于向后兼容。它没有效果。 
  

示例：

  hadoop fs -setrep -w 3 / user / hadoop / dir1 
 
 hadoop fs -setrep -R -w 100 / path / to / hive / warehouse 
  

参考： - setrep

Our current HDFS Cluster has replication factor 1.But to improve the performance and reliability(node failure) we want to increase Hive intermediate files (hive.exec.scratchdir) replication factor alone to 5. Is it possible to implement that ?

Regards, Selva

解决方案

See if -setrep helps you.

setrep

Usage:

hadoop fs -setrep [-R] [-w] <numReplicas> <path>

Changes the replication factor of a file. If path is a directory then the command recursively changes the replication factor of all files under the directory tree rooted at path.

Options:

The -w flag requests that the command wait for the replication to complete. This can potentially take a very long time.

The -R flag is accepted for backwards compatibility. It has no effect.

Example:

hadoop fs -setrep -w 3 /user/hadoop/dir1

hadoop fs -setrep -R -w 100 /path/to/hive/warehouse

Reference: -setrep

这篇关于如何单独改变HIVE的HDFS复制因子的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！