Hadoop-HDFS-用于查看文件拆分方式的命令 [英] Hadoop - HDFS - Command to see how a file's splits

问题描述

Hadoop FsShell(或hdfs命令)中是否存在命令行以查看文件的拆分情况，或查看将文件放入HDFS后如何在数据节点上拆分文件?

Is there a command line in Hadoop FsShell (or hdfs command) to see what are the splits of a file or to see how a file has been splitted on the data nodes when put in HDFS ?

推荐答案

hdfs fsck似乎就是您想要的:

hdfs fsck seems to be what you're after:

$ hdfs fsck /import/collections/part-00000 -files -blocks
Connecting to namenode via http://vm28-hulk-priv:50070
FSCK started by usrhadoop (auth:SIMPLE) from /10.237.241.28 for path /import/collections/part-00000 at Thu Mar 19 07:35:15 EDT 2015
/import/collections/part-00000 3620228 bytes, 1 block(s):  OK
0. BP-1201623000-10.237.241.28-1421858661680:blk_1074635302_894483 len=3620228 repl=3

Status: HEALTHY
 Total size:    3620228 B
 Total dirs:    0
 Total files:   1
 Total symlinks:                0
 Total blocks (validated):      1 (avg. block size 3620228 B)
 Minimally replicated blocks:   1 (100.0 %)
 Over-replicated blocks:        0 (0.0 %)
 Under-replicated blocks:       0 (0.0 %)
 Mis-replicated blocks:         0 (0.0 %)
 Default replication factor:    3
 Average block replication:     3.0
 Corrupt blocks:                0
 Missing replicas:              0 (0.0 %)
 Number of data-nodes:          4
 Number of racks:               1
FSCK ended at Thu Mar 19 07:35:15 EDT 2015 in 1 milliseconds


The filesystem under path '/import/collections/part-00000' is HEALTHY

这篇关于Hadoop-HDFS-用于查看文件拆分方式的命令的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！