HBase：我如何估算HBase表的大小？ [英] HBase: How can I estimate the size of a HBase table?

问题描述

我有多个HBase表，我怎么估计在java中使用的表的大小？

解决方案

你必须通过 / hbase 文件夹
所有表格信息使用java客户端访问hdfs。将会出现。

Hadoop shell：

您可以使用 hadoop fs -du ** hbase的路径** / hbase

在/ hbase下每个表占用多个文件夹...

hadoop fs -ls -R ** hbase **路径** / hbase

hadoop fs -du -h ** hbase的路径** / hbase / tablename

Java HDFS客户端：

通过在hbase root dir下传递每个表路径，像下面那样，您可以使用java hdfs客户端：
检查 getSizeOfPaths & getSizeOfDirectory 方法

  public class HdfsUtil {
 / ** 
 *通过获取路径的大小并除以splitSize来估计拆分的数量。 
 * 
 * @param paths 
 * @param配置
 * @param splitSize 
 * @return 
 * @throws IOException 
 * / 
 public static long getNumOfSplitsForInputs（Path [] paths，Configuration configuration，long splitSize）throws IOException 
 {
 long size = getSizeOfPaths（paths，configuration）; 
 long splits =（int）Math.ceil（size /（splitSize））; 
回报拆分; 
 
 
 public static long getSizeOfPaths（Path [] paths，Configuration configuration）throws IOException 
 {
 long totalSize = 0L; 
 
（路径路径）
 {
 totalSize + = getSizeOfDirectory（path，configuration）; 
} 
返回totalSize; 
} 
 //这里您可以给出通过shell描述的hbase路径文件夹
 public static long getSizeOfDirectory（Path path，Configuration configuration）throws IOException {
 //获取文件非注释边的大小
 FileSystem fileSystem = FileSystem.get（configuration）; 
 long size = fileSystem.getContentSummary（path）.getLength（）; 
 / **静态字符串byteCountToDisplaySize（BigInteger大小）
返回一个人类可读的文件大小版本，其中输入表示特定的字节数。** / 
 System.out。 println（FileUtils.byteCountToDisplaySize（size））
返回大小; 
} 
} 
  

I have multiple HBase tables, how can I estimate the approximate size of the tables using in java?

解决方案

One way is you have to access hdfs using java client usually under /hbase folder all the tables info. will be present.

Hadoop shell :

you can check that using hadoop fs -du -h **path to hbase**/hbase

under /hbase each table occupies one more folder...

hadoop fs -ls -R **path to hbase**/hbase

hadoop fs -du -h **path to hbase**/hbase/tablename

Java HDFS Client :

Same thing you can use java hdfs client by passing each table path under hbase root dir like below ... Check getSizeOfPaths & getSizeOfDirectory methods

public class HdfsUtil {
    /**
     * Estimates the number of splits by taking the size of the paths and dividing by the splitSize.
     *
     * @param paths
     * @param configuration
     * @param splitSize
     * @return
     * @throws IOException
     */
    public static long getNumOfSplitsForInputs(Path[] paths, Configuration configuration, long splitSize) throws IOException
    {
        long size = getSizeOfPaths(paths, configuration);
        long splits = (int) Math.ceil( size / (splitSize)) ;
        return splits;
    }

    public static long getSizeOfPaths(Path[] paths, Configuration configuration) throws IOException
    {
        long totalSize = 0L;

        for(Path path: paths)
        {
           totalSize += getSizeOfDirectory(path, configuration);
        }
        return totalSize;
    }
// here you can give hbase path folder which was described through shell
        public static long getSizeOfDirectory(Path path, Configuration configuration) throws IOException {
            //Get the file size of the unannotated Edges
            FileSystem fileSystem = FileSystem.get(configuration);
            long size  = fileSystem.getContentSummary(path).getLength();
/**static String    byteCountToDisplaySize(BigInteger size)
Returns a human-readable version of the file size, where the input represents a specific number of bytes.**/
System.out.println(FileUtils.byteCountToDisplaySize(size))
            return size;
        }
    }

这篇关于HBase：我如何估算HBase表的大小？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！