HBase连接池，用于非常频繁地扫描行 [英] HBase connection pooling for very frequent scanning of row

问题描述

我必须在一小时内非常频繁地(〜百万次)扫描表中的行.我有关于rowid(这是一个字节数组)的信息.我正在创建用于创建起始行和结束行的rowid，在我的情况下该行ID基本上是相同的.

I have to scan the table for row very frequent(~million times) in an hour. I have the information about rowid(which is a byte array). I am creating rowid for creating startrow and endrow which are essentially the same in my case.

     public String someMethod(byte[] rowid){
            if (aTable == null) {
                  aTable = new HTable(Config.getHadoopConfig(),
                  Config.getATable());     
            }
            byte[] endRow = new byte[rowId.length];
            endrow = System.copyArray(rowId, 0, endRow, 0, rowId.length)
            Scan scan = new Scan(rowId , endRow)
            //scanner implementation and iteration over the result
            (ResultScanner result = aTable.getScanner(scan);) {
                   for (Result item : result) {

                   }
            }
     }

我想知道是否可以实现一些连接池以提高性能. HBase Java API中是否有可用的池化机制?我正在使用HBase的0.96.x版本.另外，是否有任何可以提高性能的配置设置.谢谢

I am wondering whether I can implement some connection pooling for improving the performance. IS there any pooling mechanism available in HBase Java API. I am using 0.96.x version of HBase. Also, is there any configuration setting which can improve the performance. Thanks

推荐答案

来自 http: //hbase.apache.org/book.html

连接池

对于需要高端多线程访问的应用程序(例如， 可以为许多应用程序提供服务的Web服务器或应用程序服务器 单个JVM中的多个线程)，您可以预先创建一个HConnection，如图所示 在以下示例中:

For applications which require high-end multithreaded access (e.g., web-servers or application servers that may serve many application threads in a single JVM), you can pre-create an HConnection, as shown in the following example:

示例9.1.预先创建HConnection

Example 9.1. Pre-Creating a HConnection

// Create a connection to the cluster.
HConnection connection = HConnectionManager.createConnection(Configuration);
HTableInterface table = connection.getTable("myTable");
// use table as needed, the table returned is lightweight
table.close();
// use the connection for other access to the cluster
connection.close();

这篇关于HBase连接池，用于非常频繁地扫描行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！