连接到本地HBase时出错 [英] Error when connecting to local HBase

问题描述

我正试图通过Java中的一个小程序连接到本地系统中安装的HBase（使用Hortonworks 1.1.1.16），该程序执行下一个命令：

I'm trying to connect to HBase installed in the local system (using Hortonworks 1.1.1.16), by a small program in Java, which executes the next command:

HBaseAdmin.checkHBaseAvailable(conf);

值得一提的是，从命令行连接到HBase时，使用 hbase 命令。

It is worth saying that there is not problem at all when connecting to HBase from the command line, using the hbase command.

主机文件的内容是下一个（其中 example.com 包含实际的主机名）：

The content of the host file is the next one (where example.com contains the actual host name):

127.0.0.1 localhost example.com

HBase配置为以独立模式工作：

HBase is configured to work in standalone mode:

hbase.cluster.distributed=false

执行程序时抛出下一个异常：

When executing the program, the next exception is thrown:

13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52 GMT
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:host.name=localhost
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:java.version=1.7.0_19
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Oracle Corporation
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.19.x86_64/jre
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:java.class.path=[...]
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-358.2.1.el6.x86_64
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:user.name=root
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:user.home=/root
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Client environment:user.dir=/root/git/project
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=example.com:2181 sessionTimeout=60000 watcher=hconnection-0x678e4593
13/05/13 15:18:29 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is hconnection-0x678e4593
13/05/13 15:18:29 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
13/05/13 15:18:29 INFO zookeeper.ClientCnxn: Socket connection established to localhost/127.0.0.1:2181, initiating session
13/05/13 15:18:29 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/127.0.0.1:2181, sessionid = 0x13e9d6851af0046, negotiated timeout = 40000
13/05/13 15:18:29 INFO client.HConnectionManager$HConnectionImplementation: ClusterId is cccadf06-f6bf-492e-8a39-e8beac521ce6
13/05/13 15:18:29 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 1 of 1 failed; no more retrying.
com.google.protobuf.ServiceException: java.io.IOException: Broken pipe
    at org.apache.hadoop.hbase.ipc.ProtobufRpcClientEngine$Invoker.invoke(ProtobufRpcClientEngine.java:149)
    at com.sun.proxy.$Proxy5.isMasterRunning(Unknown Source)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.createMasterInterface(HConnectionManager.java:732)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.createMasterWithRetries(HConnectionManager.java:764)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterProtocol(HConnectionManager.java:1724)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterMonitor(HConnectionManager.java:1757)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:837)
    at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2010)
    at TestHBase.main(TestHBase.java:37)
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:94)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:450)
    at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55)
    at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
    at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146)
    at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
    at java.io.DataOutputStream.flush(DataOutputStream.java:123)
    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.writeConnectionHeader(HBaseClient.java:896)
    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:847)
    at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1414)
    at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1299)
    at org.apache.hadoop.hbase.ipc.ProtobufRpcClientEngine$Invoker.invoke(ProtobufRpcClientEngine.java:131)
    ... 8 more
13/05/13 15:18:29 INFO client.HConnectionManager$HConnectionImplementation: Closing zookeeper sessionid=0x13e9d6851af0046
13/05/13 15:18:29 INFO zookeeper.ZooKeeper: Session: 0x13e9d6851af0046 closed
13/05/13 15:18:29 INFO zookeeper.ClientCnxn: EventThread shut down
org.apache.hadoop.hbase.exceptions.MasterNotRunningException: com.google.protobuf.ServiceException: java.io.IOException: Broken pipe
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.createMasterWithRetries(HConnectionManager.java:793)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterProtocol(HConnectionManager.java:1724)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterMonitor(HConnectionManager.java:1757)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isMasterRunning(HConnectionManager.java:837)
    at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:2010)
    at TestHBase.main(TestHBase.java:37)
Caused by: com.google.protobuf.ServiceException: java.io.IOException: Broken pipe
    at org.apache.hadoop.hbase.ipc.ProtobufRpcClientEngine$Invoker.invoke(ProtobufRpcClientEngine.java:149)
    at com.sun.proxy.$Proxy5.isMasterRunning(Unknown Source)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.createMasterInterface(HConnectionManager.java:732)
    at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.createMasterWithRetries(HConnectionManager.java:764)
    ... 5 more
Caused by: java.io.IOException: Broken pipe
    at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
    at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
    at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:94)
    at sun.nio.ch.IOUtil.write(IOUtil.java:65)
    at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:450)
    at org.apache.hadoop.net.SocketOutputStream$Writer.performIO(SocketOutputStream.java:55)
    at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:142)
    at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:146)
    at org.apache.hadoop.net.SocketOutputStream.write(SocketOutputStream.java:107)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
    at java.io.DataOutputStream.flush(DataOutputStream.java:123)
    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.writeConnectionHeader(HBaseClient.java:896)
    at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:847)
    at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:1414)
    at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:1299)
    at org.apache.hadoop.hbase.ipc.ProtobufRpcClientEngine$Invoker.invoke(ProtobufRpcClientEngine.java:131)
    ... 8 more

此跟踪提供了一些可能实际发生的事情的证据。似乎与ZooKeeper的连接已经建立，但尝试访问主服务器时却失败了。

This trace provides some evidence of what may be actually happening. It seems that the connection to ZooKeeper is established, but something fails when tries to access the master.

尽管我花了几个小时试图在Google上找到一个解决方案似乎没有这样的例外。特别是，这个异常在互联网上发现的两件事情有所不同：

Though I've spent hours trying to find a solution in Google, I haven't seem such an exception. Particularly, this exception varies in two things from most found in the Internet:

• 每个人似乎都有错误 getMaster尝试0的1失败而不是 getMaster尝试1的1失败。我不知道这是否有所作为，但我发现它有点奇怪。

• Everybody seems to have the error getMaster attempt 0 of 1 failed rather than getMaster attempt 1 of 1 failed. I don't known whether this makes a point at all, but I find it somehow weird.

我找不到其他人获得断线错误

I can't find other people getting the Broken pipe error.

顺便说一下， ，只要我在Hortonworks管理控制台中看到。

By the way, the master is actually running, as far as I can see in the Hortonworks Management Console.

查看最近的日志时，这是输出：

When looking at the most recent logs, this is the output:

2013-05-13 15:30:07,192 WARN org.apache.hadoop.ipc.HBaseServer: Incorrect header or version mismatch from 127.0.0.1:40788 got version 0 expected version 3

由于它是一个警告而不是错误，我不知道它是否具有与实际问题有关。每个执行中的端口会有所不同。

As it is a warning rather than an error, I don't know whether it has something to do with the actual problem. The port varies in each execution.

非常感谢您的时间，希望您能为此问题带来一些亮点。

Thanks a lot for your time, I hope you can bring some light into this problem.

Regards

推荐答案

最后我们发现问题并解决了这个问题。原来是一个依赖问题。我们使用 hbase-0.95.0 和 hbase-client-0.95.0 。使用 hbase-0.94.7 或 hbase-0.94.9 似乎有效。

Finally we found the problem and solved it. It turned out to be a dependencies problem. We were using hbase-0.95.0 and hbase-client-0.95.0. Using hbase-0.94.7 or hbase-0.94.9 seemed to work.

然而，即使使用该版本的HBase库，在某些情况下也会出现一些问题。特别是在应用服务器（JBoss AS7）中运行时出现了一些问题。最后，通过删除依赖关系 hbase-client-0.95.0 ，并将其替换为 haboop-core-1.1，似乎可以解决所有问题.2 因为hbase库中不包含某些类。

Yet, some problems occurred under certain circumstances even with that versions of the HBase library. Particularly, some problems arised when running it within an application server (JBoss AS7). In the end, all problems seems to be solved by removing the dependency hbase-client-0.95.0, and replacing it by haboop-core-1.1.2 as some classes not contained in the hbase libraries were required.

请问。

这篇关于连接到本地HBase时出错的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！