连接并坚持HBase [英] Connecting and Persisting to HBase
本文介绍了连接并坚持HBase的处理方法，对大家解决问题具有一定的参考价值，需要的朋友们下面随着小编来一起学习吧！
问题描述
（192.168.56.102是vm的inet ip）我试图连接到hbase，它是cloudera-vm的一部分，
我使用虚拟主机的网络设置。
所以我可以访问hbase master的webUI @ http://192.168.56.102:60010/master.jsp
另外我的java客户端（在vm上运行得很好）建立了连接到192.168.56.102:2181
但是当它调用 getMaster 连接被拒绝查看日志：
11/09/14 11： 19:30 INFO zookeeper.ZooKeeper：启动客户端连接，connectString = 192.168.56.102：2181 sessionTimeout = 180000 watcher = hconnection
11/09/14 11:19:30 INFO zookeeper.ClientCnxn：打开套接字连接到服务器/ 192.168.56.102:2181
11/09/14 11:19:30 INFO zookeeper.ClientCnxn：建立到cloudera-vm的套接字连接/192.168.56.102:2181，启动会话
11/09/14 11:19:30 INFO zookeeper.ClientCnxn：会话建立在服务器cloudera-vm / 192.168.56.102：2181上完成，sessionid = 0x13267157f930009，协商超时= 40000
11/09/14 11:19:32 INFO client.HConnectionManager $ HConnectionImplementation：getMaster尝试0或10失败;在睡眠后重试1000
java.net.ConnectException：连接被拒绝：没有更多信息
at sun.nio.ch.SocketChannelImpl.checkConnect（Native Method）
at sun.nio.ch. SocketChannelImpl.finishConnect（SocketChannelImpl.java:567）
在org.apache.hadoop.net.SocketIOWithTimeout.connect（SocketIOWithTimeout.java:206）
在org.apache.hadoop.net.NetUtils.connect（ NetUtils.java:404）
位于org.apache.hadoop.hbase.ipc.HBaseClient $ Connection.setupIOstreams（HBaseClient.java:311）
位于org.apache.hadoop.hbase.ipc.HBaseClient。 getConnection（HBaseClient.java:865）
at org.apache.hadoop.hbase.ipc.HBaseClient.call（HBaseClient.java:732）
at org.apache.hadoop.hbase.ipc.HBaseRPC $ Invoker.invoke（HBaseRPC.java:257）
at $ Proxy9.getProtocolVersion（Unknown Source）
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy（HBaseRPC.java:419）
在org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy（HBaseRPC.java:393）
在org.apache。 hadoop.hbase.ipc.HBaseRPC.getProxy（HBaseRPC.java:444）
at org.apache.hadoop.hbase.client.HConnectionManager $ HConnectionImplementation.getMaster（HConnectionManager.java:358）
at org。 apache.hadoop.hbase.client.HBaseAdmin。< init>（HBaseAdmin.java:90）
at org.datanucleus.store.hbase.HBaseUtils $ 1.run（HBaseUtils.java:432）
at java.security.AccessController.doPrivileged（Native方法）
在org.datanucleus.store.hbase.HBaseUtils.createSchemaForClass（HBaseUtils.java:428）
在org.datanucleus.store.hbase.HBaseMetaDataListener.loaded （HBaseMetaDataListener.java:74）
at org.datanucleus.store.hbase.HBaseStoreManager。< init>（HBaseStoreManager.java:76）
at sun.reflect.NativeConstructorAccessorImpl.newInstance0（Native Method）
at sun.reflect.NativeConstructorAccessorImpl.newInstance（NativeConstructorAccessorImpl.java:39）
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance（DelegatingConstructorAccessorImpl.j ava：27）
at java.lang.reflect.Constructor.newInstance（Constructor.java:513）
at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension（NonManagedPluginRegistry.java:677）
在org.datanucleus.plugin.PluginManager.createExecutableExtension（PluginManager.java:287）
at org.datanucleus.NucleusContext.createStoreManagerForProperties（NucleusContext.java:453）
at org.datanucleus.NucleusContext.initialise（NucleusContext .java：264）
at org.datanucleus.api.jpa.JPAEntityManagerFactory.initialiseNucleusContext（JPAEntityManagerFactory.java:746）
at org.datanucleus.api.jpa.JPAEntityManagerFactory。< init>（JPAEntityManagerFactory。
at org.datanucleus.api.jpa.PersistenceProviderImpl.createEntityManagerFactory（PersistenceProviderImpl.java:91）
at javax.persistence.Persistence.createEntityManagerFactory（Persistence.java:150）
在javax.persistence.Persistence.createEntityManagerFactory（持久性.java：70）
at de.syrtec.jpa2hbase.start.TestDAO.main（TestDAO.java:13）
hbase-site.xml：
<配置>
<属性>
<名称> hbase.zookeeper.quorum< / name>
<值> 192.168.56.102< /值>
< / property>
<属性>
< name> hbase.zookeeper.property.clientPort< / name>
<值> 2181< /值>
< / property>
<属性>
<名称> hbase.master< / name>
<值> 192.168.56.102：60010< /值>
< / property>
< / configuration>
persistence.xml：
< persistence
xsi：schemaLocation =http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/ persistence_1_0.xsd
version =1.0>
< persistence-unit name =hbase-addressbook
transaction-type =RESOURCE_LOCAL>
<属性>
< property name =datanucleus.ConnectionPasswordvalue =/>
< property name =datanucleus.autoCreateSchemavalue =true/>
< property name =datanucleus.validateTablesvalue =false/>
< property name =datanucleus.Optimisticvalue =false/>
< property name =datanucleus.validateConstraintsvalue =false/>
< / properties>
< / persistence-unit>
< /余辉>
TestDAO.java：
public class TestDAO {
public static void main（String [] args）{
EntityManagerFactory emf = Persistence.createEntityManagerFactory（hbase-addressbook）;
EntityManager em = emf.createEntityManager（）;
EntityTransaction tx = null;
帐户a2 =新帐户（myPre3，mySur3，2，新登录（e，f））;
tx = em.getTransaction（）;
tx.begin（）;
em.persist（a2）;
tx.commit（）;
只需添加运行在虚拟机上的网络服务（netstat -ntpl）：
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 563 / sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 909 / cupsd
tcp 0 0 0.0.0.0:8088 0.0.0.0:* LISTEN 2004 / python2.6
tcp 0 0 127.0。 0.1：8001 0.0.0.0:* LISTEN 2345 / python2.6
tcp6 0 0 ::: 39087 ::: * LISTEN 1327 / java
tcp6 0 0 ::: 54162 ::: * LISTEN 1411 / java
tcp6 0 0 127.0.0.1:8020 ::: * LISTEN 1411 / java
tcp6 0 0 127.0.0.1:8021 ::: * LISTEN 1327 / java
tcp6 0 0： :: 50070 ::: * LISTEN 1411 / java
tcp6 0 0 ::: 22 ::: * LISTEN 563 / sshd
tcp6 0 0 :: 1：631 ::: * LISTEN 909 / cupsd
tcp6 0 0 ::: 11000 ::: * LISTEN 2077 / java
tcp6 0 0 ::: 43738 ::: * LISTEN 858 / java
tcp6 0 0 ::: 50010 ::: * LISTEN 858 / java
tcp6 0 0 ::: 51258 ::: * LISTEN 858 / java
tcp6 0 0 ::: 50075 ::: * LISTEN 858 / java
tcp6 0 0： :: 60030 ::: * LISTEN 1726 / java
tcp6 0 0 127.0.0.1:44446 ::: * LISTEN 1642 / java
tcp6 0 0 127.0.1.1:41246 ::: * LISTEN 1726 / java
tcp6 0 0 ::: 8002 ::: * LISTEN 2009 / java
tcp6 0 0 ::: 8003 ::: * LISTEN 2009 / java
tcp6 0 0 127.0.1.1:58915 ::: * LISTEN 1726 / java
tcp6 0 0 ::: 59203 ::: * LISTEN 1491 / java
tcp6 0 0： :: 50020 ::: * LISTEN 858 / java
tcp6 0 0 127.0.0.1:8005 ::: * LISTEN 2077 / java
tcp6 0 0 ::: 2181 ::: * LISTEN 1726 / java
tcp6 0 0 ::: 9290 ::: * LISTEN 1327 / java
tcp6 0 0 ::: 60010 ::: * LISTEN 1726 / java
tcp6 0 0 ::: 50090 ::: * LISTEN 1491 / java
tcp6 0 0 ::: 10090 ::: * LISTEN 1411 / java
tcp6 0 0 ::: 50060： :: * LISTEN 1642 / java
tcp6 0 0 ::: 50030 ::: * LISTEN 1327 / java
问题的根源在于Ubuntu的IPV6支持（我在这里使用）。
但只是在网络管理中停用IPV6是不够的。
相反，您必须将以下内容附加到文件中：/etc/sysctl.conf
#disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf .default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
重新启动后，您应该验证IPV6确实是关闭的：
$ p $ cat $ proc / sys / net / ipv6 / conf / all / disable_ipv6
（0 = IPV6开启; 1 =关闭IPV6）
非常感谢
这给了我至关重要的提示。
I just tried to connect to hbase which is part of the cloudera-vm using a java client.
(192.168.56.102 is the inet ip of the vm)
I use virtual box with host only network setting.
So I can access the webUI of the hbase master @ http://192.168.56.102:60010/master.jsp
Also my java client (worked well on the vm itself) established connection to 192.168.56.102:2181
But when it calls getMaster I get connection refused see log:
11/09/14 11:19:30 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=192.168.56.102:2181 sessionTimeout=180000 watcher=hconnection
11/09/14 11:19:30 INFO zookeeper.ClientCnxn: Opening socket connection to server /192.168.56.102:2181
11/09/14 11:19:30 INFO zookeeper.ClientCnxn: Socket connection established to cloudera-vm/192.168.56.102:2181, initiating session
11/09/14 11:19:30 INFO zookeeper.ClientCnxn: Session establishment complete on server cloudera-vm/192.168.56.102:2181, sessionid = 0x13267157f930009, negotiated timeout = 40000
11/09/14 11:19:32 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 0 of 10 failed; retrying after sleep of 1000
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:311)
at org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:865)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:732)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy9.getProtocolVersion(Unknown Source)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:358)
at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:90)
at org.datanucleus.store.hbase.HBaseUtils$1.run(HBaseUtils.java:432)
at java.security.AccessController.doPrivileged(Native Method)
at org.datanucleus.store.hbase.HBaseUtils.createSchemaForClass(HBaseUtils.java:428)
at org.datanucleus.store.hbase.HBaseMetaDataListener.loaded(HBaseMetaDataListener.java:74)
at org.datanucleus.store.hbase.HBaseStoreManager.<init>(HBaseStoreManager.java:76)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.datanucleus.plugin.NonManagedPluginRegistry.createExecutableExtension(NonManagedPluginRegistry.java:677)
at org.datanucleus.plugin.PluginManager.createExecutableExtension(PluginManager.java:287)
at org.datanucleus.NucleusContext.createStoreManagerForProperties(NucleusContext.java:453)
at org.datanucleus.NucleusContext.initialise(NucleusContext.java:264)
at org.datanucleus.api.jpa.JPAEntityManagerFactory.initialiseNucleusContext(JPAEntityManagerFactory.java:746)
at org.datanucleus.api.jpa.JPAEntityManagerFactory.<init>(JPAEntityManagerFactory.java:422)
at org.datanucleus.api.jpa.PersistenceProviderImpl.createEntityManagerFactory(PersistenceProviderImpl.java:91)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:150)
at javax.persistence.Persistence.createEntityManagerFactory(Persistence.java:70)
at de.syrtec.jpa2hbase.start.TestDAO.main(TestDAO.java:13)
hbase-site.xml:
<configuration>
<property>
<name>hbase.zookeeper.quorum</name>
<value>192.168.56.102</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>2181</value>
</property>
<property>
<name>hbase.master</name>
<value>192.168.56.102:60010</value>
</property>
</configuration>
persistence.xml:
<persistence
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
version="1.0">
<persistence-unit name="hbase-addressbook"
transaction-type="RESOURCE_LOCAL">
<properties>
<property name="datanucleus.ConnectionURL" value="hbase:192.168.56.102:60010" />
<property name="datanucleus.ConnectionUserName" value="" />
<property name="datanucleus.ConnectionPassword" value="" />
<property name="datanucleus.autoCreateSchema" value="true" />
<property name="datanucleus.validateTables" value="false" />
<property name="datanucleus.Optimistic" value="false" />
<property name="datanucleus.validateConstraints" value="false" />
</properties>
</persistence-unit>
</persistence>
TestDAO.java:
public class TestDAO {
public static void main(String[] args) {
EntityManagerFactory emf = Persistence.createEntityManagerFactory("hbase-addressbook");
EntityManager em = emf.createEntityManager();
EntityTransaction tx = null;
Account a2 = new Account("myPre3", "mySur3", 2, new Login("e", "f"));
tx = em.getTransaction();
tx.begin();
em.persist(a2);
tx.commit();
}
}
just an addition of the network services running on the VM (netstat -ntpl):
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 563/sshd
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 909/cupsd
tcp 0 0 0.0.0.0:8088 0.0.0.0:* LISTEN 2004/python2.6
tcp 0 0 127.0.0.1:8001 0.0.0.0:* LISTEN 2345/python2.6
tcp6 0 0 :::39087 :::* LISTEN 1327/java
tcp6 0 0 :::54162 :::* LISTEN 1411/java
tcp6 0 0 127.0.0.1:8020 :::* LISTEN 1411/java
tcp6 0 0 127.0.0.1:8021 :::* LISTEN 1327/java
tcp6 0 0 :::50070 :::* LISTEN 1411/java
tcp6 0 0 :::22 :::* LISTEN 563/sshd
tcp6 0 0 ::1:631 :::* LISTEN 909/cupsd
tcp6 0 0 :::11000 :::* LISTEN 2077/java
tcp6 0 0 :::43738 :::* LISTEN 858/java
tcp6 0 0 :::50010 :::* LISTEN 858/java
tcp6 0 0 :::51258 :::* LISTEN 858/java
tcp6 0 0 :::50075 :::* LISTEN 858/java
tcp6 0 0 :::60030 :::* LISTEN 1726/java
tcp6 0 0 127.0.0.1:44446 :::* LISTEN 1642/java
tcp6 0 0 127.0.1.1:41246 :::* LISTEN 1726/java
tcp6 0 0 :::8002 :::* LISTEN 2009/java
tcp6 0 0 :::8003 :::* LISTEN 2009/java
tcp6 0 0 127.0.1.1:58915 :::* LISTEN 1726/java
tcp6 0 0 :::59203 :::* LISTEN 1491/java
tcp6 0 0 :::50020 :::* LISTEN 858/java
tcp6 0 0 127.0.0.1:8005 :::* LISTEN 2077/java
tcp6 0 0 :::2181 :::* LISTEN 1726/java
tcp6 0 0 :::9290 :::* LISTEN 1327/java
tcp6 0 0 :::60010 :::* LISTEN 1726/java
tcp6 0 0 :::50090 :::* LISTEN 1491/java
tcp6 0 0 :::10090 :::* LISTEN 1411/java
tcp6 0 0 :::50060 :::* LISTEN 1642/java
tcp6 0 0 :::50030 :::* LISTEN 1327/java
解决方案
After many tedious days of research I can provide a solution..could save others from a lot of headaches..
The root of the problem is the IPV6 support of Ubuntu (I'm using here). But just deactivating IPV6 in the network administration wasn't enough.
Instead you have to append the following to the file: /etc/sysctl.conf
#disable ipv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
After reboot you should validate that IPV6 is really off by:
cat /proc/sys/net/ipv6/conf/all/disable_ipv6
(0 = IPV6 on ; 1 = IPV6 off)
Thanks a lot to
which gave me the crucial hint.
这篇关于连接并坚持HBase的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
查看全文
