使用主机系统上的客户端访问VM中运行的HBase [英] Accessing HBase running in VM with a client on host system
问题描述
我尝试使用客户端程序将一些数据写入hbase HBase @ Hadoop从Cloudera @ ubuntu预先配置的VM中运行。
客户端运行在托管虚拟机的系统上，并直接在虚拟机中运行客户端。
所以现在我想使用客户端在vm之外访问vm上的服务器
我正在使用NAT。
为了能够访问像HBase Master这样的服务器，HUE。在vm上运行我在虚拟框中配置了端口转发：
因此，我可以访问HBase Master，HUE的概览站点。
要运行vm上的服务器上的客户端，我创建了
hbase-site.xml，内容如下：
<结构>
<属性>
<名称> hbase.zookeeper.quorum< / name>
<值>本地主机< /值>
< / property>
<属性>
< name> hbase.zookeeper.property.clientPort< / name>
<值> 9997< /值>
< / property>
<属性>
<名称> hbase.master< / name>
< value> localhost：9999< /值>
< / property>
< / configuration>
所以我期望转发工作：
运行客户端时日志中的错误消息如下所示：
11/09/07 17:48:00信息zookeeper .ZooKeeper：启动客户端连接，connectString = localhost：2181 sessionTimeout = 180000 watcher = hconnection
11/09/07 17:48:00 INFO zookeeper.ClientCnxn：打开到服务器localhost / 127.0.0.1的套接字连接：2181
11/09/07 17:48:01警告zookeeper.ClientCnxn：服务器null的会话0x0，意外错误，关闭套接字连接并尝试重新连接
java.net.ConnectException：连接被拒绝：没有更多信息
at sun.nio.ch.SocketChannelImpl.checkConnect（本地方法）
at sun.nio.ch.SocketChannelImpl.finishConnect（SocketChannelImpl.java:567）
at org.apache.zookeeper.ClientCnxn $ SendThread.run（ClientCnxn.java:1119）
11/09/07 17:48:01信息zookeeper.ClientCnxn：打开到本地服务器的套接字连接/ 0：0：0：0：0 ：0：0：1：2181
11/09/07 17:48:01警告zookeeper.ClientCnxn：服务器null的会话0x0，意外错误，关闭套接字连接并尝试重新连接
java.net。 SocketException：协议族不支持的地址族：在sun.nio.ch.Net.connect（Native方法）上连接
（在Sun.nio.ch.SocketChannelImpl.connect中为
）（SocketChannelImpl.java:500）
at org.apache.zookeeper.ClientCnxn $ SendThread.startConnect（ClientCnxn.java:1050）
at org.apache.zookeeper.ClientCnxn $ SendThread.run（ClientCnxn.java:1077）
11/09/07 17:48:03信息zookeeper.ClientCnxn：打开到服务器localhost / 127.0.0.1的套接字连接：2181
11/09/07 17:48:04 WARN zookeeper.ClientCnxn：服务器的会话0x0 null，意外的错误，关闭套接字连接并尝试重新连接
java.net.ConnectException：连接被拒绝：没有更多信息
at sun.nio.ch.SocketChannelImpl.checkConnect（Native Method）
at sun.nio.ch.SocketChannelImpl.finishConn ect（SocketChannelImpl.java:567）
at org.apache.zookeeper.ClientCnxn $ SendThread.run（ClientCnxn.java:1119）
11/09/07 17:48:04信息zookeeper.ClientCnxn：打开与服务器localhost / 0：0：0：0：0：0：0：0：1：2181的套接字连接
11/09/07 17:48:04警告zookeeper.ClientCnxn：服务器null，会话0x0意外错误，关闭套接字连接并尝试重新连接
java.net.SocketException：协议族不支持的地址族：在sun.nio.ch.Net.connect（本地方法）连接
at sun.nio.ch.SocketChannelImpl.connect（SocketChannelImpl.java:500）
位于org.apache.zookeeper.ClientCnxn $ SendThread.startConnect（ClientCnxn.java:1050）
位于org.apache.zookeeper。 ClientCnxn $ SendThread.run（ClientCnxn.java:1077）
11/09/07 17:48:05信息zookeeper.ClientCnxn：打开与服务器localhost / 127.0.0.1的连接：2181
正确的连接日志（直接在虚拟机上运行客户端）如下所示：
11/09/07 09:05:29信息zookeeper.ZooKeeper：启动客户端连接，connectString = localhost：2181 sessionTimeout = 180000 watcher = hconnection
11/09/07 09:05:29信息zookeeper.ClientCnxn：打开与服务器localhost / 0：0：0：0：0：0：0：1：2181的套接字连接
11/09/07 09： 05:29 INFO zookeeper.ClientCnxn：建立到localhost / 0：0：0：0：0：0：0：1：2181的套接字连接，启动会话
11/09/07 09:05:29 INFO zookeeper .ClientCnxn：在服务器localhost / 0：0：0：0：0：0：0：1：2181，sessionid = 0x132449d36df0006，协商超时时间= 40000
因此，我现在只在第一个问题出现之前看到连接url不正确，因为端口被正确转发，但IP仍然是localhost而不是10.0 .2.15在端口转发设置中配置：
打开与服务器localhost / 127.0.0.1的套接字连接：2181
只有我发现的提示是di在主机（win7）和vm（Ubuntu）
中禁用IPV6 - >功能并检查端口 - >它们被正确转发了
有没有人有想法？
为了给读者提供附加价值，下面是解决方案：
无法解析cloudera-vm
的DNS名称 为了解决这个问题，我简单地添加到C：\ Windows \ System32 \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ $ c> 192.168.56.101 cloudera-vm
因此，无论何时使用cloudera-vm，它都会解析为适当的知识产权。
虚拟机现在以主机专用网络模式运行，现在分配给192.168.56.101。
因此不需要端口转发。
仅用于比较：
hbase-site.xml
<配置>
<属性>
<名称> hbase.zookeeper.quorum< / name>
<值> cloudera-vm< /值>
< / property>
< / configuration>
persistence.xml
<持久性
xsi：schemaLocation =http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0 .xsd
version =1.0>
< persistence-unit name =hbase-addressbook
transaction-type =RESOURCE_LOCAL>
<属性>
< property name =datanucleus.ConnectionURLvalue =hbase：cloudera-vm/>
< property name =datanucleus.ConnectionUserNamevalue =/>
< property name =datanucleus.ConnectionPasswordvalue =/>
< property name =datanucleus.autoCreateSchemavalue =true/>
< property name =datanucleus.validateTablesvalue =false/>
< property name =datanucleus.Optimisticvalue =false/>
< property name =datanucleus.validateConstraintsvalue =false/>
< / properties>
< / persistence-unit>
< /余辉>
之后我得到了另一个错误（连接被拒绝，尽管所有连接字符串都是正确的），因此我研究了几天原因。
解决方案是通过在文件中添加以下内容来禁用在VM上运行的Ubuntu的IPV6：/etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net .ipv6.conf.lo.disable_ipv6 = 1
重新启动后，所有连接并正常工作： p>
I try to write some data to hbase with a client program
HBase @ Hadoop runs in a preconfigured VM from Cloudera @ ubuntu.
The Client runs on the system hosting the VM and running the client directly in the VM works.
So now I want to use the client outside the vm to access the servers on the vm
I'm using NAT. To be able to access the servers like HBase Master, HUE..running on the vm I configured port forwarding in virtual box:
Thus I can reach the overview sites of the HBase Master, HUE..
To run the client against the servers on the vm I created hbase-site.xml with content:
<configuration>
<property>
<name>hbase.zookeeper.quorum</name>
<value>localhost</value>
</property>
<property>
<name>hbase.zookeeper.property.clientPort</name>
<value>9997</value>
</property>
<property>
<name>hbase.master</name>
<value>localhost:9999</value>
</property>
</configuration>
So I expected that forwarding works:
The error messages in the log when running the client looks like:
11/09/07 17:48:00 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
11/09/07 17:48:00 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181
11/09/07 17:48:01 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
11/09/07 17:48:01 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181
11/09/07 17:48:01 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Address family not supported by protocol family: connect
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:500)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1050)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1077)
11/09/07 17:48:03 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181
11/09/07 17:48:04 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119)
11/09/07 17:48:04 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181
11/09/07 17:48:04 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.SocketException: Address family not supported by protocol family: connect
at sun.nio.ch.Net.connect(Native Method)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:500)
at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1050)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1077)
11/09/07 17:48:05 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181
A correct connection log (when running the client directly on the vm) looks like:
11/09/07 09:05:29 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
11/09/07 09:05:29 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181
11/09/07 09:05:29 INFO zookeeper.ClientCnxn: Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session
11/09/07 09:05:29 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x132449d36df0006, negotiated timeout = 40000
So I only see now in the log line before the first problem that connection url isn't correct as the port is correctly forwarded but the IP is still localhost and not 10.0.2.15 as configured in the port forwarding settings:
Opening socket connection to server localhost/127.0.0.1:2181
Only hint I found is disabling IPV6 -> is disabled in host(win7) and vm(Ubuntu) and checking the port -> they are correctly forwarded
Has anyone an idea?
Just to give an added value to users reading this, here is the solution:
Could not resolve the DNS name of cloudera-vm
To fix that I simply added to C:\Windows\System32\drivers\etc\hosts :
192.168.56.101 cloudera-vm
So whenever cloudera-vm is used it is resolved to the apropriate IP. The VM is now running at Host-Only-Network Mode, and is assigned to 192.168.56.101 now. So no port forwarding is necessary.
Just for comparison:
hbase-site.xml
<configuration>
<property>
<name>hbase.zookeeper.quorum</name>
<value>cloudera-vm</value>
</property>
</configuration>
persistence.xml
<persistence
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
version="1.0">
<persistence-unit name="hbase-addressbook"
transaction-type="RESOURCE_LOCAL">
<properties>
<property name="datanucleus.ConnectionURL" value="hbase:cloudera-vm" />
<property name="datanucleus.ConnectionUserName" value="" />
<property name="datanucleus.ConnectionPassword" value="" />
<property name="datanucleus.autoCreateSchema" value="true" />
<property name="datanucleus.validateTables" value="false" />
<property name="datanucleus.Optimistic" value="false" />
<property name="datanucleus.validateConstraints" value="false" />
</properties>
</persistence-unit>
</persistence>
After that I got another error (connection refused although all connection strings were correct) thus I researched for days for the cause.
The solution was to disable IPV6 of Ubuntu running on the VM by appending the following to the file: /etc/sysctl.conf
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1
After reboot everything connects and works correctly :)
这篇关于使用主机系统上的客户端访问VM中运行的HBase的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
