HDInsight-Azure SQL DB上的Hive MetaStore-错误 [英] HDInsight - Hive MetaStore on Azure SQL DB - Error

问题描述

你好

我正在尝试使用Hive Metastore创建远程存储在Azure SQL数据库实例上的HDInsight群集.

I am trying to create a HDInsight cluster with the Hive Metastore stored remotely on an Azure SQL DB Instance.

由于任何原因..它都失败了.

For whatever reason .. it is failing.

这是我采取的步骤:

1.在新的数据库服务器上创建新的数据库

1. Create a new DataBase on a new DataBase Server

2.在数据库服务器上的主数据库上创建登录实例

2. Create a login instance on the master db on the db server

3.在要用于元存储的数据库上为该登录实例创建一个用户:其名称为"hive"

3. Create a user for that login instance on the db to be used for the metastore: the name of it is "hive"

4.为该用户授予db管理者配置单元"的权限db.

4. Grant db manager for that user for the "hive" db.

5.使用远程元存储的设置创建HDInsight服务

5. Create the HDInsight service using the settings for remote metastore

   -选择db-server/db-name

   - select the db-server/db-name

   -输入用户名

   - enter the username

   -输入密码

   - enter the password

6.完成

一旦我去Ambari，Hive服务就会出现4个错误，主要是由于Hive MetaStore.

Once I go to Ambari the Hive service has 4 errors, mostly due to Hive MetaStore.

我正在寻找相关的东西.

I am trying to find the relevant.

我可以从Ambari日志中得到的是:

What I could get from Ambari log was: 

org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:92)上的

(RetryingMetaStoreClient.java:92)在org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138)在org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java: 110)，位于org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3556) 在org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3588)在org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550)... 8更多由:java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 方法)在sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)在sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)在java.lang.reflect.Constructor.newInstance(Constructor.java:423) 在org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564)...更多14位原因:MetaException(message:无法使用提供的任何URI连接到元存储.最近的失败: org.apache.thrift.transport.TTransportException: java.net.ConnectException:在org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java)的org.apache.thrift.transport.TSocket.open(TSocket.java:226)处拒绝连接(拒绝连接): 487)，网址为org.apache.hadoop.hive.metastore.HiveMetaStoreClient.< init>(HiveMetaStoreClient.java:282) 在org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient处.< init(gt; SessionHiveMetaStoreClient.java:76) :62) 在org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java)的java.lang.reflect.Constructor.newInstance(Constructor.java:423)的sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)处:1564) 在org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138)上的org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138)上的init(RetryingMetaStoreClient.java:92). hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) 在org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3588)在org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3556) .hadoop.hive.ql.session.SessionState.start(SessionState.java:550)在org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) 在org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625)在sun.reflect.NativeMethodAccessorImpl.invoke0(本机方法)在sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) .reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 在org.apache.hadoop.util.RunJar.run(RunJar.java:243)在java.lang.reflect.Method.invoke(Method.java:498)在org.apache.hadoop.util.RunJar.main(RunJar .java:158)原因:java.net.ConnectException:java.net.PlainSocketImpl.socketConnect(Native)的连接被拒绝(连接被拒绝) 方法)at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)at org.apache.thrift.transport.TSocket.open(TSocket.java:221)上的java.net.Socket.connect(Socket.java:589)上的java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392).. .22 more)at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:534) 在org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.< init>(SessionHiveMetaStoreClient.java:76)的org.apache.hadoop.hive.metastore.HiveMetaStoreClient.(HiveMetaStoreClient.java:282) ...还有19个))" style =" font-size:0.8em; width:778.313px; text-"> Metastore 在hn1-hive-m.uycu5bzch2qu3fye4qjoio1a0a.fx.internal.cloudapp.net上失败(跟踪(最近一次调用为最后一次)):文件"/var/lib/ambari-agent/cache/common-services/HIVE/0.1.2.0.2". 0/package/alerts/alert_hive_metastore.py"，第203行，在执行timeout_kill_strategy = TerminateStrategy.KILL_PROCESS_TREE， 在__i ...</init></init></init></init>中的文件``/usr/lib/ambari-agent/lib/resource_management/core/base.py''，第166行中. ;</init>

(RetryingMetaStoreClient.java:92) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3556) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3588) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) ... 8 more Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) ... 14 more Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused (Connection refused) at org.apache.thrift.transport.TSocket.open(TSocket.java:226) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:487) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:282) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:76) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1564) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:92) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:138) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:110) at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3556) at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3588) at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:625) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.apache.hadoop.util.RunJar.run(RunJar.java:243) at org.apache.hadoop.util.RunJar.main(RunJar.java:158) Caused by: java.net.ConnectException: Connection refused (Connection refused) at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:589) at org.apache.thrift.transport.TSocket.open(TSocket.java:221) ... 22 more ) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:534) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:282) at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:76) ... 19 more )" style="font-size:0.8em;width:778.313px;text-">Metastore on hn1-hive-m.uycu5bzch2qu3fye4qjoio1a0a.fx.internal.cloudapp.net failed (Traceback (most recent call last): File "/var/lib/ambari-agent/cache/common-services/HIVE/0.12.0.2.0/package/alerts/alert_hive_metastore.py", line 203, in execute timeout_kill_strategy=TerminateStrategy.KILL_PROCESS_TREE, File "/usr/lib/ambari-agent/lib/resource_management/core/base.py", line 166, in __i...</init></init></init></init></init>

推荐答案

Hi Cristi，

我遵循了文档

I followed the documentation Use external metadata stores in Azure HDInsight and able to create a cluster with external metastore successfully.

根据错误消息，它似乎是版本问题.请注意，如果您在多个群集之间共享一个元存储，请确保所有群集都是相同的HDInsight版本.不同的Hive版本使用 不同的元存储数据库架构.例如，您不能在Hive 1.2和Hive 2.1版本的群集之间共享元存储.

As per the error message, it appears to be a version issue. Note that, if you share a metastore across multiple clusters, ensure all the clusters are the same HDInsight version. Different Hive versions use different metastore database schemas. For example, you cannot share a metastore across Hive 1.2 and Hive 2.1 versioned clusters.

有关HDInsight版本的列表，请参见 关于表格丢失错误:请确保您指定了有效的配置详细信息.

这篇关于HDInsight-Azure SQL DB上的Hive MetaStore-错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！