无法使用Brew安装将Derby初始化为配置单元 [英] Unable to initialize hive with Derby from Brew install

问题描述

我的理解是， Derby 会在当前目录中创建文件。但是没有。

所以我试图用来完成 hive 初始化， Derby ：但..看起来它已经是一个德比数据库了。

  schematool --verbose -initSchema -dbType derby 
 
 
将Metastore模式初始化启动到2.1.0 
初始化脚本hive-schema-2.1.0.derby.sql 
连接到jdbc：derby：; databaseName = metastore_db; create = true 
连接到：Apache Derby（版本10.10.2.0  - （1582446））
驱动程序：Apache Derby嵌入式JDBC驱动程序（版本10.10。 2.0  - （1582446））
事务隔离：TRANSACTION_READ_COMMITTED 
 0：jdbc：derby：> ！
上的autocommit自动提交状态：true 
 0：jdbc：derby：> CREATE FUNCTIONAPP。NUCLEUS_ASCII（C CHAR（1））返回INTEGER LANGUAGE JAVA参数STYLE JAVA READS SQL DATA调用NULL NULL INPUT EXTERNAL NAME'org.datanucleus.store.rdbms.adapter.DerbySQLFunction.ascii'
错误：FUNCTION'NUCLEUS_ASCII'已经存在。 （state = X0Y68，code = 30000）
 
关闭：0：jdbc：derby：; databaseName = metastore_db; create = true 
 org.apache.hadoop.hive.metastore.HiveMetaException：Schema初始化失败！ Metastore状态将不一致！ 
基础原因：java.io.IOException：架构脚本失败，错误代码2 
 org.apache.hadoop.hive.metastore.HiveMetaException：架构初始化失败！ Metastore状态将不一致！ 
 at org.apache.hive.beeline.HiveSchemaTool.doInit（HiveSchemaTool.java:291）
 at org.apache.hive.beeline.HiveSchemaTool.doInit（HiveSchemaTool.java:264）
 org.apache.hive.beeline.HiveSchemaTool.main（HiveSchemaTool.java:505）
 at sun.reflect.NativeMethodAccessorImpl.invoke0（Native Method）
 at sun.reflect.NativeMethodAccessorImpl.invoke（NativeMethodAccessorImpl。 
 at sun.reflect.DelegatingMethodAccessorImpl.invoke（DelegatingMethodAccessorImpl.java:43）$ b $ at java.lang.reflect.Method.invoke（Method.java:498）
 at org .apache.hadoop.util.RunJar.run（RunJar.java:221）
 at org.apache.hadoop.util.RunJar.main（RunJar.java:136）
引起：java.io .IOException：架构脚本失败，错误代码2 
 at org.apache.hive.beeline.HiveSchemaTool.runBeeLine（HiveSchemaTool.java:390）
 at org.apache.hive.beeline.HiveSchemaTool.runBeeLine（HiveSchemaTool .java：347）
在org.apache.hive.beeline.HiveSchemaTool.doInit（Hi 
  

所以......它在哪里？

更新使用

从头开始重新安装配置单元

  brew重新安装配置单元
  

和发生同样的错误。

另一个更新鉴于此错误的新方向，现在可以在另一个问题中回答：

非os / x - 但其他类似 - 问题被发现，可以在这里服务：

https://stackoverflow.com/a/40017753/1056563

 我在HomeBrew上安装了配置单元（ MacOS）在/ usr / local / Cellar / hive，并且正在运行schematool -dbType derby -initSchema我收到以下错误消息：
 
将Metastore模式初始化为2.0.0初始化脚本hive-schema-2.0 .0.derby.sql错误：FUNCTION'NUCLEUS_ASCII'已经存在。 （state = X0Y68，code = 30000）org.apache.hadoop.hive.metastore.HiveMetaException：模式初始化失败！ Metastore状态将不一致！ 
 
但是，我找不到安装路径下的metastore_db或metastore_db.tmp文件夹，所以我试过：
 
 find / usr / -name hive-schema-2.0。 0.derby.sql 
 vi /usr/local/Cellar/hive/2.0.1/libexec/scripts/metastore/upgrade/derby/hive-schema-2.0.0.derby.sql 
评论'NUCLEUS_ASCII'函数和'NUCLEUS_MATCHES'函数
重新运行schematool -dbType derby -initSchema，然后一切顺利！ 
  

解决方案

Homebrew安装Hive（版本2.3.1）未配置。默认设置是使用进程内Derby数据库（Hive已经包含所需的lib）。

在 brew install hive 之后，您必须执行的操作（ immediatelly ）是初始化数据库：

$ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $> schematool -initSchema -dbType derby

然后你可以运行 hive ，它就可以工作。但是，如果您在初始化数据库之前尝试运行 hive ，Hive实际上会半创建一个不完整的数据库，并且无法正常工作：

 显示表格; 
失败：SemanticException org.apache.hadoop.hive.ql.metadata.HiveException：java.lang.RuntimeException：无法实例化org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 
  

由于数据库是半创建的，所以 schematool 现在也会失败：

 错误：FUNCTION'NUCLEUS_ASCII'已经存在。 （state = X0Y68，code = 30000）
 org.apache.hadoop.hive.metastore.HiveMetaException：模式初始化失败！ Metastore状态将不一致！ 
  

要解决这个问题，您必须删除数据库：

  rm -Rf metastore_db 
  

并运行再次启动命令。

注意到我从当前目录中删除了metastore_db？这是另一个问题：Hive配置为在当前工作目录中创建和使用Derby数据库。这是因为它具有以下默认值'javax.jdo.option.ConnectionURL'：

  jdbc：derby：; databaseName = metastore_db; create = true 
  

要解决该问题，请创建文件 / usr / local / opt / hive / libexec / conf / hive-site.xml as



 < ;? xml version =1.0encoding =UTF-8standalone =no？> 
<？xml-stylesheet type =text / xslhref =configuration.xsl？> 
 
<配置> 
<属性> 
< name> javax.jdo.option.ConnectionURL< / name> 
< value> jdbc：derby：/ usr / local / var / hive / metastore_db; create = true< / value> 
< / property> 
< / configuration> 
  

并像以前一样重新创建数据库。现在数据库位于 / usr / local / var / hive 中，所以如果你在初始化之前再次意外地运行了 hive 数据库，删除它：

  rm -Rf / usr / local / var / hive 
  




It had been my understanding that Derby creates file(s) in the current directory. But there are none there.

So I had tried to do the hive initialization using Derby: but .. it seems there is a derby database already.

 schematool --verbose -initSchema -dbType derby


Starting metastore schema initialization to 2.1.0
Initialization script hive-schema-2.1.0.derby.sql
Connecting to jdbc:derby:;databaseName=metastore_db;create=true
Connected to: Apache Derby (version 10.10.2.0 - (1582446))
Driver: Apache Derby Embedded JDBC Driver (version 10.10.2.0 - (1582446))
Transaction isolation: TRANSACTION_READ_COMMITTED
0: jdbc:derby:> !autocommit on
Autocommit status: true
0: jdbc:derby:> CREATE FUNCTION "APP"."NUCLEUS_ASCII" (C CHAR(1)) RETURNS INTEGER LANGUAGE JAVA PARAMETER STYLE JAVA READS SQL DATA CALLED ON NULL INPUT EXTERNAL NAME 'org.datanucleus.store.rdbms.adapter.DerbySQLFunction.ascii'
Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000)

Closing: 0: jdbc:derby:;databaseName=metastore_db;create=true
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
    at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:291)
    at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:264)
    at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:505)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: Schema script failed, errorcode 2
    at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:390)
    at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:347)
    at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:287)

So .. where is it?

Update I have reinstalled hive from scratch using

  brew reinstall hive

And the same error occurs.

Another update Given the new direction of this error it now is answered by within another question:

An answer to a non-os/x - but similar otherwise - question was found that can serve here:

https://stackoverflow.com/a/40017753/1056563

I installed hive with HomeBrew(MacOS) at /usr/local/Cellar/hive and afer running schematool -dbType derby -initSchema I get the following error message:

Starting metastore schema initialization to 2.0.0 Initialization script hive-schema-2.0.0.derby.sql Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000) org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!

However, I can't find either metastore_db or metastore_db.tmp folder under install path, so I tried:

find /usr/ -name hive-schema-2.0.0.derby.sql
vi /usr/local/Cellar/hive/2.0.1/libexec/scripts/metastore/upgrade/derby/hive-schema-2.0.0.derby.sql
comment the 'NUCLEUS_ASCII' function and 'NUCLEUS_MATCHES' function
rerun schematool -dbType derby -initSchema, then everything goes well!

解决方案

Homebrew installs Hive (version 2.3.1) unconfigured. The default settings are to use in-process Derby database (Hive already includes the required lib).

The only thing you have to do (immediatelly after brew install hive) is to initialize the database:

schematool -initSchema -dbType derby

and then you can run hive, and it will work. However, if you tried to run hive before initializing the database, Hive will actually semi-create an incomplete database and will fail to work:

show tables;
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Since the database is semi-created, schematool will now fail as well:

Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!

To fix that, you will have to delete the database:

rm -Rf metastore_db

and run the initilization command again.

Noticed that I deleted the metastore_db from current directory? This is another problem: Hive is configured to create and use the Derby database in current working dir. This is because it has the following default value for ‘javax.jdo.option.ConnectionURL’:

jdbc:derby:;databaseName=metastore_db;create=true

To fix that, create file /usr/local/opt/hive/libexec/conf/hive-site.xml as

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:derby:/usr/local/var/hive/metastore_db;create=true</value>
  </property>
</configuration>

and recreate the database like before. Now the database is in /usr/local/var/hive, so in case you again accidentally ran hive before initializing the DB, delete it with:

rm -Rf /usr/local/var/hive

这篇关于无法使用Brew安装将Derby初始化为配置单元的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！