java.lang.OutOfMemoryError: Java heap space with hive [英] java.lang.OutOfMemoryError: Java heap space with hive

问题描述

我使用了 hadoop hive 0.9.0 和 1.1.2 以及 netbeans，但我遇到了这个错误，我无法解决这个问题请帮我代码:

I used hadoop hive 0.9.0 and 1.1.2 and netbeans, but I got this error and I can not solve this problem please help me code :

public class Hive_test {

private static String driverName = "org.apache.hadoop.hive.jdbc.HiveDriver";

   @SuppressWarnings("CallToThreadDumpStack")
public static void main(String[] args) throws SQLException {
    try {
        Class.forName(driverName);
    } catch (ClassNotFoundException e){
        e.printStackTrace();
        System.exit(1);
    }
            System.out.println("commencer la connexion");
    Connection con = DriverManager.getConnection("jdbc:hive://localhost:10000/default",""," ");
    Statement stmt = con.createStatement();
    ResultSet res = stmt.executeQuery("select * from STATE");
    while (res.next()){
        System.out.println(String.valueOf(res.getInt(1)) + "	" + res.getString(2));
                    System.out.println("sql terminer");
    }
}

下面的错误；

error :
commencer la connexion
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:353)
    at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
    at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
    at org.apache.hadoop.hive.service.ThriftHive$Client.recv_execute(ThriftHive.java:116)
    at org.apache.hadoop.hive.service.ThriftHive$Client.execute(ThriftHive.java:103)
    at org.apache.hadoop.hive.jdbc.HiveStatement.executeQuery(HiveStatement.java:192)
    at org.apache.hadoop.hive.jdbc.HiveStatement.execute(HiveStatement.java:132)
    at org.apache.hadoop.hive.jdbc.HiveConnection.configureConnection(HiveConnection.java:132)
    at org.apache.hadoop.hive.jdbc.HiveConnection.<init>(HiveConnection.java:122)
    at org.apache.hadoop.hive.jdbc.HiveDriver.connect(HiveDriver.java:106)
    at java.sql.DriverManager.getConnection(DriverManager.java:571)
    at java.sql.DriverManager.getConnection(DriverManager.java:215)
    at hive.Hive_test.main(Hive_test.java:22)

推荐答案

可以在Hive中设置container heapsize并解决这个错误:

You can set the container heapsize in Hive and resolve this error:

在 Hadoop MapReduce 框架之上运行的大多数工具都提供了为其作业调整这些 Hadoop 级别设置的方法.在 Hive 中有多种方法可以做到这一点.此处显示了其中三个:

Most tools that operate on top of the Hadoop MapReduce framework provide ways to tune these Hadoop level settings for its jobs. There are multiple ways to do this in Hive. Three of these are shown here:

1) 直接通过 Hive 命令行传递:

1) Pass it directly via the Hive command line:

hive -hiveconf mapreduce.map.memory.mb=4096 -hiveconf mapreduce.reduce.memory.mb=5120 -e "select count(*) from test_table;"

2) 在调用 Hive 之前设置 ENV 变量:

2) Set the ENV variable before invoking Hive:

export HIVE_OPTS="-hiveconf mapreduce.map.memory.mb=4096 -hiveconf mapreduce.reduce.memory.mb=5120"

3) 在 hive CLI 中使用set"命令.

3) Use the "set" command within the hive CLI.

hive> set mapreduce.map.memory.mb=4096;
hive> set mapreduce.reduce.memory.mb=5120;
hive> select count(*) from test_table;

这篇关于java.lang.OutOfMemoryError: Java heap space with hive的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！