从hadoop unix运行时增加jvm堆空间 [英] increase jvm heap space while runnig from hadoop unix
问题描述
我正在从hadoop命令运行java类test.java:
I am running a java class test.java from hadoop command :
$ hadoop test
我使用的是stringBuilder,其大小内存不足:
I am using a stringBuilder, and its size is going out of memory :
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder. java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572 )
at java.lang.StringBuffer.append(StringBuffer.java:320)
at java.io.StringWriter.write(StringWriter.java:60)
at org.json.JSONObject.quote(JSONObject.java:1287)
at org.json.JSONObject.writeValue(JSONObject.java:1597)
at org.json.JSONObject.write(JSONObject.java:1649)
at org.json.JSONObject.writeValue(JSONObject.java:1574)
at org.json.JSONArray.write(JSONArray.java:929)
at org.json.JSONObject.writeValue(JSONObject.java:1576)
at org.json.JSONObject.write(JSONObject.java:1649)
at org.json.JSONObject.writeValue(JSONObject.java:1574)
at org.json.JSONObject.write(JSONObject.java:1632)
at org.json.JSONObject.toString(JSONObject.java:1443)
我知道在java中我们可以通过提供堆空间大小来运行java程序:
I know in java we can run a java program by providing a heap space size :
java -Xmx4G test
如何在运行时执行此操作hadoop,如果我像以下那样运行:
How can I do this while running with hadoop, if I run it like :
$ hadoop -Xmx4G test
它抛出异常:
Caused by: java.lang.ClassNotFoundException: java
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: java. Program will exit.
Error: No command named `-Xmx4g' was found. Perhaps you meant `hadoop Xmx4g'
我想永久地改变堆大小。
I would like to change the heap size permanently.
推荐答案
通常在执行类之前在终端中执行其中一个操作或将其放在bash脚本中。
Usually you do one of these in the terminal before executing the class or put it in a bash script.
export HADOOP_HEAPSIZE=4096
hadoop test
或者,
export HADOOP_OPTS="$HADOOP_OPTS -Xmx4G"
hadoop test
或者,如果上述方法不起作用,请尝试在 $ HADOOP_HOME / conf /中修改HADOOP_OPTS hadoop-env.sh
Or, if the above doesn't work, try modifying HADOOP_OPTS in $HADOOP_HOME/conf/hadoop-env.sh
这篇关于从hadoop unix运行时增加jvm堆空间的文章就介绍到这了,希望我们推荐的答案对大家有所帮助,也希望大家多多支持IT屋!