创建<时，Java`OutOfMemoryError` 100个主题 [英] Java `OutOfMemoryError` when creating < 100 threads

问题描述

由于这个错误，我一直在阅读和测试并将我的头撞在墙上超过一天。

I've been reading and testing and banging my head on the wall for over a day because of this error.

我在一个名为 Listener 的类中有一些Java代码，看起来像这样

I have some Java code in a class called Listener that looks like this

ExecutorService executor = Executors.newFixedThreadPool(NTHREADS);
boolean listening = true;
int count = 0;
while (listening) {
    Runnable worker;
    try {
        worker = new ServerThread(serverSocket.accept()); // this is line 254
        executor.execute(worker);
        count++;
        logger.info("{} threads started", count);
    } catch (Exception e1){
        //...
    }
}

我一直在调整JVM设置 -Xmx （1到15G之间）和 -Xss （从104k到512M）。服务器有24 GB的RAM，但还必须运行支持该程序的数据库。

I have been tweaking the JVM settings -Xmx (anywhere from 1 to 15G) and -Xss (anywhere from 104k to 512M). The server has 24 GB of RAM, but must also run the database that supports the program.

创建2-20个线程后（程序中其他地方存在几十个） （）），我收到错误

After 2-20 threads are created (a few dozen exist elsewhere in the program as well), I get the error

Exception in thread "Thread-0" java.lang.OutOfMemoryError: unable to create new native thread
at java.lang.Thread.start0(Native Method)
at java.lang.Thread.start(Thread.java:657)
at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:943)
at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1325)
at xxx.Listener.run(Listener.java:254)

$ java -version 收益率：

java version "1.6.0_24"
OpenJDK Runtime Environment (IcedTea6 1.11.1) (fedora-65.1.11.1.fc16-x86_64)
OpenJDK 64-Bit Server VM (build 20.0-b12, mixed mode)

总有系统上有大量可用内存的时候这种情况发生了，其他程序继续执行得很好。是什么导致Java认为它没有新线程的内存？

There is always a large amount of free memory on the system when this happens, and other programs continue to execute fine. What is causing Java to think it has no more memory for new threads?

更新：
也许这比我想象的要大 - 当我使用 ^ C 时，我设法得到了这个错误（只有一次）：

UPDATE: Perhaps this is bigger than I thought- I managed to get this error (only one time) when I used ^C:

OpenJDK 64-Bit Server VM warning: Exception java.lang.OutOfMemoryError occurred dispatching signal SIGINT to handler- the VM may need to be forcibly terminated

当我试图杀死客户端时也是如此（也是用Java编写并在同一台服务器上运行，它是一个读取文件并将其发送到服务器通过套接字），所以JVM之外肯定有一个限制，导致一个人干扰另一个，但我无法想象如果我还有空闲内存并且根本不使用交换？服务器-Xmx1G -Xss104k客户端-Xmx10M

and the same happened when I tried to kill the client (also written in Java and running on the same server, it is a single thread that reads a file and sends it to the server over the socket), so there is definitely a limit beyond the JVM causing one to interfere with the other, but I can't imagine what if I still have free memory and am not using swap at all? Server -Xmx1G -Xss104k Client -Xmx10M

UPDATE2：
放弃perl Forks :: Super 库和从bash运行客户端让我在服务器与OOME崩溃之前获得最多34个线程，因此运行多个客户端肯定会对服务器产生影响，但同时我仍然可以一次运行超过34个（如果一个计算客户端，则为68个）java线程。哪些系统资源阻止了更多线程的创建（即我应该在哪里寻找生猪）？当所有内容（客户端，服务器，GC ......）同时耗尽内存时， top 说明了我的CPU和内存使用情况：

UPDATE2: Abandoning the perl Forks::Super library and running the clients from bash let me get up to 34 threads before the server crashed with OOME, so running multiple clients definitely had an impact on the server, but at the same time I should still be able to run more than 34 (68 if one counts the clients) java threads at a time. Which system resources are blocking the creation of more threads (i.e. where should I look to find the hog)? When everything (clients, server, GC...) runs out of memory at the same time, top says this about my CPU and memory usage:

Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  24681040k total,  1029420k used, 23651620k free,    30648k buffers
Swap: 26836988k total,        0k used, 26836988k free,   453620k cached

UPDATE3：下面的hs_error日志是否表明我的java不是64位？

UPDATE3: Does the hs_error log below indicate that my java is not 64 bit?

# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create GC thread. Out of system resources.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   In 32 bit mode, the process size limit was hit
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Use 64 bit Java on a 64 bit OS
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# JRE version: 6.0_24-b24
# Java VM: OpenJDK 64-Bit Server VM (20.0-b12 mixed mode linux-amd64 compressed oops)
# Derivative: IcedTea6 1.11.1
# Distribution: Fedora release 16 (Verne), package fedora-65.1.11.1.fc16-x86_64

推荐答案

您可以通过最大用户进程进行限制，以了解您的使用限制：

You can be limit by max user processes, to know your limit use :

ulimit -u

要更改限制：

在 /etc/security/limits.conf 中设置：

user soft nproc [your_val] 
user hard nproc [your_val]

如果不够，可能需要添加其他配置链接。

You may have to add some other config if it's not enough see this link.

注意：OP发现了这个错误报告 ch解释了编辑 /etc/security/limits.conf 的限制。

Note : The OP found this bug report in fedora and centos which explains the limitations of editing /etc/security/limits.conf.

这篇关于创建<时，Java`OutOfMemoryError` 100个主题的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！