JMeter - linux 内存不足 [英] JMeter - out of memory on linux
问题描述
我正在尝试为应用程序运行负载测试.为此，我在具有 60GB RAM 和足够多的 CPU 功率的 Ubuntu Vm 上使用 JMeter (v.2.13).目标是达到 10k 用户通过 WebSocket 连接.
I am trying to run a load test for a application. For this i am using JMeter (v.2.13) on an Ubuntu Vm with 60GB Ram and more than enough CPU power. Goal is to reach 10k Users connected via WebSocket.
但是在测试运行期间，我在 ssh 控制台上收到以下错误(大约 1.5k 到 2.5k 模拟用户)
However during the test runs i get the following errors on the ssh-console (at approx. 1.5k to 2.5k simulated users)
OpenJDK 64-Bit Server VM warning: Attempt to protect stack guard pages failed.
OpenJDK 64-Bit Server VM warning: Attempt to deallocate stack guard pages failed.
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f20ee653000, 12288, 0) failed; error='Cannot allocate memory' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 12288 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /jmetertests/jm/bin/hs_err_pid1833.log
OpenJDK 64-Bit Server VM warning: Attempt to deallocate stack guard pages failed.
OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f2218de8000, 12288, 0) failed; error='Cannot allocate memory' (errno=12)
提到的错误报告文件看起来像这样
The mentioned error report file looks like this
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 12288 bytes for committing reserved memory.
# Possible reasons:
# The system is out of physical RAM or swap space
# In 32 bit mode, the process size limit was hit
# Possible solutions:
# Reduce memory load on the system
# Increase physical memory or swap space
# Check if swap backing store is full
# Use 64 bit Java on a 64 bit OS
# Decrease Java heap size (-Xmx/-Xms)
# Decrease number of Java threads
# Decrease Java thread stack sizes (-Xss)
# Set larger code cache with -XX:ReservedCodeCacheSize=
# This output file may be truncated or incomplete.
#
# Out of Memory Error (os_linux.cpp:2798), pid=1833, tid=140472285792000
#
# JRE version: OpenJDK Runtime Environment (7.0_75-b13) (build 1.7.0_75-b13)
# Java VM: OpenJDK 64-Bit Server VM (24.75-b04 mixed mode linux-amd64 )
# Derivative: IcedTea 2.5.4
# Distribution: Ubuntu 14.04 LTS, package 7u75-2.5.4-1~trusty1
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
我试图通过在 .sh 文件中发出 java 命令之前添加以下行来修改 jmeter.sh 来分配更多内存:
I tried to allocate more memory by modifying the jmeter.sh through adding the following line right before the java-command is issued in the .sh-file:
JVM_ARGS="-Xms5g -Xmx20g -Xss300k"
我还尝试使用以下命令设置 _JAVA_OPTIONS 环境变量
I also tried setting the _JAVA_OPTIONS environment variable with the following command
export _JAVA_OPTIONS="-Xms5g -Xmx20g"
最后我在一些 SO-thread 中找到了这个命令
And finally I found this command im some SO-thread
sysctl -w vm.max_map_count=500000
top-Command 给了我以下内存信息
The top-Command gives me the following information on memory
KiB Mem: 61836576 total, 15163400 used, 46673176 free, 10636 buffers
KiB Swap: 0 total, 0 used, 0 free. 94492 cached Mem
更新
仅供参考:除了简单的数据编写器之外，我在 JMeter 中不使用任何侦听器.但是，即使我禁用了最后一个侦听器，也会引发错误.
UPDATE
FYI: I do not use any listeners in JMeter except the simple data writer. However errors are thrown even if I disable that last listener.
java -Xms40g -version
成功 - 所以我真的可以分配这么多内存
is succesful - so i can really allocate so much memory
我通过使用减少了堆栈大小
I reduced the stack size by using
-Xss300k
这至少帮助我改变了一些东西，因为我现在得到了
which helped ot at least changed something as I now get a
Uncaught Exception java.lang.OutOfMemoryError:
unable to create new native thread. See log file for details.
错误.好像系统线程数用完了?
error. It seems that the number of threads on the system are exhausted?
根据某些用户的要求，结果 uf ulimit -a
as requested by some user the results uf ulimit -a
core file size (blocks, -c) 0
scheduling priority (-e) 0
pending signals (-i) 491456
max locked memory (kbytes, -l) 64
open files (-n) 500000
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
max user processes (-u) 491456
... all other entries are set to 'unlimited'
并且limits.conf包含以下条目
and the limits.conf contains the following entries
* hard nofile 900000
* soft nofile 900000
root hard nofile 900000
root soft nofile 900000
此外，我已将 proc/sys/kernel 下的 threads-max 设置为可笑的高值，并且还增加了 proc/sys/vm/的值max_map_count
Additonally i have set the threads-max under proc/sys/kernel to something ridiculous high and also increased the value of proc/sys/vm/max_map_count
是我错过了什么还是我做错了什么?感谢您的帮助.
Did I miss something or did i do something wrong? Thank you for your help.
推荐答案
好的同时我想通了:
- 请务必阅读 Dmriti T 的帖子
- 将 java 的堆栈大小设置为较小的值 - 通常最小值就足够了.在我的情况下是 200k(另见第 4 点.)
- 增加 sysctl.conf 中的最大打开文件数(参见第 4 点)
- 增加堆大小 - 但由于 jmeter 会创建大量线程，也会为线程在 OS 级别所需的内存留出足够的空间(因此新线程需要 OS 和 Java 堆上的内存 - 因此也为 os 留出足够的内存)(见下文)
- 更新
limits.conf(见下文) - 更新 sysctl.conf 文件(见下文)
- make sure to read Dmriti T's post
- set the java's stacksize to something small - usually the smallest value will be enough. In my case 200k (see also point 4.)
- increase the maximum number of open files in the sysctl.conf (see point 4)
- increase heap size - but as jmeter will create a lot of threads also leave enough space for the memory the threads will need on OS level (so a new thread needs memory on OS and java heap - therefore leave the os enough memory too) (see below)
- update the
limits.conf(see below) - update the sysctl.conf-file (see below)
limits.conf
/etc/security/limits.conf
limits.conf
/etc/security/limits.conf
* hard nofile 900000
* soft nofile 900000
root hard nofile 900000
root soft nofile 900000
sysctl.conf
/etc/sysctl.conf
sysctl.conf
/etc/sysctl.conf
kernel.pid_max=999999
kernel.thread-max=999999
vm.max_map_count=999999
fs.file-max=999999
注意:是pid_max和thread-max
attention: is pid_max and thread-max
您将在 jmeter 安装的 /bin/ 文件夹中找到它们.
you will find those in the /bin/ folder of your jmeter installation.
linux下添加一行
JVM_ARGS="-Xmx15g -Xss250k"
在最后一行实际 java 调用之前的某个地方.这将减少为每个线程分配的堆栈大小.
somewhere before the actual java call on the last line. This will decrease the stacksize allocated for each thread.
在windows下你必须编辑jmeter.bat.我没有在 Windows 下配置它，但至少关于堆应该有一行以 set HEAP 开头.windows下不知道把-Xss参数放在哪里
under windows you have to edit the jmeter.bat.
I have no configured it under windows but at least regarding the heap there should be a line starting with set HEAP. I dont know where to put the -Xss parameter under windows.
这篇关于JMeter - linux 内存不足的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
