Linux下的Java虚拟内存使用情况，使用的内存过多 [英] Virtual Memory Usage from Java under Linux, too much memory used

问题描述

我在Linux下运行的Java应用程序有问题。

I have a problem with a Java application running under Linux.

当我启动应用程序时，使用默认的最大堆大小（64 MB），我看到使用top应用程序，240 MB的虚拟内存分配给应用程序。这会在计算机上产生一些其他软件的问题，这些软件相对资源有限。

When I launch the application, using the default maximum heap size (64 MB), I see using the tops application that 240 MB of virtual Memory are allocated to the application. This creates some issues with some other software on the computer, which is relatively resource-limited.

据我所知，保留的虚拟内存无论如何都不会被使用，因为一旦我们达到堆限制，就会抛出 OutOfMemoryError 。我在Windows下运行相同的应用程序，我发现虚拟内存大小和堆大小相似。

The reserved virtual memory will not be used anyway, as far as I understand, because once we reach the heap limit an OutOfMemoryError is thrown. I ran the same application under windows and I see that the Virtual Memory size and the Heap size are similar.

无论如何我可以配置正在使用的虚拟内存Linux下的Java进程？

Is there anyway that I can configure the Virtual Memory in use for a Java process under Linux?

编辑1 ：问题不在于堆。问题是，如果我设置128 MB的堆，例如，Linux仍然会分配210 MB的虚拟内存，这是不需要的。**

Edit 1: The problem is not the Heap. The problem is that if I set a Heap of 128 MB, for example, still Linux allocates 210 MB of Virtual Memory, which is not needed, ever.**

编辑2 ：使用 ulimit -v 允许限制虚拟内存量。如果大小设置低于204 MB，则应用程序将不会运行，即使它不需要204 MB，只需64 MB。所以我想了解为什么Java需要这么多虚拟内存。可以更改吗？

Edit 2: Using ulimit -v allows limiting the amount of virtual memory. If the size set is below 204 MB, then the application won't run even though it doesn't need 204 MB, only 64 MB. So I want to understand why Java requires so much virtual memory. Can this be changed?

编辑3 ：系统中还运行了其他几个应用程序，这些应用程序是嵌入式的。并且系统确实有虚拟内存限制（来自评论，重要细节）。

Edit 3: There are several other applications running in the system, which is embedded. And the system does have a virtual memory limit (from comments, important detail).

推荐答案

这是一个长期存在的抱怨使用Java，但它基本上没有意义，通常基于查看错误的信息。通常的措辞就像Java上的Hello World需要10兆字节！为什么需要它？好吧，这是一种让64位JVM上的Hello World占用超过4千兆字节的方法......至少通过一种测量形式。

This has been a long-standing complaint with Java, but it's largely meaningless, and usually based on looking at the wrong information. The usual phrasing is something like "Hello World on Java takes 10 megabytes! Why does it need that?" Well, here's a way to make Hello World on a 64-bit JVM claim to take over 4 gigabytes ... at least by one form of measurement.

java -Xms1024m -Xmx4096m com.example.Hello

测量内存的不同方法

在Linux上， top 命令为您提供了几个不同的内存编号。以下是关于Hello World示例的内容：

Different Ways to Measure Memory

On Linux, the top command gives you several different numbers for memory. Here's what it says about the Hello World example:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2120 kgregory  20   0 4373m  15m 7152 S    0  0.2   0:00.10 java

• VIRT是虚拟内存空间：虚拟内存映射中所有内容的总和（见下文）。它基本上没有意义，除非它没有（见下文）。


• RES是驻留集大小：当前驻留在RAM中的页数。在几乎所有情况下，这是说太大时应该使用的唯一数字。但它仍然不是一个非常好的数字，特别是在谈论Java时。


• SHR是与其他进程共享的驻留内存量。对于Java进程，这通常仅限于共享库和内存映射的JAR文件。在这个例子中，我只运行了一个Java进程，所以我怀疑7k是操作系统使用的库的结果。


• 默认情况下SWAP没有打开，并且不是这里显示。它表示当前驻留在磁盘上的虚拟内存量，它是否实际位于交换空间。操作系统非常适合将活动页面保存在RAM中，并且交换的唯一方法是（1）购买更多内存，或（2）减少进程数量，因此最好忽略此数字。


• VIRT is the virtual memory space: the sum of everything in the virtual memory map (see below). It is largely meaningless, except when it isn't (see below).
• RES is the resident set size: the number of pages that are currently resident in RAM. In almost all cases, this is the only number that you should use when saying "too big." But it's still not a very good number, especially when talking about Java.
• SHR is the amount of resident memory that is shared with other processes. For a Java process, this is typically limited to shared libraries and memory-mapped JARfiles. In this example, I only had one Java process running, so I suspect that the 7k is a result of libraries used by the OS.
• SWAP isn't turned on by default, and isn't shown here. It indicates the amount of virtual memory that is currently resident on disk, whether or not it's actually in the swap space. The OS is very good about keeping active pages in RAM, and the only cures for swapping are (1) buy more memory, or (2) reduce the number of processes, so it's best to ignore this number.

Windows任务管理器的情况有点复杂。在Windows XP下，有内存使用和虚拟内存大小列，但官方文档没有说明它们的含义。 Windows Vista和Windows 7添加了更多列，它们实际上是记录。其中，工作集测量是最有用的;它大致对应于Linux上RES和SHR的总和。

The situation for Windows Task Manager is a bit more complicated. Under Windows XP, there are "Memory Usage" and "Virtual Memory Size" columns, but the official documentation is silent on what they mean. Windows Vista and Windows 7 add more columns, and they're actually documented. Of these, the "Working Set" measurement is the most useful; it roughly corresponds to the sum of RES and SHR on Linux.

虚拟进程消耗的内存是进程内存映射中所有内容的总和。这包括数据（例如，Java堆），但也包括程序使用的所有共享库和内存映射文件。在Linux上，您可以使用 pmap 命令查看映射到流程中的所有内容空间（从现在开始我只是指Linux，因为它是我使用的;我确信有相同的Windows工具）。这是Hello World程序的内存映射的摘录;整个内存映射超过100行，并且拥有千行列表并不罕见。

The virtual memory consumed by a process is the total of everything that's in the process memory map. This includes data (eg, the Java heap), but also all of the shared libraries and memory-mapped files used by the program. On Linux, you can use the pmap command to see all of the things mapped into the process space (from here on out I'm only going to refer to Linux, because it's what I use; I'm sure there are equivalent tools for Windows). Here's an excerpt from the memory map of the "Hello World" program; the entire memory map is over 100 lines long, and it's not unusual to have a thousand-line list.

0000000040000000     36K r-x--  /usr/local/java/jdk-1.6-x64/bin/java
0000000040108000      8K rwx--  /usr/local/java/jdk-1.6-x64/bin/java
0000000040eba000    676K rwx--    [ anon ]
00000006fae00000  21248K rwx--    [ anon ]
00000006fc2c0000  62720K rwx--    [ anon ]
0000000700000000 699072K rwx--    [ anon ]
000000072aab0000 2097152K rwx--    [ anon ]
00000007aaab0000 349504K rwx--    [ anon ]
00000007c0000000 1048576K rwx--    [ anon ]
...
00007fa1ed00d000   1652K r-xs-  /usr/local/java/jdk-1.6-x64/jre/lib/rt.jar
...
00007fa1ed1d3000   1024K rwx--    [ anon ]
00007fa1ed2d3000      4K -----    [ anon ]
00007fa1ed2d4000   1024K rwx--    [ anon ]
00007fa1ed3d4000      4K -----    [ anon ]
...
00007fa1f20d3000    164K r-x--  /usr/local/java/jdk-1.6-x64/jre/lib/amd64/libjava.so
00007fa1f20fc000   1020K -----  /usr/local/java/jdk-1.6-x64/jre/lib/amd64/libjava.so
00007fa1f21fb000     28K rwx--  /usr/local/java/jdk-1.6-x64/jre/lib/amd64/libjava.so
...
00007fa1f34aa000   1576K r-x--  /lib/x86_64-linux-gnu/libc-2.13.so
00007fa1f3634000   2044K -----  /lib/x86_64-linux-gnu/libc-2.13.so
00007fa1f3833000     16K r-x--  /lib/x86_64-linux-gnu/libc-2.13.so
00007fa1f3837000      4K rwx--  /lib/x86_64-linux-gnu/libc-2.13.so
...

格式的简要说明：每行开始与段的虚拟内存地址。接下来是段大小，权限和段的来源。最后一项是文件或anon，表示通过 mmap 分配的内存块a>。

A quick explanation of the format: each row starts with the virtual memory address of the segment. This is followed by the segment size, permissions, and the source of the segment. This last item is either a file or "anon", which indicates a block of memory allocated via mmap.

从顶部开始，我们有



• JVM加载程序（即，当您键入 java 时运行的程序）。这个很小;所有这一切都是在存储真实JVM代码的共享库中加载。


• 一堆anon块阻塞了Java堆和内部数据。这是一个Sun JVM，因此堆被分成多代，每一代都是它自己的内存块。请注意，JVM根据 -Xmx 值分配虚拟内存空间;这允许它有一个连续的堆。内部使用 -Xms 值来表示程序启动时正在使用中有多少堆，并在接近该限制时触发垃圾收集。


• 内存映射的JAR文件，在这种情况下是包含JDK类的文件。当您对JAR进行内存映射时，您可以非常有效地访问其中的文件（而不是每次从头开始读取它）。 Sun JVM将对类路径上的所有JAR进行内存映射;如果你的应用程序代码需要访问JAR，你也可以对它进行内存映射。


• 两个线程的每线程数据。 1M块是一个线程堆栈;我不知道4K块会发生什么。对于真正的应用程序，您将看到通过内存映射重复的数十个（如果不是数百个）条目。


• 其中一个包含实际JVM代码的共享库。其中有几个。


• C标准库的共享库。这只是JVM加载的许多内容之一，它们并不是Java的严格组成部分。


• The JVM loader (ie, the program that gets run when you type java). This is very small; all it does is load in the shared libraries where the real JVM code is stored.
• A bunch of anon blocks holding the Java heap and internal data. This is a Sun JVM, so the heap is broken into multiple generations, each of which is its own memory block. Note that the JVM allocates virtual memory space based on the -Xmx value; this allows it to have a contiguous heap. The -Xms value is used internally to say how much of the heap is "in use" when the program starts, and to trigger garbage collection as that limit is approached.
• A memory-mapped JARfile, in this case the file that holds the "JDK classes." When you memory-map a JAR, you can access the files within it very efficiently (versus reading it from the start each time). The Sun JVM will memory-map all JARs on the classpath; if your application code needs to access a JAR, you can also memory-map it.
• Per-thread data for two threads. The 1M block is a thread stack; I don't know what goes into the 4K block. For a real app, you will see dozens if not hundreds of these entries repeated through the memory map.
• One of the shared libraries that holds the actual JVM code. There are several of these.
• The shared library for the C standard library. This is just one of many things that the JVM loads that are not strictly part of Java.

共享库特别有趣：每个共享库至少有两个段：一个包含库代码的只读段，以及一个包含库的全局每个进程数据的读写段（我不知道没有权限的段是什么;我'我们只在x64 Linux上看过它。库的只读部分可以在使用该库的所有进程之间共享;例如， libc 具有可共享的1.5M虚拟内存空间。

The shared libraries are particularly interesting: each shared library has at least two segments: a read-only segment containing the library code, and a read-write segment that contains global per-process data for the library (I don't know what the segment with no permissions is; I've only seen it on x64 Linux). The read-only portion of the library can be shared between all processes that use the library; for example, libc has 1.5M of virtual memory space that can be shared.

虚拟内存映射包含很多内容。其中一些是只读的，一些是共享的，一些是分配但从未触及过（例如，在这个例子中几乎所有的4Gb堆）。但操作系统非常智能，只能加载它所需的内容，因此虚拟内存大小在很大程度上无关紧要。

The virtual memory map contains a lot of stuff. Some of it is read-only, some of it is shared, and some of it is allocated but never touched (eg, almost all of the 4Gb of heap in this example). But the operating system is smart enough to only load what it needs, so the virtual memory size is largely irrelevant.

虚拟内存大小非常重要的是你运行的时候在32位操作系统上，您只能分配2Gb（或在某些情况下，3Gb）的进程地址空间。在这种情况下，您正在处理稀缺资源，并且可能必须进行权衡，例如减少堆大小以便对大文件进行内存映射或创建大量线程。

Where virtual memory size is important is if you're running on a 32-bit operating system, where you can only allocate 2Gb (or, in some cases, 3Gb) of process address space. In that case you're dealing with a scarce resource, and might have to make tradeoffs, such as reducing your heap size in order to memory-map a large file or create lots of threads.

但是，鉴于64位机器无处不在，我认为虚拟内存大小与完全不相关的统计数据之前不会很久。

But, given that 64-bit machines are ubiquitous, I don't think it will be long before Virtual Memory Size is a completely irrelevant statistic.

驻留集大小是实际在RAM中的虚拟内存空间的一部分。如果你的RSS增长到你的总物理内存的很大一部分，可能是时候开始担心。如果你的RSS增长占用你的所有物理内存，并且你的系统开始交换，那么开始担心就好了。

Resident Set size is that portion of the virtual memory space that is actually in RAM. If your RSS grows to be a significant portion of your total physical memory, it might be time to start worrying. If your RSS grows to take up all your physical memory, and your system starts swapping, it's well past time to start worrying.

但是RSS也有误导性，特别是在轻装机。操作系统不会花费大量精力来回收进程使用的页面。这样做几乎没有什么好处，如果进程在未来触及页面，则可能会出现代价高昂的页面错误。因此，RSS统计信息可能包含许多未处于活动状态的页面。

But RSS is also misleading, especially on a lightly loaded machine. The operating system doesn't expend a lot of effort to reclaiming the pages used by a process. There's little benefit to be gained by doing so, and the potential for an expensive page fault if the process touches the page in the future. As a result, the RSS statistic may include lots of pages that aren't in active use.

除非你交换，否则不要过分关注各种内存统计信息告诉你。需要注意的是，不断增长的RSS可能表明某种内存泄漏。

Unless you're swapping, don't get overly concerned about what the various memory statistics are telling you. With the caveat that an ever-growing RSS may indicate some sort of memory leak.

使用Java程序时，注意堆中发生的事情更为重要。消耗的空间总量很重要，您可以采取一些措施来减少这种情况。更重要的是你在垃圾收集中花费的时间，以及收集堆的哪些部分。

With a Java program, it's far more important to pay attention to what's happening in the heap. The total amount of space consumed is important, and there are some steps that you can take to reduce that. More important is the amount of time that you spend in garbage collection, and which parts of the heap are getting collected.

访问磁盘（即数据库）是昂贵的，记忆很便宜。如果你可以换一个换另一个，那就这样做。

Accessing the disk (ie, a database) is expensive, and memory is cheap. If you can trade one for the other, do so.

这篇关于Linux下的Java虚拟内存使用情况，使用的内存过多的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！