Linux下Java的虚拟内存使用，使用了太多内存 [英] Virtual Memory Usage from Java under Linux, too much memory used

问题描述

我在 Linux 下运行 Java 应用程序时遇到问题.

当我使用默认的最大堆大小 (64 MB) 启动应用程序时，我看到使用 tops 应用程序为应用程序分配了 240 MB 的虚拟内存.这会导致计算机上的一些其他软件出现一些问题，而这些软件相对资源有限.

保留的虚拟内存无论如何都不会被使用，据我所知，因为一旦我们达到堆限制，就会抛出一个OutOfMemoryError.我在windows下运行了同一个应用，发现虚拟内存大小和堆大小差不多.

无论如何，我可以为 Linux 下的 Java 进程配置正在使用的虚拟内存吗?

编辑 1:问题不在于堆.问题是，如果我设置一个 128 MB 的堆，例如，Linux 仍然分配 210 MB 的虚拟内存，这是不需要的.**

Edit 2:使用 ulimit -v 可以限制虚拟内存的数量.如果设置的大小低于 204 MB，那么即使不需要 204 MB，只需要 64 MB，应用程序也不会运行.所以我想了解为什么Java需要这么多的虚拟内存.可以改吗?

编辑 3:系统中还有其他几个应用程序在运行，这些应用程序是嵌入式的.并且系统确实有虚拟内存限制(来自评论，重要细节).

解决方案

这是对 Java 的长期抱怨，但它在很大程度上毫无意义，并且通常基于查看错误的信息.通常的措辞类似于Java 上的 Hello World 需要 10 兆字节！为什么需要那个?"好吧，这里有一种方法可以让 64 位 JVM 上的 Hello World 占用超过 4 GB 的空间……至少通过一种测量形式.

<前>java -Xms1024m -Xmx4096m com.example.Hello

测量内存的不同方法

在 Linux 上，top 命令为您提供了几个不同的内存数字.以下是关于 Hello World 示例的说明:

<前>PID 用户 PR NI VIRT RES SHR S %CPU %MEM TIME+ 命令2120 kgregory 20 0 4373m 15m 7152 S 0 0.2 0:00.10 java

• VIRT 是虚拟内存空间:虚拟内存映射中所有内容的总和(见下文).它基本上没有意义，除非它不是(见下文).
• RES 是常驻集大小:当前驻留在 RAM 中的页数.在几乎所有情况下，这是您在说太大"时应该使用的唯一数字.但这仍然不是一个很好的数字，尤其是在谈到 Java 时.
• SHR 是与其他进程共享的常驻内存量.对于 Java 进程，这通常仅限于共享库和内存映射 JAR 文件.在这个例子中，我只运行了一个 Java 进程，所以我怀疑 7k 是操作系统使用的库的结果.
• 默认情况下未启用 SWAP，此处未显示.它表示当前驻留在磁盘上的虚拟内存量，无论它是否实际位于交换空间中.操作系统非常擅长将活动页面保留在 RAM 中，交换的唯一解决方法是 (1) 购买更多内存，或 (2) 减少进程数量，因此最好忽略此数字.

Windows 任务管理器的情况有点复杂.在 Windows XP 下，有内存使用"和虚拟内存大小"列，但 官方文档 没有说明它们的意思.Windows Vista 和 Windows 7 添加了更多列，它们实际上是 记录.其中，工作集"测量是最有用的；大致相当于Linux上的RES和SHR之和.

了解虚拟内存映射

进程消耗的虚拟内存是进程内存映射中所有内容的总和.这包括数据(例如，Java 堆)，但也包括程序使用的所有共享库和内存映射文件.在 Linux 上，您可以使用 pmap 命令查看映射到进程中的所有内容空间(从这里开始，我将只提及 Linux，因为我使用它；我确信 Windows 有等效的工具).这是Hello World"程序内存映射的摘录；整个内存映射长达100多行，千行列表也不少见.

<前>0000000040000000 36K r-x--/usr/local/java/jdk-1.6-x64/bin/java0000000040108000 8K rwx--/usr/local/java/jdk-1.6-x64/bin/java0000000040eba000 676K rwx-- [匿名]00000006fae00000 21248K rwx-- [匿名]00000006fc2c0000 62720K rwx-- [匿名]0000000700000000 699072K rwx-- [匿名]000000072aab0000 2097152K rwx-- [匿名]00000007aaab0000 349504K rwx-- [匿名]00000007c0000000 1048576K rwx-- [匿名]...00007fa1ed00d000 1652K r-xs-/usr/local/java/jdk-1.6-x64/jre/lib/rt.jar...00007fa1ed1d3000 1024K rwx-- [匿名]00007fa1ed2d3000 4K ----- [匿名]00007fa1ed2d4000 1024K rwx-- [匿名]00007fa1ed3d4000 4K ----- [匿名]...00007fa1f20d3000 164K r-x--/usr/local/java/jdk-1.6-x64/jre/lib/amd64/libjava.so00007fa1f20fc000 1020K -----/usr/local/java/jdk-1.6-x64/jre/lib/amd64/libjava.so00007fa1f21fb000 28K rwx--/usr/local/java/jdk-1.6-x64/jre/lib/amd64/libjava.so...00007fa1f34aa000 1576K r-x--/lib/x86_64-linux-gnu/libc-2.13.so00007fa1f3634000 2044K -----/lib/x86_64-linux-gnu/libc-2.13.so00007fa1f3833000 16K r-x--/lib/x86_64-linux-gnu/libc-2.13.so00007fa1f3837000 4K rwx--/lib/x86_64-linux-gnu/libc-2.13.so...

格式的快速解释:每一行都以段的虚拟内存地址开始.接下来是段大小、权限和段的来源.最后一项是文件或anon"，表示通过 mmap.

从顶部开始，我们有

• JVM 加载器(即，当您键入 java 时运行的程序).这是非常小的；它所做的只是加载到存储真实 JVM 代码的共享库中.
• 一堆保存 Java 堆和内部数据的匿名块.这是一个Sun JVM，所以堆被分成多代，每一代都是自己的内存块.注意JVM根据-Xmx值分配虚拟内存空间；这允许它有一个连续的堆.-Xms 值在内部用于说明程序启动时正在使用"的堆有多少，并在接近该限制时触发垃圾收集.
• 一个内存映射的 JAR 文件，在本例中是保存JDK 类"的文件.当您对 JAR 进行内存映射时，您可以非常有效地访问其中的文件(而不是每次从头开始读取).Sun JVM 将对类路径上的所有 JAR 进行内存映射；如果您的应用程序代码需要访问 JAR，您还可以对其进行内存映射.
• 两个线程的每线程数据.1M 块是线程堆栈.我对 4k 块没有很好的解释，但 @ericsoe 将其标识为保护块":它没有读/写权限，因此如果访问会导致段错误，JVM 会捕获并翻译将其转换为 StackOverFlowError.对于真正的应用程序，您会看到数十个甚至数百个这样的条目在内存映射中重复.
• 保存实际 JVM 代码的共享库之一.其中有几个.
• C 标准库的共享库.这只是 JVM 加载的许多不属于 Java 的严格部分之一.

共享库特别有趣:每个共享库至少有两个段:一个包含库代码的只读段，一个包含库的全局每进程数据的读写段(我不知道没有权限的段是什么；我只在 x64 Linux 上见过它).库的只读部分可以在使用库的所有进程之间共享；例如，libc 有 1.5M 的虚拟内存空间可以共享.

虚拟内存大小何时重要?

虚拟内存映射包含很多东西.其中一些是只读的，一些是共享的，还有一些已分配但从未接触过(例如，在此示例中几乎所有的 4Gb 堆).但是操作系统足够智能，只加载它需要的内容，因此虚拟内存大小在很大程度上无关紧要.

如果您在 32 位操作系统上运行，虚拟内存大小很重要，您只能分配 2Gb(或在某些情况下，3Gb)的进程地址空间.在这种情况下，您正在处理稀缺资源，并且可能必须进行权衡，例如减少堆大小以对大文件进行内存映射或创建大量线程.

但是，鉴于 64 位机器无处不在，我认为不久之后虚拟内存大小将成为一个完全无关的统计数据.

居民集大小何时重要?

常驻集大小是实际在 RAM 中的虚拟内存空间部分.如果您的 RSS 增长到您的总物理内存的很大一部分，则可能是时候开始担心了.如果您的 RSS 增长到占用您所有的物理内存，并且您的系统开始交换，那么该担心的时间已经过去了.

但 RSS 也具有误导性，尤其是在负载较轻的机器上.操作系统不会花费大量精力来回收进程使用的页面.这样做几乎没有什么好处，而且如果进程在将来接触到页面，可能会发生代价高昂的页面错误.因此，RSS 统计信息可能包含许多未处于活动状态的页面.

底线

除非您正在交换，否则不要过分担心各种内存统计信息告诉您什么.需要注意的是，不断增长的 RSS 可能表明存在某种内存泄漏.

对于 Java 程序，关注堆中发生的事情要重要得多.消耗的空间总量很重要，您可以采取一些步骤来减少它.更重要的是您花在垃圾收集上的时间，以及堆的哪些部分正在被收集.

访问磁盘(即数据库)很昂贵，而内存很便宜.如果你可以用一个换另一个，那就这样做吧.

I have a problem with a Java application running under Linux.

When I launch the application, using the default maximum heap size (64 MB), I see using the tops application that 240 MB of virtual Memory are allocated to the application. This creates some issues with some other software on the computer, which is relatively resource-limited.

The reserved virtual memory will not be used anyway, as far as I understand, because once we reach the heap limit an OutOfMemoryError is thrown. I ran the same application under windows and I see that the Virtual Memory size and the Heap size are similar.

Is there anyway that I can configure the Virtual Memory in use for a Java process under Linux?

Edit 1: The problem is not the Heap. The problem is that if I set a Heap of 128 MB, for example, still Linux allocates 210 MB of Virtual Memory, which is not needed, ever.**

Edit 2: Using ulimit -v allows limiting the amount of virtual memory. If the size set is below 204 MB, then the application won't run even though it doesn't need 204 MB, only 64 MB. So I want to understand why Java requires so much virtual memory. Can this be changed?

Edit 3: There are several other applications running in the system, which is embedded. And the system does have a virtual memory limit (from comments, important detail).

解决方案

This has been a long-standing complaint with Java, but it's largely meaningless, and usually based on looking at the wrong information. The usual phrasing is something like "Hello World on Java takes 10 megabytes! Why does it need that?" Well, here's a way to make Hello World on a 64-bit JVM claim to take over 4 gigabytes ... at least by one form of measurement.

java -Xms1024m -Xmx4096m com.example.Hello

Different Ways to Measure Memory

On Linux, the top command gives you several different numbers for memory. Here's what it says about the Hello World example:

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 2120 kgregory  20   0 4373m  15m 7152 S    0  0.2   0:00.10 java

• VIRT is the virtual memory space: the sum of everything in the virtual memory map (see below). It is largely meaningless, except when it isn't (see below).
• RES is the resident set size: the number of pages that are currently resident in RAM. In almost all cases, this is the only number that you should use when saying "too big." But it's still not a very good number, especially when talking about Java.
• SHR is the amount of resident memory that is shared with other processes. For a Java process, this is typically limited to shared libraries and memory-mapped JARfiles. In this example, I only had one Java process running, so I suspect that the 7k is a result of libraries used by the OS.
• SWAP isn't turned on by default, and isn't shown here. It indicates the amount of virtual memory that is currently resident on disk, whether or not it's actually in the swap space. The OS is very good about keeping active pages in RAM, and the only cures for swapping are (1) buy more memory, or (2) reduce the number of processes, so it's best to ignore this number.

The situation for Windows Task Manager is a bit more complicated. Under Windows XP, there are "Memory Usage" and "Virtual Memory Size" columns, but the official documentation is silent on what they mean. Windows Vista and Windows 7 add more columns, and they're actually documented. Of these, the "Working Set" measurement is the most useful; it roughly corresponds to the sum of RES and SHR on Linux.

Understanding the Virtual Memory Map

The virtual memory consumed by a process is the total of everything that's in the process memory map. This includes data (eg, the Java heap), but also all of the shared libraries and memory-mapped files used by the program. On Linux, you can use the pmap command to see all of the things mapped into the process space (from here on out I'm only going to refer to Linux, because it's what I use; I'm sure there are equivalent tools for Windows). Here's an excerpt from the memory map of the "Hello World" program; the entire memory map is over 100 lines long, and it's not unusual to have a thousand-line list.

0000000040000000     36K r-x--  /usr/local/java/jdk-1.6-x64/bin/java
0000000040108000      8K rwx--  /usr/local/java/jdk-1.6-x64/bin/java
0000000040eba000    676K rwx--    [ anon ]
00000006fae00000  21248K rwx--    [ anon ]
00000006fc2c0000  62720K rwx--    [ anon ]
0000000700000000 699072K rwx--    [ anon ]
000000072aab0000 2097152K rwx--    [ anon ]
00000007aaab0000 349504K rwx--    [ anon ]
00000007c0000000 1048576K rwx--    [ anon ]
...
00007fa1ed00d000   1652K r-xs-  /usr/local/java/jdk-1.6-x64/jre/lib/rt.jar
...
00007fa1ed1d3000   1024K rwx--    [ anon ]
00007fa1ed2d3000      4K -----    [ anon ]
00007fa1ed2d4000   1024K rwx--    [ anon ]
00007fa1ed3d4000      4K -----    [ anon ]
...
00007fa1f20d3000    164K r-x--  /usr/local/java/jdk-1.6-x64/jre/lib/amd64/libjava.so
00007fa1f20fc000   1020K -----  /usr/local/java/jdk-1.6-x64/jre/lib/amd64/libjava.so
00007fa1f21fb000     28K rwx--  /usr/local/java/jdk-1.6-x64/jre/lib/amd64/libjava.so
...
00007fa1f34aa000   1576K r-x--  /lib/x86_64-linux-gnu/libc-2.13.so
00007fa1f3634000   2044K -----  /lib/x86_64-linux-gnu/libc-2.13.so
00007fa1f3833000     16K r-x--  /lib/x86_64-linux-gnu/libc-2.13.so
00007fa1f3837000      4K rwx--  /lib/x86_64-linux-gnu/libc-2.13.so
...

A quick explanation of the format: each row starts with the virtual memory address of the segment. This is followed by the segment size, permissions, and the source of the segment. This last item is either a file or "anon", which indicates a block of memory allocated via mmap.

Starting from the top, we have

• The JVM loader (ie, the program that gets run when you type java). This is very small; all it does is load in the shared libraries where the real JVM code is stored.
• A bunch of anon blocks holding the Java heap and internal data. This is a Sun JVM, so the heap is broken into multiple generations, each of which is its own memory block. Note that the JVM allocates virtual memory space based on the -Xmx value; this allows it to have a contiguous heap. The -Xms value is used internally to say how much of the heap is "in use" when the program starts, and to trigger garbage collection as that limit is approached.
• A memory-mapped JARfile, in this case the file that holds the "JDK classes." When you memory-map a JAR, you can access the files within it very efficiently (versus reading it from the start each time). The Sun JVM will memory-map all JARs on the classpath; if your application code needs to access a JAR, you can also memory-map it.
• Per-thread data for two threads. The 1M block is the thread stack. I didn't have a good explanation for the 4k block, but @ericsoe identified it as a "guard block": it does not have read/write permissions, so will cause a segment fault if accessed, and the JVM catches that and translates it to a StackOverFlowError. For a real app, you will see dozens if not hundreds of these entries repeated through the memory map.
• One of the shared libraries that holds the actual JVM code. There are several of these.
• The shared library for the C standard library. This is just one of many things that the JVM loads that are not strictly part of Java.

The shared libraries are particularly interesting: each shared library has at least two segments: a read-only segment containing the library code, and a read-write segment that contains global per-process data for the library (I don't know what the segment with no permissions is; I've only seen it on x64 Linux). The read-only portion of the library can be shared between all processes that use the library; for example, libc has 1.5M of virtual memory space that can be shared.

When is Virtual Memory Size Important?

The virtual memory map contains a lot of stuff. Some of it is read-only, some of it is shared, and some of it is allocated but never touched (eg, almost all of the 4Gb of heap in this example). But the operating system is smart enough to only load what it needs, so the virtual memory size is largely irrelevant.

Where virtual memory size is important is if you're running on a 32-bit operating system, where you can only allocate 2Gb (or, in some cases, 3Gb) of process address space. In that case you're dealing with a scarce resource, and might have to make tradeoffs, such as reducing your heap size in order to memory-map a large file or create lots of threads.

But, given that 64-bit machines are ubiquitous, I don't think it will be long before Virtual Memory Size is a completely irrelevant statistic.

When is Resident Set Size Important?

Resident Set size is that portion of the virtual memory space that is actually in RAM. If your RSS grows to be a significant portion of your total physical memory, it might be time to start worrying. If your RSS grows to take up all your physical memory, and your system starts swapping, it's well past time to start worrying.

But RSS is also misleading, especially on a lightly loaded machine. The operating system doesn't expend a lot of effort to reclaiming the pages used by a process. There's little benefit to be gained by doing so, and the potential for an expensive page fault if the process touches the page in the future. As a result, the RSS statistic may include lots of pages that aren't in active use.

Bottom Line

Unless you're swapping, don't get overly concerned about what the various memory statistics are telling you. With the caveat that an ever-growing RSS may indicate some sort of memory leak.

With a Java program, it's far more important to pay attention to what's happening in the heap. The total amount of space consumed is important, and there are some steps that you can take to reduce that. More important is the amount of time that you spend in garbage collection, and which parts of the heap are getting collected.

Accessing the disk (ie, a database) is expensive, and memory is cheap. If you can trade one for the other, do so.

这篇关于Linux下Java的虚拟内存使用，使用了太多内存的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！