为什么unix和Java RunTime显示的统计数据存在差异 [英] why is there difference in statistics shown by unix and Java RunTime

问题描述

我的应用程序存在内存问题，需要帮助了解这些统计信息。

I am having some memory issues with my application and need help understanding these statistics.

Unix'top'显示我的进程的这些统计信息 -

Unix 'top' shows these stats for my process-

VSZ: 37.4g
RSS: 20.0g 

因此，这意味着20g目前已交换进程并正在使用中。

So, this means 20g is currently swapped in for the process and in use.

然而，当我使用Runtime类从我的应用程序中打印统计数据时，我得到了这个：

However, when I print stats from within my application using Runtime class, I get this:

Runtime.totalMemory() : 9.8G
Runtime.freeMemory()  : 3.6G
Runtime.maxMemory()  : 14.3G

为什么[Runtime.totalMemory（） - Runtime.freeMemory（）]与RSS不匹配？这是该进程当前使用的内存。这两个数字之间存在巨大差异。

Why doesn't [Runtime.totalMemory() - Runtime.freeMemory()] match RSS? This is the memory currently in use by the process. There is a huge difference between the two numbers.

此外，运行时是否将未使用的内存（Runtime.freeMemory（））返回给OS以供其他进程使用？

Also, does the runtime give back the unused memory (Runtime.freeMemory()) back to OS for use by other processes?

请注意，我的应用程序运行在使用共享和复制缓存设置的对等GemFire缓存系统中。我需要优化应用程序以减少内存占用。

Note that my applications are running in a peer to peer GemFire caching system set up with shared and replicated caches. I need to optimize the application to reduce memory footprint.

推荐答案

Runtime.totalMemory显示当前可用的内存。 Java懒惰地分配内存。

Runtime.totalMemory shows currently available memory. Java allocates memory lazily.

此外，运行时是否将未使用的内存（Runtime.freeMemory（））返回给OS以供其他流程？

Also, does the runtime give back the unused memory (Runtime.freeMemory()) back to OS for use by other processes?

否。如果Java分配内存（totalMemory）它现在在java进程中。

No. If Java allocated memory (totalMemory) it is in java process now.

RSS：20.0g
14.3G

RSS: 20.0g 14.3G

如前所述，Java除堆外还使用内存。
Gemfire也使用了堆内存（检查这个） 。
尝试在VisualVM-Buffer Monitor中查看它们。

As already mentioned Java uses memory besides heap. Also Gemfire uses off heap memory (check this). Try to look at them in VisualVM-Buffer Monitor.

你的基础设施（操作系统，虚拟机）是什么？

What is your infrastracture (OS, VMs)?

如果你不能使用标准工具，你可能应该使用JMX编写自己的可维护性代理（例如）

If you can't use standard tools, you probably should write own serviceability agent using JMX (for example)

UPDATE

根据doc，默认情况下Gemfire使用JVM堆。

As per doc, by default Gemfire uses JVM Heap.

确定

为什么RSS始终如一显示20G用于仅使用~10G堆内存的进程。

Why is RSS consistently showing 20G for a process that is using only ~10G of heap memory.

如果您仍然问，我会提供更多详细信息。什么是java内存使用？

If you still ask, I would provide more details. What are java memory usage?

• 堆内存（ Xmx ）;


• 堆栈内存（每个线程都有自己的堆栈 ThreadStackSize ， VMThreadStackSize ）;


• MaxPermSize或MaxMetaspaceSize（ MaxPermGenSize ， MaxMetaspaceSize ）;


• 直接字节缓冲区（ MaxDirectMemorySize ）;


• 内存池（Par Eden Space，Par Survivor Space，CMS Old Gen，Metaspace / PermGen，Code Cache，Compressed Class Space（ CompressedClassSpaceSize ））;


• String table（所有String.intern（）将在哪里， StringTableSize ）;


• PerfDataMemorySize ;


• MarkStackSize ;


• CompilerThreadStackSize ;


• 常量池;


• 我可能错过了某事;

• Heap memory (Xmx);
• Stack memory (each thread has own stack ThreadStackSize, VMThreadStackSize);
• MaxPermSize or MaxMetaspaceSize (MaxPermGenSize, MaxMetaspaceSize);
• Direct byte buffers (MaxDirectMemorySize);
• Memory Pools (Par Eden Space, Par Survivor Space, CMS Old Gen, Metaspace/PermGen, Code Cache, Compressed Class Space (CompressedClassSpaceSize));
• String table (where all String.intern() will be, StringTableSize);
• PerfDataMemorySize;
• MarkStackSize;
• CompilerThreadStackSize;
• Constant Pool;
• I likely missed something;

要查看默认值，可以运行：

To see default values, you can run:

 java -XX:+UnlockDiagnosticVMOptions -XX:+PrintFlagsFinal -version

要查看特定流程的哪些选项，您可以运行：

To see which options for your particular process, you can run:

 jps -lvm

是否有命令行unix中的工具确定其余用法？

Is there an command line tool in unix to determine the rest usage?

其中一些是的。对于他们中的一些没有。

For some of them yes. For some of them no.

对于直接内存，你可以试试sun.misc.SharedSecrets.getJavaNioAccess（）。getDirectBufferPool（）。getMemoryUsed（）。

For direct memory you could try sun.misc.SharedSecrets.getJavaNioAccess().getDirectBufferPool().getMemoryUsed().

使用cmd，例如：

 jcmd <pid> VM.native_memory baseline

但这取决于运行设置（ NativeMemoryTracking ）。您可以阅读如何启用本机内存跟踪此处。

But it depends on running settings (NativeMemoryTracking). You could read how to enable Native Memory tracking here.

在linux中你也可以使用：

In linux you also could use:

 pmap -x <pid>

总之这可能无关紧要，因为你的任务：

In conclusion this is probably no matter, because your task:

请注意，我的应用程序运行在使用共享和复制缓存设置的对等GemFire缓存系统中。我需要优化应用程序以减少内存占用。

Note that my applications are running in a peer to peer GemFire caching system set up with shared and replicated caches. I need to optimize the application to reduce memory footprint.

并且你不能影响本机内存使用。我想看一下 jmap util，它可以显示类直方图。您应该检查GemFire中具有大尺寸的内容并查看这些对象，可能是您存储在缓存数据中，而不应该存在。我的意思是在我的优化缓存实践中，我会查看对象和字段，看看哪些字段非常频繁，哪些字段不是。另一种方法是检查序列化机制和对象布局。

And you can not impact on native memory usage. I suppose look at jmap util, which can show you class histogram. You should check what has big size in GemFire and review those objects, probably you store in cache data, which are not should be there. I mean in my practice for optimizing cache I review object and fields and see what fields are really frequent, what are not. Another approach is check your serialization mechanism and object layouts.

正如我所提到的，你可以使用可维护性代理：

As I mentioned you can use serviceability agents:

import com.sun.tools.attach.VirtualMachine;
import sun.tools.attach.HotSpotVirtualMachine;

import java.io.InputStream;

public class JMemoryMain {

    public static void main(String[] args) throws Exception {
        final int pid = JMemoryMainUtils.getPid(args);
        final HotSpotVirtualMachine vm = (HotSpotVirtualMachine) VirtualMachine.attach(String.valueOf(pid));
        final byte[] buffer = new byte[1024];
        try (InputStream is = vm.heapHisto()) {
            for (int read; (read = is.read(buffer)) > 0;) {
                System.out.write(buffer, 0, read);
            }
        }
        vm.detach();
    }

}

你需要JDK的tools.jar在依赖项中运行它。它也可以打印你的类直方图，但有时候工作，当jmap没有。
此外，当你应该等待很多时间，在计算直方图时，你可以使用VM.getSystemDictionary（）并只查找你的类。
而且，如果由于开销而无法启用NMT，那将非常有用。

You need tools.jar from JDK in dependencies to run this. It could print you class histogram too, but sometimes works, when jmap does not. Also, when you should wait a lot of time, while histogram is calculated, you could use VM.getSystemDictionary() and find only your classes. And, it would be usefull, if you can not enable NMT, because of overhead.

这篇关于为什么unix和Java RunTime显示的统计数据存在差异的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！