Java定期挂在futex上并且IO输出非常低 [英] Java periodically hangs at futex and very low IO output

问题描述

当前，我的应用程序定期在IO中被阻止，并且输出非常低.我使用一些命令来跟踪该过程.

Currently my application periodically blocked in IO , and the output is very low . I use some command to trace the process.

通过使用 jstack ，我发现该应用程序挂在FileOutputStream.writeBytes上.

By using jstack i found that the app is hanging at FileOutputStream.writeBytes.

通过使用 strace -f -c -p pid 收集系统调用信息，我发现了.在正常情况下，它同时具有futex和write syscall.但是，当它变得不正常时，只有futex系统调用.该应用程序不断调用futex，但全部失败并抛出ETIMEDOUT，如下所示:

By using strace -f -c -p pid to collect syscall info, i found that. For normal situation, it has both futex and write syscalls. But when it went unnormal, there are only futex syscalls. The app keeps calling futex but all failed and throw ETIMEDOUT, just like this:

<futex resumed>  =-1 ETIMEDOUT (Connecton timed out)
futex(Ox7f823, FUTEX_WAKE_PRIVATE,1)=0
futex(Ox7f824, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME) =-1<unfinished>
<futex resumed>  =-1 ETIMEDOUT (Connecton timed out)
futex(Ox7f823, FUTEX_WAKE_PRIVATE,1)=0
futex(Ox7f824, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME) =-1<unfinished>

此问题定期发生，持续几分钟或几个小时，然后又恢复正常.

This issue happens periodically ,and continues for mintues or hours, and go normal again.

通常，当在IO中被阻止时， echo 3>/proc/sys/vm/drop_caches 总是使它暂时正常. 我在Google上搜索了一下，发现了一些类似的问题，在下面列出.

Escipally, when blocked in IO, echo 3 > /proc/sys/vm/drop_caches always makes it go normal temporarily. I googled it and found some similiar proleam, listing below.

1. le跃.不起作用，我们系统的ntpd已停止.
2. 透明的大页面错误. https://bugzilla.redhat.com/show_bug.cgi?id=879801 这与我的问题非常相似，但是我的khugepaged进程是正常的，并且负载始终接近于零.通常， drop_caches 也适用于我的应用程序.而且我的系统也是多核和大内存的.它对我不起作用. 那么有人遇到同样的问题或熟悉此问题吗?

1. leap second. Doesn't work, our system's ntpd is stopped.
2. transparent hugepage bug. https://bugzilla.redhat.com/show_bug.cgi?id=879801 This is very similar to my probleam, but my khugepaged process is normal, and the load is always nearly zero. Escipally drop_caches works for my application too. And my system is also multi core and large memory. It donsn't work for me. So anyone met the same probleam or familiar with this issue?

有关我的系统的一些信息. 操作系统:Redhat 6.1，内核版本2.6.31

Some info about my system. OS:Redhat 6.1, kernal version 2.6.31

JDK:1.7.0_05

JDK:1.7.0_05

CPU:X5650，24核

CPU:X5650, 24cores

内存:24GB和48GB

Memory :24GB and 48GB

推荐答案

也许是futex_wait()中的内核错误?

Maybe the kernel bug in futex_wait()?

您可以在此处阅读有关内容: https://groups.google.com/forum/#!topic/mechanical -sympathy/QbmpZxp6C64

You can read about it here: https://groups.google.com/forum/#!topic/mechanical-sympathy/QbmpZxp6C64

这篇关于Java定期挂在futex上并且IO输出非常低的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！