SocketInputStream.socketRead0（）中CPU使用率过高的原因 [英] Reasons for high CPU usage in SocketInputStream.socketRead0()

问题描述

在对自行开发的网络应用程序进行分析时，我发现了非常奇怪的（至少对我来说）观察。

While profiling homegrown web-application I came across following very strange (at least for me) observation.

几乎所有的时间花费在 socketRead0（） SocketInputStream 类的方法。这并不奇怪，因为我的应用程序在每个请求上都使用远程服务进行联网。奇怪的是，此方法不仅挂钟时间使用率很高， CPU时钟时间也非常高。我无法理解为什么CPU时间很长，因为如果我的应用程序等待远程服务回复（实际上并不是那么快），那么应用程序本身就没有什么可做的了。所以CPU时间应该显然很低。

Almost all time is spent in socketRead0() method of a SocketInputStream class. It's not surprising because my application do a networking with a remote service on each request. What is strange that not only wall clock time usage is high for this method, CPU clock time also very high. I can't understand why CPU time is high, because if my application wait for remote service to reply (which is not so fast in fact), there is nothing to do left for application itself. So CPU time should be apparently low.

更多观察：

• VisualVM在采样模式下显示方法 SocketInputStream.socketRead0（）正在吃掉高达95％的时间（挂钟时间和 CPU时间）;


• mpstat （我们使用Linux作为操作系统）显示约90％的用户时间和约1-3％的系统时间（其余的）是空闲时间）;


• 应用程序部署在专用服务器上;


• 远程服务也是HTTP Web应用程序。平均响应时间约为100毫秒。平均响应大小约为2Kb。


• 我的应用程序使用spring RestTemplate 与远程服务进行交互，而不是 SocketInputStream 直接。

• VisualVM in sampling mode shows that method SocketInputStream.socketRead0() is eating up to 95% of time (both wall clock time and CPU time);
• mpstat (we use Linux as a OS) shows around ~90% user time and ~1-3% system time (the rest is idle time);
• application deployed on the dedicated server;
• remote service is HTTP web-application also. Average response time is about 100ms. Average response size is about 2Kb.
• my application use spring RestTemplate to interact with remote service, not the SocketInputStream directly.

现在我只有一个想法 - 也许这是调用本机方法的开销JVM（ SocketInputStream.socketRead0（）是原生的）？

For now I have only one idea — maybe this is overhead of calling native methods in JVM (SocketInputStream.socketRead0() is native)?

你怎么看？还有其他原因吗？

What do you think? Is there any other reasons to this?

推荐答案

我面临同样的问题。我的应用程序有一个非常高的qps，每个请求将使我发送多个thrift调用，使用这个本机api： socketRead0

I am facing the same problem. My application has a very high qps and each request will make me send multiple thrift calls, which use this native api : socketRead0

所以我决定做一个实验。我在返回之前制作一个带有api睡眠30s的模拟服务器，并且客户端调用此api。我的目的是在net io发生时测试线程状态。基于我的线程转储，线程状态是 RUNNABLE 。

So I decide to do an experiment. I make a mock server with an api sleep 30s before return and a client calls this api. My purpose is to test the thread status when the net io happening. Based on my thread dump, the thread status is RUNNABLE.

这解释了两件事：

1. 具有高qps阻塞的应用程序io将面临高CPU负载值

1. application with high qps blocking io will face a high cpu load value

你的java线程仍然在jvm中运行，因为线程状态是 RUNNABLE 这将有助于提高用户空间cpu利用率

your java thread is still running in jvm since the thread state is RUNNABLE which will contribute to high user space cpu utilization

这两个都会让你的cpu忙。

both of these will make your cpu busy.

我在实验中注意到系统空间的cpu利用率低。我认为这与jvm和os之间的线程调度策略差异有关。我们知道hotspot线程模型是1：1，意味着一个jvm线程到一个os线程。当阻塞io发生时，例如 socketRead0 内核线程将设置为状态 S 并且不会阻塞cpu，但是用户空间线程阻塞（等待）。当发生这种情况时，我认为我们需要在我们的应用程序中重新考虑基本的I / O模型。

I noticed during the experiment, the system space cpu utilization is low. I think this something relates to the thread scheduling strategy difference between jvm and os. We know hotspot threading model is 1:1, meaning one jvm thread to one os thread. when a blocking io happened, such as socketRead0 the kernel thread will set to state S and will not blocking cpu, but user space thread is blocking(waiting). when this happened, I think we need to rethink the fundamental I/O model in our application.

这篇关于SocketInputStream.socketRead0（）中CPU使用率过高的原因的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！