JMeter或Apache ab哪种测量方法正确? [英] Which gets the measurements right, JMeter or Apache ab?

问题描述

我开始用JMeter编写一些基本测试，但感到惊讶的是，这些测量结果与Apache ab的测量结果如此不同.

I started writing some basic tests in JMeter and was surprised that the measurements are so different from those from Apache ab.

我有一个千兆局域网，用于连接运行Nginx的Intel i7服务器和运行JMeter或ab的i5测试计算机.最初，我只是测试现成的Nginx主页响应率.

I have a gigabit LAN connecting an Intel i7 server running Nginx and an i5 test machine running JMeter or ab. Initially, I am simply testing the out-of-the box Nginx home page response rate.

ab -c 1 -n 100 http://testserver.local/

给予

Document Path:          /
Document Length:        151 bytes

Concurrency Level:      1
Time taken for tests:   0.078 seconds
Complete requests:      100
Failed requests:        0
Write errors:           0
Total transferred:      38400 bytes
HTML transferred:       15100 bytes
Requests per second:    1280.77 [#/sec] (mean)
Time per request:       0.781 [ms] (mean)
Time per request:       0.781 [ms] (mean, across all concurrent requests)
Transfer rate:          480.29 [Kbytes/sec] received

此结果始终可重复，+/-几个百分点.

This result is consistently reproducible, +/- a few percent.

在JMeter中，我有一个包含1个用户的100循环线程组:

In JMeter, I have a 1-user 100-loop thread group containing:

• HTTP标头管理器设置接受编码:gzip
• HTTP Get/sampler
• 摘要报告侦听器

只有100个样本，每次运行时都会产生不一致的结果.但是，最令人吃惊的事实是，据报道吞吐量低至每秒40个请求(而不是1280个).记录的最高速率是1030，只有当我增加到10,000个样本时才实现.

With only 100 samples, this gives wildly inconsistent results each time I run it. But the most startling fact is that the throughput is reported as low as 40 requests per second (not 1280). The highest recorded rate was 1030, and this was achieved only when I increased to 10,000 samples.

我是否正确地认为JMeter对于简单的负载测试是错误的工具，因为它的开销太高而无法进行精确的测量?

Am I right in thinking that JMeter is the wrong tool for simple load tests because its overheads are too high to allow accurate measurements?

推荐答案

Jmeter告诉您每个请求实际上花费了多长时间. AB只是做一些非常基本的数学运算以获得总平均值.因此，对您问题的直接答案是jmeter正确无误，而ab只是通过为您提供所有数据的均值来进行粗略的猜测.

Jmeter tells you how long each request actually took. AB just does some very basic math to get the overall average. So, the direct answer to your question is that jmeter gets it right and ab just makes a rough guess by giving you the mean across everything.

但是，可以肯定的是，如果您将两个工具并排放置并对其速度进行评级，那么很明显，ab将会表现出更好的性能. Jmeter只是做更多的事情，它记录了更多的数据并且正在处理更多的逻辑，因此处理单个请求花费的时间更长.一个简单的事实是，Jmeter是功能齐全的负载测试工具，而AB则不是.

But, sure, if you put the two tools side by side and rate them for speed then it is clearly the case that ab is going to out perform jmeter. Jmeter just does more, it records more data and is processing more logic so it takes longer to turn around a single request. The simple fact is that Jmeter is a fully featured load testing tool, AB is, well, not.

问题是，负载测试工具的目的不是成为障碍最快的孩子，而是要能够构建逼真的表示您的应用程序的负载类型上线时可能会受到打击.在这方面，jmeter可以胜任，所以这实际上取决于您的要求.如果您只想使用最少的硬件来生成尽可能多的请求，那么ab是一个不错的选择，但是如果您想构建一个具有事务性旅程，条件逻辑和各种其他有用内容的代表性测试，那么jmeter是要走的路.可以这样想:它们都是Apache项目，但我认为AB旨在测试apache网络服务器，而JMeter则旨在测试Tomcat.

The thing is, the aim of a load testing tool is not to be the fastest kid on the block, instead it is about being able to build a realistic representation of the sort of load your app might be hit with when it goes live. In this respect jmeter wins hands down, so it really depends on what your requirements are. If you just want to generate as many requests as possible using the least amount of hardware then ab is a nice choice but if you want to build a representative test, with transactional journeys, conditional logic and all sorts of other useful stuff, then jmeter is the way to go. Think of it like this: they are both Apache projects but AB was, I think, designed to test the apache web server, JMeter, however, was designed to test Tomcat.

现在，我猜测jmeter产生不一致的结果，因为它达到了运行它的机器的极限.我敢打赌，您正在GUI模式下运行，并且至少有一个监听器处于活动状态，像这样，您正在要求该工具执行很多操作.如果您需要很高的请求率，则Jmeter具有精益和均值模式.通常，对于大批量，最佳实践是在命令行上以很少的侦听器执行测试. apache jmeter网站上有很多关于此主题的信息.

Now, I'm guessing that jmeter was producing inconsistent results because it was hitting a limit on the machine it was running on. I'm betting you were running in GUI mode and had at least one listener active, like this you are asking the tool to do a lot. If you need a high rate of requests then Jmeter has a lean and mean mode. Typically, for large volumes the best practice is to execute tests at the command line with very few listeners; there's lots of info about this subject on the apache jmeter site.

如果您真的要进行负载测试，则应该考虑的另一点是，要真正从此类事情中受益，您需要首先确定站点需要支持的负载类型，然后再确定您是否应该设计一个代表这一点的测试.使用起搏和模拟的等待时间可以实现这一点.告诉线程应该尽可能快地消失并尽可能快地运行的问题是，它将在其本地条件允许的情况下尽可能快地进行迭代，但是总是存在 中断，甚至ab都是有限的；不管工具多么轻巧，它仍然会执行某事.但是，如果您加快请求的速度，则可以消除此问题，并且作为相当有用的额外好处，最终可以在运行之间和代码构建之间保持一致，因此即使服务器加速或减速(更改代码库)也是如此您的测试仍将以相同的请求速率进行请求-对于基准测试非常有用.

Another point you should consider, if you're really getting into load testing, is that in order to really get benefit from this sort of thing you need to first decide what sort of load you need your site to support and only then should you design a test that represents this. This is achieved using pacing and simulated wait times. The problem with telling a thread that is should just go away and run as fast as it possibly can is that it will iterate as fast as its local conditions allow it to, but there will always be something that puts the breaks on, even ab is limited; no matter how lightweight a tool is it still does something. But if you pace your requests then you remove this problem and as a rather useful added bonus you end up with consistency between runs and between builds of the code, so even if your server speeds up or slows down (with changes to the code base) your test will still make the same rate of requests - which is pretty useful for benchmarking.

如果您想进一步提高JMeter的性能，请查看常量吞吐量计时器，然后使用多个线程来构建您需要表示的流量级别.

If you want to take JMeter further then have a look at the Constant Throughput Timer and then use multiple threads to build the level of traffic you need to represent.

这篇关于JMeter或Apache ab哪种测量方法正确?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！