如何测量鲁棒性? [英] How to measure robustness?

问题描述

我正在撰写有关衡量产品质量的论文.在这种情况下，该产品是一个网站.我已经确定了几种质量属性和测量技术.

I am working on a thesis about meassuring quality of a product. The product in this case is a website. I have identified several quality attributes and meassurement techniques.

一个质量属性是坚固性".我想以某种方式进行测量，但是我找不到任何有用的信息来客观地实现这一目标.

One quality attribute is "Robustness". I want to meassure that somehow, but I can't find any useful information how to do this in an objective manner.

是否有任何静态或动态指标可以衡量稳健性?即，像单元测试覆盖率一样，是否有办法测量这种鲁棒性?如果是这样，是否有任何(免费)工具可以做到这一点?

Is there any static or dynamic metric that could meassure robustness? Ie, like unit test coverage, is there a way to meassure robustness like that? If so, is there any (free) tool that can do such a thing?

有人有使用这种工具的经验吗?

Does anyone have any experience with such tooling?

最后但并非最不重要的一点是，如果您有任何想法让我不知所措，也许还有其他方法可以确定稳健性.

Last but not least, perhaps there are other ways to determine robustness, if you have any ideas about that I am all ears.

非常感谢.

推荐答案

好吧，简短的回答是否".健壮可能意味着很多事情，但是我能想到的最好的定义是在每种情况下都能正确执行".如果您将错误的HTTP标头发送到健壮的Web服务器，则它应该不会崩溃.它应该返回正确的错误类型，并且应该以某种可配置的方式将事件记录在某个地方.如果功能强大的Web服务器可以长时间运行，则其内存占用量应保持不变.

Well, the short answer is "no." Robust can mean a lot of things, but the best definition I can come up with is "performing correctly in every situation." If you send a bad HTTP header to a robust web server, it shouldn't crash. It should return exactly the right kind of error, and it should log the event somewhere, perhaps in a configurable way. If a robust web server runs for a very long time, its memory footprint should stay the same.

使系统变得强大的很多原因是其对边缘情况的处理.好的单元测试是其中的一部分，但是很可能不会对系统存在的任何问题进行单元测试(如果知道了这些问题，开发人员可能会修复它们，然后再添加一个测试)

A lot of what makes a system robust is its handling of edge cases. Good unit tests are a part of that, but it's quite likely that there will not be unit tests for any of the problems that a system has (if those problems were known, the developers probably would have fixed them and only then added a test).

不幸的是，几乎不可能测量任意程序的健壮性，因为要执行此操作，您需要知道该程序应该执行的操作.如果您有规范，则可以编写大量测试，然后对任何客户端运行它们作为测试.例如，看一下Acid2浏览器测试.它以简单，可重复的方式仔细衡量任何给定的Web浏览器符合标准的程度.那差不多就可以了，人们已经指出了这种方法的许多缺陷(例如，一个程序崩溃的次数更多，但是根据规范，是否还有另外一件事更健壮?)

Unfortunately, it's nearly impossible to measure the robustness of an arbitrary program because in order to do that you need to know what that program is supposed to do. If you had a specification, you could write a huge number of tests and then run them against any client as a test. For example, look at the Acid2 browser test. It carefully measures how well any given web browser complies with a standard in an easy, repeatable fashion. That's about as close as you can get, and people have pointed out many flaws with such an approach (for instance, is a program that crashes more often but does one extra thing according to spec more robust?)

但是，您可以使用各种检查来粗略地估计系统的运行状况.单元测试的覆盖范围是一个非常标准的标准，它的同级，分支覆盖范围，功能覆盖范围，语句覆盖范围等也是一个不错的选择.另一个不错的选择是"lint"程序，例如FindBugs.这些可以表明潜在的问题.开源项目通常根据提交和发布的频率以及最近发布的频率来判断.如果项目具有错误系统，则可以测量已修复的错误数量和百分比.如果您要测量的是程序的特定实例，尤其是活动频繁的实例，则MTBF(平均故障间隔时间)是衡量鲁棒性的好方法(请参阅

There are, though, various checks that you could use as a rough, numerical estimate of the health of a system. Unit test coverage is a pretty standard one, as are its siblings, branch coverage, function coverage, statement coverage, etc. Another good choice is "lint" programs like FindBugs. These can indicate the potential for problems. Open source projects are often judged by how frequently and recently commits are made or releases released. If a project has a bug system, you can measure how many bugs have been fixed and the percentage. If there's a specific instance of the program you're measuring, especially one with a lot of activity, MTBF (Mean Time Between Failures) is a good measure of robustness (See Philip's Answer)

但是，这些度量并不能真正告诉您程序的健壮性.它们只是猜测的方式.如果很容易确定程序是否健壮，我们可能只是让编译器进行检查.

These measurements, though, don't really tell you how robust a program is. They're merely ways to guess at it. If it were easy to figure out if a program was robust, we'd probably just make the compiler check for it.

祝您论文顺利！希望您能提出一些很棒的新方法！

Good luck with your thesis! I hope you come up with some cool new measurements!

这篇关于如何测量鲁棒性?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！