单元测试期间调试断言的最佳实践 [英] Best practice for debug Asserts during Unit testing
问题描述
大量使用单元测试会阻止调试断言的使用吗?似乎在要测试的代码中触发调试断言暗示着单元测试不应该存在或调试断言不应该存在. 只能有一个"似乎是一个合理的原则.这是常见的做法吗?还是在单元测试时禁用调试断言，以便它们可以进行集成测试?
Does heavy use of unit tests discourage the use of debug asserts? It seems like a debug assert firing in the code under test implies the unit test shouldn't exist or the debug assert shouldn't exist. "There can be only one" seems like a reasonable principle. Is this the common practice? Or do you disable your debug asserts when unit testing, so they can be around for integration testing?
Edit:我更新了"Assert"以调试断言，以区分被测代码中的断言与单元测试中测试运行后检查状态的行.
I updated 'Assert' to debug assert to distinguish an assert in the code under test from the lines in the unit test that check state after the test has run.
这也是我认为显示困境的一个例子: 单元测试为受保护的功能传递了无效的输入，该功能断言它的输入是有效的.单元测试应该不存在吗?这不是公共功能.也许检查输入会杀死性能?还是断言不存在?该功能受保护不是私有的，因此应检查其输入是否安全.
Also here is an example that I believe shows the dilema: A unit test passes invalid inputs for a protected function that asserts it's inputs are valid. Should the unit test not exist? It's not a public function. Perhaps checking the inputs would kill perf? Or should the assert not exist? The function is protected not private so it should be checking it's inputs for safety.
推荐答案
这是一个非常有效的问题.
This is a perfectly valid question.
首先，许多人建议您错误地使用了断言.我认为许多调试专家会不同意.尽管用断言检查不变量是一种好习惯，但是断言不应局限于状态不变量.实际上，除了检查不变式以外，许多专家调试器还会告诉您声明可能导致异常的任何条件.
First of all, many people are suggesting that your are using assertions wrongly. I think many debugging experts would disagree. Although it is good practice to check invariants with assertions, assertions shouldn't be limited to state invariants. In fact, many expert debuggers will tell you to assert any conditions that may cause an exception in addition to checking invariants.
例如，考虑以下代码:
if (param1 == null)
throw new ArgumentNullException("param1");
很好.但是，当引发异常时，堆栈将解开堆栈，直到有东西处理异常(可能是某些顶级默认处理程序)为止.如果此时执行暂停(您可能在Windows应用程序中有一个模式异常对话框)，则有机会附加调试器，但是您可能丢失了很多信息，这些信息可以帮助您解决此问题，因为大部分堆栈都已展开.
That's fine. But when the exception is thrown, the stack gets unwound until something handles the exception (probably some top level default handler). If execution pauses at that point (you may have a modal exception dialog in a Windows app), you have the chance to attach a debugger, but you have probably lost a lot of the information that could have helped you to fix the issue, because most of the stack has been unwound.
现在考虑以下几点:
if (param1 == null)
{
Debug.Fail("param1 == null");
throw new ArgumentNullException("param1");
}
现在，如果出现问题，将弹出模态断言对话框.执行立即暂停.您可以自由地连接所选的调试器，并在故障的确切点准确检查堆栈中的内容以及系统的所有状态.在发行版本中，您仍然会遇到异常.
Now if the problem occurs, the modal assert dialog pops up. Execution is paused instantaneously. You are free to attach your chosen debugger and investigate exactly what's on the stack and all the state of the system at the exact point of failure. In a release build, you still get an exception.
现在我们如何处理您的单元测试?
Now how do we handle your unit tests?
考虑一个单元测试，测试上面包含断言的代码.您想检查param1为null时是否引发了异常.您希望特定的断言失败，但是任何其他断言失败都将表明有问题.您想允许特定测试的特定断言失败.
Consider a unit test that tests the code above that includes the assertion. You want to check that the exception is thrown when param1 is null. You expect that particular assertion to fail, but any other assertion failures would indicate that something is wrong. You want to allow particular assertion failures for particular tests.
解决此问题的方式取决于您使用的语言等.但是，如果您使用的是.NET，我有一些建议(我实际上没有尝试过，但是以后会更新帖子):
The way you solve this will depend upon what languages etc. you're using. However, I have some suggestions if you are using .NET (I haven't actually tried this, but I will in the future and update the post):
- 检查Trace.Listeners.查找DefaultTraceListener的任何实例，并将AssertUiEnabled设置为false.这将阻止弹出模式对话框.您还可以清除listeners集合，但不会进行任何跟踪.
- 编写您自己的TraceListener来记录断言.您如何记录断言取决于您.记录失败消息可能还不够好，因此您可能需要遍历堆栈以查找断言来自的方法并进行记录.
- 测试结束后，检查发生的唯一断言失败是否是您期望的失败.如果还有其他情况发生，请通过测试.
对于一个TraceListener的示例，其中包含执行这样的堆栈遍历的代码，我将搜索SUPERASSERT.NET的SuperAssertListener并检查其代码. (如果您真的很想使用断言进行调试，那么集成SUPERASSERT.NET也很值得.)
For an example of a TraceListener that contains the code to do a stack walk like that, I'd search for SUPERASSERT.NET's SuperAssertListener and check its code. (It's also worth integrating SUPERASSERT.NET if you're really serious about debugging using assertions).
大多数单元测试框架都支持测试设置/拆卸方法.您可能想要添加代码以重置跟踪侦听器，并断言那些区域中没有任何意外的断言失败，以最大程度地减少重复并防止错误.
Most unit test frameworks support test setup/teardown methods. You may want to add code to reset the trace listener and to assert that there aren't any unexpected assertion failures in those areas to mimimize duplication and prevent mistakes.
更新:
这里是一个示例TraceListener，可用于对测试断言进行单元化.您应该将一个实例添加到Trace.Listeners集合中.您可能还希望提供一些简单的方法，使您的测试可以掌握监听器.
Here is an example TraceListener that can be used to unit test assertions. You should add an instance to the Trace.Listeners collection. You'll probably also want to provide some easy way that your tests can get hold of the listener.
注意:这要归功于约翰·罗宾斯(John Robbins)的SUPERASSERT.NET.
NOTE: This owes an awful lot to John Robbins' SUPERASSERT.NET.
/// <summary>
/// TraceListener used for trapping assertion failures during unit tests.
/// </summary>
public class DebugAssertUnitTestTraceListener : DefaultTraceListener
{
/// <summary>
/// Defines an assertion by the method it failed in and the messages it
/// provided.
/// </summary>
public class Assertion
{
/// <summary>
/// Gets the message provided by the assertion.
/// </summary>
public String Message { get; private set; }
/// <summary>
/// Gets the detailed message provided by the assertion.
/// </summary>
public String DetailedMessage { get; private set; }
/// <summary>
/// Gets the name of the method the assertion failed in.
/// </summary>
public String MethodName { get; private set; }
/// <summary>
/// Creates a new Assertion definition.
/// </summary>
/// <param name="message"></param>
/// <param name="detailedMessage"></param>
/// <param name="methodName"></param>
public Assertion(String message, String detailedMessage, String methodName)
{
if (methodName == null)
{
throw new ArgumentNullException("methodName");
}
Message = message;
DetailedMessage = detailedMessage;
MethodName = methodName;
}
/// <summary>
/// Gets a string representation of this instance.
/// </summary>
/// <returns></returns>
public override string ToString()
{
return String.Format("Message: {0}{1}Detail: {2}{1}Method: {3}{1}",
Message ?? "<No Message>",
Environment.NewLine,
DetailedMessage ?? "<No Detail>",
MethodName);
}
/// <summary>
/// Tests this object and another object for equality.
/// </summary>
/// <param name="obj"></param>
/// <returns></returns>
public override bool Equals(object obj)
{
var other = obj as Assertion;
if (other == null)
{
return false;
}
return
this.Message == other.Message &&
this.DetailedMessage == other.DetailedMessage &&
this.MethodName == other.MethodName;
}
/// <summary>
/// Gets a hash code for this instance.
/// Calculated as recommended at http://msdn.microsoft.com/en-us/library/system.object.gethashcode.aspx
/// </summary>
/// <returns></returns>
public override int GetHashCode()
{
return
MethodName.GetHashCode() ^
(DetailedMessage == null ? 0 : DetailedMessage.GetHashCode()) ^
(Message == null ? 0 : Message.GetHashCode());
}
}
/// <summary>
/// Records the assertions that failed.
/// </summary>
private readonly List<Assertion> assertionFailures;
/// <summary>
/// Gets the assertions that failed since the last call to Clear().
/// </summary>
public ReadOnlyCollection<Assertion> AssertionFailures { get { return new ReadOnlyCollection<Assertion>(assertionFailures); } }
/// <summary>
/// Gets the assertions that are allowed to fail.
/// </summary>
public List<Assertion> AllowedFailures { get; private set; }
/// <summary>
/// Creates a new instance of this trace listener with the default name
/// DebugAssertUnitTestTraceListener.
/// </summary>
public DebugAssertUnitTestTraceListener() : this("DebugAssertUnitTestListener") { }
/// <summary>
/// Creates a new instance of this trace listener with the specified name.
/// </summary>
/// <param name="name"></param>
public DebugAssertUnitTestTraceListener(String name) : base()
{
AssertUiEnabled = false;
Name = name;
AllowedFailures = new List<Assertion>();
assertionFailures = new List<Assertion>();
}
/// <summary>
/// Records assertion failures.
/// </summary>
/// <param name="message"></param>
/// <param name="detailMessage"></param>
public override void Fail(string message, string detailMessage)
{
var failure = new Assertion(message, detailMessage, GetAssertionMethodName());
if (!AllowedFailures.Contains(failure))
{
assertionFailures.Add(failure);
}
}
/// <summary>
/// Records assertion failures.
/// </summary>
/// <param name="message"></param>
public override void Fail(string message)
{
Fail(message, null);
}
/// <summary>
/// Gets rid of any assertions that have been recorded.
/// </summary>
public void ClearAssertions()
{
assertionFailures.Clear();
}
/// <summary>
/// Gets the full name of the method that causes the assertion failure.
///
/// Credit goes to John Robbins of Wintellect for the code in this method,
/// which was taken from his excellent SuperAssertTraceListener.
/// </summary>
/// <returns></returns>
private String GetAssertionMethodName()
{
StackTrace stk = new StackTrace();
int i = 0;
for (; i < stk.FrameCount; i++)
{
StackFrame frame = stk.GetFrame(i);
MethodBase method = frame.GetMethod();
if (null != method)
{
if(method.ReflectedType.ToString().Equals("System.Diagnostics.Debug"))
{
if (method.Name.Equals("Assert") || method.Name.Equals("Fail"))
{
i++;
break;
}
}
}
}
// Now walk the stack but only get the real parts.
stk = new StackTrace(i, true);
// Get the fully qualified name of the method that made the assertion.
StackFrame hitFrame = stk.GetFrame(0);
StringBuilder sbKey = new StringBuilder();
sbKey.AppendFormat("{0}.{1}",
hitFrame.GetMethod().ReflectedType.FullName,
hitFrame.GetMethod().Name);
return sbKey.ToString();
}
}
您可以在每次测试开始时针对期望的断言将断言添加到AllowedFailures集合中.
You can add Assertions to the AllowedFailures collection at the start of each test for the assertions you expect.
在每个测试结束时(希望您的单元测试框架支持测试拆解方法):
At the end of every test (hopefully your unit testing framework supports a test teardown method) do:
if (DebugAssertListener.AssertionFailures.Count > 0)
{
// TODO: Create a message for the failure.
DebugAssertListener.ClearAssertions();
DebugAssertListener.AllowedFailures.Clear();
// TODO: Fail the test using the message created above.
}
这篇关于单元测试期间调试断言的最佳实践的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
