测试大容量 SMTP 电子邮件发送代码的最佳方法? [英] Best way to test high-volume SMTP email sending code?

问题描述

我在 Windows 服务 (C#) 中编写了一个组件，该组件有时负责发送大量电子邮件.这些电子邮件将发送给许多域的收件人 –真的，任何领域.(是的，收件人想要这封电子邮件.不，我不是垃圾邮件.是的，我正在向 CAN-SPAM 投诉.是的，我知道 从代码发送电子邮件很糟糕.) 许多电子邮件是事务性的(响应用户操作而生成)；有些是批量的(基本上是邮件合并).

I've written a component in a Windows service (C#) which is responsible for sending sometimes large volumes of emails. These emails will go to recipients on many domains – really, any domain. (Yes, the recipients want the email. No, I'm not spamming. Yes, I'm in complaince with CAN-SPAM. Yes, I'm aware sending email from code sucks.) Many of the emails are transactional (generated in response to user actions); some are bulk (mail-merges basically).

我不想依赖外部 SMTP 服务器.(除其他考虑之外，必须检查邮箱是否有退回邮件并尝试解析它们的想法让我感觉很糟糕.)

I do not want to rely on an external SMTP server. (Among other considerations, the thought of having to check a mailbox for bounce messages and trying to parse them gives me bad feelings.)

我的设计相当简单.事务和批量消息都生成并插入到数据库表中.此表包含电子邮件信封和内容，以及尝试次数和重试日期.

My design is fairly simple. Both the transactional and bulk messages are generated and inserted into a DB table. This table contains the email envelope and content, plus an attempt count and retry-after date.

该服务运行几个工作线程，一次抓取 20 行并循环遍历每行.使用 Simple DNS Plus 库，我获取 MX 记录收件人的域，然后使用 System.Net.Mail.SmtpClient 同步发送电子邮件.如果对 Send() 的调用成功，我可以将电子邮件出列.如果它暂时失败，我可以增加尝试次数并设置适当的重试日期.如果它永久失败，我可以出列并处理失败.

The service runs a few worker threads which grab 20 rows at a time and loop through each. Using the Simple DNS Plus library, I grab the MX record(s) of the recipient's domain and then use System.Net.Mail.SmtpClient to synchronously send the email. If the call to Send() succeeds, I can dequeue the email. If it temporarily fails, I can increment the attempt count and set an appropriate retry-after date. If it permanently fails, I can dequeue and handle the failure.

显然，向数百个不同的实际域发送数千封测试电子邮件是一个非常糟糕的主意.但是，我肯定需要对我的多线程发送代码进行压力测试.我也不太确定模拟 SMTP 的各种故障模式的最佳方法是什么.另外，我想确保通过各种垃圾邮件控制方法(灰名单以命名与网络层最相关的方法).

Obviously, sending thousands of test emails to hundreds of different actual domains is a Very Bad Idea. However, I definitely need to stress-test my multi-threaded send code. I'm also not quite sure what the best way is to simulate the various failure modes of SMTP. Plus, I want to make sure I get past the various spam control methods (graylisting to name the most relevant to the network layer of things).

我最近发现我的 ISP 阻止了与我的 ISP 的 SMTP 服务器以外的任何服务器上的端口 25 的连接，这加剧了我的小规模测试困难.(在生产中，这个东西当然会在端口 25 未被阻塞的适当服务器上.这无助于我在我的开发机器上进行测试.)

Even my small-scale testing difficulties are exacerbated by my recent discovery of my ISP blocking connections to port 25 on any server other than my ISP's SMTP server. (In production, this thing will of course be on a proper server where port 25 isn't blocked. That does not help me test from my dev machine.)

所以，我最好奇的两件事是:

So, the two things I'm most curious about:

1. 我应该如何测试我的代码?
2. SmtpClient.Send() 失败的各种方式有哪些?列出了六个例外；SmtpException 和 SmtpFailedRecipientsException 似乎是最相关的.

1. How should I go about testing my code?
2. What are the various ways that SmtpClient.Send() can fail? Six exceptions are listed; SmtpException and SmtpFailedRecipientsException seem to be the most relevant.

<小时>

更新: Marc B 的回答 指出我基本上是在创建自己的 SMTP 服务器.他提出了我正在重新发明轮子的有效观点，所以这是我不使用实际"轮子(后缀等)的理由:

---

Update: Marc B's answer points out that I'm basically creating my own SMTP server. He makes the valid point that I'm reinventing the wheel, so here's my rationale for not using an 'actual' one (Postfix, etc) instead:

1. 电子邮件具有不同的发送优先级(尽管这与信封的 X-Priority 无关).批量电子邮件是低优先级；交易量高.(并且任何电子邮件或电子邮件组都可以进一步配置为具有任意优先级.)我需要能够暂停发送优先级较低的电子邮件，以便可以首先发送优先级较高的电子邮件.(为了实现这一点，工作线程只需在每次获得另外 20 个时从队列中获取最高优先级的项目.)

1. Emails have different send priorities (though this is unrelated to the envelope's X-Priority). Bulk email is low priority; transactional is high. (And any email or group of emails can be further configured to have an arbitrary priority.) I need to be able to suspend the sending of lower-priority emails so higher-priority emails can be delivered first. (To accomplish this, the worker threads simply pick up the highest priority items from the queue each time they get another 20.)

如果我已经向外部 SMTP 服务器提交了数千个批量项目，我无法在我希望提交的项目现在被发送时暂停这些项目.粗略的 Google 搜索 显示 Postfix 并不真正支持优先级;Sendmail 优先考虑信封中的信息，不能满足我的需求.

If I've already submitted several thousand bulk items to an external SMTP server, I have no way of putting those on hold while the items I wish to submit now get sent. A cursory Google search shows Postfix doesn't really support priorities; Sendmail prioritizes on information in the envelope, which does not meet my needs.

我需要能够向我的用户显示爆炸(一组批量电子邮件)的发送过程的进度.如果我只是将所有电子邮件都交给外部服务器，我不知道实际交付的进度有多远.

I need to be able to display the progress of the send process of a blast (group of bulk emails) to my users. If I've simply handed all of my emails off to an external server, I have no idea how far along in actual delivery it is.

我对解析退回邮件犹豫不决，因为每个 MTA 的退回邮件都不同.Sendmail 与 Exchange 不同，与 [...] 不同.另外，我检查退回收件箱的频率是多少?如果退回邮件本身没有送达怎么办?

I'm hesitant to parse bounce messages because each MTA's bounce message is different. Sendmail's is different from Exchange's is different from [...]. Also, at what frequency do I check my bounce inbox? What if a bounce message itself isn't delivered?

我不太担心爆炸中途失败.

I'm not too terribly concerned with a blast failing half-way through.

如果我们谈论的是灾难性故障(应用程序终止未处理的异常、电源故障等):由于工作线程在成功发送每封电子邮件时将其从数据库中出列，因此我可以知道谁收到了爆炸，谁没有收到不.此外，当服务在失败后重置时，它只是从队列中中断的地方开始.

If we're talking catastrophic failure (app-terminating unhandled exception, power failure, whatever): Since the worker threads dequeue each email from the database when it is successfully delivered, I can know who has received the blast and who hasn't. Further, when the service resets after a failure, it simply picks up where it left off in the queue.

如果我们谈论的是本地故障(SmtpException、DNS 故障等):我只是记录故障，增加电子邮件的尝试计数器，然后再试一次.(这基本上是 SMTP 规范所要求的.)在 n 次尝试之后，我可以永久地使消息失败(将其出列)并记录失败以供稍后检查.这样，我可以找到我的代码没有处理的奇怪的边缘情况 –即使我的代码第一次不是 100% 完美.(说实话，不会的.)

If we're talking local failure (a SmtpException, DNS failure, or so forth): I just log the failure, increment the email's attempt counter, and try again later. (Which is basically what the SMTP spec calls for.) After n attempts, I can permanently fail the message (dequeue it) and log the failure for examination later. This way, I can find weird edge cases that my code isn't handling – even if my code isn't 100% perfect the first time. (And let's be honest, it won't be.)

我希望自己滚动路由最终能够让我比依赖外部 SMTP 服务器更快地发送电子邮件.如果服务器不在我的控制之下，我将不得不担心速率限制；即使是这样，它仍然是一个瓶颈.我采用的多线程架构意味着我可以并行连接到多个远程服务器，从而减少了传递 n 条消息所需的总时间.

I'm hoping the roll-my-own route will ultimately allow me to get emails out faster than if I had to rely on an external SMTP server. I'd have to worry about rate-limiting if the server weren't under my control; even if it were, it's still a bottleneck. The multithreaded architecture I've gone with means I'm connecting to multiple remote servers in parallel, decreasing the total amount of time it takes to deliver n messages.

推荐答案

假设您有两台服务器可用.一个是发送者，一个是接收者.您可以使用一长串假域名在两者上设置 DNS(甚至只是主机文件).就两台服务器而言，这些域完全有效，因为本地 DNS 服务器对它们具有权威性，但就网络的其余部分而言完全无效.只需确保解析器在 DNS 之前检查主机文件即可.

Assume you've got two servers available. One will be the sender, one will be the receiver. You can set up DNS (or even just hosts files) on both with a long series of fake domains. As far as the two servers are concerned, those domains are perfectly valid as the local DNS servers are authoritative for them, but are completely invalid as far as the rest of the net is concerned. Just make sure the resolver checks the hosts file before DNS.

完成后，您可以让发送服务器向接收服务器发送垃圾邮件，让接收服务器随心所欲地发送垃圾邮件，接收服务器也可以执行各种操作来测试您的代码反应.灰名单、TCP 延迟、硬退回、ICMP 无法访问、超出 ICMP 跃点等...

Once that's done, you can have the sending server spam the receiving server to your heart's content, as have the receiver do various things to test your code's reactions. Greylisting, TCP delays, hard bounces, ICMP unreachables, ICMP hops exceeded, etc...

当然，鉴于您必须测试所有这些条件，您基本上是在创建自己的 SMTP 服务器，那么为什么不使用实际的开始呢?我猜想对退回邮件进行一些基本解析所需的工作将远远少于必须提出代码块来处理 postfix/sendmail/exim/etc 的所有故障模式......已经很好地处理了他们的拥有.

Of course, given you have to test all these conditions, you're basically creating your own SMTP server, so why not use an actual one to begin with? I'd guess the effort required to do some basic parseing of bounce messages will be far less than having to come up with code chunks to handle all the failure modes that postfix/sendmail/exim/etc... already handle perfectly well on their own.

当您认为您的发送代码必须从一开始就完美时，尤其如此.如果电子邮件爆炸中途失败并且只有一半的收件人列表收到了邮件，那么与几百或几千封邮件被退回的情况相比，你的漏洞要大得多.或者更糟糕的是，以多种不同的方式失败(一些服务器无法访问，一些服务器因流量过多而将您列入灰名单等......).而在您手动处理它们或修补您的反弹解析器来处理它们之前，反弹会很高兴地坐在传入队列中.

And this is especially true when you consider your sending code has to be perfect from the get-go. If an email blast fails part-way through and only half the recipient list gets the message, you're in a far bigger hole than if a few hundred or a few thousand messages bounce. Or worse yet, fails in multiple different ways (some servers unreachable, some greylisting you for excessive traffic, etc...). Whereas bounces will happily sit in the incoming queue until you process them manually, or patch up your bounce parser to handle them.

这篇关于测试大容量 SMTP 电子邮件发送代码的最佳方法?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！