我们可以通过哈希对通过IMAP检索的电子邮件进行重复数据删除吗? [英] Can we deduplicate emails retrieved through IMAP by hash?

问题描述

我正在尝试最多处理一次通过IMAP检索到的电子邮件. (我问了一个相关问题.)

计算通过IMAP检索到的MIME消息的加密哈希码以进行重复数据删除是否可靠?

换句话说，为什么通过IMAP多次检索同一封电子邮件会导致不同的结果?电子邮件是否可以更改其内容(例如，当它在文件夹之间移动或被标记为已读或出于其他原因)?

我正在Windows上以Mailkit.NET作为客户端使用hMailserver.不确定这是否重要.

解决方案

许多邮件列表都会附加页脚，因此发送给我和列表的邮件都有两个不同的签名.

大多数人认为这只是一条消息.

我建议使用message-id标头字段进行一次处理.在过去的十年中，AFAICT一直是可靠的独特产品(我见过的最后一次碰撞是在2000年左右).

I'm trying to achieve at-most-once processing of email messages retrieved over IMAP. (I asked a related question about it.)

Is it reliable to compute a cryptographic hash code of the MIME messages retrieved over IMAP to deduplicate them?

In other words, why would the same email result in a different result when retrieved over IMAP multiple times? Can an email change it's contents for example when it's moved across folders, or marked as read or for some other reason?

I'm using hMailserver on Windows with Mailkit.NET as the client. Not sure this matters, though.

解决方案

Many mailing lists append a footer, so mail sent both to me and a list arrives with two different signatures.

Most people consider this to be one message.

I suggest using the message-id header field for at-most-once processing. AFAICT it's been reliably unique for the last ten years (the last collision I've seen was from around 2000).

这篇关于我们可以通过哈希对通过IMAP检索的电子邮件进行重复数据删除吗?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！