JavaMail，IMAP，具有大量文件夹的性能 [英] JavaMail, IMAP, performance with large number of folders

问题描述

我们正在为IMAP帐户开发基于Java的邮件客户端，并使用最新的Java邮件API（1.5.6）。我们的客户拥有超过400个文件夹的邮件帐户。用户在文件夹上执行检查邮件，并对每个文件夹进行迭代并获取新消息，例如

  folder.getMessagesByUid（lastStoredUID，UIDFolder。 LastUID）
  

或获取未读消息的数量需要太多时间，可以使用大量的文件夹。 （我们必须遍历400个文件夹）

为了提高性能，我们在线程中使用了paralel工作连接，我们有一个SESSION实例，但每个线程（连接）都有自己的STORE实例。当我们必须获取新消息时，我们会自动生成10个线程与其STORE实例，分别连接/登录，在STORE上打开文件夹实例，打开文件夹，执行文件夹操作，关闭文件夹，然后关闭SESSION。但是对于我来说，我不清楚，我是否要保持文件夹的意图，而且我不清楚 是否应该打开文件夹，或者我们明确地关闭，或者我们可能会将所有打开的> （关闭操作到mailserver）。

在我目前的实现中，

  new Thread（）{
 public void run（）{
 //获取新的存储实例
存储tempStore = MySingleSession.getStore（imap）; 
 tempStore.connect（..）; 
 
 //获取一个文件夹，例如inbox 
文件夹inbox = tempStore.get（INBOX）; 
 
 inbox.open（Folder.READ）; 
 
 //执行获取新消息
 inbox.getMessagesByUd（lastUID，UIDFolder.MaxUID）; 
 
 inbox.close（）; 
 
 tempStore.close（）; 
 .. 
  

检查性能时，我看到打开/关闭文件夹需要很长时间特别是在超过100.000条消息的文件夹上。我看到即使我关闭了一个文件夹的存储，文件夹仍然打开，如果我们没有明确地关闭该文件夹，我们可以对它进行操作。如果我更改了实现并且没有明确地关闭文件夹并保持打开状态，那么这种机制的缺点是什么？

• 将文件夹引用保留在以其全名为键名的哈希表中，例如
  INBOX - > IMAPFolder，并在执行任何操作（getMessages）之前执行isOpened检查，但完成后，不要像我当前的实现一样明确关闭。当这样做时，我们可能打开了400个文件夹，邮件服务器和邮件服务器关闭它们可能看起来不好，但是如果打开或不打开，我将首先检查每个文件夹操作。因此，文件夹将尽可能保持打开（直到mailserver关闭连接），如果需要再次使用同一个文件夹，则不必再打开它。




• 我不明白，为什么我仍然可以在一个关闭商店的文件夹上操作？




• mail.imap.connectionpoolsize的默认值是1，这有助于增加这个值吗？




• 我不明白下面这种情况会发生什么？

   存储tempStore = MySingleSession.getStore（imap）; 
   
   tempStore.connect（..）; 
   
  文件夹f = tempStore.getFolder（INBOX）; 
   
   f.open（Folder.READ_ONLY）; 
   
   //关闭商店！ 
   tempStore.close（）; 
   
   //现在文件夹f已关闭，
   //但是即使其Store已断开，我们也可以再次打开它！ 
   
   f.open（Folder.READ_ONLY）; 
   
   //我们可以继续操作文件夹f 
    




• 其中是更好的方法，保持文件夹引用在一个hastable由全名键入或执行tempStore.getFolder（文件夹名称）每次？例如，我们必须处理INBOX，首先使用

IMAPFolder f = folderCache.get（INBOX）; //假设已经放入hashtable folderCache

或

IMAPFolder f = tempStore.getFolder（INBOX） ;

我按需生成Store实例，然后关闭它们，以免在邮件服务器上使用庞大的资源。但是再次打开文件夹并且手动关闭不是一个很好的方法，它使用仍然资源？或使用这种方式是可以接受的，因为邮件服务器可能会在需要时关闭它，我们只需检查

• 如果文件夹已经打开，继续操作，


• 如果文件夹关闭，首先打开，然后操作，保持打开以备进一步使用？

解决方案

一个帖子的问题太多了。让我尝试回答其中的一些...

关闭文件夹后，不应该从文件夹中使用任何Message对象，所以只关闭文件夹当您完成使用其消息时。

如果单个用户具有400个文件夹，则可以使用单个存储连接，因为每个打开的文件夹都将获得自己的连接。 / p>

如果您经常打开和关闭文件夹，则增加连接池大小可以帮助，因为关闭的文件夹的连接将添加到池中，并在您打开新文件夹时重新使用，而不需要创建一个新的连接。

关闭商店应关闭该商店的所有文件夹，但如果涉及多个线程，则存在固有的竞争条件。关闭商店是完成使用后清理所有连接的一种方法，而不是阻止其他线程重新打开文件夹的方法。

缓存文件夹关闭文件夹的对象不太可能比每次调用getFolder更大的优点。

We are developing a java based mail client for IMAP accounts and use latest java mail api (1.5.6). Our customers have mail accounts with more than 400 folders. Users perform check mails on folders and iterating on each folder and getting new messages for example,

folder.getMessagesByUid(lastStoredUID, UIDFolder.LastUID)

or getting count of unread messages takes too much time beacuse huge number of folders. (We have to iterate on 400 folders)

To increase the performance we used paralel working connections in threads and we have a single SESSION instance but each thread(connection) has its own STORE instance. When we have to fetch new messages, we generate 10 threads with their STORE instances on the fly, connect/login them separately, get folder instance on that STORE, open folder, perform folder operations, close the folder and close the SESSION afterwards. But for me its not clear, whether I have to keep the folders intances and also I it is not clear whether the folders should be kept opened or should we explicity close aftwerwards or we may leave all opened (leave the close operation to mailserver).

In my current implementation,

new Thread() {
    public void run() {
        //Get a new store instance
        Store tempStore = MySingleSession.getStore("imap");
        tempStore.connect(..);

        //Get a folder for example inbox
        Folder inbox = tempStore.get("INBOX");

        inbox.open(Folder.READ);

        // Perform get new messages
        inbox.getMessagesByUd(lastUID, UIDFolder.MaxUID);

        inbox.close();

        tempStore.close();
        ..

When checked the performance, I see that opening/closing folders takes a long time especially on folders with more than 100.000 messages. And I see that even if I close the store of a folder, folder remains opened and we can operate on it if we do not explicitly close that folder. What are the disadvantages for a such mechanism if I change my implementation and do not close the folder explicitly and keep them open?

• Keep the folder references in a hashtable keyed by their fullname, for example "INBOX" -> IMAPFolder and perform isOpened check before any operation (getMessages), but when finished, do not close explicitly as in my current implementation. When doing like that we may have 400 folders opened, it may seem bad for mail server and mail server close them, but I will check for each folder opeation first, if it is opened or not. As a result folders will be kept opened as possible (until mailserver closes connection) and if the same folder is needed again we do not have to open it again.

• I do not understand, why I can still operate on a folder which has its store closed?

• Default value for mail.imap.connectionpoolsize is 1, does it help to increase this value in this case?

• And I do not understand what happens with this scenario below?

  Store tempStore = MySingleSession.getStore("imap");
  
  tempStore.connect(..);
  
  Folder f = tempStore.getFolder("INBOX");
  
  f.open(Folder.READ_ONLY);
  
  // close the store!!!
  tempStore.close();
  
  // now folder f becomes closed,
  // but we may open it again even if its Store is disconnected!
  
  f.open(Folder.READ_ONLY);
  
  // we can continue operating on folder f
  

• Which is better approach, keep folder references in an hastable keyed by fullname or perform tempStore.getFolder(foldername) each time? For example we have to handle INBOX, first option use

IMAPFolder f = folderCache.get("INBOX"); // suppose already put in hashtable folderCache

or

IMAPFolder f = tempStore.getFolder("INBOX");

I generate the Store instances on demand and close them afterwards in order not to use huge resources on the mail server. But opening the folder again and not closing manually is not a good approach beacuse it uses still resource? or using that way is acceptable because mailserver may close it when needed and we have just to check whether

• if the folder is already opened continue operating,
• if the folder is closed perform open first, then operate, keep opened for further usage?

解决方案

Way too many questions for one post. Let me try to answer a few of them...

After you close a Folder, you shouldn't use any Message objects from the Folder, so only close the Folder when you're done using its Messages.

If a single user has 400 folders, you can use a single Store connection since each open Folder will get its own connection.

If you open and close Folders frequently, increasing the connection pool size can help since the connection for a closed Folder will be added to the pool and reused when you open a new Folder, without needing to create a new connection.

Closing the Store should close all the Folders for that Store, but there's an inherent race condition if multiple threads are involved. Closing the Store is a way to clean up all the connections when you're done using it, not a way to prevent other threads from re-opening Folders.

Caching Folder objects for closed Folders is not likely to be a big advantage over calling getFolder each time.

这篇关于JavaMail，IMAP，具有大量文件夹的性能的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！