Mongo打开了太多连接 [英] Mongo opens too many connections

问题描述

我试图在Java循环中向MongoDB写入大量数据。我根据打开的连接数得到错误。

I am trying to write a lot of data to MongoDB, in a Java loop. I am getting errors based on the number of connections open.

我的理论是，由于MongoDB不是事务性的，因此可以同时打开许多连接。然而，Java代码也能够非常快地循环，在一段时间之后循环迭代的次数开始超过可用连接的数量并且Mongo碰到墙。

My theory is that since MongoDB is not transactional, lots of connections can be opened simultaneously. However the Java code is also able to loop very fast, after a certain time the number of loop iterations starts overtaking the number of available connections and Mongo hits a wall.

My代码看起来像这样。我已经看到它建议不要 m.close（）但是你只是更快地得到错误。

My code looks like this. I've seen it recommended to not do m.close() but then you just get the error even faster.

public static void upsert(){
    Mongo m = null;
    DB db = null;

    try {
    m = new Mongo("localhost");
    db = m.getDB("sempedia");    } catch (UnknownHostException e1) { e1.printStackTrace(); } catch (MongoException e1) { e1.printStackTrace(); }

    // create documents
    // I am doing an upsert - hence the doc, doc
    DBCollection triples;
try {
        triples = db.getCollection("triples");
        triples.update(doc,doc,true,false); 
    } catch (MongoException e) { e.printStackTrace(); }

    m.close();
}

在我的java控制台中，我收到此错误：

In my java console I get this error:

警告：使用0
java.net.SocketException确定maxBSON大小的异常：连接重置

WARNING: Exception determining maxBSON size using0 java.net.SocketException: Connection reset

并且mongodb给出了这个错误：

And mongodb gives this error:

Tue Oct 25 22:31:39 [initandlisten]连接被拒绝因为
许多开放式连接：204 of 204

Tue Oct 25 22:31:39 [initandlisten] connection refused because too many open connections: 204 of 204

处理此问题最优雅的方法是什么？

What would be the most elegant way to deal with this issue?

推荐答案

您正在为每个单独的操作创建Mongo类的实例。这不起作用，因为每个实例将创建并保持至少一个连接（但默认情况下为10），并且只有在Java GC清理Mongo实例或调用close（）时才会删除这些连接。

You are creating an instance of the Mongo class for each individual operation. That won't work since each instance will create and hold at least one connection (but by default, 10) and those connections will only be removed if the Java GC cleans up your Mongo instance or when you invoke close().

问题在于，在这两种情况下，即使使用单个线程，您创建它们的速度也比关闭时更快。这将匆忙耗尽最大量的连接。正确的解决方法是使用单例模式保留一个Mongo实例（Mongo.Holder为此提供功能，请尝试Mongo.Holder.connect（..））。快速修复是增加计算机上的文件描述符限制，因此最大连接数量要高得多，但显然最终可能达到相同的限制。您可以使用（在shell中）检查当前最大值：

The problem is that in both cases you're creating them faster than they are being closed even using a single thread. This will exhaust the maximum amount of connections in a hurry. The right fix is to keep one Mongo instance around using the singleton pattern (Mongo.Holder provides functionality for this, try Mongo.Holder.connect(..)). A quick "fix" is to increase the file descriptor limit on your machine so the maximum amount of connections is considerably higher but obviously you eventually might hit the same limit. You can check your current max using (in shell) :

db.serverStatus().connections

TL; DR：将Mongo实例视为单例，并使它们尽可能长寿，你就是金色的。使用静态方法getInstance（）实现一个MongoFactory，返回一个延迟创建的实例就可以了。祝你好运。

TL;DR : Treat a Mongo instance as a singleton and make them as long-lived as possible and you're golden. Implementing a MongoFactory with a static method getInstance() that returns a lazily created instance will do the trick just fine. Good luck.

这篇关于Mongo打开了太多连接的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！