是否有MongoDB的最大bson大小可解决? [英] Is there a MongoDB maximum bson size work around?

问题描述

我正在处理的文档非常大.它从一个非常长的调查(例如调查猴子)中收集用户输入，并将答案存储在mongodb数据库中.

The document I am working on is extremely large. It collects user input from an extremely long survey (like survey monkey) and stores the answers in a mongodb database.

我很意外地收到以下错误

I am unsurprisingly getting the following error

Error: Document exceeds maximal allowed bson size of 16777216 bytes

如果我不能更改文档中的字段，我可以做些什么?是否有某种方法可以通过删除空白或类似内容来压缩文档?

If I cannot change the fields in my document is there anything I can do? Is there some way to compress down the document, by removing white space or something like that?

修改

这是文档的结构

Schema({
    id : { type: Number, required: true },
    created: { type: Date, default: Date.now },
    last_modified: { type: Date, default: Date.now },
    data : { type: Schema.Types.Mixed, required: true }
});

数据字段的示例:

{
    id: 65,
    question: {
        test: "some questions",
        answers: [2,5,6]
    }
    // there could be thousands of these question objects
}

推荐答案

您可以做的一件事是构建自己的mongoDB :-). Mongodb是一个开放的源，并且文档大小的限制对于强制执行此行并为其构建你自己.对此要小心.

One thing you can do is to build your own mongoDB :-). Mongodb is an open source and the limitation about the size of a document is rather arbitrary to enforce a better schema design. You can just modify this line and build it for yourself. Be careful with this.

最直接的想法是在不同的文档中每个小问题，并在其字段中引用其上级.

The most straight forward idea is to have each small question in a different document with a field which reference to its parent.

另一种想法是限制父文档中的文档数量.可以说，如果限制为N个元素，则父元素如下所示:

Another idea is to limit number of documents in the parent. Lets say you limit is N elements then the parent looks like this:

{
  _id : ObjectId(),
  id : { type: Number, required: true },
  created: { type: Date, default: Date.now },  // you can store it only for the first element
  last_modified: { type: Date, default: Date.now }, // the same here
  data : [{
    id: 65,
    question: {
        test: "some questions",
        answers: [2,5,6]
    }
  }, ... up to N of such things {}
  ]
}

通过这种方式修改数字N，可以确保您位于16 MB的BSON中.为了阅读整个调查表，您可以选择

This way modifying number N you can make sure that you will be in 16 MB of BSON. And in order to read the whole survey you can select

db.coll.find({id: the Id you need})，然后在应用程序级别上合并整个调查.同样不要忘记在id上确保确保索引.

db.coll.find({id: the Id you need}) and then combine the whole survey on the application level. Also do not forget to ensureIndex on id.

尝试不同的方法，对数据进行基准测试，看看哪种方法对您有用.

Try different things, do a benchmark on your data and see what works for you.

这篇关于是否有MongoDB的最大bson大小可解决?的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！