无法将缓冲区写入MongoDB GridFS [英] Unable to write buffer to MongoDB GridFS
问题描述
大家好，我花了将近一天的时间来解决这个问题.我正在使用multer的inMemory标志从网页上载图像.显而易见的尝试是将从multer接收的缓冲区写入GridFs(更具体地说是GridStore).
Hi guys i have spent almost a day trying to figure this one out. I'm using multer's inMemory flag to upload an image from my webpage. The obvious attempt is to write the buffer received from multer to GridFs (GridStore more specifically).
这是我的multer代码
here is my multer code
product.js(路由/控制器)
var DB = rek('database');
var router = express.Router();
var multer = require("multer");
var uploadOptions = {inMemory:true, onFileUploadComplete: uploadDone}
router.post('/product/:productId/images/', multer(uploadOptions), uploadImageResponse);
function uploadDone(file) {
var options = {
name:file.name,
mode:"w+",
content_type:file.mimetype,
w: "majority",
chunkSize: 1024,
metadata:file
};
var GridStore = new DB.GridStore(DB.connection, file.name, "w+", options);
GridStore.open(function(err, GS){
if(err) throw err;
return GS.write(file.buffer, function(err, GS){
console.log("file written");
if(err) throw err;
return GS.close(function(err, result){
if(err) throw err
console.log(result);
})
})
});
}
我的数据库对象来自哪里?我只是在初始化期间将其添加到猫鼬对象中.这就是代码的样子
where is my DB object coming from? i'm just adding it to the mongoose object during initialisation. this is what the code looks like
database.js
var mongoose = require("mongoose"),
mongooseTimestamps = require("mongoose-concrete-timestamps"),
autoIncrement = require("mongoose-auto-increment"),
config = require("../config"),
Grid = require("gridfs-stream");
mongoose.connect( config.database['development'].url + "" + config.database['development'].name );
var db = mongoose.connection;
db.once("open", function(err){
if(err) throw err
mongoose.GridStore = mongoose.mongo.GridStore
})
db.on("error",function(errMsg){
console.log("Error Connecting to Mongo: " + errMsg);
});
mongoose.set('debug', true);
mongoose.plugin(mongooseTimestamps);
autoIncrement.initialize(db);
module.exports = mongoose;
因此，无数次更改代码并获得相同的结果-无写，无错误
So this is currently myself up after changing my code countless number of times and arriving at the same result - No Writes, No errors
我每次都会从猫鼬的输出日志中得到这个
I do get this everytime though from the mongoose output log
POST /product/1000/images 200 4.494 ms - 22
Mongoose: fs.chunks.ensureIndex([ [ 'files_id', 1 ], [ 'n', 1 ] ]) { w: 'majority' }
Mongoose: fs.files.find({ filename: '2b08f506ed277eda45f9fc400c098aa1.jpg' }) { readPreference: 'primary', w: 'majority' }
Mongoose: fs.chunks.find({ n: 0, files_id: ObjectId("54bb87aaabf2c0416a50c068") }) { readPreference: 'primary', w: 'majority' }
如果我错了，请更正我，但是为什么当我插入/写入" GridFS时为什么要查找.到目前为止，我得到此输出，而断点仅使我得到对GridStore.open的调用，我使流正确返回，但随后写入从未发生且未引发任何错误.
Correct me if i'm wrong but why is it doing a find when i'm "inserting/writing" to GridFS. So far i get this output and my breakpoints only get me as far as the call to GridStore.open, I get the stream back correctly but then the write never happens and no error is thrown.
到目前为止我尝试过的事情
What i have tried so far
- 使用Multer的内存-相同结果
- 使用Multer的dest属性，并将流从fs传递到gridfs-相同的结果.
- 使用gridfs-stream模块-结果相同
- 使用了本机GridFS/GridStoire-结果相同.
任何帮助将不胜感激.
推荐答案
现在，您错过的是"inMemory"选项中的"buffer"不是"or/or"，也不意味着内容是而是改为在内存中".实际上，它是数据的副本"，它也发送到磁盘上的临时文件.
Now, what you missed here is that the "buffer" from the "inMemory" option is not "either/or" and does not mean that the content is held "In Memory" instead. It is in fact a "copy" of the data that is also sent to the temporary file on disk.
因此，是否设置"inMemory"并不重要，因为仍会创建文件(默认情况下在/tmp目录中)，但是当超出范围时，这些文件当然会取消链接:
So it really doesn't matter if you set "inMemory" or not as the files will still be created ( by default in the /tmp directory ) but these will of course unlink when out of scope:
var async = require('async'),
express = require('express'),
multer = require('multer'),
fs = require('fs'),
mongoose = require('mongoose'),
Grid = require('gridfs-stream'),
Schema = mongoose.Schema;
Grid.mongo = mongoose.mongo;
var app = express(),
gfs = {};
// Set up multer middleware
app.use(
multer({
//inMemory: true
})
);
// Register handler
app.post('/',function (req,res) {
async.eachLimit(Object.keys(req.files), 10, function(file,callback) {
var fileobj = req.files[file];
var writeStream = gfs.createWriteStream({
"filename": fileobj.fieldname
});
fs.createReadStream(fileobj.path).pipe(writeStream);
writeStream.on('close',function() {
console.log('done');
callback();
});
writeStream.on('error',callback);
},function(err) {
if (err) {
console.log(err);
res.status(500).end();
}
res.status(200).end();
});
});
mongoose.connect('mongodb://localhost/test');
// Start app listen and events
var server = app.listen(3000,function() {
mongoose.connection.on('open',function(err) {
if (err) throw err;
// Set up connection
gfs = Grid(mongoose.connection.db);
console.log('listening and connected');
});
});
当然还有一个简单的测试:
And of course a simple test:
var FormData = require('form-data'),
fs = require('fs'),
http = require('http');
var fname = 'GearsLogo.png';
var form = new FormData();
form.append(fname,fs.createReadStream(fname))
var request = http.request({
method: 'post',
port: 3000,
headers: form.getHeaders()
});
form.pipe(request);
request.on('response',function(res) {
console.log(res.statusCode);
});
或者使用您的请求方法内联调用中间件，和/或设置onFileUploadComplete()处理程序，而不是迭代req.files的内容. "gridfs = stream"包可能是您必须上载内容的最简单的选择，并且尝试从作为副本的缓冲区工作并不会真正带来任何实际好处，因为IO成本和存储总会存在.
Alternately call the middle-ware in-line with your request method, and/or set up the onFileUploadComplete() handler rather than iterate the content of req.files. The "gridfs=stream" package is probably the simplest option you have to upload content and trying to work from a buffer that is a copy is not really going to offer any real advantage since the IO cost and storage is always going to be there.
这篇关于无法将缓冲区写入MongoDB GridFS的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
