Node.js 代码加载到 mongodb 不一致 [英] Node.js code loading inconsistently to mongodb

问题描述

以下代码尝试从 csv 文件加载到在本地主机上运行的 mongodb 实例.

The following code tries to load from a csv file to a mongodb instance running on localhost.

问题 - 每次运行时加载不同数量的文档(始终少于 csv 中的记录总数).

Problem - It loads a different number of documents on every run (always less than total number of records in csv).

var csv = require('csv');

var server = new Server('localhost', 27017, {auto_reconnect: true, poolSize: 1});
var db = new Db('test', server);

db.open(function(err, db, onemore) {
  if(!err) {
    //Database connection is established.
    db.collection('teststocks', function(err, collection) {
      if(!err) {
        // Stocks collection is connected, open the file and insert the doc
        console.log("Trying to load from " + process.argv[2]);
        csv()
          .fromPath(process.argv[2], {
            columns: true
          })
          .on('data', function(data, index) {
            //data.stock = process.argv[2].substring(process.argv[2].lastIndexOf('/') + 1, process.argv[2].lastIndexOf('.'));
            collection.insert(data, {safe: true}, function(error, collection){
                    if ( error ) { console.log("Error inserting record : " + error); }
            });
            console.log("Inserted data for " + index);
          })
          .on('error', function(error) {
            db.close();
            console.log("Error: " + error);
          })
          .on('end', function(count) {
            console.log("Finished all writing.");
            db.close();
          });
      }
  });
  }
});

P.S:我可以使用 mongoimport 实用程序加载数据，但我是 node.js 和 mongodb 的新手；我想了解我在上面的代码中犯的错误.

推荐答案

这可能是因为您在 csv() 完成读取后关闭了数据库连接.但是，由于 collection.insert 是异步的，不能保证每次调用都在 csv 完成和 db.close() 被调用之前完成.

It's likely because you're closing the database connection after the csv() finishes reading. But, as collection.insert is asynchronous, there's no guarantee each call has finished before the csv is done and db.close() is called.

一种选择是将 csv data 收集到一个 Array 中，然后一次 insert 所有这些:

One option is to collect the csv data into an Array, then insert all of them at once:

var docs = [];

csv()
  // ...
  .on('data', function (data, index) {
    docs.push(data); // or possibly: docs[index] = data;
  })
  // ...
  .on('end', function () {
    console.log("Finished reading CSV.");    

    collection.insert(docs, { safe: true }, function (error, inserted) {
      console.log("Finished all writing.");
      db.close();
    });
  });

这篇关于Node.js 代码加载到 mongodb 不一致的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！