如何使用Firebase / Node.js迭代大型数据集？ [英] How do I iterate through a large dataset with Firebase / Node.js?

问题描述

<$ p $目前我正在使用 Fireproof 来使它成为承诺的基础。 （函数（questionSnapshot））{
返回console.log（questionSnapshot.val（））;
}）;
}）;

但是 questions 是巨大的。 20,000多条记录。不知道 - 有没有更好的方法？

解决方案

这个问题真的是Node中的迭代性能，而不是Firebase ，如果我理解正确。问题是，一旦你有这个20000+的对象数组，什么是最好的方式来遍历它们，并执行每一个操作？

有几个可能的性能优化。首先，你可以使用来代替循环而不是 Array.prototype.forEach（），对于非常大的数组，往往会更快。

  ... 
 var len = questionsSnapshot.length; 
 for（var i = 0; i< len; i ++）{
 console.log（questionsSnapshot [i] .val（））; 
} 
  

或者，使用ES6 ...



  for（let questionsnapshot）{
 console.log（question.val（））; 
 
 $ / code> 

然而，在使用节点上的nodejs.org/api/console.html#console_console_time_labelrel =nofollow>我注意到没有一致的性能增益。实际上， Array.prototype.forEach（）通常比使用循环更快。

另一个优化可能是将传递给 forEach（）的嵌套回调移动到模块范围中。这样，它只会被创建一次，而不是每次调用其父功能时重新创建。

  fbase.child（ （value）。then（function（questionsSnapshot）{
 return questionsSnapshot.forEach（logValue）; 
}）; 
 
函数logValue（questionSnapshot）{
 return console.log（questionSnapshot.val（））; 
 
 $ / code> 

然而，如果您拨打很多电话，这只会导致显着的性能提升到父功能，即，如果经常执行您的Firebase查询。在我看来，这个问题更多地是在你已经拥有了一个大数组的时候进行迭代，而不是关于多次执行相同的查询。



所以，如果你想要为你的数组中的每个项目执行一个操作，我想你在做什么是好的。 20000个物体并不是那么庞大，在我的不起眼的计算机上迭代这样一个数组需要大约5ms。实际上，无论你对每个项目做什么都可能比遍历数组花费更长的时间。

如果你注意到一个特定的性能滞后，你可能想要通过编辑您的帖子来阐述。否则，我建议不要担心。

Currently I'm using Fireproof to make it promise-based and here's what I have:

fbase.child("questions").once("value").then(function(questionsSnapshot) {
  return questionsSnapshot.forEach(function(questionSnapshot) {
    return console.log(questionSnapshot.val());
  });
});

But questions is huge. 20,000+ records. Not sure - is there a better way?

解决方案

This question is really about iteration performance in Node rather than anything to do with Firebase, if I understand correctly. The question is, once you have this 20000+ array of objects, what is the best way to iterate through them and perform an operation on each one?

There are a few possible performance optimizations you could make. First, you could use a for loop rather than Array.prototype.forEach(), which tends to be faster for very large arrays.

...
var len = questionsSnapshot.length;
for (var i = 0; i < len; i++) {
  console.log(questionsSnapshot[i].val());
}

Or, using ES6 for...of syntax:

for (let question of questionsSnapshot) {
  console.log(question.val());
}

However, in a quick test using console.time() in Node on an array of 20000 objects, I noticed no consistent performance gain. In fact Array.prototype.forEach() was often faster than using a for loop.

The other optimization you might make is to move the nested callback passed to forEach() into the module scope. That way, it will only be created once, rather than being recreated each time its parent function is called.

fbase.child("questions").once("value").then(function(questionsSnapshot) {
  return questionsSnapshot.forEach(logValue);
});

function logValue(questionSnapshot) {
  return console.log(questionSnapshot.val());
}

However, this will only result in a significant performance gain if you make many calls to the parent function, i.e., if perform your Firebase query very often. It sounds to me like the question is more about iterating over a large array once you already have it, not about performing the same query many times over.

So, if you want to perform an operation for each item in your array, I think what you are doing is fine. 20000 objects is not all that huge; it takes about 5ms to iterate through such an array on my unimpressive computer. In fact, whatever it is you are doing to each item is likely to take longer than iterating through the array.

If you notice a particular lag in performance, you might want to elaborate by editing your post. Otherwise, I would suggest not worrying about it.

这篇关于如何使用Firebase / Node.js迭代大型数据集？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！