批量写入 Firebase Cloud Firestore [英] Batch write to firebase cloud firestore

问题描述

我想创建一个新集合并向其中添加数千个大小约为 1-2K 的文档.我已经在 json 中有数据，所以我认为这很容易.

I want to create a new collection and add thousands of documents sized ~ 1-2K to it. I already have data in json so I thought this would be easy.

我知道批处理一次可以进行 500 次写入，因此为了将其分成 500 次，我编写了以下代码.尽管出于测试目的，我使用 20 个块运行它，而我的测试 json 有 72 个对象.

I understand that batch can have 500 writes at a time so to break it into chunks of 500 I wrote the following code. Though for testing purpose I am running it with chunks of 20 and my test json has 72 objects.

但我不断收到以下错误

[e-mail]:148
  throw new Error('Cannot modify a WriteBatch that has been committed.');
  ^

Error: Cannot modify a WriteBatch that has been committed.

我的代码如下

var dataObj = JSON.parse(fs.readFileSync('./bigt.json'))
var tmpdd = dataObj.slice(0, 72)
var batch = db.batch();

console.log(tmpdd.length)

let tc = tmpdd.length
let lc = 0
let upperLimit = 20, dd = null

while(lc<=tc){

    dd = tmpdd.slice(lc, upperLimit )

    console.log(lc, upperLimit)
    dd.map(
    o => batch.set(db.collection('nseStocks').doc(o.Date+o.variable), o)
    )

    batch.commit().then(function () {
        console.log('Written to firestore', lc, lc + upperLimit)
    })
    .catch(
        (err) => console.log('Fail', err)
    )

    lc = upperLimit
    upperLimit = upperLimit + 20 

}

另外，在循环的每次迭代中似乎都没有提交批处理也很奇怪.理想情况下，我会让 Firestore 确定文档 ID，但显然批处理没有添加功能.

Also it's weird that batch doesn't seem to be committed in every iteration of the loop. Ideally I would let Firestore determine document ids but apparently batch does not have add function.

我尝试在循环中添加文档而不是批量写入.但是在添加几个文件后它给了我超时错误.当然，对于大量文档来说，这是不切实际的.

I have tried adding documents in a loop instead of doing batch writes. But it gives me timeout error after adding a few documents. And of course it's not practical for large number of documents.

你可以说我对 Firestore 很陌生，这是我玩它的第二天.

You could tell I am very new to Firestore and it's my second day playing with it.

如果有任何明显的错误或更好的方法来完成这个看似简单的任务，请告诉我.

Please let me know if there are any obvious mistakes or better ways of achieving this seemingly simple task.

谢谢

推荐答案

您正在为程序顶层的所有写入创建一个批处理.它正在为您为所有批量写入所做的所有对 batch.set() 的调用重复使用.

You're creating a single batch for all writes at the top level of your program. It's getting reused for all the calls to batch.set() that you make for all your batch writes.

var batch = db.batch();

相反，您应该为每组写入创建一个新批次.您可以在 while 循环的顶部执行此操作:

Instead, you should create a new batch for each set of writes. You can do that at the top of your while loop:

while(lc<=tc) {
    var batch = db.batch();
    // use the new batch here
}

这篇关于批量写入 Firebase Cloud Firestore的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！