Google App Maker将记录保存到数据库的时间为每条记录240毫秒 [英] Google App Maker saving records to DB is taking 240ms per record

问题描述

我们有一个使用Google Cloud SQL表的Google App Maker应用.我们的位置在中欧布拉格，保存一项记录至少需要240毫秒，令人难以置信.

We are having a Google App Maker app using Google Cloud SQL tables. Our location is Prague, Central Europe and saving one record takes incredible 240ms at minimum.

Google App Maker似乎有一个代理靠近我的位置(延迟约20毫秒).但是，我们在美国中部1(Google推荐使用此实例来为Google App Maker应用提供最佳性能)中设置的实例显示〜120ms ping.

It looks like Google App Maker has a proxy just close to my location (latency ~20ms). However our instance set up in US-Central1 (which is recommened by Google to provide Google App Maker apps with the best performance) shows ~120ms ping.

我们尝试了很多事情，包括将实例位置更改为德国法兰克福(约25毫秒).在这种情况下，它花费的时间甚至更长(每条记录500毫秒).我们的查询似乎是往返于这种复杂的方式: 我们的位置(代理)→美国(Google App Maker主用户)→法兰克福(SQL实例)

We have tried many things including changing the instance location to Frankfurt, Germany (~25 ms). In this case, it took even longer (500ms per record). Our queries seemed to be round-tripping this complicated way: Our location (proxy) → US (Google App Maker master) → Frankfurt (SQL instance)

我们非常绝望，因为我们目前无法承受从Google App Maker迁移的风险.我坚信，无论现在还是将来，这种解决方案都可以解决.

We are quite desperate as we cannot afford to migrate away from Google App Maker at the moment. And I strongly believe this has some solution either now or in the future.

我的资源: db-f1-micro(vCPU:1)，内存:614.4 MB，存储容量:10 GB 数据库版本:MySQL 2nd Gen 5.7，实例区域:us-central1-a 连接性:公共IP，我的位置:捷克共和国布拉格

My resources: db-f1-micro (vCPUs: 1), Memory: 614.4 MB, Storage capacity: 10 GB Database version: MySQL 2nd Gen 5.7, Instance zone: us-central1-a Connectivity: Public IP, My location: Prague, Czech Republic

function serverSaveData() {  
  var recordData = [];
  recordData[0] = ['NewField_String']
  recordData[1] = ['NewField1_String'];

  for (var i = 0 ; i < recordData.length; i ++) {
    var newProduct = app.models.testDB.newRecord();
    newProduct.NewField = recordData[i][0];
    newProduct.NewField1 = recordData[i][1];
    var time = new Date().toString();
    app.saveRecords([newProduct]);
    console.log('Product saved: ' + time);
  }

}

我们需要每条记录的最大速度为〜25ms(实际速度的十分之一).我们有10000种产品，导入时间不会超过几分钟.

We need the speed to be maximum ~25ms per record (one tenth of the actual speed). We have 10000 products and importing should not take more than several minutes.

请问您有什么解决方案，还是可以将记录批量保存到数据库中?非常感谢！

Would you guys please see any solution, or is there a way to save records to the database in bulk? Thank you so much!

推荐答案

App Maker起起伏伏，您只需要查找起伏的变通办法.我只是处理了一个庞大的数据库的导入，该数据库包含超过50万条记录，这是一个很大的挑战.

App Maker has ups and downs and you just need to find workarounds for the downs. I just dealt with the import of a huge database that contained over a half million records and it was quite a challenge.

对于您而言，我建议分批保存项目以加快流程.我认为批量保存2000条记录就足够了.

In your case, I'd recommend to save items in batches to speed up the process. I think that saving records in batches of 2000 should be enough.

您可以执行以下操作:

function saveData(){

    //get the data
    var data = getData();

    var totalRecords = data.length;
    var batchLimit = 2000;
    var totalRounds = Math.ceil(totalRecords / batchLimit);
    var round = 1;
    var roundlimit = batchLimit;
    var currentItem = 0;

    do{
        var recordsToSave = [];
        for(var i=currentItem; i<roundlimit; i++){
            var recordData = data[i];
            var newProduct = app.models.testDB.newRecord();
            newProduct.newField = recordData.newFielddData;
            newProduct.newFiled2 = recordData.newField2Data;
            recordsToSave.push(newProduct);
        }
        if(recordsToSave.length){
            app.saveRecords(recordsToSave);
        }
        currentItem += batchLimit;
        roundlimit += batchLimit;
        round++;

    } while(round <= totalRounds);

}

我使用了与上述类似的解决方案，以完成将680,000条以上的记录导入到appmaker中.它可能需要更多的调整，但是您明白了.至于连接的延迟，很抱歉，我帮不上忙.希望有一位Google App Maker工程师可以参加，但就我目前而言，这是您最好的选择.

I used a similar solution like the one above to complete the import of 680,000+ records to appmaker. It might need a little bit more of tweaking but you get the idea. As for the latency with the connection, sorry I can't help. Hopefully a Google App Maker engineer can pitch in but as far as I see now, this is your best shot.

这篇关于Google App Maker将记录保存到数据库的时间为每条记录240毫秒的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！