使用云代码一次在Parse Server上更新和保存数千个对象.代码不适用于2个以上的对象. (Parse.Object.saveAll()) [英] Updating and saving thousands of objects at once on Parse Server using cloud code. Code doesn't work for more than 2 objects. (Parse.Object.saveAll())
问题描述
在解析服务器中，我有一个名为Stats的类，其中包含列secondsPlayed(数字)和timeScore(数字)
In my parse server I have a class called Stats which contains the columns secondsPlayed (number) and timeScore (number)
我正在使用云代码更新列timeScore
I am using cloud code to update all the rows in the column timeScore
下面的代码仅在更新和保存1个或2个对象results.length时有效.如果Parse.Query返回的结果超过2个，则代码崩溃，并且出现以下错误.
The code below works only when updating and saving 1 or 2 objects results.length. If Parse.Query returns more than 2 results the code crashes and I get the following error.
error: Failed running cloud function timeScore for user undefined with:
Input: {}
Error: {"code":101,"message":"Object not found."} functionName=timeScore, code=101, message=Object not found., , user=undefined
error: Error generating response. ParseError { code: 101, message: 'Object not found.' } code=101, message=Object not found.
error: Object not found. code=101, message=Object not found.
这是个问题，因为我需要更新和保存数千个对象.最好和最快的方法是什么?
This is a problem as I need to update and save thousands of objects. What is the best and fastest way to do this?
为什么我的代码只能用于2个对象，但不能同时用于2个以上的对象?我怎样才能解决这个问题?
Why does my code work for 2 objects but not for more than 2? How can I fix this?
这是我的代码
var _ = require("underscore");
Parse.Cloud.define("timeScore", function(request, response) {
var query = new Parse.Query("Stats");
query.greaterThan("secondsPlayed", 1000);
query.find().then(function(results) {
_.each(results, function(result) {
var secondsPlayed = result.get("secondsPlayed") || 0;
result.set("timeScore", secondsPlayed*2);
});
return Parse.Object.saveAll(results);
}).then(function(results) {
response.success(results);
}, function(error) {
response.error(error);
}); });
这就是我的称呼
#!/usr/bin/env node
var Parse = require("parse/node");
Parse.initialize("xx", "xx");
Parse.serverURL = "http://randomapp.herokuapp.com/parse";
Parse.Cloud.run('timeScore');
更新:
下面是我的最新代码.一切正常，除了我没有明显原因而收到以下错误的事实.
Below is my latest code. Everything works well except for the fact that I get the following error for no apparent reason.
heroku[router]: at=error code=H12 desc="Request timeout" method=POST path="/parse/functions/timeScore
无论选择什么batchSize，我都会收到超时错误，并且每30秒就会收到一次.我一共得到了5次.第五次之后，我不再明白了.我在处理过程中大约2.5分钟(30秒* 5)出现了第5个错误，也是最后一个错误.此错误不会以任何方式影响该过程.不管batchSize如何，所有250k对象都将更新和保存.
I get the timeout error regardless of what batchSize I choose and I get it every 30 seconds. I get it a total of 5 times. After the 5th time I dont get it anymore. I get the 5th and last error at around 2.5 minutes into the process (30seconds*5). This error does not effect the process in any way. All 250k objects are updated and saved regardless of batchSize.
我想也许是因为我从未调用过results.error或results.success，服务器认为我仍在做一些工作并显示错误.但是，我按照以下方式更新了代码，但仍然收到超时错误.
I thought maybe because I never call results.error or results.success the server thinks I am still doing some work and shows the error. However I updated my code as the following and I still get the timeout errors.
还在每个超时错误之后，从开始处再次调用processBatch().由于我收到5个超时错误，processBatch()被调用了5次.因此，在第5次超时错误之后，有5个processBatch()函数同时运行(我通过日志确认了这一点).
Also after each timeout error processBatch() is called once again from the beggining. Since I get 5 timeout errors processBatch() gets called 5 times. So after the 5th timeout error there is 5 processBatch() functions running simultaneously (I confirmed this with logs).
是什么导致我收到的heroku超时错误?我该如何解决?
What is causing the heroku timeout errors I am getting? How do I fix it?
var _ = require("underscore");
Parse.Cloud.define("timeScore", function(request, response) {
var counter = 0;
function processBatch(query, batchSize, startingAt, process) {
query.limit(batchSize);
query.skip(startingAt);
return query.find().then(results => {
return process(results).then(() => results.length);
}).then(length => {
return (length === batchSize)? processBatch(query, batchSize, startingAt+length, process) : {};
});
}
function setTimeScores(stats) {
console.log("LENGTH " + stats.length);
_.each(stats, stat => {
counter ++;
stat.set("timeScore", counter);
});
return Parse.Object.saveAll(stats);
}
var query = new Parse.Query("Stats");
processBatch(query, 2500, 0, setTimeScores).then(results => {
response.success(results);
}).catch(error => {
response.error(error);
});
});
推荐答案
要处理大于最大查询限制的对象数，请构建一个更抽象的函数，该函数使用查询的limit()和skip()来游标数据:
To handle numbers of objects greater than the max query limit, build a more abstract function that uses query's limit() and skip() to cursor through the data:
function processBatch(query, batchSize, startingAt, process) {
query.limit(batchSize);
query.skip(startingAt);
return query.find().then(results => {
return process(results).then(() => results.length);
}).then(length => {
return (length === batchSize)? processBatch(query, batchSize, startingAt+length, process) : {};
});
}
这表示，得到一个由query指定的batchSize个长对象，然后对检索到的对象进行处理，然后，如果可能更多，请再次执行相同的操作，跳过我们已经处理过的对象已处理.
This says, get one, batchSize-long batch of objects specified by query, then do something with the retrieved objects, then, if there might be more, do the same thing again, skipping objects we've processed already.
您的处理步骤如下:
function setTimeScores(stats) {
_.each(stats, stat => {
var secondsPlayed = stat.get("secondsPlayed") || 0;
stat.set("timeScore", secondsPlayed*2);
});
return Parse.Object.saveAll(stats);
}
这样称呼:
let query = new Parse.Query("Stats");
query.greaterThan("secondsPlayed", 1000);
processBatch(query, 100, 0, setTimeScores);
EDIT 在云功能的上下文中，这样称呼它...
EDIT in the context of a cloud function, call it like this...
Parse.Cloud.define("computeStats", function(request, response) {
let query = new Parse.Query("Stats");
query.greaterThan("secondsPlayed", 1000);
processBatch(query, 100, 0, setTimeScores).then(results => {
response.success(results);
}).catch(error => {
response.error(error);
});
});
这篇关于使用云代码一次在Parse Server上更新和保存数千个对象.代码不适用于2个以上的对象. (Parse.Object.saveAll())的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
