Parse.com的现实数据备份方法 [英] Realistic Data Backup method for Parse.com
问题描述
我们正在使用Parse.com构建一个iOS应用，但仍无法找到有效备份数据的正确方法.
作为前提，我们将有很多数据存储行. 假设我们有一个包含一百万行的类，假设我们已经对其进行了备份，然后在发生危险情况(例如生产中的数据丢失)之后将其返回到Parse.
我们考虑的几种解决方案如下:
1)使用外部服务器进行备份
备份: -使用REST API不断将数据备份到远程MySQL服务器(我们选择使用MySQL作为自定义分析目的，因为使用MySQL可以更快，更轻松地为我们处理数据)
导入后: a)-从MySQL备份重新创建JSON对象，并使用REST API发送回Parse. 假设我们使用批处理操作，该操作允许通过1个查询创建50个并发对象，并且假设每个查询需要1秒，那么100万个数据集将需要5.5个小时才能传输到Parse.
b)-从MySQL备份重新创建一个JSON文件，并使用仪表板手动导入数据. 我们刚刚使用这种方法尝试处理了700,000条记录文件:加载指示器停止并在左窗格中显示行数大约花费了2个小时，但现在它从未在右窗格中打开(它说操作超时" )，距离上传开始已超过6个小时.
因此，我们不能依靠1.b，而1.a似乎需要太长时间才能从灾难中恢复(如果我们有1000万条记录，那将是55小时= 2.2天). >
现在我们正在考虑以下问题:
2)不断将数据复制到另一个应用程序
在Parse中创建以下内容: -生产应用:A -复制应用程序:B 因此，当A在生产中时，每个查询都将被复制到B(不断使用后台作业). 缺点当然是它会吃掉A的突发限制，因为它只会使查询量增加一倍.因此，并不是理想的扩大规模的想法.
我们想要的是类似AWS RDS的东西，它提供了每天自动备份的选项. 我想知道，因为它基于AWS infra，所以对于Parse来说很难做到这一点.
如果您对此有任何想法，请告诉我，我们将很乐意分享专门知识.
P.S.:
我们注意到上述2)想法中的一个重要缺陷.
如果我们使用REST API复制，所有类的所有objectIds都会更改，因此每个1to1或1toMany关系都会中断.
因此，我们考虑为每个对象类放置一个uuid.
此方法是否有问题? 我们要实现的一件事是 query.include("ObjectName") (或在Obj-C"includeKey"中)， 但我想如果我们不将应用程序逻辑基于objectId，那将是不可能的.
正在寻找解决此问题的方法； 但是基于uuid的管理是否可以在Parse的数据存储区逻辑下发挥作用?
我可以确认今天Parse确实丢失了我的数据.或者至少看起来是这样.
在多个应用程序上检测到多个错误(Parse Status Twitter帐户同意)后，我们无法检索应用程序的数据，没有任何错误.
这是因为我们的一个类(类型指针)的整个列都消失了，并且仪表板中不再存在数据.
我们正在使用此指针列来过滤/检索数据，因此返回的查询和集合为空.
因此，我们决定手动重新创建该列.一次偶然的机会，用相同的名称和类型重新创建了该列，从而解决了该问题，并且数据仍然存在.我无法解释，但我真的认为，应用程序的反应就像是丢失了数据.
因此，自动备份和还原选项是必需的，而不是必需的.
We are building an iOS app with Parse.com, but still can't figure out the right way to backup data efficiently.
As a premise, we have and will have a LOT of data store rows. Say we have a class with 1million rows, assume we have it backed up, then want to bring it back to Parse, after a hazardous situation (like data loss on production).
The few solutions we have considered are the following:
1) Use external server for backup
BackUp: - use the REST API to constantly back up data to a remote MySQL server (we chose MySQL for customized analytics purpose, since it's way faster and easier to handle data with MySQL for us)
ImportBack: a) - recreate JSON objects from MySQL backup and use the REST API to send back to Parse. Say we use the batch operation which permits 50 simultaneous objects to be created with 1 query, and assume it takes 1 sec for every query, 1million data sets will take 5.5hours to transfer to Parse.
b) - recreate one JSON file from MySQL backup and use the Dashboard to import data manually. We just tried with 700,000 records file with this method: it took about 2 hours for the loading indicator to stop and show the number of rows in the left pane, but now it never opens in the right pane (it says "operation time out") and it's over 6hours since the upload started.
So we can't rely on 1.b, and 1.a seems to take too long to recover from a disaster (if we have 10 million records, it'll be like 55 hours = 2.2 days).
Now we are thinking about the following:
2) Constantly replicate data to another app
Create the following in Parse: - Production App: A - Replication App: B So while A is in production, every single query will be duplicated to B (using background job constantly). The downside is of course that it'll eat up the burst limit of A as it'll simply double the amount of query. So not ideal thinking of scaling up.
What we want is something like AWS RDS which gives an option to automatically backup daily. I wonder how this could be difficult for Parse since it's based on AWS infra.
Please let me know if you have any idea on this, will be happy to share know-hows.
P.S.:
We’ve noticed an important flaw in the above 2) idea.
If we replicate using REST API, all the objectIds of all Classes will be changed, so every 1to1 or 1toMany relations will be broken.
So we think about putting a uuid for every object class.
Is there any problem about this method? One thing we want to achieve is query.include("ObjectName") ( or in Obj-C "includeKey"), but I suppose that won’t be possible if we don’t base our app logic on objectId.
Looking for a work around for this issue; but will uuid-based management be functional under Parse’s Datastore logic?
I can confirm that today, Parse did lost my data. Or at least it appeared to be so.
After several errors where detected on multiple apps (agreed by Parse Status twitter account), we could not retrieve data for an app, without any error.
It was because an entire column of one of our class (type pointer) disappeared and data was not present anymore in the dashboard.
We are using this pointer column to filter / retrieve data, so the returned queries and collections were empty.
So we decided to recreate the column manually. By chance, recreating the column, with the same name and type, solved the issue and the data was still there... I can't explain it but I really thought, and the app reacted as if, data were lost.
So an automated backup and restore option is mandatory, it is not an option.
这篇关于Parse.com的现实数据备份方法的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！
