如何将损坏的数据上传到生产的Google App Engine服务器？ [英] How to finish a broken data upload to the production Google App Engine server?

问题描述

我是上传数据到App Engine （不是开发服务器），通过loader类和远程API，并且我在CSV文件的中间点击了配额。根据日志和进度sqllite db，我怎样才能选择剩余部分数据上传？

I was uploading the data to App Engine (not dev server) through loader class and remote api, and I hit the quota in the middle of a CSV file. Based on logs and progress sqllite db, how can I select remaining portion of data to be uploaded?

通过数十条记录来确定哪些数据是未被传输的，是没有吸引力的任务，所以我寻找一些方法来限制我需要检查的记录数量。

Going through tens of records to determine which was and which was not transfered, is not appealing task, so I look for some way to limit the number of record I need to check.

这里是相关的（IMO）日志部分，如何解释工作项目数字？

Here's relevant (IMO) log portion, how to interpret work item numbers?

[DEBUG    2010-03-30 03:22:51,757 bulkloader.py] [Thread-2] [1041-1050] Transferred 10 entities in 3.9 seconds
[DEBUG    2010-03-30 03:22:51,757 adaptive_thread_pool.py] [Thread-2] Got work item [1071-1080]
<cut>
[DEBUG    2010-03-30 03:23:09,194 bulkloader.py] [Thread-1] [1141-1150] Transferred 10 entities in 4.6 seconds
[DEBUG    2010-03-30 03:23:09,194 adaptive_thread_pool.py] [Thread-1] Got work item [1161-1170]
<cut>
[DEBUG    2010-03-30 03:23:09,226 bulkloader.py] [Thread-3] [1151-1160] Transferred 10 entities in 4.2 seconds
[DEBUG    2010-03-30 03:23:09,226 adaptive_thread_pool.py] [Thread-3] Got work item [1171-1180]
[ERROR    2010-03-30 03:23:10,174 bulkloader.py] Retrying on non-fatal HTTP error: 503 Service Unavailable

推荐答案

您可以 resume>上传中断：

You can resume a broken upload:

如果转账中断，您的
可以使用--db_filename = ...
参数从
中止的地方恢复转账。该值是工具创建的
进度文件的名称，
可以是您在
开始传输时提供
的名称和--db_filename参数，或者包含时间戳的默认
名称。这
假定你已经安装了sqlite3，
并且没有用--db_filename = skip禁用进程文件
。

If the transfer is interrupted, you can resume the transfer from where it left off using the --db_filename=... argument. The value is the name of the progress file created by the tool, which is either a name you provided with the --db_filename argument when you started the transfer, or a default name that includes a timestamp. This assumes you have sqlite3 installed, and did not disable the progress file with --db_filename=skip.

这篇关于如何将损坏的数据上传到生产的Google App Engine服务器？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！