处理App Engine中的模式迁移 [英] Handling Schema Migrations in App Engine

问题描述

我在其中一个App Engine应用程序中通过移动它们之间的一些属性来更新了几个重度使用的NDB模型。因此，现在有些模型包含的数据表明我的应用程序的以前版本会在不同的地方查找（并且不这样做）。我已经更新了我的处理程序，以确保在执行与我的发行版相关的迁移后不会发生这种情况。

然而，我担心模式正在迁移。我已经用几百个实体在本地测试了我的迁移，并且任务耗时约1秒（使用deferred）。由于我在生产上有1000多个实体，我想这个任务实际上需要几秒钟的时间。与此同时，我确信用户会遇到服务问题。

从其他问题（如），我收集了一个很好的做法是将用户重定向到一个页面，提醒他们计划停机维护。然而，这对我来说并不是一个真正的选择 - 我们需要尽可能保持正常运行时间。

因此，我的问题是：有没有办法让我的应用程序保持在线状态并仍然执行可能长时间的迁移？我在考虑使用App Engine的流量分割功能在新应用版本迁移时将用户保留在旧应用版本上，但这仍然会导致服务问题。

解决方案

您可以执行2次迁移（添加 + 删除），而不是单个（移动）。

第一次迁移添加要移入模型中新位置的属性。

更新读取属性的代码以首先检查新位置，如果属性不存在，则回退到旧位置。对于这两个位置查询可能需要加倍，并添加逻辑以识别和跳过重复结果。几乎是新旧代码的组合。

然后，您可以更新编写属性的代码以使用新位置。过去这个时刻创建的实体将不再拥有旧的位置。如果您想更安全地使用它（可以回滚到较旧的代码版本），或者如果您想让较旧的应用程序版本运行一段时间，则可以使编写代码在旧位置和新位置都可以写入。

然后运行一次性工作，将旧位置（如果存在）的属性内容复制到新位置。这确保所有属性都存在于所有实体的新位置。

此时，您可以删除访问旧位置的所有代码。

这是不归之处。在下一步之前，您需要停止运行旧版应用程序代码的任何GAE实例 - 它们将无法正常运行，并且无法在没有其他数据库更新作业的情况下回滚代码。

然后运行一次性清理工作，从旧地点为所有实体移除属性 - 它们不再被最新版本的代码访问。

使用第二次迁移完成从模型中旧位置删除的属性。

它可能但是它应该可以实现零停机时间（针对最新的应用程序版本），而不考虑迁移的持续时间。

更新：

更好的迁移算法（避免了 OR 'd doubled queries）的问题是描述在如何重命名数据存储实体字段，但能够通过旧的和新的属性名称检索记录？

I've updated several of the heavily-used NDB models in one of my App Engine applications by moving a few properties between them. As such, some models now contain data that previous versions of my application would look to find in different places (and fail in doing so). I have updated my handlers to make sure that this will not happen after I run a migration that will accompany my release.

However, I am concerned about what will happen while the schema is migrating. I've tested my migration locally with a few hundred entities, and the task took about 1 second (using deferred's). As I have well over 1000 entities on production, I imagine that this task will actually take a few seconds. In the meantime, I'm sure that users will experience service problems.

From other questions (such as this one), I've gathered that a good practice is to redirect users to a page that alerts them of planned downtime for maintenance. However, this isn't really an option for me -- we need to keep our uptime as high as possible.

Thus, my question is this: is there a way to keep my application online and still execute a potentially-lengthy migration? I was thinking about using App Engine's "Traffic Splitting" to keep users on the old app version while the new app version migrates, but this will still cause a service problem.

解决方案

You could do 2 migrations (add + delete) instead of a single (move) one.

A first migration just adds the properties to be moved into their new locations in the models.

Update the code reading the properties to check the new locations first and, if the properties don't exist, fallback to the old locations. Queries will likely need to be doubled for the 2 locations and with logic added to identify and skip the duplicate results. Pretty much a combination of the new and the old code.

Then you can update the code writing the properties to use the new locations. Entities created past this moment won't have the old locations anymore. If you want to play it even safer (able to rollback to older code versions) or if you want to keep your older app versions running for a little while longer you can make the writing code write in both old and new locations.

Then run a one-time job copying the attribute contents from the old locations (if they exist) to the new locations. This ensures all properties are present in the new locations for all entities.

At this moment you can drop all code accessing the old locations.

This is the point of no return. Before the next step you need to stop any GAE instance running your older app code - they won't run properly anymore and you won't be able to rollback your code without another DB-updating job.

Then run a one-time cleanup job removing the properties from the old locations for all entities - they're no longer accessed by the lastest version of the code.

Finalize with a 2nd migration deleting the properties from the old location in the models.

It might be a lot of work, but it should be possible to achieve zero downtime (for the latest app version) irrespective of the duration of the migrations.

Update:

A potentially even better migration algorithm (avoids problematic OR'd doubled queries) is described in How to rename a Datastore entity field but be able to retrieve records via old and new property names?

这篇关于处理App Engine中的模式迁移的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！