如何设计一个可以有效地动态管理配置的系统？ [英] How to design a system that can manage configurations in a dynamic way efficiently?

问题描述

我正在设计一个系统，该系统需要跨一堆应用程序服务器以动态方式管理配置（配置文件）。我正在与Consul密钥值存储一起管理配置。

I am working on designing a system where I need to manage configuration (config files) in a dynamic way across bunch of application servers. I am working with Consul key value store to manage configurations.

我在consul kv存储中的节点下面创建了一个用于管理配置的

I created below node in consul kv store for the purpose of managing configurations.

{"remoteConfig":"abc-123.tgz", "...."}

此处 remoteConfig 包含所有应用程序服务器将使用的配置文件（至少这是我得到的设计）。

Here remoteConfig contains the config file that all the app servers will use (atleast this is the design I got).

以下是我要执行的操作：

Below is what I am trying to do:

• 所有应用服务器都会在Consul的上面的节点上监视，并且只要 remoteConfig 密钥更改，将通知他们，然后他们将下载此配置并将其存储在磁盘上。


• 现在，一旦所有应用程序服务器集群中的所有成员均已下载了新配置，然后仅。我们应该切换为在集群中所有框内的内存中使用新配​​置。如果很少有应用服务器无法下载，那么我们不应在成功的其余框中切换到使用最新的配置。

• All the app servers keep a watch on above node in Consul and as soon as value of remoteConfig key changes, they will be notified and then they will download this config and store it on disk.
• Now once all the app servers in the cluster have downloaded the new config then only we should switch to use new configs in memory across all the boxes in the cluster. If few app servers failed to download then we should not switch to use latest configs in remaining boxes where it was successful.

我能够做的第一点很容易，但是我对如何有效地设计第二点感到困惑，这仅在所有应用服务器都下载了特定配置后才能帮助我切换到最新配置。我确实知道如何通过在Consul中获取和释放锁来原子地更新节点，但困惑在于如何有效地设计节点以轻松处理这些情况。

I am able to do first point easily but I am confuse on how to design my second point efficiently which can help me to switch to latest configs only when all the app servers have downloaded that particular config. I do know on how to atomically update a node by acquiring and releasing lock in Consul but confusion is on how to design it efficiently to handle these cases easily.

问题：

• 如何设计节点，以便更容易看到所有机器都已成功下载此特定配置？现在是时候在所有选项卡上切换到最新的配置。


• 如果某些计算机无法下载特定配置，则从读取中可以清楚地看出该应用服务器无法下载也许它也可以显示时间戳，例如此应用程序服务器在此时间戳下下载了此配置，并且他们在此时间戳下切换到了新配置。

我不必须保留每台计算机的所有配置状态的历史记录，仅使用最新的就足够了。以上设计中还欢迎进行任何其他改进，以动态方式管理配置。

I don’t have to keep history for all the configs status for each machine, just the latest one will be sufficient. Any other improvements are also welcome in above design to manage the configuration in a dynamic way.

（注意：我也可以有很多其他节点（例如状态节点）只是为了方便地执行此操作。除了Consul，我们还可以使用Zookeeper，也可以在两种技术中使用bcoz锁定/领导者的东西，但现在我要坚持使用Consul）

(Note: I can have bunch of other nodes as well (like status node) to do this exercise just fyi. Also instead of Consul, we can use Zookeeper also bcoz lock/leader stuff can be done in both the technologies but for now I am gonna stick to Consul)

推荐答案

我无法回答您的问题，但是我担心如果您找到一种实现既定目标的方法，可能会导致潜在的竞赛情况。

I can't answer your question, but I am concerned about a potential race condition that might occur if you find a way to achieve your stated goal.

p>

假设您有5台服务器，并且所有服务器都使用版本1的配置文件。然后，要求服务器下载配置文件的版本2。当所有5台服务器都完成此操作后，您（以某种方式）向所有5台服务器发送信号，以告知它们从配置文件的版本1切换到版本2。这是可能发生竞争状况的地方。不能保证在5台服务器中的每台服务器上都在同一时间点从配置文件的版本1切换到版本2。因此，在短时间内（可能仅几毫秒），某些服务器仍将使用版本1，而其他服务器将使用版本2。在这短暂的时间内，您的服务器上的配置将不一致。

Let's assume you have 5 servers and all are using version 1 of the configuration files. Then the servers are asked to download version 2 of the configuration files. When all 5 servers have done that, you (somehow) send a signal to all 5 servers to tell them to switch from version 1 to version 2 of the configuration files. This is where the race condition can occur. Switching from version 1 to version 2 of the configuration file is not guaranteed to occur at the same point in time in each of the 5 servers. Thus, for a brief period of time (perhaps just a few milliseconds) some servers will still be using version 1 while other servers will be using version 2. During that brief period of time, you will have inconsistent configuration on your servers.

如果短暂的不一致会给您带来麻烦，那么我认为您将需要一个不同的从配置的版本1切换到版本2。该机制实质上可以归结为：（1）请求所有服务器进程终止； （2）等待它们全部终止，然后（3）使用配置的版本2重新启动它们。显然，这种方法需要在较短的时间内不运行服务器，这是不理想的，但至少可以避免竞争。

If that brief inconsistency can cause problems for you, then I think you will need a different "switch from version 1 to version 2 of configuration" mechanism, which in essence boils down to: (1) ask all the server processes to terminate; (2) wait for all of them to terminate, and (3) restart them with version 2 of the configuration. Obviously, this approach necessitates a brief period during which servers are not running, which is not ideal, but at least it avoids the race condition.

这篇关于如何设计一个可以有效地动态管理配置的系统？的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！