Memcached一致性哈希无法与4个服务器中的3个一起运行 [英] Memcached consistent hashing not working with 3 of 4 servers down

问题描述

我有3个Memcached服务器在运行，我在其中一个或另一个关闭了服务器，以研究PHP-memcached在服务器无法访问时的行为.

I have 3 memcached servers running where I shutdown the one or the other to investigate how PHP-memcached behaves upon a server not beeing reachable.

我在PHP中定义了4台服务器，其中1台用于模拟大部分脱机的服务器(备用服务器).当我关闭1台服务器(=> 2仍在线)时，第三个->get()给我结果.

I have defined 4 servers in PHP, 1 to simulate a server that is mostly offline (spare server). When I shutdown 1 server (=> 2 are still online), the third ->get() gives me a result.

当我关闭另一台服务器(=> 1仍处于联机状态)时，它将找不到推送到最后一台服务器的对象.

When I shutdown one more server (=> 1 is still online), it won't find objects pushed to that last server.

首次运行，共4台服务器中的3台:

Entity not found in cache on 1st try: NOT FOUND
Entity not found in cache on 2nd try: NOT FOUND
Entity not found in cache on 3rd try: NOT FOUND
Entity not found in cache on 4th try: NOT FOUND

第二次运行，最多4台服务器中的3台:

Entity found in Cache: SUCCESS

第三次运行，共有4台服务器中的2台:

Entity not found in cache on 1st try: CONNECTION FAILURE
Entity not found in cache on 2nd try: SERVER IS MARKED DEAD
Entity not found in cache on 3rd try: NOT FOUND
Entity not found in cache on 4th try: NOT FOUND

第四次运行，共4台服务器中的1台:

Entity not found in cache on 1st try: CONNECTION FAILURE
Entity not found in cache on 2nd try: SERVER IS MARKED DEAD
Entity not found in cache on 3rd try: CONNECTION FAILURE
Entity not found in cache on 4th try: SERVER IS MARKED DEAD

尽管有一台服务器处于联机状态，并且每次在缓存中找不到任何对象时，我都会将其推送到内存缓存中，但是它不再能够找到密钥.

Although there is one server left online and I do push my object to memcached everytime it does not find any in cache, it is not able to find the key anymore.

我认为它也应该只支持一台服务器.

I think it should also work with only a single server left.

您能向我解释这种行为吗?

即使关闭20台服务器中的19台，似乎也无法实现安全的操作.

It looks like it is not possible to implement something that is safe even when I shutdown 19 of 20 servers.

侧面问题:libketama不再真正维护了，使用它仍然很好吗? lib背后的逻辑相当不错，并且还用于清漆缓存服务器中.

Sidequestion: libketama is not really maintained anymore, is it still good to use it? The logic behind the lib was rather good and is also used in the varnish caching server.

我的脚本:

<?php
require_once 'CachableEntity.php';
require_once 'TestEntity.php';

echo PHP_EOL;

$cache = new Memcached();
$cache->setOption(Memcached::OPT_LIBKETAMA_COMPATIBLE, true);
$cache->setOption(Memcached::OPT_DISTRIBUTION, Memcached::DISTRIBUTION_CONSISTENT);
$cache->setOption(Memcached::OPT_SERVER_FAILURE_LIMIT, 1);
$cache->setOption(Memcached::OPT_REMOVE_FAILED_SERVERS, true);
$cache->setOption(Memcached::OPT_AUTO_EJECT_HOSTS, true);

$cache->setOption(Memcached::OPT_TCP_NODELAY, true);
//$cache->setOption(Memcached::OPT_RETRY_TIMEOUT, 10);

$cache->addServers([
    ['localhost', '11212'],
    ['localhost', '11213'],
    ['localhost', '11214'],
    ['localhost', '11215'], // always offline
]);


$entityId = '/test/test/article_123456789.test';

$entity = new TestEntity($entityId);

$found = false;

$cacheKey = $entity->getCacheKey();

$cacheResult = $cache->get($cacheKey);
if (empty($cacheResult)) {
    echo 'Entity not found in cache on 1st try: ' . $cache->getResultMessage(), PHP_EOL;

    $cacheResult = $cache->get($cacheKey);
    if (empty($cacheResult)) {
        echo 'Entity not found in cache on 2nd try: ' . $cache->getResultMessage(), PHP_EOL;

        $cacheResult = $cache->get($cacheKey);
        if (empty($cacheResult)) {
            echo 'Entity not found in cache on 3rd try: ' . $cache->getResultMessage(), PHP_EOL;

            $cacheResult = $cache->get($cacheKey);
            if (empty($cacheResult)) {
                echo 'Entity not found in cache on 4th try: ' . $cache->getResultMessage(), PHP_EOL;

                $entity
                    ->setTitle('TEST')
                    ->setText('Hellow w0rld. Lorem Orem Rem Em M IpsuM')
                    ->setUrl('http://www.google.com/content-123456789.html');

                $cache->set($cacheKey, $entity->serialize(), 120);
            }
        }
        else { $found = true; }
    }
    else { $found = true; }
}
else { $found = true; }


if ($found === true) {
    echo 'Entity found in Cache: ' . $cache->getResultMessage(), PHP_EOL;
    $entity->unserialize($cacheResult);
    echo 'Title: ' . $entity->getTitle(), PHP_EOL;
}

echo PHP_EOL;

推荐答案

• 您遇到的行为是一致的.当服务器不可用时，首先将其标记为故障，然后标记为无效.

问题是，显然，只有将Memcached::OPT_SERVER_FAILURE_LIMIT值设置为2而又将它设置为1时，它才是连贯的.这可以解释为什么每个不可达的服务器都有两个错误行(CONNECTION FAILURE，SERVER IS MARKED AS DEAD)

The problem is that apparently it would only be coherent if you had set Memcached::OPT_SERVER_FAILURE_LIMIT value to 2 while you have set this to 1. This would have explained why you have two error lines per unreachable server (CONNECTION FAILURE, SERVER IS MARKED AS DEAD)

这似乎与超时有关.在失败后添加具有匹配的OPT_RETRY_TIMEOUT值的usleep()将使该服务器能够从列表中删除(请参见

This seems to be related with timeout. Adding a usleep() after a failure with a matching OPT_RETRY_TIMEOUT value will enable the server to be dropped from the list (see the following bug comment)

• 该值不会复制到下一个服务器，因为仅分配了密钥.

• The value does not replicate to the next server because only keys are distributed.

请注意，OPT_LIBKETAMA_COMPATIBLE不使用libketama，而仅重现相同的算法，这意味着libketama不再处于活动状态并不重要，这是建议的配置

Note that OPT_LIBKETAMA_COMPATIBLE does not use libketama, but only reproduces the same algorithm, which means that it does not matter if libketama is no longer active while this is the recommended configuration in PHP documentation:

如果您要使用一致的哈希，强烈建议启用此选项，并且在将来的版本中可能默认启用此选项.

It is highly recommended to enable this option if you want to use consistent hashing, and it may be enabled by default in future releases.

在我对您的帖子的理解中，消息在缓存中找到实体:成功"仅在第二次运行(1个服务器处于脱机状态)上出现，因为与上一个命令没有任何更改，并且托管此密钥的服务器仍然可用(因此memcached考虑从值存储在第一，第二或第三服务器上的键).我们称这些服务器为John，George，Ringo和Paul.

In my understanding of your post, the message "Entity found in Cache: SUCCESS" only appears on the second run (1 server offline) because there's no change from the previous command and the server hosting this key is still available (so memcached consider from the key that the value is stored on either the 1st, 2nd or 3rd server). Let's call those servers John, George, Ringo and Paul.

在第三次运行中，从头开始，memcached从密钥中推论出四个服务器之一拥有该值(例如John).在放弃之前，它问了约翰两次，因为它已经关闭了.然后，其算法仅考虑3个服务器(不知道Paul已经死了)，并推论George应该包含该值.

In the third run, at start, memcached deduces from the key which one of the four servers owns the value (e.g. John). It asks John twice before giving up because it's now off. Its algorithm then only considers 3 servers (not knowing that Paul is already dead) and deduces that George should contain the value.

乔治回答两次，它不包含该值，然后将其存储.

George answers twice that it does not contain the value and then store it.

但是在第四轮中，约翰，乔治和保罗离开了. Memcached尝试两次John，然后两次尝试George.然后将其存储在Ringo中.

But on the fourth run, John, George and Paul are off. Memcached tries John twice, and then tries George twice. It then stores in Ringo.

这里的问题是不可用的服务器不会存储在不同的运行之间，并且在同一运行中，您必须询问一台服务器两次，然后再将其删除.

The problem here is that the unavailable servers are not memorized between different runs, and that within the same run you have to ask twice a server before it's removed.

这篇关于Memcached一致性哈希无法与4个服务器中的3个一起运行的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！