redis服务器重启/扩展后如何重新连接redis客户端 [英] How to reconnect redis client after redis server reboot/scale

问题描述

我有一个使用 Redis 作为缓存内存的 azure 应用服务(基于 Docker).当我重新启动/缩放 redis 服务器时，azure 应用服务中的 redis 客户端与服务器失去连接并抛出以下异常:

<块引用>

超时等待响应(outbound=0KiB, inbound=0KiB, 2852ms elapsed, timeout 2000ms), command=SETEX, next: GET test, inst: 0, qu: 0, qs: 45, aw: False, rs:ReadAsync, ws: Idle, in: 0, serverEndpoint: Unspecified/redis-server-com:6380, mgr: 10 of 10 available, clientName: wallet-api, IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=1,Free=32766,Min=4,Max=32767), v: 2.0.601.3402(请查看这篇文章，了解一些可能导致超时的常见客户端问题:

就在再次建立连接之前，我得到了以下两个异常，我猜每个连接一个:

<块引用>

SocketFailure on redis-server.com:6380/Interactive, Idle/Faulted, last: GET, origin: ReadFromPipe,outstanding: 52, last-read: 982s ago, last-write: 6s ago, unanswered-write: 938s前，保持活动状态:60 秒，状态:已连接已建立，管理器:10 个中的 9 个可用，在:0，最后一次心跳:0 秒前，最后一次心跳:0 秒前，全局:0 秒前，v:2.0.601.3402 <--- 无法从传输连接读取数据:连接超时.<--- 连接超时

<块引用>

SocketFailure on redis-server.com:6380/Subscription, Idle/Faulted, last: PING, origin: ReadFromPipe,outstanding: 16, last-read: 998s ago, last-write: 36s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 9 of 10 available, in: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.0.601.3402 <--- 无法从中读取数据传输连接:连接超时.<--- 连接超时

为什么连接没有刷新?有什么办法可以改善重连吗?15 分钟对于生产环境来说太长了.

更新 03/09/2020.我做了一个快速测试，使用相同的客户端重新启动 redis 服务器，但使用通过 SSL(端口 6380)和普通连接(端口 6379)的安全连接.使用普通连接检查 netstat (netstat -ptona)，redis 客户端重新连接成功.但是在启用 SSL 的情况下再次检查，连接保持建立但 redis 服务器没有响应.

可能的解决方法:它看起来与框架有关.正如@Json Pan 在他的回复中所建议的，我会尝试升级到 netcore 3.1 并强制应用程序定期刷新连接.

解决方案

UPDATE

读完这篇博客，我修改了源代码，将项目从.net core 1.0升级到3.1.

我建议你可以在你的项目中尝试或修改它，以测试重新连接的时间.

您可以下载我的示例代码.

隐私

我建议您使用使用懒惰模式重新连接.

ConnectionMultiplexer 如何处理断开连接?对你有用.

I have an azure app service (based on Docker) that uses Redis as a cache memory. When I reboot/scale redis server, redis client inside azure app service lose connection with server and throws the following exception:

Timeout awaiting response (outbound=0KiB, inbound=0KiB, 2852ms elapsed, timeout is 2000ms), command=SETEX, next: GET test, inst: 0, qu: 0, qs: 45, aw: False, rs: ReadAsync, ws: Idle, in: 0, serverEndpoint: Unspecified/redis-server-com:6380, mgr: 10 of 10 available, clientName: wallet-api, IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=1,Free=32766,Min=4,Max=32767), v: 2.0.601.3402 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts)

It takes up to 15min to reconnect with redis server from azure app service, however if I restart app service as soon as app is up, redis client connection is established successfully. From documentation, ConnectionMultiplexor object should manage reconnection, but it does not look like he is doing his job.

Here the redis client code:

public class RedisStore : IRedisStore, IDisposable
{

    private readonly ConfigurationOptions _options;
    private static IConnectionMultiplexer _connection;

    public RedisStore(RedisConfiguration redisConfiguration)
    {
        _options = ConfigurationOptions.Parse(redisConfiguration.ConnectionString);
        _options.ReconnectRetryPolicy = new ExponentialRetry(redisConfiguration.RetryFromMilliSeconds);
    }

    async Task IRedisStore.InitializeConnection()
    {
        if (_connection == null)
        {
            _connection = await ConnectionMultiplexer.ConnectAsync(_options);
        }
    }

    async Task<T> IRedisStore.SetGet<T>(string key)
    {
        var value = await _connection.GetDatabase().StringGetAsync(key);

        if (value.IsNull)
            return default(T);

        return JsonConvert.DeserializeObject<T>(value);
    }

    async Task IRedisStore.SetStore<T>(string key, T value)
    {
        var serialized = JsonConvert.SerializeObject(value);
        await _connection.GetDatabase().StringSetAsync(key, serialized);
    }

    void IDisposable.Dispose()
    {
        _connection.Dispose();
    }
}

The redis connection is initialized from bootstrap code:

private async Task InitializeRedis()
    {
        var redis = Container.GetInstance<IRedisStore>();
        await redis.InitializeConnection();
    }

Also, while app service is throwing redis timeout exceptions, netstat displayed that redis connection is established:

Just before to establish connection again, I got the following 2 exceptions, I guess one for each connection:

SocketFailure on redis-server.com:6380/Interactive, Idle/Faulted, last: GET, origin: ReadFromPipe, outstanding: 52, last-read: 982s ago, last-write: 6s ago, unanswered-write: 938s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 9 of 10 available, in: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.0.601.3402 <--- Unable to read data from the transport connection: Connection timed out. <--- Connection timed out

SocketFailure on redis-server.com:6380/Subscription, Idle/Faulted, last: PING, origin: ReadFromPipe, outstanding: 16, last-read: 998s ago, last-write: 36s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 9 of 10 available, in: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.0.601.3402 <--- Unable to read data from the transport connection: Connection timed out. <--- Connection timed out

Why connection is not refreshed? Is there any way to improve reconnection? 15min is too much for a production environment.

UPDATE 03/09/2020. I did a quick test rebooting redis server with same client but using a secured connection via SSL (port 6380) and a plain connection (port 6379). Checking netstat (netstat -ptona) with a plain connection, redis client reconnect successfully. However checking again with SSL enabled, connection keeps established but there is no response from redis server.

Possible workaround: It looks like something related to framework. As @Json Pan suggested in his reply, I will try upgrading to netcore 3.1 and force app to refresh connection periodically.

解决方案

UPDATE

After read this blog, I modify the source code, upgrade the project from .net core 1.0 to 3.1.

I suggest you can try it or modify it in your project, to test reconnect time.

You can download my sample code.

PRIVIOUS

I recommand you use Reconnecting with Lazy pattern.

And the answer in How does ConnectionMultiplexer deal with disconnects?, will useful to you.

这篇关于redis服务器重启/扩展后如何重新连接redis客户端的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持IT屋！