Frequent RedisConnectionExecption errors popping up after a certain uptime in Docker containers (Swarm)

[Hedy88]

Hi, we've been dealing with tons of these errors randomly popping up after 24+ hours of container uptime. I don't know if this is an issue on our end as everything we use with Valkey / Redis with other clients on the same docker network configuration and manages to hold up well without issues for our users.

Error example:

[13:25:27 ERR] An unhandled exception has occurred while executing the request.StackExchange.Redis.RedisConnectionException: No connection (requires writable - not eligible for replica) is active/available to service this operation: SET user:id:<REDACTED>:lock, mc: 1/1/0, mgr: 10 of 10 available, clientName: b3-backend, IOCP: (Busy=0,Free=1000,Min=1,Max=1000), WORKER: (Busy=2,Free=32765,Min=3,Max=32767), POOL: (Threads=4,QueuedItems=0,CompletedItems=161269,Timers=6), v: 2.9.32.54708

Connection setup

builder.Services.AddSingleton<IConnectionMultiplexer>(sp =>
{
    var config = builder.Configuration.GetConnectionString("Valkey")
                 ?? throw new ArgumentNullException("Valkey connection string");

    var options = ConfigurationOptions.Parse(config);
    
    options.AbortOnConnectFail = false;
    options.ConnectRetry = 3;
    options.ReconnectRetryPolicy = new ExponentialRetry(5000);
    options.ClientName = "b3-backend";
    options.KeepAlive = 30;
    options.AllowAdmin = true;

    return ConnectionMultiplexer.Connect(options);
});