Skip to content

ConnectionFailed isn't raised when single server node goes down #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
nayato opened this issue Mar 19, 2014 · 2 comments
Closed

ConnectionFailed isn't raised when single server node goes down #1

nayato opened this issue Mar 19, 2014 · 2 comments
Assignees
Labels

Comments

@nayato
Copy link

nayato commented Mar 19, 2014

StackExchange.Redis v1.0.187 (from nuget & from repo on March 18).
Started Redis locally, ran the following piece:

using (var c = await ConnectionMultiplexer.ConnectAsync("127.0.0.1"))
{
    c.ConnectionFailed += (s, e) => Console.Write("failed");
    c.ErrorMessage += (s, e) => Console.Write("error");
    c.ConnectionRestored += (s,e) => Console.Write("restored");
    var db = c.GetDatabase();
    while (true)
    {
        try
        {
            byte[] value = await db.StringGetAsync("123");
            Console.Write(".");
        }
        catch (Exception ex)
        {
            Console.Write("!");
        }
        await Task.Delay(TimeSpan.FromSeconds(1));
    }
}

When Redis gets stopped while running the code, event handlers for ConnectionFailed and ErrorMessage are not raised. I get RedisConnectionException on StringGetAsync call. When Redis is started again, ConnectionRestored event handler is called twice (I guess there's two connections being kept around per node?).

@mgravell
Copy link
Collaborator

Thanks, I will investigate; and yes, there are two connections per node - one interactive, one pub/sub (which is initiated early in case of configuration change broadcasts)

@mgravell
Copy link
Collaborator

Fixed

@mgravell mgravell self-assigned this Mar 19, 2014
@mgravell mgravell added the bug label Mar 19, 2014
mgravell pushed a commit that referenced this issue Sep 26, 2014
TimLovellSmith added a commit to TimLovellSmith/StackExchange.Redis that referenced this issue May 19, 2021
…be processed:

[xUnit.net 00:00:28.94]     PubSub.PubSubGetAllCorrectOrder_OnMessage_Sync [FAIL]
  Failed PubSub.PubSubGetAllCorrectOrder_OnMessage_Sync [20 s]
  Error Message:
   System.AggregateException : One or more errors occurred. (Items: 757) (There were 1 private and 0 ambient exceptions; expected 0.)
---- System.TimeoutException : Items: 757
---- StackExchange.Redis.Tests.SkipTestException : There were 1 private and 0 ambient exceptions; expected 0.
  Stack Trace:

----- Inner Stack Trace StackExchange#1 (System.TimeoutException) -----
   at StackExchange.Redis.Tests.PubSub.PubSubGetAllCorrectOrder_OnMessage_Sync() in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\PubSub.cs:line 452
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace StackExchange#2 (StackExchange.Redis.Tests.SkipTestException) -----
   at StackExchange.Redis.Tests.Skip.Inconclusive(String message) in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\Helpers\Skip.cs:line 8
   at StackExchange.Redis.Tests.TestBase.Teardown() in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\TestBase.cs:line 190
   at StackExchange.Redis.Tests.TestBase.Dispose() in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\TestBase.cs:line 86
   at ReflectionAbstractionExtensions.DisposeTestClass(ITest test, Object testClass, IMessageBus messageBus, ExecutionTimer timer, CancellationTokenSource cancellationTokenSource) in C:\Dev\xunit\xunit\src\xunit.execution\Extensions\ReflectionAbstractionExtensions.cs:line 79
  Standard Output Messages:
 Running under .NET Core 3.1.12 (3.1.12)
   Running on: Microsoft Windows 10.0.19043
 05:30:32.722: 05:30:32.722: Connect took: 1ms
 05:30:32.724: Published: 99
 05:30:32.725: Published: 199
 05:30:32.725: Published: 299
 05:30:32.726: Received: 99
 05:30:32.726: Published: 399
 05:30:32.726: Received: 199
 05:30:32.727: Published: 499
 05:30:32.727: Published: 599
 05:30:32.727: Received: 299
 05:30:32.728: Published: 699
 05:30:32.728: Received: 399
 05:30:32.728: Published: 799
 05:30:32.728: Received: 499
 05:30:32.728: Published: 899
 05:30:32.729: Published: 999
 05:30:32.729: Send loop complete.
 05:30:32.729: Received: 599
 05:30:32.730: Received: 699
 05:30:52.735: 05:30:52.735: Service Counts: (Scheduler) Queue: 42219, Pool: 0
 05:30:52.735: 05:30:32.731: Connection failed (SocketFailure): 127.0.0.1:6379/Subscription: StackExchange.Redis.RedisConnectionException: SocketFailure (ReadSocketError/ConnectionReset, last-recv: 5111) on 127.0.0.1:6379/Subscription, Idle/Faulted, last: SUBSCRIBE, origin: ReadFromPipe, outstanding: 0, last-read: 0s ago, last-write: 0s ago, keep-alive: 3s, state: ConnectedEstablished, mgr: 8 of 10 available, in: 0, in-pipe: 5133, out-pipe: 0, last-heartbeat: never, last-mbeat: 0s ago, global: 0s ago, v: 2.2.22.52840
  ---> Pipelines.Sockets.Unofficial.ConnectionResetException: An existing connection was forcibly closed by the remote host.
  ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
    at Pipelines.Sockets.Unofficial.Internal.Throw.Socket(Int32 errorCode) in /_/src/Pipelines.Sockets.Unofficial/Internal/Throw.cs:line 59
    at Pipelines.Sockets.Unofficial.SocketAwaitableEventArgs.<GetResult>g__ThrowSocketException|10_0(SocketError e) in /_/src/Pipelines.Sockets.Unofficial/SocketAwaitableEventArgs.cs:line 87
    at Pipelines.Sockets.Unofficial.SocketAwaitableEventArgs.GetResult() in /_/src/Pipelines.Sockets.Unofficial/SocketAwaitableEventArgs.cs:line 82
    at Pipelines.Sockets.Unofficial.SocketConnection.DoReceiveAsync() in /_/src/Pipelines.Sockets.Unofficial/SocketConnection.Receive.cs:line 64
    --- End of inner exception stack trace ---
    at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
    at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
    at System.IO.Pipelines.Pipe.TryRead(ReadResult& result)
    at System.IO.Pipelines.Pipe.DefaultPipeReader.TryRead(ReadResult& result)
    at Pipelines.Sockets.Unofficial.SocketConnection.WrappedReader.TryRead(ReadResult& result) in /_/src/Pipelines.Sockets.Unofficial/SocketConnection.cs:line 404
    at StackExchange.Redis.PhysicalConnection.ReadFromPipe() in /_/src/StackExchange.Redis/PhysicalConnection.cs:line 1493
    --- End of inner exception stack trace ---

[xUnit.net 00:00:53.77]     PubSub.PubSubGetAllCorrectOrder_OnMessage_Async [FAIL]
  Failed PubSub.PubSubGetAllCorrectOrder_OnMessage_Async [20 s]
  Error Message:
   System.AggregateException : One or more errors occurred. (Items: 800) (There were 1 private and 0 ambient exceptions; expected 0.)
---- System.TimeoutException : Items: 800
---- StackExchange.Redis.Tests.SkipTestException : There were 1 private and 0 ambient exceptions; expected 0.
  Stack Trace:

----- Inner Stack Trace StackExchange#1 (System.TimeoutException) -----
   at StackExchange.Redis.Tests.PubSub.PubSubGetAllCorrectOrder_OnMessage_Async() in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\PubSub.cs:line 522
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace StackExchange#2 (StackExchange.Redis.Tests.SkipTestException) -----
   at StackExchange.Redis.Tests.Skip.Inconclusive(String message) in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\Helpers\Skip.cs:line 8
   at StackExchange.Redis.Tests.TestBase.Teardown() in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\TestBase.cs:line 190
   at StackExchange.Redis.Tests.TestBase.Dispose() in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\TestBase.cs:line 86
   at ReflectionAbstractionExtensions.DisposeTestClass(ITest test, Object testClass, IMessageBus messageBus, ExecutionTimer timer, CancellationTokenSource cancellationTokenSource) in C:\Dev\xunit\xunit\src\xunit.execution\Extensions\ReflectionAbstractionExtensions.cs:line 79
  Standard Output Messages:
 Running under .NET Core 3.1.12 (3.1.12)
   Running on: Microsoft Windows 10.0.19043
 05:30:57.567: 05:30:57.567: Connect took: 1ms
 05:30:57.569: Published: 99
 05:30:57.569: Published: 199
 05:30:57.569: Published: 299
 05:30:57.570: Received: 99
 05:30:57.570: Published: 399
 05:30:57.570: Published: 499
 05:30:57.570: Received: 199
 05:30:57.570: Published: 599
 05:30:57.570: Published: 699
 05:30:57.570: Published: 799
 05:30:57.571: Received: 299
 05:30:57.571: Published: 899
 05:30:57.571: Received: 399
 05:30:57.571: Published: 999
 05:30:57.571: Send loop complete.
 05:30:57.572: Received: 499
 05:30:57.573: Received: 599
 05:30:57.573: Received: 699
 05:30:57.574: Received: 799
 05:31:17.565: 05:31:17.565: Service Counts: (Scheduler) Queue: 44359, Pool: 0
 05:31:17.565: 05:30:57.575: Connection failed (SocketFailure): 127.0.0.1:6379/Subscription: StackExchange.Redis.RedisConnectionException: SocketFailure (ReadSocketError/ConnectionReset, last-recv: 4923) on 127.0.0.1:6379/Subscription, Idle/Faulted, last: SUBSCRIBE, origin: ReadFromPipe, outstanding: 0, last-read: 0s ago, last-write: 0s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 7 of 10 available, in: 0, in-pipe: 0, out-pipe: 0, last-heartbeat: never, last-mbeat: 0s ago, global: 0s ago, v: 2.2.22.52840
  ---> Pipelines.Sockets.Unofficial.ConnectionResetException: An existing connection was forcibly closed by the remote host.
  ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
    at Pipelines.Sockets.Unofficial.Internal.Throw.Socket(Int32 errorCode) in /_/src/Pipelines.Sockets.Unofficial/Internal/Throw.cs:line 59
    at Pipelines.Sockets.Unofficial.SocketAwaitableEventArgs.GetResult() in /_/src/Pipelines.Sockets.Unofficial/SocketAwaitableEventArgs.cs:line 74
    at Pipelines.Sockets.Unofficial.SocketConnection.DoReceiveAsync() in /_/src/Pipelines.Sockets.Unofficial/SocketConnection.Receive.cs:line 64
    --- End of inner exception stack trace ---
    at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
    at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
    at System.IO.Pipelines.Pipe.GetReadAsyncResult()
    at StackExchange.Redis.PhysicalConnection.ReadFromPipe()
    --- End of inner exception stack trace ---
TimLovellSmith added a commit to TimLovellSmith/StackExchange.Redis that referenced this issue May 19, 2021
…ubSub.PubSubGetAllCorrectOrder_OnMessage_Sync [FAIL]

  Failed PubSub.PubSubGetAllCorrectOrder_OnMessage_Sync [20 s]
  Error Message:
   System.AggregateException : One or more errors occurred. (Items: 632) (There were 1 private and 0 ambient exceptions; expected 0.)
---- System.TimeoutException : Items: 632
---- StackExchange.Redis.Tests.SkipTestException : There were 1 private and 0 ambient exceptions; expected 0.
  Stack Trace:

----- Inner Stack Trace StackExchange#1 (System.TimeoutException) -----
   at StackExchange.Redis.Tests.PubSub.PubSubGetAllCorrectOrder_OnMessage_Sync() in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\PubSub.cs:line 452
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace StackExchange#2 (StackExchange.Redis.Tests.SkipTestException) -----
   at StackExchange.Redis.Tests.Skip.Inconclusive(String message) in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\Helpers\Skip.cs:line 8
   at StackExchange.Redis.Tests.TestBase.Teardown() in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\TestBase.cs:line 190
   at StackExchange.Redis.Tests.TestBase.Dispose() in C:\repo\StackExchange.Redis\tests\StackExchange.Redis.Tests\TestBase.cs:line 86
   at ReflectionAbstractionExtensions.DisposeTestClass(ITest test, Object testClass, IMessageBus messageBus, ExecutionTimer timer, CancellationTokenSource cancellationTokenSource) in C:\Dev\xunit\xunit\src\xunit.execution\Extensions\ReflectionAbstractionExtensions.cs:line 79
  Standard Output Messages:
 Running under .NET Core 3.1.12 (3.1.12)
   Running on: Microsoft Windows 10.0.19043
 05:59:50.998: 05:59:50.998: Connect took: 2ms
 05:59:51.000: Published: 99
 05:59:51.000: Published: 199
 05:59:51.001: Published: 299
 05:59:51.001: Published: 399
 05:59:51.001: Received: 99
 05:59:51.002: Published: 499
 05:59:51.002: Received: 199
 05:59:51.002: Published: 599
 05:59:51.002: Received: 299
 05:59:51.002: Published: 699
 05:59:51.003: Received: 399
 05:59:51.003: Published: 799
 05:59:51.003: Received: 499
 05:59:51.003: Published: 899
 05:59:51.003: Received: 599
 05:59:51.003: Published: 999
 05:59:51.004: Send loop complete.
 06:00:11.015: 06:00:11.015: Service Counts: (Scheduler) Queue: 41597, Pool: 0
 06:00:11.015: 05:59:51.004: Connection failed (SocketFailure): 127.0.0.1:6379/Subscription: StackExchange.Redis.RedisConnectionException: SocketFailure (ReadSocketError/ConnectionReset, last-recv: 5890) on 127.0.0.1:6379/Subscription, Idle/Faulted, last: SUBSCRIBE, origin: ReadFromPipe, outstanding: 0, last-read: 0s ago, last-write: 0s ago, keep-alive: 3s, state: ConnectedEstablished, mgr: 9 of 10 available, in: 0, in-pipe: 0, out-pipe: 0, last-heartbeat: never, last-mbeat: 0s ago, global: 0s ago, v: 2.2.23.28554
  ---> Pipelines.Sockets.Unofficial.ConnectionResetException: An existing connection was forcibly closed by the remote host.
  ---> System.Net.Sockets.SocketException (10054): An existing connection was forcibly closed by the remote host.
    at Pipelines.Sockets.Unofficial.Internal.Throw.Socket(Int32 errorCode) in /_/src/Pipelines.Sockets.Unofficial/Internal/Throw.cs:line 59
    at Pipelines.Sockets.Unofficial.SocketAwaitableEventArgs.GetResult() in /_/src/Pipelines.Sockets.Unofficial/SocketAwaitableEventArgs.cs:line 74
    at Pipelines.Sockets.Unofficial.SocketConnection.DoReceiveAsync() in /_/src/Pipelines.Sockets.Unofficial/SocketConnection.Receive.cs:line 64
    --- End of inner exception stack trace ---
    at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
    at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
    at System.IO.Pipelines.Pipe.GetReadAsyncResult()
    at System.IO.Pipelines.Pipe.DefaultPipeReader.GetResult(Int16 token)
    at StackExchange.Redis.PhysicalConnection.ReadFromPipe()
    --- End of inner exception stack trace ---
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants