Skip to content

Node restart doesn't seem to work correctly #171

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
clintongormley opened this issue May 13, 2010 · 4 comments
Closed

Node restart doesn't seem to work correctly #171

clintongormley opened this issue May 13, 2010 · 4 comments

Comments

@clintongormley
Copy link
Contributor

Hiya

I tried starting two nodes on my local machine (not using the java service wrapper, because that way I can only start one node), then issued a restart request to one of the nodes.

I get this in the logs:

[09:27:54,697][INFO ][action.admin.cluster.node.restart] [Silver Scorpion] Restarting in [1s]
[09:27:55,699][INFO ][action.admin.cluster.node.restart] [Silver Scorpion] Initiating requested restart
[09:27:55,700][INFO ][node                     ] [Silver Scorpion] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-12T12:56:28}[3538]: Stopping ...
[09:27:55,706][INFO ][cluster.service          ] [Invisible Girl] Removed {[Silver Scorpion][fba92d81-ef41-44cf-bead-308919a12e2f][inet[getafix.traveljury.com/127.0.0.2:9302]],}
[09:27:55,735][INFO ][cluster.service          ] [Answer] Removed {[Silver Scorpion][fba92d81-ef41-44cf-bead-308919a12e2f][inet[getafix.traveljury.com/127.0.0.2:9302]],}
[09:27:55,873][INFO ][node                     ] [Silver Scorpion] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-12T12:56:28}[3538]: Stopped
[09:27:55,873][INFO ][node                     ] [Silver Scorpion] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-12T12:56:28}[3538]: Starting ...
[09:27:55,875][INFO ][transport                ] [Silver Scorpion] bound_address[inet[/0.0.0.0:9302]], publish_address[inet[getafix.traveljury.com/127.0.0.2:9302]]
[09:27:58,922][INFO ][cluster.service          ] [Invisible Girl] Added {[Silver Scorpion][b038b644-5038-426b-a0bf-beefe8ebec77][inet[getafix.traveljury.com/127.0.0.2:9302]],}
[09:27:58,954][WARN ][discovery.zen            ] [Silver Scorpion] Received a cluster state from [[Invisible Girl][c71ce4ef-446e-4217-b240-f31b5e88becd][inet[getafix.traveljury.com/127.0.0.2:9300]]] and not part of the cluster, should not happen
[09:27:58,955][INFO ][cluster.service          ] [Answer] Added {[Silver Scorpion][b038b644-5038-426b-a0bf-beefe8ebec77][inet[getafix.traveljury.com/127.0.0.2:9302]],}
[09:28:00,938][WARN ][discovery.zen            ] [Silver Scorpion] Received a cluster state from [[Invisible Girl][c71ce4ef-446e-4217-b240-f31b5e88becd][inet[getafix.traveljury.com/127.0.0.2:9300]]] and not part of the cluster, should not happen
[09:28:28,930][WARN ][discovery                ] [Silver Scorpion] Waited for 30s and no initial state was set by the discovery
[09:28:28,933][INFO ][discovery                ] [Silver Scorpion] iAnnounce/b038b644-5038-426b-a0bf-beefe8ebec77
[09:28:28,940][INFO ][http                     ] [Silver Scorpion] bound_address[inet[/0.0.0.0:9202]], publish_address[inet[getafix.traveljury.com/127.0.0.2:9202]]
[09:28:28,968][INFO ][jmx                      ] [Silver Scorpion] bound_address[service:jmx:rmi:///jndi/rmi://:9403/jmxrmi], publish_address[service:jmx:rmi:///jndi/rmi://127.0.0.2:9403/jmxrmi]
[09:28:28,972][INFO ][node                     ] [Silver Scorpion] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-12T12:56:28}[3538]: Started
@kimchy
Copy link
Member

kimchy commented May 13, 2010

I just pushed a fix, can you try it. I don't really understand how it changed its name though (the [Name] just after the logger component.... . I just tested and it works for me...

@clintongormley
Copy link
Contributor Author

OK, this throws a number of warnings, but appears to work:

[17:52:07,335][INFO ][action.admin.cluster.node.restart] [Terraxia] Restarting in [1s]
[17:52:07,335][INFO ][action.admin.cluster.node.restart] [Terraxia] Restarting in [1s]
[17:52:08,336][INFO ][action.admin.cluster.node.restart] [Terraxia] Initiating requested restart
[17:52:08,336][INFO ][node                     ] [Terraxia] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-13T15:40:14}[3337]: Stopping ...
[17:52:08,357][INFO ][discovery.zen            ] [Crimson Cowl] Master [[Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]]] left, reason [shut_down]
[17:52:08,361][INFO ][cluster.service          ] [Crimson Cowl] Master {New [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], Previous [Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]]}, Removed {[Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]],}
[17:52:08,436][INFO ][node                     ] [Terraxia] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-13T15:40:14}[3337]: Stopped
[17:52:08,436][INFO ][node                     ] [Terraxia] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-13T15:40:14}[3337]: Starting ...
[17:52:08,438][INFO ][transport                ] [Terraxia] bound_address[inet[/0.0.0.0:9300]], publish_address[inet[getafix.traveljury.com/127.0.0.2:9300]]
[17:52:08,336][INFO ][action.admin.cluster.node.restart] [Terraxia] Initiating requested restart
[17:52:08,336][INFO ][node                     ] [Terraxia] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-13T15:40:14}[3337]: Stopping ...
[17:52:08,357][INFO ][discovery.zen            ] [Crimson Cowl] Master [[Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]]] left, reason [shut_down]
[17:52:08,361][INFO ][cluster.service          ] [Crimson Cowl] Master {New [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], Previous [Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]]}, Removed {[Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]],}
[17:52:08,436][INFO ][node                     ] [Terraxia] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-13T15:40:14}[3337]: Stopped
[17:52:08,436][INFO ][node                     ] [Terraxia] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-13T15:40:14}[3337]: Starting ...
[17:52:08,438][INFO ][transport                ] [Terraxia] bound_address[inet[/0.0.0.0:9300]], publish_address[inet[getafix.traveljury.com/127.0.0.2:9300]]
[17:52:11,449][INFO ][cluster.service          ] [Crimson Cowl] Added {[Terraxia][61dbb9be-e707-438a-83f7-514924cbdb77][inet[getafix.traveljury.com/127.0.0.2:9300]],}
[17:52:11,472][INFO ][cluster.service          ] [Terraxia] Master {New [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], Previous [Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]]}, Removed {[Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]],}
[17:52:12,181][INFO ][discovery                ] [Terraxia] iAnnounce/61dbb9be-e707-438a-83f7-514924cbdb77
[17:52:11,449][INFO ][cluster.service          ] [Crimson Cowl] Added {[Terraxia][61dbb9be-e707-438a-83f7-514924cbdb77][inet[getafix.traveljury.com/127.0.0.2:9300]],}
[17:52:11,472][INFO ][cluster.service          ] [Terraxia] Master {New [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], Previous [Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]]}, Removed {[Terraxia][75afba3e-9ed6-4fe6-a730-ea43e55efda8][inet[getafix.traveljury.com/127.0.0.2:9300]],}
[17:52:12,181][INFO ][discovery                ] [Terraxia] iAnnounce/61dbb9be-e707-438a-83f7-514924cbdb77
[17:52:12,186][INFO ][http                     ] [Terraxia] bound_address[inet[/0.0.0.0:9200]], publish_address[inet[getafix.traveljury.com/127.0.0.2:9200]]
[17:52:12,197][INFO ][jmx                      ] [Terraxia] bound_address[service:jmx:rmi:///jndi/rmi://:9403/jmxrmi], publish_address[service:jmx:rmi:///jndi/rmi://127.0.0.2:9403/jmxrmi]
[17:52:12,201][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709
[17:52:12,201][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=ia_object_1273725818
[17:52:12,201][INFO ][node                     ] [Terraxia] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-13T15:40:14}[3337]: Started
[17:52:12,186][INFO ][http                     ] [Terraxia] bound_address[inet[/0.0.0.0:9200]], publish_address[inet[getafix.traveljury.com/127.0.0.2:9200]]
[17:52:12,197][INFO ][jmx                      ] [Terraxia] bound_address[service:jmx:rmi:///jndi/rmi://:9403/jmxrmi], publish_address[service:jmx:rmi:///jndi/rmi://127.0.0.2:9403/jmxrmi]
[17:52:12,201][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709
[17:52:12,201][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=ia_object_1273725818
[17:52:12,201][INFO ][node                     ] [Terraxia] {ElasticSearch/0.7.0-SNAPSHOT/2010-05-13T15:40:14}[3337]: Started
[17:52:18,405][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=0
[17:52:18,407][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=0,shardType=store
[17:52:18,408][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][0] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:18,465][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=1
[17:52:18,466][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=1,shardType=store
[17:52:18,468][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][1] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:18,405][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=0
[17:52:18,407][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=0,shardType=store
[17:52:18,408][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][0] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:18,465][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=1
[17:52:18,466][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=1,shardType=store
[17:52:18,468][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][1] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:19,330][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=2
[17:52:19,331][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=2,shardType=store
[17:52:19,332][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][2] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:19,345][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=3
[17:52:19,346][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=3,shardType=store
[17:52:19,348][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][3] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:19,388][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=4
[17:52:19,891][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=4,shardType=store
[17:52:19,896][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][4] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,120][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][0] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:19,330][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=2
[17:52:19,331][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=2,shardType=store
[17:52:19,332][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][2] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:19,345][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=3
[17:52:19,346][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=3,shardType=store
[17:52:19,348][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][3] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:19,388][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=4
[17:52:19,891][WARN ][jmx                      ] [Terraxia] Could not register object with name: {elasticsearch}:type=Terraxia [iAnnounce_61dbb9be-e707-438a-83f7-514924cbdb77],service=indices,index=iannounce_object_1273723709,subService=shards,shard=4,shardType=store
[17:52:19,896][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][4] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,120][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][0] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,197][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][1] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,257][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][2] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,559][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][3] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,571][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][4] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,197][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][1] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,257][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][2] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,559][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][3] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:20,571][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][4] Starting recovery from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]
[17:52:21,781][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][0] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[1.6s]
   Phase1: recovered [1] files with total size of [32], took [2ms]
   Phase2: recovered [725] transaction log operations, took [1.2s]
   Phase3: recovered [0] transaction log operations, took [368ms]
[17:52:21,828][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][1] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[1.6s]
   Phase1: recovered [1] files with total size of [32], took [60ms]
   Phase2: recovered [725] transaction log operations, took [1.1s]
   Phase3: recovered [0] transaction log operations, took [426ms]
[17:52:22,069][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][1] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[3.6s]
   Phase1: recovered [1] files with total size of [32], took [1ms]
   Phase2: recovered [1271] transaction log operations, took [2.7s]
   Phase3: recovered [0] transaction log operations, took [839ms]
[17:52:21,781][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][0] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[1.6s]
   Phase1: recovered [1] files with total size of [32], took [2ms]
   Phase2: recovered [725] transaction log operations, took [1.2s]
   Phase3: recovered [0] transaction log operations, took [368ms]
[17:52:21,828][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][1] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[1.6s]
   Phase1: recovered [1] files with total size of [32], took [60ms]
   Phase2: recovered [725] transaction log operations, took [1.1s]
   Phase3: recovered [0] transaction log operations, took [426ms]
[17:52:22,069][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][1] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[3.6s]
   Phase1: recovered [1] files with total size of [32], took [1ms]
   Phase2: recovered [1271] transaction log operations, took [2.7s]
   Phase3: recovered [0] transaction log operations, took [839ms]
[17:52:22,303][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][3] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[1.7s]
   Phase1: recovered [1] files with total size of [32], took [4ms]
   Phase2: recovered [726] transaction log operations, took [1.4s]
   Phase3: recovered [0] transaction log operations, took [216ms]
[17:52:22,360][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][2] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[2.1s]
   Phase1: recovered [1] files with total size of [32], took [150ms]
   Phase2: recovered [726] transaction log operations, took [1.4s]
   Phase3: recovered [0] transaction log operations, took [246ms]
[17:52:22,612][INFO ][cluster.service          ] [Terraxia] Got old cluster state [51<52] from source [zen-disco-receive(from [[Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]])], ignoring
[17:52:22,697][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][4] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[2.1s]
   Phase1: recovered [1] files with total size of [32], took [115ms]
   Phase2: recovered [725] transaction log operations, took [1.5s]
   Phase3: recovered [0] transaction log operations, took [367ms]
[17:52:22,812][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][2] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[3.4s]
   Phase1: recovered [1] files with total size of [32], took [2ms]
   Phase2: recovered [1285] transaction log operations, took [2.8s]
   Phase3: recovered [0] transaction log operations, took [574ms]
[17:52:22,864][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][3] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[3.5s]
   Phase1: recovered [1] files with total size of [32], took [37ms]
   Phase2: recovered [1294] transaction log operations, took [2.9s]
   Phase3: recovered [0] transaction log operations, took [475ms]
[17:52:22,865][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][4] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[2.9s]
   Phase1: recovered [1] files with total size of [32], took [244ms]
   Phase2: recovered [1280] transaction log operations, took [2.1s]
   Phase3: recovered [0] transaction log operations, took [507ms]
[17:52:22,871][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][0] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[4.4s]
   Phase1: recovered [1] files with total size of [32], took [97ms]
   Phase2: recovered [1336] transaction log operations, took [3.8s]
   Phase3: recovered [0] transaction log operations, took [457ms]
[17:52:22,303][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][3] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[1.7s]
   Phase1: recovered [1] files with total size of [32], took [4ms]
   Phase2: recovered [726] transaction log operations, took [1.4s]
   Phase3: recovered [0] transaction log operations, took [216ms]
[17:52:22,360][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][2] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[2.1s]
   Phase1: recovered [1] files with total size of [32], took [150ms]
   Phase2: recovered [726] transaction log operations, took [1.4s]
   Phase3: recovered [0] transaction log operations, took [246ms]
[17:52:22,612][INFO ][cluster.service          ] [Terraxia] Got old cluster state [51<52] from source [zen-disco-receive(from [[Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]]])], ignoring
[17:52:22,697][DEBUG][index.shard.recovery     ] [Terraxia][ia_object_1273725818][4] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[2.1s]
   Phase1: recovered [1] files with total size of [32], took [115ms]
   Phase2: recovered [725] transaction log operations, took [1.5s]
   Phase3: recovered [0] transaction log operations, took [367ms]
[17:52:22,812][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][2] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[3.4s]
   Phase1: recovered [1] files with total size of [32], took [2ms]
   Phase2: recovered [1285] transaction log operations, took [2.8s]
   Phase3: recovered [0] transaction log operations, took [574ms]
[17:52:22,864][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][3] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[3.5s]
   Phase1: recovered [1] files with total size of [32], took [37ms]
   Phase2: recovered [1294] transaction log operations, took [2.9s]
   Phase3: recovered [0] transaction log operations, took [475ms]
[17:52:22,865][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][4] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[2.9s]
   Phase1: recovered [1] files with total size of [32], took [244ms]
   Phase2: recovered [1280] transaction log operations, took [2.1s]
   Phase3: recovered [0] transaction log operations, took [507ms]
[17:52:22,871][DEBUG][index.shard.recovery     ] [Terraxia][iannounce_object_1273723709][0] Recovery completed from [Crimson Cowl][f0fdde25-126e-455f-95a4-b7afb9dd4669][inet[getafix.traveljury.com/127.0.0.2:9301]], took[4.4s]
   Phase1: recovered [1] files with total size of [32], took [97ms]
   Phase2: recovered [1336] transaction log operations, took [3.8s]
   Phase3: recovered [0] transaction log operations, took [457ms]

@kimchy
Copy link
Member

kimchy commented May 13, 2010

I see that you send more than one restart request concurrently, you shouldn't... . Let me fix so it will ignore that...

@clintongormley
Copy link
Contributor Author

This looks to be fixed

@bluelu bluelu mentioned this issue Dec 11, 2014
Closed
@makeyang makeyang mentioned this issue Sep 7, 2015
Closed
@wuyunfeng wuyunfeng mentioned this issue Aug 21, 2018
Closed
costin pushed a commit that referenced this issue Dec 6, 2022
@elasticsearchmachine
Merge pull request #171 from elastic/master
e00a56b 
🤖 ESQL: Merge upstream
This issue was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kimchy @clintongormley