Skip to content

fix: resilient ogmios v6 tx submission #971

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 6, 2023

Conversation

mkazlauskas
Copy link
Member

@mkazlauskas mkazlauskas commented Oct 31, 2023

Context

OgmiosTxSubmitProvider is bricked whenever websocket connection drops.

Proposed Solution

Replace OgmiosTxSubmitProvider with ObservableCardanoNode.submitTx and a new NodeTxSubmitProvider. See commit messages for more details.

Note

I didn't run e2e tests, but they seem to be broken in general in conway-era branch. This kind of change would be great to test e2e, but we'll probably have to defer it until later, after this is merged to conway-era.

Follow-up work

Consider removing non-observable variant of CardanoNode

interactionType no longer exists

also, remove an unnecessary eslint-disable
@mkazlauskas mkazlauskas force-pushed the fix/resilient-ogmios-v6-tx-submission branch 2 times, most recently from ae8a1e3 to 1d8596a Compare October 31, 2023 14:43
implement it for OgmiosObservableCardanoNode
OgmiosTxSubmitProvider was unreliable as it didn't
have any re-connection logic. ObservableCardanoNode
already had re-connection logic in-place.

Furthermore, this refactor reduces the scope of Ogmios
dependency, because ogmios package is no longer Provider-aware.
This is a step towards refining this package as a thin wrapper
over ogmios client libraries.

OgmiosTxSubmitProvider is replaced by a combination of:
- ObservableCardanoNode.submitTx
- a new NodeTxSubmitProvider implementation that depends
  on ObservableCardanNode and HandleProvider

Note: re-enabling rabbitMqTxSubmitProvider and TxSubmitWorker tests
is out of scope of this commit. Those tests were already disabled.
This commit modified the test files in order for tests to compile.
@mkazlauskas mkazlauskas force-pushed the fix/resilient-ogmios-v6-tx-submission branch from 1d8596a to 8c56c5e Compare October 31, 2023 14:54
Copy link
Member

@rhyslbw rhyslbw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @mkazlauskas, that's a solid improvement

Copy link
Contributor

@mirceahasegan mirceahasegan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic work, @mkazlauskas ! 🚀

Copy link

Standard DiffPost

This PR would generate the following kubectl diff:

Preview
diff -u -N /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-backend /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-backend
--- /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-backend	2023-10-31 15:10:26.836182505 +0000
+++ /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-backend	2023-10-31 15:10:26.836182505 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "58"
   creationTimestamp: "2023-08-11T18:16:12Z"
-  generation: 58
+  generation: 59
   labels:
     app: backend
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-backend
@@ -89,7 +89,7 @@
           value: "false"
         - name: USE_KORA_LABS
           value: "true"
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:0x9q4ii9h02ccw1i178hbh8djxgajfgm
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:mfff879ifg4h6zww6xw7cbjd6ygzvmv2
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-coingecko-proxy /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-coingecko-proxy
--- /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-10-31 15:10:27.180197482 +0000
+++ /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-10-31 15:10:27.180197482 +0000
@@ -8,7 +8,7 @@
   labels:
     app: coingecko-proxy
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-coingecko-proxy
diff -u -N /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-projector /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-projector
--- /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-projector	2023-10-31 15:10:27.524212460 +0000
+++ /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-projector	2023-10-31 15:10:27.524212460 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "58"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 58
+  generation: 59
   labels:
     app: handle-projector
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-handle-projector
@@ -73,7 +73,7 @@
               name: handle-owner-user.dev-preview-dbsync-db.credentials.postgresql.acid.zalan.do
         - name: PROJECTION_NAMES
           value: handle
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:0x9q4ii9h02ccw1i178hbh8djxgajfgm
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:mfff879ifg4h6zww6xw7cbjd6ygzvmv2
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-provider /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-provider
--- /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-provider	2023-10-31 15:10:27.864227263 +0000
+++ /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-handle-provider	2023-10-31 15:10:27.864227263 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "25"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 25
+  generation: 26
   labels:
     app: handle-provider
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-handle-provider
@@ -81,7 +81,7 @@
           value: handle
         - name: USE_KORA_LABS
           value: "true"
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:0x9q4ii9h02ccw1i178hbh8djxgajfgm
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:mfff879ifg4h6zww6xw7cbjd6ygzvmv2
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-pg-boss-worker /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-pg-boss-worker
--- /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-pg-boss-worker	2023-10-31 15:10:28.208242241 +0000
+++ /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-pg-boss-worker	2023-10-31 15:10:28.208242241 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "58"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 58
+  generation: 59
   labels:
     app: pg-boss-worker
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-pg-boss-worker
@@ -92,10 +92,10 @@
               key: username
               name: stakepool-owner-user.dev-preview-dbsync-db.credentials.postgresql.acid.zalan.do
         - name: QUEUES
-          value: pool-delist-schedule,pool-metadata,pool-metrics,pool-rewards
+          value: pool-metadata,pool-metrics
         - name: STAKE_POOL_PROVIDER_URL
           value: http://dev-preview-cardanojs-backend.dev-preview.svc.cluster.local
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:0x9q4ii9h02ccw1i178hbh8djxgajfgm
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:mfff879ifg4h6zww6xw7cbjd6ygzvmv2
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-projector /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-projector
--- /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-projector	2023-10-31 15:10:28.568257915 +0000
+++ /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-projector	2023-10-31 15:10:28.568257915 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "58"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 58
+  generation: 59
   labels:
     app: stake-pool-projector
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-stake-pool-projector
@@ -72,8 +72,8 @@
               key: username
               name: stakepool-owner-user.dev-preview-dbsync-db.credentials.postgresql.acid.zalan.do
         - name: PROJECTION_NAMES
-          value: stake-pool,stake-pool-metadata-job,stake-pool-metrics-job,stake-pool-rewards-job
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:0x9q4ii9h02ccw1i178hbh8djxgajfgm
+          value: stake-pool,stake-pool-metadata-job,stake-pool-metrics-job
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:mfff879ifg4h6zww6xw7cbjd6ygzvmv2
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-provider /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-provider
--- /tmp/LIVE-1534627760/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-provider	2023-10-31 15:10:28.908272719 +0000
+++ /tmp/MERGED-1375714260/apps.v1.Deployment.dev-preview.dev-preview-cardanojs-stake-pool-provider	2023-10-31 15:10:28.908272719 +0000
@@ -4,11 +4,11 @@
   annotations:
     deployment.kubernetes.io/revision: "55"
   creationTimestamp: "2023-08-11T18:16:13Z"
-  generation: 55
+  generation: 56
   labels:
     app: stake-pool-provider
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-stake-pool-provider
@@ -83,7 +83,7 @@
           value: http://dev-preview-cardano-stack-metadata.dev-preview.svc.cluster.local
         - name: USE_TYPEORM_STAKE_POOL_PROVIDER
           value: "true"
-        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:0x9q4ii9h02ccw1i178hbh8djxgajfgm
+        image: 926093910549.dkr.ecr.us-east-1.amazonaws.com/cardano-services:mfff879ifg4h6zww6xw7cbjd6ygzvmv2
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 3
diff -u -N /tmp/LIVE-1534627760/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-backend-monitor /tmp/MERGED-1375714260/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-backend-monitor
--- /tmp/LIVE-1534627760/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-backend-monitor	2023-10-31 15:10:29.248287522 +0000
+++ /tmp/MERGED-1375714260/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-backend-monitor	2023-10-31 15:10:29.248287522 +0000
@@ -5,7 +5,7 @@
   generation: 2
   labels:
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     instance: primary
   name: lace-backend-monitor
   namespace: dev-preview
diff -u -N /tmp/LIVE-1534627760/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-handle-provider-monitor /tmp/MERGED-1375714260/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-handle-provider-monitor
--- /tmp/LIVE-1534627760/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-handle-provider-monitor	2023-10-31 15:10:29.588302334 +0000
+++ /tmp/MERGED-1375714260/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-handle-provider-monitor	2023-10-31 15:10:29.592302508 +0000
@@ -5,7 +5,7 @@
   generation: 2
   labels:
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     instance: primary
   name: lace-handle-provider-monitor
   namespace: dev-preview
diff -u -N /tmp/LIVE-1534627760/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-stake-pool-provider-monitor /tmp/MERGED-1375714260/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-stake-pool-provider-monitor
--- /tmp/LIVE-1534627760/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-stake-pool-provider-monitor	2023-10-31 15:10:29.928317148 +0000
+++ /tmp/MERGED-1375714260/monitoring.coreos.com.v1.ServiceMonitor.dev-preview.lace-stake-pool-provider-monitor	2023-10-31 15:10:29.932317322 +0000
@@ -5,7 +5,7 @@
   generation: 2
   labels:
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     instance: primary
   name: lace-stake-pool-provider-monitor
   namespace: dev-preview
diff -u -N /tmp/LIVE-1534627760/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-backend /tmp/MERGED-1375714260/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-backend
--- /tmp/LIVE-1534627760/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-backend	2023-10-31 15:10:30.284332659 +0000
+++ /tmp/MERGED-1375714260/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-backend	2023-10-31 15:10:30.284332659 +0000
@@ -19,7 +19,7 @@
   labels:
     app: backend
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-backend
diff -u -N /tmp/LIVE-1534627760/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-coingecko-proxy /tmp/MERGED-1375714260/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-coingecko-proxy
--- /tmp/LIVE-1534627760/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-10-31 15:10:30.636347997 +0000
+++ /tmp/MERGED-1375714260/networking.k8s.io.v1.Ingress.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-10-31 15:10:30.636347997 +0000
@@ -16,7 +16,7 @@
   labels:
     app: coingecko-proxy
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-coingecko-proxy
diff -u -N /tmp/LIVE-1534627760/v1.Service.dev-preview.dev-preview-cardanojs-backend /tmp/MERGED-1375714260/v1.Service.dev-preview.dev-preview-cardanojs-backend
--- /tmp/LIVE-1534627760/v1.Service.dev-preview.dev-preview-cardanojs-backend	2023-10-31 15:10:25.468122943 +0000
+++ /tmp/MERGED-1375714260/v1.Service.dev-preview.dev-preview-cardanojs-backend	2023-10-31 15:10:25.468122943 +0000
@@ -5,7 +5,7 @@
   labels:
     app: backend
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-backend
diff -u -N /tmp/LIVE-1534627760/v1.Service.dev-preview.dev-preview-cardanojs-coingecko-proxy /tmp/MERGED-1375714260/v1.Service.dev-preview.dev-preview-cardanojs-coingecko-proxy
--- /tmp/LIVE-1534627760/v1.Service.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-10-31 15:10:25.808137746 +0000
+++ /tmp/MERGED-1375714260/v1.Service.dev-preview.dev-preview-cardanojs-coingecko-proxy	2023-10-31 15:10:25.808137746 +0000
@@ -5,7 +5,7 @@
   labels:
     app: coingecko-proxy
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-coingecko-proxy
diff -u -N /tmp/LIVE-1534627760/v1.Service.dev-preview.dev-preview-cardanojs-handle-provider /tmp/MERGED-1375714260/v1.Service.dev-preview.dev-preview-cardanojs-handle-provider
--- /tmp/LIVE-1534627760/v1.Service.dev-preview.dev-preview-cardanojs-handle-provider	2023-10-31 15:10:26.144152375 +0000
+++ /tmp/MERGED-1375714260/v1.Service.dev-preview.dev-preview-cardanojs-handle-provider	2023-10-31 15:10:26.144152375 +0000
@@ -5,7 +5,7 @@
   labels:
     app: handle-provider
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-handle-provider
diff -u -N /tmp/LIVE-1534627760/v1.Service.dev-preview.dev-preview-cardanojs-stake-pool-provider /tmp/MERGED-1375714260/v1.Service.dev-preview.dev-preview-cardanojs-stake-pool-provider
--- /tmp/LIVE-1534627760/v1.Service.dev-preview.dev-preview-cardanojs-stake-pool-provider	2023-10-31 15:10:26.488167353 +0000
+++ /tmp/MERGED-1375714260/v1.Service.dev-preview.dev-preview-cardanojs-stake-pool-provider	2023-10-31 15:10:26.488167353 +0000
@@ -5,7 +5,7 @@
   labels:
     app: stake-pool-provider
     app.kubernetes.io/managed-by: std-kubectl
-    app.kubernetes.io/version: c424d00cb716bb2b26607fa9aee43c7555482e6b
+    app.kubernetes.io/version: cf0919da765f25186db50e59e5c0fe340983a459
     network: preview
     release: dev-preview-cardanojs
   name: dev-preview-cardanojs-stake-pool-provider

@mkazlauskas mkazlauskas merged commit e45b264 into conway-era Nov 6, 2023
@mkazlauskas mkazlauskas deleted the fix/resilient-ogmios-v6-tx-submission branch November 6, 2023 11:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants