Full AWS gp3 support for iops and througput config. #1261

Jan-M · 2020-12-16T14:04:36Z

Enable AWS gp3 support in manifest, allowing configuration of IOPS and Throughput and Size.

Potentially includes full support for EBS volume types with that. Thus ìo2.

zalando-robot · 2021-01-08T19:19:47Z

Docker image "registry-write.opensource.zalan.do/acid/postgres-operator-ui-test:v1.5.0-106-g0760b94-pr-1261-6" is not based on an approved base image. Currently, this will have no impact on your deployments, but builds using non-allowed images will be blocked from deployment from February 1st 2021.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Open Source Registry namespace library and use a recommended version as listed in the documentation.

zalando-robot · 2021-01-14T17:31:16Z

Docker image "registry-write.opensource.zalan.do/acid/postgres-operator-ui-test:v1.5.0-108-g89886a9-pr-1261-8" is not based on an approved base image. Currently, this will have no impact on your deployments, but builds using non-allowed images will be blocked from deployment from February 1st 2021.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Open Source Registry namespace library and use a recommended version as listed in the documentation.

go.mod

pkg/cluster/volumes.go

sdudoladov · 2021-01-18T17:20:58Z

pkg/cluster/volumes.go

+	}
+
+	if c.OpConfig.StorageResizeMode == "mixed" {
+		// mixed op uses AWS API to adjust size,throughput,iops and calls pvc chance for file system resize


This phrase is documentation rather than comment.

I assume "chance" should really be change

pkg/cluster/volumes.go

sdudoladov · 2021-01-18T17:28:21Z

pkg/cluster/volumes.go

+	var err error
+
+	targetValue := c.Spec.Volume
+	newSize, err := resource.ParseQuantity(targetValue.Size)


please do not ignore errors.

pkg/cluster/volumes.go

zalando-robot · 2021-01-19T20:23:22Z

Docker image "registry-write.opensource.zalan.do/acid/postgres-operator-test:v1.5.0-110-g259a5d6-dirty-pr-1261-10" is not based on an approved base image. Currently, this will have no impact on your deployments, but builds using non-allowed images will be blocked from deployment from March 1st 2021.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Open Source Registry namespace library and use a recommended version as listed in the documentation.

zalando-robot · 2021-01-19T20:30:23Z

Docker image "registry-write.opensource.zalan.do/acid/postgres-operator-ui-test:v1.5.0-110-g5af81cf-pr-1261-10" is not based on an approved base image. Currently, this will have no impact on your deployments, but builds using non-allowed images will be blocked from deployment from March 1st 2021.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Open Source Registry namespace library and use a recommended version as listed in the documentation.

zalando-robot · 2021-01-19T21:09:34Z

Docker image "registry-write.opensource.zalan.do/acid/postgres-operator-test:v1.5.0-111-ge3a6b32-dirty-pr-1261-11" is not based on an approved base image. Currently, this will have no impact on your deployments, but builds using non-allowed images will be blocked from deployment from March 1st 2021.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Open Source Registry namespace library and use a recommended version as listed in the documentation.

zalando-robot · 2021-01-19T21:16:52Z

Docker image "registry-write.opensource.zalan.do/acid/postgres-operator-ui-test:v1.5.0-111-g2cc71bf-pr-1261-11" is not based on an approved base image. Currently, this will have no impact on your deployments, but builds using non-allowed images will be blocked from deployment from March 1st 2021.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Open Source Registry namespace library and use a recommended version as listed in the documentation.

zalando-robot · 2021-01-20T12:43:13Z

Docker image "registry-write.opensource.zalan.do/acid/postgres-operator-test:v1.5.0-114-g3063d73-dirty-pr-1261-14" is not based on an approved base image. Currently, this will have no impact on your deployments, but builds using non-allowed images will be blocked from deployment from March 1st 2021.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Open Source Registry namespace library and use a recommended version as listed in the documentation.

zalando-robot · 2021-01-20T12:49:48Z

Docker image "registry-write.opensource.zalan.do/acid/postgres-operator-ui-test:v1.5.0-114-g757f284-pr-1261-14" is not based on an approved base image. Currently, this will have no impact on your deployments, but builds using non-allowed images will be blocked from deployment from March 1st 2021.

To create a compliant Docker image of your application, you must reference an allowed Docker image as its base image in your Dockerfile. This base image must come from the Open Source Registry namespace library and use a recommended version as listed in the documentation.

docs/developer.md

sdudoladov · 2021-01-21T13:38:00Z

docs/reference/cluster_manifest.md

@@ -356,6 +356,14 @@ properties of the persistent storage that stores Postgres data.
 * **subPath**
  Subpath to use when mounting volume into Spilo container. Optional.

+* **iops**


and the defaults for iopsand throughput are ... ?

docs/reference/operator_parameters.md

manifests/complete-postgres-manifest.yaml

sdudoladov · 2021-01-21T13:59:34Z

pkg/cluster/volumes.go

+	return nil
+}
+
+func (c *Cluster) syncUnderlyingEBSVolume() error {


imo Underlying is self-evident. Omitting the word from the method name would make it more readable

pkg/cluster/volumes.go

sdudoladov · 2021-01-22T16:15:57Z

👍

Jan-M · 2021-01-25T08:40:21Z

👍

Jan-M · 2021-01-25T09:06:46Z

👍

Jan-M requested review from avaczi, CyberDem0n, erthalion, FxKu, RafiaSabih and sdudoladov as code owners December 16, 2020 14:04

Jan-M added enhancement zalando labels Dec 16, 2020

Jan-M self-assigned this Dec 16, 2020

FxKu added this to the 1.7 milestone Dec 18, 2020

sdudoladov reviewed Jan 18, 2021

View reviewed changes

Jan-M changed the title ~~Initial commit for full AWS gp3 support.~~ Full AWS gp3 support for iops and througput config. Jan 20, 2021

Jan-M and others added 10 commits January 20, 2021 14:04

Initial commit for full AWS gp3 support.

da04480

Work in progress.

679a497

add documentation and validation for new manifest fields

a3eaa98

More code for ebs migrations.

d4b5e6a

Make interface optional like aws api.

bff8b09

Remove unnecessary check. Fix test with pointers.

8ad5894

Fixing test, problem was first arg. no pointer.

35e0e5f

Lets see if test works.

1a528a4

Doing the test and references right.

ec5e1f5

Extact pod and pvc creation.

e56a522

Jan-M added 5 commits January 20, 2021 14:04

Reuse function in other test.

c671ee8

Add tests for not touching unchanged properties.

0ceb554

More logging, some logic change around errors.

f308260

Doc changes.

0a7965c

Fix non execute of ebs modifications.

4cee9b8

Jan-M force-pushed the full-gp3-support branch from 81d867b to 4cee9b8 Compare January 20, 2021 13:07

Remove outdated version.

cccf369

sdudoladov reviewed Jan 21, 2021

View reviewed changes

Jan-M added 2 commits January 22, 2021 17:07

Address review input.

d7f2e58

Same order.

d97f836

Jan-M merged commit 4a88f00 into master Jan 25, 2021

Full AWS gp3 support for iops and througput config. #1261

Full AWS gp3 support for iops and througput config. #1261

Uh oh!

Conversation

Jan-M commented Dec 16, 2020

Uh oh!

zalando-robot commented Jan 8, 2021

Uh oh!

zalando-robot commented Jan 14, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sdudoladov Jan 18, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sdudoladov Jan 18, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zalando-robot commented Jan 19, 2021

Uh oh!

zalando-robot commented Jan 19, 2021

Uh oh!

zalando-robot commented Jan 19, 2021

Uh oh!

zalando-robot commented Jan 19, 2021

Uh oh!

zalando-robot commented Jan 20, 2021

Uh oh!

zalando-robot commented Jan 20, 2021

Uh oh!

Uh oh!

sdudoladov Jan 21, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sdudoladov Jan 21, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sdudoladov commented Jan 22, 2021

Uh oh!

Jan-M commented Jan 25, 2021

Uh oh!

Jan-M commented Jan 25, 2021

Uh oh!

Uh oh!