Skip to content

Add TableReplicas Support for DynamoDB Table Replicas #120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Mar 25, 2025
Merged

Add TableReplicas Support for DynamoDB Table Replicas #120

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
3cc1a09
add replica update to crd
rushmash91 Mar 7, 2025
0185830
add tests
rushmash91 Mar 13, 2025
534418a
Rename to table replicas
rushmash91 Mar 21, 2025
08ebf41
delete fixes
rushmash91 Mar 24, 2025
652f3f4
change update-replica style
rushmash91 Mar 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions apis/v1alpha1/ack-generate-metadata.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
ack_generate_info:
build_date: "2025-02-20T18:05:49Z"
build_hash: a326346bd3a6973254d247c9ab2dc76790c36241
build_date: "2025-03-24T15:20:40Z"
build_hash: 3722729cebe6d3c03c7e442655ef0846f91566a2
go_version: go1.24.0
version: v0.43.2
api_directory_checksum: 3bc0637159c94a74d2402d9a707f9c21339e9b45
version: v0.43.2-7-g3722729
api_directory_checksum: cb49386ebd7bb50e2521072a76262c72b9dbd285
api_version: v1alpha1
aws_sdk_go_version: v1.32.6
generator_config_info:
file_checksum: f6d68afa724d9e1d8fb6ce58da11ed0e5635f9d5
file_checksum: 4533fa8aca3b134b5895ad6ce9a093c3446d99da
original_file_name: generator.yaml
last_modification:
reason: API generation
6 changes: 6 additions & 0 deletions apis/v1alpha1/generator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ operations:
resources:
Table:
fields:
TableReplicas:
custom_field:
list_of: CreateReplicationGroupMemberAction
compare:
is_ignored: true
GlobalSecondaryIndexesDescriptions:
custom_field:
list_of: GlobalSecondaryIndexDescription
Expand Down Expand Up @@ -71,6 +76,7 @@ resources:
code: ResourceNotFoundException
terminal_codes:
- InvalidParameter
- ValidationException
update_operation:
custom_method_name: customUpdateTable
hooks:
Expand Down
3 changes: 2 additions & 1 deletion apis/v1alpha1/table.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

11 changes: 11 additions & 0 deletions apis/v1alpha1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

39 changes: 39 additions & 0 deletions config/crd/bases/dynamodb.services.k8s.aws_tables.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -376,6 +376,45 @@ spec:
The name of the table to create. You can also provide the Amazon Resource
Name (ARN) of the table in this parameter.
type: string
tableReplicas:
items:
description: Represents a replica to be created.
properties:
globalSecondaryIndexes:
items:
description: Represents the properties of a replica global
secondary index.
properties:
indexName:
type: string
provisionedThroughputOverride:
description: |-
Replica-specific provisioned throughput settings. If not specified, uses
the source table's provisioned throughput settings.
properties:
readCapacityUnits:
format: int64
type: integer
type: object
type: object
type: array
kmsMasterKeyID:
type: string
provisionedThroughputOverride:
description: |-
Replica-specific provisioned throughput settings. If not specified, uses
the source table's provisioned throughput settings.
properties:
readCapacityUnits:
format: int64
type: integer
type: object
regionName:
type: string
tableClassOverride:
type: string
type: object
type: array
tags:
description: |-
A list of key-value pairs to label the table. For more information, see Tagging
Expand Down
6 changes: 6 additions & 0 deletions generator.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@ operations:
resources:
Table:
fields:
TableReplicas:
custom_field:
list_of: CreateReplicationGroupMemberAction
compare:
is_ignored: true
GlobalSecondaryIndexesDescriptions:
custom_field:
list_of: GlobalSecondaryIndexDescription
Expand Down Expand Up @@ -71,6 +76,7 @@ resources:
code: ResourceNotFoundException
terminal_codes:
- InvalidParameter
- ValidationException
update_operation:
custom_method_name: customUpdateTable
hooks:
Expand Down
39 changes: 39 additions & 0 deletions helm/crds/dynamodb.services.k8s.aws_tables.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -380,6 +380,45 @@ spec:
The name of the table to create. You can also provide the Amazon Resource
Name (ARN) of the table in this parameter.
type: string
tableReplicas:
items:
description: Represents a replica to be created.
properties:
globalSecondaryIndexes:
items:
description: Represents the properties of a replica global
secondary index.
properties:
indexName:
type: string
provisionedThroughputOverride:
description: |-
Replica-specific provisioned throughput settings. If not specified, uses
the source table's provisioned throughput settings.
properties:
readCapacityUnits:
format: int64
type: integer
type: object
type: object
type: array
kmsMasterKeyID:
type: string
provisionedThroughputOverride:
description: |-
Replica-specific provisioned throughput settings. If not specified, uses
the source table's provisioned throughput settings.
properties:
readCapacityUnits:
format: int64
type: integer
type: object
regionName:
type: string
tableClassOverride:
type: string
type: object
type: array
tags:
description: |-
A list of key-value pairs to label the table. For more information, see Tagging
Expand Down
39 changes: 34 additions & 5 deletions pkg/resource/table/hooks.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ package table

import (
"context"
"errors"
"fmt"
"strings"
"time"
Expand All @@ -34,21 +35,25 @@ import (

var (
ErrTableDeleting = fmt.Errorf(
"Table in '%v' state, cannot be modified or deleted",
"table in '%v' state, cannot be modified or deleted",
svcsdktypes.TableStatusDeleting,
)
ErrTableCreating = fmt.Errorf(
"Table in '%v' state, cannot be modified or deleted",
"table in '%v' state, cannot be modified or deleted",
svcsdktypes.TableStatusCreating,
)
ErrTableUpdating = fmt.Errorf(
"Table in '%v' state, cannot be modified or deleted",
"table in '%v' state, cannot be modified or deleted",
svcsdktypes.TableStatusUpdating,
)
ErrTableGSIsUpdating = fmt.Errorf(
"Table GSIs in '%v' state, cannot be modified or deleted",
"table GSIs in '%v' state, cannot be modified or deleted",
svcsdktypes.IndexStatusCreating,
)
ErrTableReplicasUpdating = fmt.Errorf(
"table replica in '%v' state, cannot be modified or deleted",
svcsdktypes.ReplicaStatusUpdating,
)
)

// TerminalStatuses are the status strings that are terminal states for a
Expand All @@ -74,12 +79,16 @@ var (
)
requeueWaitWhileUpdating = ackrequeue.NeededAfter(
ErrTableUpdating,
5*time.Second,
10*time.Second,
)
requeueWaitGSIReady = ackrequeue.NeededAfter(
ErrTableGSIsUpdating,
10*time.Second,
)
requeueWaitReplicasActive = ackrequeue.NeededAfter(
ErrTableReplicasUpdating,
10*time.Second,
)
)

// tableHasTerminalStatus returns whether the supplied Dynamodb table is in a
Expand Down Expand Up @@ -224,6 +233,17 @@ func (rm *resourceManager) customUpdateTable(
}
return nil, err
}
case delta.DifferentAt("Spec.TableReplicas"):
// Enabling replicas required streams enabled and StreamViewType to be NEW_AND_OLD_IMAGES
// Version 2019.11.21 TableUpdate API requirement
if !hasStreamSpecificationWithNewAndOldImages(desired) {
msg := "table must have DynamoDB Streams enabled with StreamViewType set to NEW_AND_OLD_IMAGES for replica updates"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this actually affect it? it would be nice to have comments explaining why...maybe documentation for the table as well

rlog.Debug(msg)
return nil, ackerr.NewTerminalError(errors.New(msg))
}
if err := rm.syncReplicas(ctx, latest, desired); err != nil {
return nil, err
}
}
}

Expand Down Expand Up @@ -558,6 +578,15 @@ func customPreCompare(
}
}

// Handle ReplicaUpdates API comparison
if len(a.ko.Spec.TableReplicas) != len(b.ko.Spec.TableReplicas) {
delta.Add("Spec.TableReplicas", a.ko.Spec.TableReplicas, b.ko.Spec.TableReplicas)
} else if a.ko.Spec.TableReplicas != nil && b.ko.Spec.TableReplicas != nil {
if !equalReplicaArrays(a.ko.Spec.TableReplicas, b.ko.Spec.TableReplicas) {
delta.Add("Spec.TableReplicas", a.ko.Spec.TableReplicas, b.ko.Spec.TableReplicas)
}
}

if a.ko.Spec.DeletionProtectionEnabled == nil {
a.ko.Spec.DeletionProtectionEnabled = aws.Bool(false)
}
Expand Down
Loading