Skip to content

Commit dd0d62f

Browse files
feat(pipes-alpha): support for customer-managed KMS keys to encrypt pipe data (#33546)
### Issue # (if applicable) Closes #31453 ### Reason for this change AWS Pipes supports for encrypting data by customer managed KMS key instead of Amazon managed key. https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-encryption-pipes-cmkey.html The L2 Pipe construct does not support this feature now. ### Description of changes - Add `kmsKey` prop to `PipeProps` - ### Describe any new or updated permissions being added - Add KMS key policy which enables pipes to access to the key. https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-encryption-key-policy.html#eb-encryption-key-policy-pipe ### Description of how you validated changes Add both unit and integ tests. ### Checklist - [x] My code adheres to the [CONTRIBUTING GUIDE](https://github.com/aws/aws-cdk/blob/main/CONTRIBUTING.md) and [DESIGN GUIDELINES](https://github.com/aws/aws-cdk/blob/main/docs/DESIGN_GUIDELINES.md) ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
1 parent 7ebb92c commit dd0d62f

File tree

14 files changed

+36766
-15
lines changed

14 files changed

+36766
-15
lines changed

packages/@aws-cdk/aws-pipes-alpha/README.md

+34-12
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ can be filtered, transformed and enriched.
2323

2424
![diagram of pipes](https://d1.awsstatic.com/product-marketing/EventBridge/Product-Page-Diagram_Amazon-EventBridge-Pipes.cd7961854be4432d63f6158ffd18271d6c9fa3ec.png)
2525

26-
For more details see the [service documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes.html).
26+
For more details see the [service documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes.html).
2727

2828
## Pipe
2929

@@ -65,7 +65,7 @@ A source is a AWS Service that is polled. The following sources are possible:
6565
- [Amazon SQS queue](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-sqs.html)
6666
- [Apache Kafka stream](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-kafka.html)
6767

68-
Currently, DynamoDB, Kinesis, and SQS are supported. If you are interested in support for additional sources,
68+
Currently, DynamoDB, Kinesis, and SQS are supported. If you are interested in support for additional sources,
6969
kindly let us know by opening a GitHub issue or raising a PR.
7070

7171
### Example source
@@ -107,7 +107,7 @@ const pipe = new pipes.Pipe(this, 'Pipe', {
107107

108108
This example shows a filter that only forwards events with the `customerType` B2B or B2C from the source messages. Messages that are not matching the filter are not forwarded to the enrichment or target step.
109109

110-
You can define multiple filter pattern which are combined with a logical `OR`.
110+
You can define multiple filter pattern which are combined with a logical `OR`.
111111

112112
Additional filter pattern and details can be found in the EventBridge pipes [docs](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-event-filtering.html).
113113

@@ -117,7 +117,7 @@ For enrichments and targets the input event can be transformed. The transformati
117117
A transformation has access to the input event as well to some context information of the pipe itself like the name of the pipe.
118118
See [docs](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-input-transformation.html) for details.
119119

120-
### Example - input transformation from object
120+
### Example - input transformation from object
121121

122122
The input transformation can be created from an object. The object can contain static values, dynamic values or pipe variables.
123123

@@ -185,7 +185,7 @@ If the transformation is applied to a target it might be converted to a string r
185185

186186
In cases where you want to forward only a part of the event to the target you can use the transformation event path.
187187

188-
> This only works for targets because the enrichment needs to have a valid json as input.
188+
> This only works for targets because the enrichment needs to have a valid json as input.
189189
190190

191191
```ts
@@ -204,7 +204,7 @@ const pipe = new pipes.Pipe(this, 'Pipe', {
204204

205205
This transformation extracts the body of the event.
206206

207-
So when the following batch of input events is processed by the pipe
207+
So when the following batch of input events is processed by the pipe
208208

209209
```json
210210
[
@@ -357,7 +357,7 @@ For example a lambda function that returns a concatenation of the static field,
357357
export async function handler (event: any) {
358358
return event.staticField + "-" + event.dynamicField + "-" + event.pipeVariable;
359359
};
360-
```
360+
```
361361

362362
will produce the following target message in the target SQS queue.
363363

@@ -407,9 +407,9 @@ const pipeTarget = new SqsTarget(targetQueue);
407407

408408
## Log destination
409409

410-
A pipe can produce log events that are forwarded to different log destinations.
410+
A pipe can produce log events that are forwarded to different log destinations.
411411
You can configure multiple destinations, but all the destination share the same log level and log data.
412-
For details check the official [documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-logs.html).
412+
For details check the official [documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-pipes-logs.html).
413413

414414
The log level and data that is included in the log events is configured on the pipe class itself.
415415
The actual destination is defined independently, and there are three options:
@@ -436,6 +436,28 @@ const pipe = new pipes.Pipe(this, 'Pipe', {
436436
});
437437
```
438438

439-
This example uses a CloudWatch Logs log group to store the log emitted during a pipe execution.
440-
The log level is set to `TRACE` so all steps of the pipe are logged.
441-
Additionally all execution data is logged as well.
439+
This example uses a CloudWatch Logs log group to store the log emitted during a pipe execution.
440+
The log level is set to `TRACE` so all steps of the pipe are logged.
441+
Additionally all execution data is logged as well.
442+
443+
## Encrypt pipe data with KMS
444+
445+
You can specify that EventBridge use a customer managed key to encrypt pipe data stored at rest,
446+
rather than use an AWS owned key as is the default.
447+
Details can be found in the [documentation](https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-encryption-pipes-cmkey.html).
448+
449+
To do this, you need to specify the key in the `kmsKey` property of the pipe.
450+
451+
```ts
452+
declare const sourceQueue: sqs.Queue;
453+
declare const targetQueue: sqs.Queue;
454+
declare const kmsKey: kms.Key;
455+
456+
const pipe = new pipes.Pipe(this, 'Pipe', {
457+
source: new SqsSource(sourceQueue),
458+
target: new SqsTarget(targetQueue),
459+
kmsKey,
460+
// pipeName is required when using a KMS key
461+
pipeName: 'MyPipe',
462+
});
463+
```

packages/@aws-cdk/aws-pipes-alpha/lib/pipe.ts

+40-3
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
1-
import { IResource, Resource, Stack } from 'aws-cdk-lib';
2-
import { IRole, Role, ServicePrincipal } from 'aws-cdk-lib/aws-iam';
1+
import { IResource, Resource, Stack, ValidationError } from 'aws-cdk-lib';
2+
import { ArnPrincipal, IRole, PolicyStatement, Role, ServicePrincipal } from 'aws-cdk-lib/aws-iam';
3+
import * as kms from 'aws-cdk-lib/aws-kms';
34
import { CfnPipe } from 'aws-cdk-lib/aws-pipes';
45
import { addConstructMetadata } from 'aws-cdk-lib/core/lib/metadata-resource';
56
import { Construct } from 'constructs';
@@ -165,6 +166,13 @@ export interface PipeProps {
165166
readonly tags?: {
166167
[key: string]: string;
167168
};
169+
170+
/**
171+
* The AWS KMS customer managed key to encrypt pipe data.
172+
*
173+
* @default undefined - AWS managed key is used
174+
*/
175+
readonly kmsKey?: kms.IKey;
168176
}
169177

170178
abstract class PipeBase extends Resource implements IPipe {
@@ -279,10 +287,38 @@ export class Pipe extends PipeBase {
279287
return { ...currentLogConfiguration, ...additionalLogConfiguration };
280288
}, initialLogConfiguration);
281289

290+
if (props.kmsKey) {
291+
if (!props.pipeName) {
292+
throw new ValidationError('`pipeName` is required when specifying a `kmsKey` prop.', this);
293+
}
294+
// Add permissions to the KMS key
295+
// see https://docs.aws.amazon.com/eventbridge/latest/userguide/eb-encryption-pipes-cmkey.html#eb-encryption-key-policy-pipe
296+
props.kmsKey.addToResourcePolicy(
297+
new PolicyStatement({
298+
actions: ['kms:Decrypt', 'kms:DescribeKey', 'kms:GenerateDataKey'],
299+
resources: ['*'],
300+
principals: [new ArnPrincipal(this.pipeRole.roleArn)],
301+
conditions: {
302+
'ArnLike': {
303+
'kms:EncryptionContext:aws:pipe:arn': Stack.of(this).formatArn({
304+
service: 'pipes',
305+
resource: 'pipe',
306+
resourceName: props.pipeName,
307+
}),
308+
},
309+
'ForAnyValue:StringEquals': {
310+
'kms:EncryptionContextKeys': [
311+
'aws:pipe:arn',
312+
],
313+
},
314+
},
315+
}),
316+
);
317+
}
318+
282319
/**
283320
* Pipe resource
284321
*/
285-
286322
const resource = new CfnPipe(this, 'Resource', {
287323
name: props.pipeName,
288324
description: props.description,
@@ -295,6 +331,7 @@ export class Pipe extends PipeBase {
295331
targetParameters: target.targetParameters,
296332
desiredState: props.desiredState,
297333
logConfiguration: logConfiguration,
334+
kmsKeyIdentifier: props.kmsKey?.keyArn,
298335
tags: props.tags,
299336
});
300337

packages/@aws-cdk/aws-pipes-alpha/rosetta/default.ts-fixture

+1
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,7 @@ import * as cdk from 'aws-cdk-lib';
33
import * as sqs from 'aws-cdk-lib/aws-sqs';
44
import * as lambda from 'aws-cdk-lib/aws-lambda';
55
import * as logs from 'aws-cdk-lib/aws-logs';
6+
import * as kms from 'aws-cdk-lib/aws-kms';
67
import { Construct } from 'constructs';
78
import * as pipes from '@aws-cdk/aws-pipes-alpha';
89
import { SqsSource } from '@aws-cdk/aws-pipes-sources-alpha';

packages/@aws-cdk/aws-pipes-alpha/test/integ.pipe-kmskey.js.snapshot/PipesKmsKeyTestDefaultTestDeployAssert1065A2A1.assets.json

+32
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)