Skip to content

Commit 29a3b2a

Browse files
authored
Merge pull request #591 from mickychetta/fargate-s3
feat(aws-fargate-s3): Created new construct
2 parents f8f9b6f + f14cc54 commit 29a3b2a

12 files changed

+3708
-0
lines changed
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
lib/*.js
2+
test/*.js
3+
*.d.ts
4+
coverage
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
lib/*.js
2+
test/*.js
3+
*.js.map
4+
*.d.ts
5+
node_modules
6+
*.generated.ts
7+
dist
8+
.jsii
9+
10+
.LAST_BUILD
11+
.nyc_output
12+
coverage
13+
.nycrc
14+
.LAST_PACKAGE
15+
*.snk
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
# Exclude typescript source and config
2+
*.ts
3+
tsconfig.json
4+
coverage
5+
.nyc_output
6+
*.tgz
7+
*.snk
8+
*.tsbuildinfo
9+
10+
# Include javascript files and typescript declarations
11+
!*.js
12+
!*.d.ts
13+
14+
# Exclude jsii outdir
15+
dist
16+
17+
# Include .jsii
18+
!.jsii
19+
20+
# Include .jsii
21+
!.jsii
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# aws-fargate-s3 module
2+
<!--BEGIN STABILITY BANNER-->
3+
4+
---
5+
6+
![Stability: Experimental](https://img.shields.io/badge/stability-Experimental-important.svg?style=for-the-badge)
7+
8+
> All classes are under active development and subject to non-backward compatible changes or removal in any
9+
> future version. These are not subject to the [Semantic Versioning](https://semver.org/) model.
10+
> This means that while you may use them, you may need to update your source code when upgrading to a newer version of this package.
11+
12+
---
13+
<!--END STABILITY BANNER-->
14+
15+
| **Reference Documentation**:| <span style="font-weight: normal">https://docs.aws.amazon.com/solutions/latest/constructs/</span>|
16+
|:-------------|:-------------|
17+
<div style="height:8px"></div>
18+
19+
| **Language** | **Package** |
20+
|:-------------|-----------------|
21+
|![Python Logo](https://docs.aws.amazon.com/cdk/api/latest/img/python32.png) Python|`aws_solutions_constructs.aws_fargate_s3`|
22+
|![Typescript Logo](https://docs.aws.amazon.com/cdk/api/latest/img/typescript32.png) Typescript|`@aws-solutions-constructs/aws-fargate-s3`|
23+
|![Java Logo](https://docs.aws.amazon.com/cdk/api/latest/img/java32.png) Java|`software.amazon.awsconstructs.services.fargates3`|
24+
25+
This AWS Solutions Construct implements an AWS Fargate service that can write/read to an Amazon S3 Bucket
26+
27+
Here is a minimal deployable pattern definition in Typescript:
28+
29+
``` typescript
30+
import { FargateToS3, FargateToS3Props } from '@aws-solutions-constructs/aws-fargate-s3';
31+
32+
const props: FargateToS3Props = {
33+
publicApi: true,
34+
ecrRepositoryArn: "arn of a repo in ECR in your account",
35+
});
36+
37+
new FargateToS3(stack, 'test-construct', props);
38+
```
39+
40+
## Pattern Construct Props
41+
42+
| **Name** | **Type** | **Description** |
43+
|:-------------|:----------------|-----------------|
44+
| publicApi | boolean | Whether the construct is deploying a private or public API. This has implications for the VPC and ALB. |
45+
| vpcProps? | [ec2.VpcProps](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ec2.VpcProps.html) | Optional custom properties for a VPC the construct will create. This VPC will be used by the new ALB and any Private Hosted Zone the construct creates (that's why loadBalancerProps and privateHostedZoneProps can't include a VPC). Providing both this and existingVpc is an error. |
46+
| existingVpc? | [ec2.IVpc](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ec2.IVpc.html) | An existing VPC in which to deploy the construct. Providing both this and vpcProps is an error. If the client provides an existing load balancer and/or existing Private Hosted Zone, those constructs must exist in this VPC. |
47+
| clusterProps? | [ecs.ClusterProps](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.ClusterProps.html) | Optional properties to create a new ECS cluster. To provide an existing cluster, use the cluster attribute of fargateServiceProps. |
48+
| ecrRepositoryArn? | string | The arn of an ECR Repository containing the image to use to generate the containers. Either this or the image property of containerDefinitionProps must be provided. format: arn:aws:ecr:*region*:*account number*:repository/*Repository Name* |
49+
| ecrImageVersion? | string | The version of the image to use from the repository. Defaults to 'Latest' |
50+
| containerDefinitionProps? | [ecs.ContainerDefinitionProps \| any](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.ContainerDefinitionProps.html) | Optional props to define the container created for the Fargate Service (defaults found in fargate-defaults.ts) |
51+
| fargateTaskDefinitionProps? | [ecs.FargateTaskDefinitionProps \| any](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.FargateTaskDefinitionProps.html) | Optional props to define the Fargate Task Definition for this construct (defaults found in fargate-defaults.ts) |
52+
| fargateServiceProps? | [ecs.FargateServiceProps \| any](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.FargateServiceProps.html) | Optional values to override default Fargate Task definition properties (fargate-defaults.ts). The construct will default to launching the service is the most isolated subnets available (precedence: Isolated, Private and Public). Override those and other defaults here. |
53+
| existingFargateServiceObject? | [ecs.FargateService](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.FargateService.html) | A Fargate Service already instantiated (probably by another Solutions Construct). If this is specified, then no props defining a new service can be provided, including: existingImageObject, ecrImageVersion, containerDefintionProps, fargateTaskDefinitionProps, ecrRepositoryArn, fargateServiceProps, clusterProps, existingClusterInterface |
54+
| existingContainerDefinitionObject? | [ecs.ContainerDefinition](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.ContainerDefinition.html) | A container definition already instantiated as part of a Fargate service. This must be the container in the existingFargateServiceObject |
55+
|existingBucketInterface?|[`s3.IBucket`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.IBucket.html)|Existing S3 Bucket interface. Providing this property and `bucketProps` results in an error.|
56+
|bucketProps?|[`s3.BucketProps`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.BucketProps.html)|Optional user provided props to override the default props for the S3 Bucket.|
57+
|loggingBucketProps?|[`s3.BucketProps`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.BucketProps.html)|Optional user provided props to override the default props for the S3 Logging Bucket.|
58+
|logS3AccessLogs?| boolean|Whether to turn on Access Logging for the S3 bucket. Creates an S3 bucket with associated storage costs for the logs. Enabling Access Logging is a best practice. default - true|
59+
|bucketPermissions?|`string[]`|Optional bucket permissions to grant to the Fargate service. One or more of the following may be specified: `Delete`, `Read`, and `Write`. Default is `ReadWrite` which includes `[s3:GetObject*, s3:GetBucket*, s3:List*, s3:DeleteObject*, s3:PutObject*, s3:Abort*]`.|
60+
|bucketArnEnvironmentVariableName?|string|Optional Name for the S3 bucket arn environment variable set for the container.|
61+
|bucketEnvironmentVariableName?|string|Optional Name for the S3 bucket name environment variable set for the container.|
62+
63+
## Pattern Properties
64+
65+
| **Name** | **Type** | **Description** |
66+
|:-------------|:----------------|-----------------|
67+
| vpc | [ec2.IVpc](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ec2.IVpc.html) | The VPC used by the construct (whether created by the construct or providedb by the client) |
68+
| service | [ecs.FargateService](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.FargateService.html) | The AWS Fargate service used by this construct (whether created by this construct or passed to this construct at initialization) |
69+
| container | [ecs.ContainerDefinition](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.ContainerDefinition.html) | The container associated with the AWS Fargate service in the service property. |
70+
| s3Bucket? |[s3.IBucket](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.IBucket.html)|Returns an instance of s3.Bucket created by the construct|
71+
| s3BucketInterface |[`s3.IBucket`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.IBucket.html)|Returns an instance of s3.IBucket created by the construct|
72+
| s3LoggingBucket? | [s3.Bucket](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.Bucket.html)|Returns an instance of s3.Bucket created by the construct|
73+
74+
## Default settings
75+
76+
Out of the box implementation of the Construct without any override will set the following defaults:
77+
78+
### AWS Fargate Service
79+
* Sets up an AWS Fargate service
80+
* Uses the existing service if provided
81+
* Creates a new service if none provided.
82+
* Service will run in isolated subnets if available, then private subnets if available and finally public subnets
83+
* Adds environment variables to the container with the ARN and Name of the S3 Bucket
84+
* Add permissions to the container IAM role allowing it to publish to the S3 Bucket
85+
86+
### Amazon S3 Bucket
87+
* Sets up an Amazon S3 Bucket
88+
* Uses an existing bucket if one is provided, otherwise creates a new one
89+
* Adds an Interface Endpoint to the VPC for S3 (the service by default runs in Isolated or Private subnets)
90+
91+
## Architecture
92+
![Architecture Diagram](architecture.png)
93+
94+
***
95+
&copy; Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
Loading
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,239 @@
1+
/**
2+
* Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
5+
* with the License. A copy of the License is located at
6+
*
7+
* http://www.apache.org/licenses/LICENSE-2.0
8+
*
9+
* or in the 'license' file accompanying this file. This file is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES
10+
* OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions
11+
* and limitations under the License.
12+
*/
13+
14+
import * as ec2 from "@aws-cdk/aws-ec2";
15+
import * as s3 from "@aws-cdk/aws-s3";
16+
// Note: To ensure CDKv2 compatibility, keep the import statement for Construct separate
17+
import { Construct } from "@aws-cdk/core";
18+
import * as defaults from "@aws-solutions-constructs/core";
19+
import * as ecs from "@aws-cdk/aws-ecs";
20+
import * as iam from "@aws-cdk/aws-iam";
21+
22+
export interface FargateToS3Props {
23+
/**
24+
* Optional custom properties for a VPC the construct will create. This VPC will
25+
* be used by the new Fargate service the construct creates (that's
26+
* why targetGroupProps can't include a VPC). Providing
27+
* both this and existingVpc is an error. An S3 Interface
28+
* endpoint will be included in this VPC.
29+
*
30+
* @default - none
31+
*/
32+
readonly vpcProps?: ec2.VpcProps;
33+
/**
34+
* An existing VPC in which to deploy the construct. Providing both this and
35+
* vpcProps is an error. If the client provides an existing Fargate service,
36+
* this value must be the VPC where the service is running. An S3 Interface
37+
* endpoint will be added to this VPC.
38+
*
39+
* @default - none
40+
*/
41+
readonly existingVpc?: ec2.IVpc;
42+
/**
43+
* Whether the construct is deploying a private or public API. This has implications for the VPC deployed
44+
* by this construct.
45+
*
46+
* @default - none
47+
*/
48+
readonly publicApi: boolean;
49+
/**
50+
* Optional properties to create a new ECS cluster
51+
*/
52+
readonly clusterProps?: ecs.ClusterProps;
53+
/**
54+
* The arn of an ECR Repository containing the image to use
55+
* to generate the containers
56+
*
57+
* format:
58+
* arn:aws:ecr:[region]:[account number]:repository/[Repository Name]
59+
*/
60+
readonly ecrRepositoryArn?: string;
61+
/**
62+
* The version of the image to use from the repository
63+
*
64+
* @default - 'latest'
65+
*/
66+
readonly ecrImageVersion?: string;
67+
/*
68+
* Optional props to define the container created for the Fargate Service
69+
*
70+
* defaults - fargate-defaults.ts
71+
*/
72+
readonly containerDefinitionProps?: ecs.ContainerDefinitionProps | any;
73+
/*
74+
* Optional props to define the Fargate Task Definition for this construct
75+
*
76+
* defaults - fargate-defaults.ts
77+
*/
78+
readonly fargateTaskDefinitionProps?: ecs.FargateTaskDefinitionProps | any;
79+
/**
80+
* Optional values to override default Fargate Task definition properties
81+
* (fargate-defaults.ts). The construct will default to launching the service
82+
* is the most isolated subnets available (precedence: Isolated, Private and
83+
* Public). Override those and other defaults here.
84+
*
85+
* defaults - fargate-defaults.ts
86+
*/
87+
readonly fargateServiceProps?: ecs.FargateServiceProps | any;
88+
/**
89+
* A Fargate Service already instantiated (probably by another Solutions Construct). If
90+
* this is specified, then no props defining a new service can be provided, including:
91+
* existingImageObject, ecrImageVersion, containerDefintionProps, fargateTaskDefinitionProps,
92+
* ecrRepositoryArn, fargateServiceProps, clusterProps, existingClusterInterface. If this value
93+
* is provided, then existingContainerDefinitionObject must be provided as well.
94+
*
95+
* @default - none
96+
*/
97+
readonly existingFargateServiceObject?: ecs.FargateService;
98+
/**
99+
* Existing instance of S3 Bucket object, providing both this and `bucketProps` will cause an error.
100+
*
101+
* @default - None
102+
*/
103+
readonly existingBucketObj?: s3.IBucket;
104+
/**
105+
* Optional user provided props to override the default props for the S3 Bucket.
106+
*
107+
* @default - Default props are used
108+
*/
109+
readonly bucketProps?: s3.BucketProps;
110+
/**
111+
* Optional user provided props to override the default props for the S3 Logging Bucket.
112+
*
113+
* @default - Default props are used
114+
*/
115+
readonly loggingBucketProps?: s3.BucketProps
116+
/**
117+
* Whether to turn on Access Logs for the S3 bucket with the associated storage costs.
118+
* Enabling Access Logging is a best practice.
119+
*
120+
* @default - true
121+
*/
122+
readonly logS3AccessLogs?: boolean;
123+
/**
124+
* Optional bucket permissions to grant to the Fargate service.
125+
* One or more of the following may be specified: "Delete", "Put", "Read", "ReadWrite", "Write".
126+
*
127+
* @default - Read/write access is given to the Fargate service if no value is specified.
128+
*/
129+
readonly bucketPermissions?: string[];
130+
/**
131+
* Optional Name for the S3 bucket arn environment variable set for the container.
132+
*
133+
* @default - None
134+
*/
135+
readonly bucketArnEnvironmentVariableName?: string;
136+
/**
137+
* Optional Name for the S3 bucket name environment variable set for the container.
138+
*
139+
* @default - None
140+
*/
141+
readonly bucketEnvironmentVariableName?: string;
142+
/*
143+
* A container definition already instantiated as part of a Fargate service. This must
144+
* be the container in the existingFargateServiceObject.
145+
*
146+
* @default - None
147+
*/
148+
readonly existingContainerDefinitionObject?: ecs.ContainerDefinition;
149+
}
150+
151+
export class FargateToS3 extends Construct {
152+
public readonly vpc: ec2.IVpc;
153+
public readonly service: ecs.FargateService;
154+
public readonly container: ecs.ContainerDefinition;
155+
public readonly s3BucketInterface: s3.IBucket;
156+
public readonly s3Bucket?: s3.Bucket;
157+
public readonly s3LoggingBucket?: s3.Bucket;
158+
159+
constructor(scope: Construct, id: string, props: FargateToS3Props) {
160+
super(scope, id);
161+
defaults.CheckProps(props);
162+
defaults.CheckFargateProps(props);
163+
164+
this.vpc = defaults.buildVpc(scope, {
165+
existingVpc: props.existingVpc,
166+
defaultVpcProps: props.publicApi ? defaults.DefaultPublicPrivateVpcProps() : defaults.DefaultIsolatedVpcProps(),
167+
userVpcProps: props.vpcProps,
168+
constructVpcProps: { enableDnsHostnames: true, enableDnsSupport: true }
169+
});
170+
171+
defaults.AddAwsServiceEndpoint(scope, this.vpc, defaults.ServiceEndpointTypes.S3);
172+
173+
if (props.existingFargateServiceObject) {
174+
this.service = props.existingFargateServiceObject;
175+
// CheckFargateProps confirms that the container is provided
176+
this.container = props.existingContainerDefinitionObject!;
177+
} else {
178+
[this.service, this.container] = defaults.CreateFargateService(
179+
scope,
180+
id,
181+
this.vpc,
182+
props.clusterProps,
183+
props.ecrRepositoryArn,
184+
props.ecrImageVersion,
185+
props.fargateTaskDefinitionProps,
186+
props.containerDefinitionProps,
187+
props.fargateServiceProps
188+
);
189+
}
190+
191+
// Setup the S3 Bucket
192+
let bucket: s3.IBucket;
193+
194+
if (!props.existingBucketObj) {
195+
[this.s3Bucket, this.s3LoggingBucket] = defaults.buildS3Bucket(this, {
196+
bucketProps: props.bucketProps,
197+
loggingBucketProps: props.loggingBucketProps,
198+
logS3AccessLogs: props.logS3AccessLogs
199+
});
200+
bucket = this.s3Bucket;
201+
} else {
202+
bucket = props.existingBucketObj;
203+
}
204+
205+
this.s3BucketInterface = bucket;
206+
207+
// Add the requested or default bucket permissions
208+
if (props.bucketPermissions) {
209+
if (props.bucketPermissions.includes('Delete')) {
210+
bucket.grantDelete(this.service.taskDefinition.taskRole);
211+
}
212+
if (props.bucketPermissions.includes('Read')) {
213+
bucket.grantRead(this.service.taskDefinition.taskRole);
214+
}
215+
// Sticking with legacy v1 permissions s3:PutObject* instead of CDK v2 s3:PutObject
216+
// to prevent build failures for both versions
217+
if (props.bucketPermissions.includes('Write')) {
218+
this.service.taskDefinition.taskRole.addToPrincipalPolicy(new iam.PolicyStatement({
219+
effect: iam.Effect.ALLOW,
220+
resources: [bucket.bucketArn, `${bucket.bucketArn}/*`],
221+
actions: ['s3:DeleteObject*', 's3:PutObject*', 's3:Abort*']
222+
}));
223+
}
224+
} else {
225+
this.service.taskDefinition.taskRole.addToPrincipalPolicy(new iam.PolicyStatement({
226+
effect: iam.Effect.ALLOW,
227+
resources: [bucket.bucketArn, `${bucket.bucketArn}/*` ],
228+
actions: ['s3:GetObject*', 's3:GetBucket*', 's3:List*', 's3:DeleteObject*', 's3:PutObject*', 's3:Abort*']
229+
}));
230+
}
231+
232+
// Add environment variables
233+
const bucketArnEnvironmentVariableName = props.bucketArnEnvironmentVariableName || 'S3_BUCKET_ARN';
234+
this.container.addEnvironment(bucketArnEnvironmentVariableName, this.s3BucketInterface.bucketArn);
235+
const bucketEnvironmentVariableName = props.bucketEnvironmentVariableName || 'S3_BUCKET_NAME';
236+
this.container.addEnvironment(bucketEnvironmentVariableName, this.s3BucketInterface.bucketName);
237+
238+
}
239+
}

0 commit comments

Comments
 (0)