Skip to content

feat(aws-fargate-s3): Created new construct #591

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Feb 19, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
lib/*.js
test/*.js
*.d.ts
coverage
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
lib/*.js
test/*.js
*.js.map
*.d.ts
node_modules
*.generated.ts
dist
.jsii

.LAST_BUILD
.nyc_output
coverage
.nycrc
.LAST_PACKAGE
*.snk
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# Exclude typescript source and config
*.ts
tsconfig.json
coverage
.nyc_output
*.tgz
*.snk
*.tsbuildinfo

# Include javascript files and typescript declarations
!*.js
!*.d.ts

# Exclude jsii outdir
dist

# Include .jsii
!.jsii

# Include .jsii
!.jsii
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# aws-fargate-s3 module
<!--BEGIN STABILITY BANNER-->

---

![Stability: Experimental](https://img.shields.io/badge/stability-Experimental-important.svg?style=for-the-badge)

> All classes are under active development and subject to non-backward compatible changes or removal in any
> future version. These are not subject to the [Semantic Versioning](https://semver.org/) model.
> This means that while you may use them, you may need to update your source code when upgrading to a newer version of this package.

---
<!--END STABILITY BANNER-->

| **Reference Documentation**:| <span style="font-weight: normal">https://docs.aws.amazon.com/solutions/latest/constructs/</span>|
|:-------------|:-------------|
<div style="height:8px"></div>

| **Language** | **Package** |
|:-------------|-----------------|
|![Python Logo](https://docs.aws.amazon.com/cdk/api/latest/img/python32.png) Python|`aws_solutions_constructs.aws_fargate_s3`|
|![Typescript Logo](https://docs.aws.amazon.com/cdk/api/latest/img/typescript32.png) Typescript|`@aws-solutions-constructs/aws-fargate-s3`|
|![Java Logo](https://docs.aws.amazon.com/cdk/api/latest/img/java32.png) Java|`software.amazon.awsconstructs.services.fargates3`|

This AWS Solutions Construct implements an AWS Fargate service that can write/read to an Amazon S3 Bucket

Here is a minimal deployable pattern definition in Typescript:

``` typescript
import { FargateToS3, FargateToS3Props } from '@aws-solutions-constructs/aws-fargate-s3';

const props: FargateToS3Props = {
publicApi: true,
ecrRepositoryArn: "arn of a repo in ECR in your account",
});

new FargateToS3(stack, 'test-construct', props);
```

## Pattern Construct Props

| **Name** | **Type** | **Description** |
|:-------------|:----------------|-----------------|
| publicApi | boolean | Whether the construct is deploying a private or public API. This has implications for the VPC and ALB. |
| vpcProps? | [ec2.VpcProps](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ec2.VpcProps.html) | Optional custom properties for a VPC the construct will create. This VPC will be used by the new ALB and any Private Hosted Zone the construct creates (that's why loadBalancerProps and privateHostedZoneProps can't include a VPC). Providing both this and existingVpc is an error. |
| existingVpc? | [ec2.IVpc](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ec2.IVpc.html) | An existing VPC in which to deploy the construct. Providing both this and vpcProps is an error. If the client provides an existing load balancer and/or existing Private Hosted Zone, those constructs must exist in this VPC. |
| clusterProps? | [ecs.ClusterProps](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.ClusterProps.html) | Optional properties to create a new ECS cluster. To provide an existing cluster, use the cluster attribute of fargateServiceProps. |
| ecrRepositoryArn? | string | The arn of an ECR Repository containing the image to use to generate the containers. Either this or the image property of containerDefinitionProps must be provided. format: arn:aws:ecr:*region*:*account number*:repository/*Repository Name* |
| ecrImageVersion? | string | The version of the image to use from the repository. Defaults to 'Latest' |
| containerDefinitionProps? | [ecs.ContainerDefinitionProps \| any](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.ContainerDefinitionProps.html) | Optional props to define the container created for the Fargate Service (defaults found in fargate-defaults.ts) |
| fargateTaskDefinitionProps? | [ecs.FargateTaskDefinitionProps \| any](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.FargateTaskDefinitionProps.html) | Optional props to define the Fargate Task Definition for this construct (defaults found in fargate-defaults.ts) |
| fargateServiceProps? | [ecs.FargateServiceProps \| any](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.FargateServiceProps.html) | Optional values to override default Fargate Task definition properties (fargate-defaults.ts). The construct will default to launching the service is the most isolated subnets available (precedence: Isolated, Private and Public). Override those and other defaults here. |
| existingFargateServiceObject? | [ecs.FargateService](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.FargateService.html) | A Fargate Service already instantiated (probably by another Solutions Construct). If this is specified, then no props defining a new service can be provided, including: existingImageObject, ecrImageVersion, containerDefintionProps, fargateTaskDefinitionProps, ecrRepositoryArn, fargateServiceProps, clusterProps, existingClusterInterface |
| existingContainerDefinitionObject? | [ecs.ContainerDefinition](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.ContainerDefinition.html) | A container definition already instantiated as part of a Fargate service. This must be the container in the existingFargateServiceObject |
|existingBucketInterface?|[`s3.IBucket`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.IBucket.html)|Existing S3 Bucket interface. Providing this property and `bucketProps` results in an error.|
|bucketProps?|[`s3.BucketProps`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.BucketProps.html)|Optional user provided props to override the default props for the S3 Bucket.|
|loggingBucketProps?|[`s3.BucketProps`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.BucketProps.html)|Optional user provided props to override the default props for the S3 Logging Bucket.|
|logS3AccessLogs?| boolean|Whether to turn on Access Logging for the S3 bucket. Creates an S3 bucket with associated storage costs for the logs. Enabling Access Logging is a best practice. default - true|
|bucketPermissions?|`string[]`|Optional bucket permissions to grant to the Fargate service. One or more of the following may be specified: `Delete`, `Read`, and `Write`. Default is `ReadWrite` which includes `[s3:GetObject*, s3:GetBucket*, s3:List*, s3:DeleteObject*, s3:PutObject*, s3:Abort*]`.|
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to change this to Default is 'Read' and 'Write' as we don't want to give the impression that ReadWrite is not deprecated. I like that we don't list it here - I assume the code still allows it for backwards compatibility.

If the code looks good I can approve this now and we can make a quick inline change of this comment later.

|bucketArnEnvironmentVariableName?|string|Optional Name for the S3 bucket arn environment variable set for the container.|
|bucketEnvironmentVariableName?|string|Optional Name for the S3 bucket name environment variable set for the container.|

## Pattern Properties

| **Name** | **Type** | **Description** |
|:-------------|:----------------|-----------------|
| vpc | [ec2.IVpc](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-ec2.IVpc.html) | The VPC used by the construct (whether created by the construct or providedb by the client) |
| service | [ecs.FargateService](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.FargateService.html) | The AWS Fargate service used by this construct (whether created by this construct or passed to this construct at initialization) |
| container | [ecs.ContainerDefinition](https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-ecs.ContainerDefinition.html) | The container associated with the AWS Fargate service in the service property. |
| s3Bucket? |[s3.IBucket](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.IBucket.html)|Returns an instance of s3.Bucket created by the construct|
| s3BucketInterface |[`s3.IBucket`](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.IBucket.html)|Returns an instance of s3.IBucket created by the construct|
| s3LoggingBucket? | [s3.Bucket](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-s3.Bucket.html)|Returns an instance of s3.Bucket created by the construct|

## Default settings

Out of the box implementation of the Construct without any override will set the following defaults:

### AWS Fargate Service
* Sets up an AWS Fargate service
* Uses the existing service if provided
* Creates a new service if none provided.
* Service will run in isolated subnets if available, then private subnets if available and finally public subnets
* Adds environment variables to the container with the ARN and Name of the S3 Bucket
* Add permissions to the container IAM role allowing it to publish to the S3 Bucket

### Amazon S3 Bucket
* Sets up an Amazon S3 Bucket
* Uses an existing bucket if one is provided, otherwise creates a new one
* Adds an Interface Endpoint to the VPC for S3 (the service by default runs in Isolated or Private subnets)

## Architecture
![Architecture Diagram](architecture.png)

***
&copy; Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,239 @@
/**
* Copyright 2022 Amazon.com, Inc. or its affiliates. All Rights Reserved.
*
* Licensed under the Apache License, Version 2.0 (the "License"). You may not use this file except in compliance
* with the License. A copy of the License is located at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* or in the 'license' file accompanying this file. This file is distributed on an 'AS IS' BASIS, WITHOUT WARRANTIES
* OR CONDITIONS OF ANY KIND, express or implied. See the License for the specific language governing permissions
* and limitations under the License.
*/

import * as ec2 from "@aws-cdk/aws-ec2";
import * as s3 from "@aws-cdk/aws-s3";
// Note: To ensure CDKv2 compatibility, keep the import statement for Construct separate
import { Construct } from "@aws-cdk/core";
import * as defaults from "@aws-solutions-constructs/core";
import * as ecs from "@aws-cdk/aws-ecs";
import * as iam from "@aws-cdk/aws-iam";

export interface FargateToS3Props {
/**
* Optional custom properties for a VPC the construct will create. This VPC will
* be used by the new Fargate service the construct creates (that's
* why targetGroupProps can't include a VPC). Providing
* both this and existingVpc is an error. An S3 Interface
* endpoint will be included in this VPC.
*
* @default - none
*/
readonly vpcProps?: ec2.VpcProps;
/**
* An existing VPC in which to deploy the construct. Providing both this and
* vpcProps is an error. If the client provides an existing Fargate service,
* this value must be the VPC where the service is running. An S3 Interface
* endpoint will be added to this VPC.
*
* @default - none
*/
readonly existingVpc?: ec2.IVpc;
/**
* Whether the construct is deploying a private or public API. This has implications for the VPC deployed
* by this construct.
*
* @default - none
*/
readonly publicApi: boolean;
/**
* Optional properties to create a new ECS cluster
*/
readonly clusterProps?: ecs.ClusterProps;
/**
* The arn of an ECR Repository containing the image to use
* to generate the containers
*
* format:
* arn:aws:ecr:[region]:[account number]:repository/[Repository Name]
*/
readonly ecrRepositoryArn?: string;
/**
* The version of the image to use from the repository
*
* @default - 'latest'
*/
readonly ecrImageVersion?: string;
/*
* Optional props to define the container created for the Fargate Service
*
* defaults - fargate-defaults.ts
*/
readonly containerDefinitionProps?: ecs.ContainerDefinitionProps | any;
/*
* Optional props to define the Fargate Task Definition for this construct
*
* defaults - fargate-defaults.ts
*/
readonly fargateTaskDefinitionProps?: ecs.FargateTaskDefinitionProps | any;
/**
* Optional values to override default Fargate Task definition properties
* (fargate-defaults.ts). The construct will default to launching the service
* is the most isolated subnets available (precedence: Isolated, Private and
* Public). Override those and other defaults here.
*
* defaults - fargate-defaults.ts
*/
readonly fargateServiceProps?: ecs.FargateServiceProps | any;
/**
* A Fargate Service already instantiated (probably by another Solutions Construct). If
* this is specified, then no props defining a new service can be provided, including:
* existingImageObject, ecrImageVersion, containerDefintionProps, fargateTaskDefinitionProps,
* ecrRepositoryArn, fargateServiceProps, clusterProps, existingClusterInterface. If this value
* is provided, then existingContainerDefinitionObject must be provided as well.
*
* @default - none
*/
readonly existingFargateServiceObject?: ecs.FargateService;
/**
* Existing instance of S3 Bucket object, providing both this and `bucketProps` will cause an error.
*
* @default - None
*/
readonly existingBucketObj?: s3.IBucket;
/**
* Optional user provided props to override the default props for the S3 Bucket.
*
* @default - Default props are used
*/
readonly bucketProps?: s3.BucketProps;
/**
* Optional user provided props to override the default props for the S3 Logging Bucket.
*
* @default - Default props are used
*/
readonly loggingBucketProps?: s3.BucketProps
/**
* Whether to turn on Access Logs for the S3 bucket with the associated storage costs.
* Enabling Access Logging is a best practice.
*
* @default - true
*/
readonly logS3AccessLogs?: boolean;
/**
* Optional bucket permissions to grant to the Fargate service.
* One or more of the following may be specified: "Delete", "Put", "Read", "ReadWrite", "Write".
*
* @default - Read/write access is given to the Fargate service if no value is specified.
*/
readonly bucketPermissions?: string[];
/**
* Optional Name for the S3 bucket arn environment variable set for the container.
*
* @default - None
*/
readonly bucketArnEnvironmentVariableName?: string;
/**
* Optional Name for the S3 bucket name environment variable set for the container.
*
* @default - None
*/
readonly bucketEnvironmentVariableName?: string;
/*
* A container definition already instantiated as part of a Fargate service. This must
* be the container in the existingFargateServiceObject.
*
* @default - None
*/
readonly existingContainerDefinitionObject?: ecs.ContainerDefinition;
}

export class FargateToS3 extends Construct {
public readonly vpc: ec2.IVpc;
public readonly service: ecs.FargateService;
public readonly container: ecs.ContainerDefinition;
public readonly s3BucketInterface: s3.IBucket;
public readonly s3Bucket?: s3.Bucket;
public readonly s3LoggingBucket?: s3.Bucket;

constructor(scope: Construct, id: string, props: FargateToS3Props) {
super(scope, id);
defaults.CheckProps(props);
defaults.CheckFargateProps(props);

this.vpc = defaults.buildVpc(scope, {
existingVpc: props.existingVpc,
defaultVpcProps: props.publicApi ? defaults.DefaultPublicPrivateVpcProps() : defaults.DefaultIsolatedVpcProps(),
userVpcProps: props.vpcProps,
constructVpcProps: { enableDnsHostnames: true, enableDnsSupport: true }
});

defaults.AddAwsServiceEndpoint(scope, this.vpc, defaults.ServiceEndpointTypes.S3);

if (props.existingFargateServiceObject) {
this.service = props.existingFargateServiceObject;
// CheckFargateProps confirms that the container is provided
this.container = props.existingContainerDefinitionObject!;
} else {
[this.service, this.container] = defaults.CreateFargateService(
scope,
id,
this.vpc,
props.clusterProps,
props.ecrRepositoryArn,
props.ecrImageVersion,
props.fargateTaskDefinitionProps,
props.containerDefinitionProps,
props.fargateServiceProps
);
}

// Setup the S3 Bucket
let bucket: s3.IBucket;

if (!props.existingBucketObj) {
[this.s3Bucket, this.s3LoggingBucket] = defaults.buildS3Bucket(this, {
bucketProps: props.bucketProps,
loggingBucketProps: props.loggingBucketProps,
logS3AccessLogs: props.logS3AccessLogs
});
bucket = this.s3Bucket;
} else {
bucket = props.existingBucketObj;
}

this.s3BucketInterface = bucket;

// Add the requested or default bucket permissions
if (props.bucketPermissions) {
if (props.bucketPermissions.includes('Delete')) {
bucket.grantDelete(this.service.taskDefinition.taskRole);
}
if (props.bucketPermissions.includes('Read')) {
bucket.grantRead(this.service.taskDefinition.taskRole);
}
// Sticking with legacy v1 permissions s3:PutObject* instead of CDK v2 s3:PutObject
// to prevent build failures for both versions
if (props.bucketPermissions.includes('Write')) {
this.service.taskDefinition.taskRole.addToPrincipalPolicy(new iam.PolicyStatement({
effect: iam.Effect.ALLOW,
resources: [bucket.bucketArn, `${bucket.bucketArn}/*`],
actions: ['s3:DeleteObject*', 's3:PutObject*', 's3:Abort*']
}));
}
} else {
this.service.taskDefinition.taskRole.addToPrincipalPolicy(new iam.PolicyStatement({
effect: iam.Effect.ALLOW,
resources: [bucket.bucketArn, `${bucket.bucketArn}/*` ],
actions: ['s3:GetObject*', 's3:GetBucket*', 's3:List*', 's3:DeleteObject*', 's3:PutObject*', 's3:Abort*']
}));
}

// Add environment variables
const bucketArnEnvironmentVariableName = props.bucketArnEnvironmentVariableName || 'S3_BUCKET_ARN';
this.container.addEnvironment(bucketArnEnvironmentVariableName, this.s3BucketInterface.bucketArn);
const bucketEnvironmentVariableName = props.bucketEnvironmentVariableName || 'S3_BUCKET_NAME';
this.container.addEnvironment(bucketEnvironmentVariableName, this.s3BucketInterface.bucketName);

}
}
Loading