Cannot configure ECS load balancer to use HTTP instead of HTTPS

[sjl2]

Description

While attempting to use the features in this comment (PR: #589) with regards to the ecs-plugin, I've run into an AWS 400 creating an ELB connecting to a port configured for http because I don't provide a certificate.

A certificate must be specified for HTTPS listeners (Service: AmazonElasticLoadBalancing; Status Code: 400; Error Code: ValidationError; Request ID: 01ee4ec3-de4c-4805-a903-0cd413067338; Proxy: null)

Upon further investigation, this is because the CloudFormation stack is being configured for https despite attempting to make it http.

Steps to reproduce the issue:

1. Configure a service's ports as such.

   ports:
     - target: 8000
       published: 8000
       protocol: tcp
       x-aws-protocol: http     

2. Run docker --context <ecs-context> compose up with a context for ECS.

Describe the results you received:
The command cleaned-up the initial deploy and exited with the following error.

A certificate must be specified for HTTPS listeners (Service: AmazonElasticLoadBalancing; Status Code: 400; Error Code: ValidationError; Request ID: 01ee4ec3-de4c-4805-a903-0cd413067338; Proxy: null)

The following is included in my exported cloud formation stack:

{
    "ApiTCP8000Listener": {
      "Properties": {
        "DefaultActions": [
          {
            "ForwardConfig": {
              "TargetGroups": [
                {
                  "TargetGroupArn": {
                    "Ref": "ApiTCP8000TargetGroup"
                  }
                }
              ]
            },
            "Type": "forward"
          }
        ],
        "LoadBalancerArn": {
          "Fn::If": [
            "CreateLoadBalancer",
            {
              "Ref": "DemoLoadBalancer"
            },
            {
              "Ref": "ParameterLoadBalancerARN"
            }
          ]
        },
        "Port": 8000,
        "Protocol": "HTTPS"
      },
      "Type": "AWS::ElasticLoadBalancingV2::Listener"
    },

    "ApiTCP8000TargetGroup": {
      "Properties": {
        "Port": 8000,
        "Protocol": "HTTPS",
        "Tags": [
          {
            "Key": "com.docker.compose.project",
            "Value": "demo"
          }
        ],
        "TargetType": "ip",
        "VpcId": {
          "Ref": "ParameterVPCId"
        }
      },
      "Type": "AWS::ElasticLoadBalancingV2::TargetGroup"
    },

}

However, I would expect that protocol to have been HTTP.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client: Docker Engine - Community
 Azure integration  0.1.15
 Version:           19.03.13-beta2
 API version:       1.40
 Go version:        go1.13.14
 Git commit:        ff3fbc9d55
 Built:             Mon Aug  3 14:58:48 2020
 OS/Arch:           darwin/amd64
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13-beta2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.14
  Git commit:       ff3fbc9d55
  Built:            Mon Aug  3 15:06:50 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker context show:
You can also run docker context inspect context-name to give us more details but don't forget to remove sensitive content.

[
    {
        "Name": "production",
        "Metadata": {
            "Description": "us-west-1",
            "Type": "ecs"
        },
        "Endpoints": {
            "docker": {
                "SkipTLSVerify": false
            },
            "ecs": {
                "Profile": "production",
                "Region": "us-west-1"
            }
        },
        "TLSMaterial": {},
        "Storage": {
            "MetadataPath": "<censored>",
            "TLSPath": "<censored>"
        }
    }
]

Output of docker info:

Client:
 Debug Mode: false
 Plugins:
  app: Docker Application (Docker Inc., v0.8.0)
  buildx: Build with BuildKit (Docker Inc., v0.3.1-tp-docker)
  scan: Docker Scan (Docker Inc., v0.3.3)

Server:
 Containers: 7
  Running: 3
  Paused: 0
  Stopped: 4
 Images: 79
 Server Version: 19.03.13-beta2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 4.19.76-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 4
 Total Memory: 5.811GiB
 Name: docker-desktop
 ID: SHAU:LFXX:BJMW:A3DN:NSWX:QGTX:QTXU:FTGP:B3IW:ZE56:4NLX:4JAU
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: gateway.docker.internal:3128
 HTTPS Proxy: gateway.docker.internal:3129
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: true
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

Additional environment details (AWS ECS, Azure ACI, local, etc.):

Using new features of the ECS Plugin to deploy with Docker Compose AWS ECS.

It may also be worth noting I tried replicating the tests found here, but got errors saying that IPProtocol required.

[flaviostutz]

Guys, is this still happening. I've just run docker compose with ECS with the following

    ports:
      - target: 5000
        published: 5000
        protocol: tcp
        x-aws-protocol: http

And it worked well. No certificate issues (the LB was created just with plain ports).

[sjl2]

Glad to hear! I'll try again and see if what happens.

[ndeloof]

Cloudformation template overlays have been introduce to support this requirement in a flexible way, see example on https://deploy-preview-12080--docsdocker.netlify.app//cloud/ecs-integration/#setting-ssl-termination-by-load-balancer