Skip to content

[gitpod-network-check] Introduce lambda and local runners #18

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 40 additions & 0 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: CI

on:
pull_request:
types: [ opened, edited ]
push:

jobs:
ci:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: "1.23.1"
cache: true

- name: Check formatting
run: |
cd gitpod-network-check
test -z "$(gofmt -l .)"

- name: Run tests
run: |
cd gitpod-network-check
go test -v ./...

- name: Build with GoReleaser
uses: goreleaser/goreleaser-action@v3
with:
distribution: goreleaser
version: v2.8.2
args: build --snapshot --clean
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
5 changes: 1 addition & 4 deletions .gitpod.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1 @@
image: gitpod/workspace-full

checkoutLocation: enterprise-deployment-toolkit
workspaceLocation: enterprise-deployment-toolkit/enterprise-deployment-toolkit.code-workspace
image: gitpod/workspace-full
4 changes: 4 additions & 0 deletions .goreleaser.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@ builds:
ignore:
- goos: windows
goarch: arm64
flags:
- -a
ldflags:
- -s -w -extldflags=-static
binary: gitpod-network-check

archives:
Expand Down
1 change: 0 additions & 1 deletion enterprise-deployment-toolkit.code-workspace
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@
},
"go.lintTool": "golangci-lint",
"gopls": {
"allowModfileModifications": true
},
}
}
Expand Down
2 changes: 2 additions & 0 deletions gitpod-network-check/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
gitpod-network-check
*.zip
4 changes: 4 additions & 0 deletions gitpod-network-check/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
.PHONY: build

build:
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -a -ldflags="-s -w -extldflags=-static" -o gitpod-network-check main.go
70 changes: 55 additions & 15 deletions gitpod-network-check/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,17 +44,28 @@ A CLI to check if your network setup is suitable for the installation of Gitpod.
pod-subnets: subnet-028d11dce93b8eefc, subnet-04ec8257d95c434b7,subnet-00a83550ce709f39c
https-hosts: accounts.google.com, github.com
instance-ami: # put your custom ami id here if you want to use it, otherwise it will using latest ubuntu AMI from aws
api-endpoint: # optional, put your API endpoint regional sub-domain here to test connectivity, like when the execute-api vpc endpoint is not in the same account as Gitpod
api-endpoint: # optional, put your API endpoint regional sub-domain here to test connectivity, like when the execute-api vpc endpoint is not in the same account as Gitpod
# lambda-role-arn: arn:aws:iam::123456789012:role/MyExistingLambdaRole # Optional: Use existing IAM Role for Lambda mode
# lambda-sg-id: sg-0123456789abcdef0 # Optional: Use existing Security Group for Lambda mode
```

note: if using a custom AMI, please ensure the [SSM agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/manually-install-ssm-agent-linux.html) and [curl](https://curl.se/) are both installed. We rely on SSM's [SendCommand](https://docs.aws.amazon.com/code-library/latest/ug/ssm_example_ssm_SendCommand_section.html) to test HTTPS connectivity.
**Note:** The `lambda-role-arn` and `lambda-sg-id` fields correspond to the `--lambda-role-arn` and `--lambda-sg-id` command-line flags, respectively. Setting them in the config file or via environment variables (e.g., `NTCHK_LAMBDA_ROLE_ARN`) achieves the same result.

**EC2 Mode Note:** If using a custom AMI (`instance-ami`), please ensure the [SSM agent](https://docs.aws.amazon.com/systems-manager/latest/userguide/manually-install-ssm-agent-linux.html) and [curl](https://curl.se/) are both installed. We rely on SSM's [SendCommand](https://docs.aws.amazon.com/code-library/latest/ug/ssm_example_ssm_SendCommand_section.html) to test HTTPS connectivity in EC2 mode.

2. Run the network diagnosis

To start the diagnosis, the the command: `./gitpod-network-check diagnose`
The tool supports different runners for executing the checks, specified by the `--runner` flag (`ec2`, `lambda`, `local`).

**Using EC2 Runner (Default):**

This mode launches temporary EC2 instances in your specified subnets to perform the network checks. This most closely simulates the environment where Gitpod components will run.

To start the diagnosis using the EC2 runner: `./gitpod-network-check diagnose --runner ec2` (or simply `./gitpod-network-check diagnose` as EC2 is the default).

```console
./gitpod-network-check diagnose
# Example output for EC2 runner
./gitpod-network-check diagnose --runner ec2
INFO[0000] ℹ️ Running with region `eu-central-1`, main subnet `[subnet-0ed211f14362b224f subnet-041703e62a05d2024]`, pod subnet `[subnet-075c44edead3b062f subnet-06eb311c6b92e0f29]`, hosts `[accounts.google.com https://github.com]`, ami ``, and API endpoint ``
INFO[0000] ✅ Main Subnets are valid
INFO[0000] ✅ Pod Subnets are valid
Expand Down Expand Up @@ -116,22 +127,51 @@ A CLI to check if your network setup is suitable for the installation of Gitpod.
INFO[0306] ✅ Security group 'sg-00d4a66a7840ebd67' deleted
```

**Using Lambda Runner:**

This mode uses AWS Lambda functions deployed into your specified subnets to perform the network checks. It avoids the need to launch full EC2 instances but has its own prerequisites.

* **Prerequisites for Lambda Mode:**
* **IAM Permissions:** The AWS credentials used to run `gitpod-network-check` need permissions to manage Lambda functions, IAM roles, security groups, and CloudWatch Logs. Specifically, it needs to perform actions like: `lambda:CreateFunction`, `lambda:GetFunction`, `lambda:DeleteFunction`, `lambda:InvokeFunction`, `iam:CreateRole`, `iam:GetRole`, `iam:DeleteRole`, `iam:AttachRolePolicy`, `iam:DetachRolePolicy`, `ec2:CreateSecurityGroup`, `ec2:DescribeSecurityGroups`, `ec2:DeleteSecurityGroup`, `ec2:AuthorizeSecurityGroupEgress`, `ec2:DescribeSubnets`, `logs:DeleteLogGroup`.
* **Network Connectivity:** Lambda functions running within a VPC need a route to the internet or required AWS service endpoints. This typically requires a **NAT Gateway** in your VPC or **VPC Endpoints** for all necessary services (e.g., STS, CloudWatch Logs, ECR, S3, DynamoDB, and any target HTTPS hosts). Without proper outbound connectivity, the Lambda checks will fail.

* **Running Lambda Runner:**
To start the diagnosis using the Lambda runner:
```bash
./gitpod-network-check diagnose --runner lambda
```

* **Using Existing Resources (Lambda Runner):**
If you have pre-existing IAM roles or Security Groups you want the Lambda functions to use, you can specify them using flags. This will prevent the tool from creating or deleting these specific resources.
```bash
./gitpod-network-check diagnose --runner lambda \
--lambda-role-arn arn:aws:iam::123456789012:role/MyExistingLambdaRole \
--lambda-sg-id sg-0123456789abcdef0
```

* **Example Output (Lambda Runner):**
The output will be similar to EC2 runner but will show Lambda function creation/invocation instead of EC2 instance management.

**Using Local Runner:**

This mode runs the checks directly from the machine where you execute the CLI. It's useful for basic outbound connectivity tests but **does not** accurately reflect the network environment within your AWS subnets.

To start the diagnosis using the local runner: `./gitpod-network-check diagnose --runner local`

3. Clean up after network diagnosis

Dianosis is designed to do clean-up before it finishes. However, if the process terminates unexpectedly, you may clean-up AWS resources it creates like so:
The `diagnose` command is designed to clean up the AWS resources it creates (EC2 instances, Lambda functions, IAM roles, Security Groups, CloudWatch Log groups) before it finishes. However, if the process terminates unexpectedly, you can manually trigger cleanup using the `clean` command. This command respects the `--runner` flag to clean up resources specific to that runner.

```console
./gitpod-network-check clean
INFO[0000] ✅ Main Subnets are valid
INFO[0000] ✅ Pod Subnets are valid
INFO[0000] ✅ Instances terminated
INFO[0000] Cleaning up: Waiting for 2 minutes so network interfaces are deleted
INFO[0121] ✅ Role 'GitpodNetworkCheck' deleted
INFO[0121] ✅ Instance profile deleted
INFO[0122] ✅ Security group 'sg-0a6119dcb6a564fc1' deleted
INFO[0122] ✅ Security group 'sg-07373362953212e54' deleted
```bash
# Clean up resources potentially left by the EC2 runner
./gitpod-network-check clean --runner ec2

# Clean up resources potentially left by the Lambda runner
./gitpod-network-check clean --runner lambda
```

**Note:** The `clean` command will *not* delete IAM roles or Security Groups if they were provided using the `--lambda-role-arn` or `--lambda-sg-id` flags during the `diagnose` run.

## FAQ

If the EC2 instances are timing out, or you cannot connect to them with Session Manager, be sure to add the following policies.
Expand Down
Loading
Loading