AWS CDK Toolkit Library - Developer Preview

[mrgrain]

The AWS Cloud Development Kit (AWS CDK) Toolkit Library enables you to perform CDK actions requiring programmatic access on AWS. You can use the AWS CDK Toolkit Library to implement actions such as bootstrapping, synthesizing, and deploying through code rather than command-line interface (CLI) commands. With this library, you can create custom tools, build specialized CLI applications, and integrate CDK programmatic access capabilities into your development workflows

📜 Reference documentation: https://docs.aws.amazon.com/cdk/api/toolkit-lib/
📦 npm package: https://www.npmjs.com/package/@aws-cdk/toolkit-lib

---

We are excited to launch the CDK Toolkit Library in Developer Preview. We want to hear from you and appreciate all feedback!

Please use this discussion for 🗣️ general feedback, 💡 ideas and 🆕 feature requests

🐛 For bug reports, please create a new issue

[dreamorosi]

Just wanted to say that I am really excited about this release. I have just finished migrating our integration test harness from @aws-cdk/cli-lib-alpha to this new toolkit and everything works with no issues.

I have opened two issues related to improvement I think could benefit the toolkit:

• @aws-cdk/toolkit-lib: unable to destroy stack without custom ioHost #157
• @aws-cdk/toolkit-lib: include NodejsFunction esbuild logs in ioHost notifications #158

other than that, great work 👏

[kyptov]

Hi, decided to try this tool. We have two accounts in two regions, one for prod and one for stage. Previously we used CDK_DEFAULT_REGION and CDK_DEFAULT_ACCOUNT to use in our stacks. How to get this values in toolkit?

[mrgrain]

That's right CDK_DEFAULT_REGION & CDK_DEFAULT_ACCOUNT should be available to a CDK app no matter if you are using cdk deploy and await toolkit.deploy(cx).

Are you saying this isn't work for await toolkit.deploy(cx)?

[kyptov]

We are using synth and deploy with CloudFormation. With cdk synth there is CDK_DEFAULT_* env, but with toolkit.synth() there is no.

Here is a reproducable snippet:

class MyStack extends Stack {
  constructor(scope: Construct, props?: StackProps) {
    super(scope, 'MyStack', props)

    StringParameter.valueFromLookup(this, 'test-param')
  }
}

const cdk = new Toolkit()

const cx = await cdk.fromAssemblyBuilder(async () => {
  const app = new App({})

  new MyStack(app, {
    // env: { account, region }, // if uncomment and provide values - synth is working
    synthesizer: new DefaultStackSynthesizer()
  })

  return app.synth()
})

await cdk.synth(cx)

This code will fail, but if you uncomment env in MyStack with valid values than it works.

The fromAssemblyBuilder method calls the callback with AssemblyBuilderProps, which includes context. This was the first place I expected CDK_DEFAULT_* values.

[kyptov]

Oh, my bad. There is process.env.CDK_DEFAULT_* inside the callback. Sorry.

[mrgrain]

I cannot reproduce this, the code fails for me regardless if I use @aws-cdk/toolkit-lib or aws-cdk with the following error:

Error: Cannot retrieve value from context provider ssm since account/region are not specified at the stack level. Configure "env" with an account and region when you define your stack.

However if I adapt either version to be:

  const account = process.env.CDK_DEFAULT_ACCOUNT;
  const region = process.env.CDK_DEFAULT_REGION;
  new MyStack(app, {
    env: { account, region },
    synthesizer: new DefaultStackSynthesizer(),
  });

it works for both @aws-cdk/toolkit-lib and aws-cdk.

[mrgrain]

Note that CDK_DEFAULT_REGION & CDK_DEFAULT_ACCOUNT env variables are only available if credentials can be resolved. Also when using fromAssemblyBuilder() they are only available inside the builder closure.

[AllanOricil]

thank you for creating this :)

[mrgrain]

Glad you like it. If you have any feedback, please let us know.

[AllanOricil]

@mrgrain there is a typo in this source code https://docs.aws.amazon.com/cdk/api/toolkit-lib/#how-to-handle-errors

The variable passed to cdk.deploy method should be cx but the source code is displaying cloudAssembly

[cmaster11]

@mrgrain, this is one of the best things you could have ever done about the CDK because it adds so much more programmaticability around all processes 🙏🏻

I suppose this is not yet a bug, but it is more of a possible optimization. When running the diff action, the result lacks information about the number of stacks with changes, despite the toolkit previously providing this information via IoHost.

The CDK CLI uses its formatter to calculate the diff, but other programmatic implementations can benefit from this precalculated info.

At https://github.com/aws/aws-cdk-cli/blob/52457cfc25dce58e8b3d1da8ff20c87cfd336034/packages/%40aws-cdk/toolkit-lib/lib/toolkit/toolkit.ts#L368C1-L371C8 we have the diff span end, which contains the formatted diff, but could also include the number of stacks with diffs. To figure out the number of stacks with diffs, you'd have to parse the log span message or re-do the whole calculation yourself.

I think we can benefit from:

1. Providing access to the currently private https://github.com/aws/aws-cdk-cli/blob/main/packages/%40aws-cdk/toolkit-lib/lib/api/io/private/messages.ts#L527 IO and SPAN enums, which would allow us to extract this info from the IoHost messages without having to hardcode the various IO codes (like CDK_TOOLKIT_I4001) in the host code.
2. Expose the number of stacks with differences in the log span
3. Expose the number of changed stacks info also as part of the diff function result, maybe as a boolean inside each stack result object (an overall hasChanges: true/false for each stack)

And one additional note, just reading the code at

aws-cdk-cli/packages/@aws-cdk/toolkit-lib/lib/toolkit/toolkit.ts

Line 363 in 52457cf

diffs = diff.numStacksWithChanges;

, looks like there is an overwriting of the diffs var. Whether it's expected because of how the whole flow works, it seems odd inside this for cycle. Maybe a bug, maybe expected, reporting just in case.

Diff log:

// Here we have the stack changes count, but it is provided in the message string only
[03:39:00.242] INFO (CDK/44572): ✨ Number of stacks with differences: 0 
    code: "CDK_TOOLKIT_I4001"
    data: {
      "duration": 9383,
      "formattedSecurityDiff": "",
      "formattedStackDiff": "Stack \u001b[1mExample-Stack\u001b[22m\n\u001b[32mThere were no differences\u001b[39m\n\u001b[32m\u001b[39m"
    }
    span: "055e1fcb-9f57-4f7f-b88b-e830883a8c1b"
    action: "diff"

// This is the result of the `.diff` command
// This needs to be re-parsed to figure out if there have been diffs
[03:39:00.243] INFO (44572): Diff result
    diffResult: {
      "Example-Stack": {
        "conditions": {
          "diffs": {}
        },
        "mappings": {
          "diffs": {}
        },
        "metadata": {
          "diffs": {}
        },
        "outputs": {
          "diffs": {}
        },
        "parameters": {
          "diffs": {}
        },
        "resources": {
          "diffs": {}
        },
        "unknown": {
          "diffs": {}
        },
        "iamChanges": {
          "statements": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "managedPolicies": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "ssoPermissionSets": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "ssoAssignments": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "ssoInstanceACAConfigs": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          }
        },
        "securityGroupChanges": {
          "ingress": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          },
          "egress": {
            "additions": [],
            "removals": [],
            "oldElements": [],
            "newElements": []
          }
        }
      }
    }

[mrgrain]

Awesome, glad you like it!

Would you mind opening a feature request for the additional info please? And if there's any other info you're missing you can just open an issue right away.

I'll look into diff variable being overwritten. 👀

[cmaster11]

Thanks! Opened at #494