Release separate parts of the whole SDK to npm?

[jakubzitny]

Hey there.

Have you ever thought about / do you think you could / would it be possible to release separate working parts of the whole SDK, possibly configurable, e.g. separate AWS.S3, to NPM?

Similarly as it is possible to Build your own version of the AWS SDK for JavaScript at amazonaws.com, it would be cool to have it on NPM and programatically just connect the separate parts of the whole SDK. The whole SDK is quite large and most of the programmers are not using most of it anyway.

Would you be interested in collaboration on this?

[chrisradek]

Today, the bulk of the SDK is in the part that's shared among all the services. Each service increases the size by a range of about ~8KB - 40KB, which can certainly add up.

Splitting up the SDK is a cool idea, and we're open to pull requests (realizing this would require the creation of multiple new packages.) I think to really take advantage of some tools' ability to perform "tree-shaking" to further reduce the footprint, the core of the SDK would need to be refactored as well, which would likely introduce breaking changes.

Assuming you're using node.js, can you share your use-case for wanting to reduce the SDK's footprint? There are definitely some very valid reasons (less space on IoT devices, faster Lambda cold boots when using a custom SDK), I just want a better feel for where the demand is at in the node space.

[jakubzitny]

Our use-case is not that space-sensitive as IoTs or Lambdas, but we're counting each MB as well. We have an Electron app (with Node included) and we're uploading files to S3.

[AdityaManohar]

@jakubzitny looks like this landed in v2.6.1 of the SDK.

[jakubzitny]

@AdityaManohar really? Can you link related MR, commit, docs page or something? I looked into Readme and Changelog and couldn't find anything. Thanks.

[AdityaManohar]

@jakubzitny Here are the relevant PRs: #1123, #1126. The CHANGELOG does seem to have an entry for v2.6.1 and v2.6.0, though I think this may be missing some documentation.

@chrisradek
Do you know if there is any public documentation for this update?

[chrisradek]

@jakubzitny
#1123 added SDK support for bundling the SDK with webpack and browserify. You can now also require individual services, and if you're using one of these tools to create a bundle, they will only include those services you've imported.

We didn't create new npm packages for all of the services, but we did enable importing individual services when bundling your code with a 3rd party tool.

Are you using the AWS SDK from a renderer process or the main process?

We're working on improving our docs around this now, but are happy to answer questions here in the meantime.

[jakubzitny]

@chrisradek Sorry for late response, thanks for the info and for implementing this.

I don't really think it matters, but if it's any help to you, we ran this from renderer and we've alse been thinking about creating a special hidden BrowserWindow with this. Using webworkers or calling it from main process is also an option.

[jeskew]

#1391 was meant to automatically close #1039, not this issue.

[mechanicals]

@chrisradek Although you said that I could Import each service singularly.
I am trying to use AWS Javascript SDK, but I am getting some errors.
I am using the latest version of AWS SDK & using Webpack to build.

I am not able to use

import S3 from 'aws-sdk/clients/s3';
gives me S3 undefined.

Interestingly this works perfectly

import AWS from 'aws-sdk';
const S3 = new AWS.S3();
But I only want to import S3 not the whole AWS SDK. Any ideas what am i doing wrong.

[ghost]

I really wish all the ways of importing and using this SDK were documented in one single place, including small but complete examples.

[cantuket]

Googles doing it right...

const { Storage } = require('@google-cloud/storage');

Ironically the aws-sdk is what put us over the 262144000 bytes limit on Lambda after replacing S3 with GCStorage.

[patgoley]

@cantuket The Lambda execution environment comes with the aws-sdk pre-installed, so no need to include it in your bundle. You can install it in devDependencies to test locally.
See here:
https://docs.aws.amazon.com/lambda/latest/dg/current-supported-versions.html

[srchase]

@cantuket

Thanks for pointing that out.

Also, V3 of the SDK, now in Developer Preview, has been modularized solving for the originally requested issue here.

[cantuket]

@patgoley Oh wow, that's game changing for us since we're trying to cram multiple binaries on there. Thanks for pointing this out! Don't how we didn't figure that out by now :/

@srchase That's great to hear for use in the rest of our environments! Love the typescript decision as well.

[himharsh1997]

In my lambda function, the total size of zip is 76.48MB(created on sls deploy function --function functionname) and aws-sdk is taking 57.3MB out of 76.48MB total. This big size is unabling me to deploy that function and also to add move feature to same package. if aws is putting such limit on functionUpdate operation they should look into their package also.

[ajredniwja]

Please check out js-sdk-v3 which has modular packages. Closing this issue now.