lambda layer > 60 seem to break with "Error: Cannot find module './async'"

[kitsunde]

Is there an existing issue for this?

• I have checked for existing issues https://github.com/getsentry/sentry-javascript/issues
• I have reviewed the documentation https://docs.sentry.io/
• I am using the latest SDK release https://github.com/getsentry/sentry-javascript/releases

How do you use Sentry?

Sentry Saas (sentry.io)

Which package are you using?

@sentry/serverless

SDK Version

6.19.4

Framework Version

N/A

Link to Sentry event

No response

Steps to Reproduce

Cloudformation:

Globals:
  Function:
    Layers:
      - !Sub 'arn:aws:lambda:${AWS::Region}:943013980633:layer:SentryNodeServerlessSDK:61'

In lambda:

require('@sentry/serverless');

I am in ap-southeast-1 if that matters. Basically 61, 62 fail on an async import. 63 fails on some other import.

Expected Result

Is functional.

Actual Result

{
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module './async'\nRequire stack:\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/utils/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/hub/build/dist/scope.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/hub/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/minimal/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/core/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/node/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/build/dist/awslambda.js\n- /opt/nodejs/node_modules/@sentry/serverless/build/dist/index.js\n- /var/task/onconnect.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
  "trace": [
    "Runtime.ImportModuleError: Error: Cannot find module './async'",
    "Require stack:",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/utils/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/hub/build/dist/scope.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/hub/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/minimal/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/core/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/node/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/build/dist/awslambda.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/build/dist/index.js",
    "- /var/task/onconnect.js",
    "- /var/runtime/UserFunction.js",
    "- /var/runtime/index.js",
    "    at _loadUserApp (/var/runtime/UserFunction.js:202:13)",
    "    at Object.module.exports.load (/var/runtime/UserFunction.js:242:17)",
    "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
    "    at Module._compile (internal/modules/cjs/loader.js:1085:14)",
    "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)",
    "    at Module.load (internal/modules/cjs/loader.js:950:32)",
    "    at Function.Module._load (internal/modules/cjs/loader.js:790:12)",
    "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:75:12)",
    "    at internal/main/run_main_module.js:17:47"
  ]
}

[Lms24]

Hi @kitsunde, thanks for reporting.

Without getting too much into debugging this (we're currently actively working on v7, see #4882), I have a feeling that this might be related to #4868 where we made some build and directory structure changes. I'll bring this up with the team.

I have a few more questions:

• The error message you posted happens with 61?
• Could you also post the other import error you're getting with 63?
• Does 64 still have the same issue?

I quickly did a check w.r.t. AWS lambda layer versions vs. Sentry versions.
Just to have this written down somewhere:

Lambda Layer version -> Sentry version
60 -> 6.19.4
61 -> 6.19.5
62 -> 6.19.6
63 -> 7.0.0-alpha.0
64 -> 7.0.1-alpha.1

[kitsunde]

• The error message you posted happens with 61?

I explicitly copied it from 61 and it was the same from 62. I also believe it's related to those changes #4868. In testing we have 6.19.6 in devDependencies, so it's isolated to the lambda layer.

I quickly did a check w.r.t. AWS lambda layer versions vs. Sentry versions. Just to have this written down somewhere:

Since I have you, I've asked here: #4853 if this could be populated somewhere previously because it's quite opaque.

Right now I only really get the version by switching the layer and getting it out of the Sentry variable itself. so if it could be edited into the releases here that would be nice.

I will get back to you on 63 and 64.

Additionally I was just adding a lambda to handle websocket ping responses and so importing Sentry in isolation and renaming the whole node_modules folder didn't stop the error. The only other dependency was the official lambdainsights layer.

[Lms24]

Hey @kitsunde , thanks for the information. We've discussed this internally and did some research and it seems like the change to build-awslambda-layer.js in combination with the directory structure changes is the cause. We'll fix this in the near future but probably only release it with v7 (be sure to check out our upcoming alphas).

Also, would you mind still sharing the logs from 63/64 just to be sure we didn't miss something important? Cause I think it's interesting that the error seems to be different with those alphas.

Regarding versioning differences between our Sentry SDK and Lamda layer versions, we're still discussing how to document them better.

[kitsunde]

62

{
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module './async'\nRequire stack:\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/utils/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/hub/build/dist/scope.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/hub/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/minimal/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/core/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/node/build/dist/index.js\n- /opt/nodejs/node_modules/@sentry/serverless/build/dist/awslambda.js\n- /opt/nodejs/node_modules/@sentry/serverless/build/dist/index.js\n- /var/task/ping.js\n- /var/runtime/UserFunction.js\n- /var/runtime/index.js",
  "trace": [
    "Runtime.ImportModuleError: Error: Cannot find module './async'",
    "Require stack:",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/utils/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/hub/build/dist/scope.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/hub/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/minimal/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/core/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/node/build/dist/index.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/build/dist/awslambda.js",
    "- /opt/nodejs/node_modules/@sentry/serverless/build/dist/index.js",
    "- /var/task/ping.js",
    "- /var/runtime/UserFunction.js",
    "- /var/runtime/index.js",
    "    at _loadUserApp (/var/runtime/UserFunction.js:202:13)",
    "    at Object.module.exports.load (/var/runtime/UserFunction.js:242:17)",
    "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
    "    at Module._compile (internal/modules/cjs/loader.js:1085:14)",
    "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)",
    "    at Module.load (internal/modules/cjs/loader.js:950:32)",
    "    at Function.Module._load (internal/modules/cjs/loader.js:790:12)",
    "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:75:12)",
    "    at internal/main/run_main_module.js:17:47"
  ]
}

63

{
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module '/opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/tracing/build/npm/cjs/index.js'. Please verify that the package.json has a valid \"main\" entry",
  "trace": [
    "Runtime.ImportModuleError: Error: Cannot find module '/opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/tracing/build/npm/cjs/index.js'. Please verify that the package.json has a valid \"main\" entry",
    "    at _loadUserApp (/var/runtime/UserFunction.js:202:13)",
    "    at Object.module.exports.load (/var/runtime/UserFunction.js:242:17)",
    "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
    "    at Module._compile (internal/modules/cjs/loader.js:1085:14)",
    "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)",
    "    at Module.load (internal/modules/cjs/loader.js:950:32)",
    "    at Function.Module._load (internal/modules/cjs/loader.js:790:12)",
    "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:75:12)",
    "    at internal/main/run_main_module.js:17:47"
  ]
}

64

{
  "errorType": "Runtime.ImportModuleError",
  "errorMessage": "Error: Cannot find module '/opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/tracing/build/npm/cjs/index.js'. Please verify that the package.json has a valid \"main\" entry",
  "trace": [
    "Runtime.ImportModuleError: Error: Cannot find module '/opt/nodejs/node_modules/@sentry/serverless/node_modules/@sentry/tracing/build/npm/cjs/index.js'. Please verify that the package.json has a valid \"main\" entry",
    "    at _loadUserApp (/var/runtime/UserFunction.js:202:13)",
    "    at Object.module.exports.load (/var/runtime/UserFunction.js:242:17)",
    "    at Object.<anonymous> (/var/runtime/index.js:43:30)",
    "    at Module._compile (internal/modules/cjs/loader.js:1085:14)",
    "    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)",
    "    at Module.load (internal/modules/cjs/loader.js:950:32)",
    "    at Function.Module._load (internal/modules/cjs/loader.js:790:12)",
    "    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:75:12)",
    "    at internal/main/run_main_module.js:17:47"
  ]
}

ping.js

const Sentry = require('@sentry/serverless');

Sentry.AWSLambda.init({
  sampleRate: 0,
});

async function main() {
  console.log(Sentry.VERSION)
  return { statusCode: 200, body: 'pong' };
}

exports.handler = Sentry.AWSLambda.wrapHandler(main, {
  ignoreSentryErrors: true,
});

package.json

{
  "name": "ws-ping",
  "version": "1.0.0",
  "main": "ping.js",
  "devDependencies": {
    "@sentry/serverless": "6.19.6",
    "eslint": "8.13.0",
    "eslint-config-airbnb-base": "15.0.0",
    "eslint-plugin-import": "2.26.0"
  },
  "dependencies": {},
  "scripts": {
    "test:lint": "eslint ./"
  }
}

While you're working on V7 maybe you guys want to remove 62 from https://docs.sentry.io/platforms/node/guides/aws-lambda/layer/ if it directly causes issues.

[Lms24]

Hey @kitsunde and everyone else,

we found a fix (#4982) for the problem and we just started releasing 6.19.7, so keep an eye out for it. We decided to make this a patch release in v6 as it is affecting quite a few people and it should be fixed before v7 is released.

Please make sure to update your layer to the new version once it's out (65 for ap-southeast-1).

Hope this helps and let us know if this solves your problem. Thank you for bringing it to our attention!

[dijonkitchen]

Was seeing Cannot find module '@sentry/serverless/dist/awslambda-auto' as well in our logs, could it also be related?

[Lms24]

@dijonkitchen yes that sounds familiar. Is it still happening in the latest version (6.19.7) (was released around 30 minutes ago)?

[dijonkitchen]

We turned off the lambdas that weren't working (they got auto-updated on toggling). We need to try turning them back on to see if these fixes are OK.

[vladanpaunovic]

@kitsunde, a new version is out as @Lms24 pointed out here.

With the lambda layer, I would encourage a simpler approach to save some computing power and introduce some zen.

With the Sentry lambda layer, you don't need to import Sentry (const Sentry = require('@sentry/serverless');). The Sentry lambda layer does this all for you.
We will make sure to improve how we communicate this in our docs to make it clearer.

- const Sentry = require('@sentry/serverless');

- Sentry.AWSLambda.init({
-   sampleRate: 0,
- });

- async function main() {
+ exports.handler = function(event, context, callback) {
  console.log(Sentry.VERSION)
  return { statusCode: 200, body: 'pong' };
}

- exports.handler = Sentry.AWSLambda.wrapHandler(main, {
-   ignoreSentryErrors: true,
- });

@dijonkitchen I had the same error before updating. Make sure to use the newest Sentry lambda layer. This fixed it for me.

[kitsunde]

@kitsunde, a new version is out as @Lms24 pointed out here.

With the lambda layer, I would encourage a simpler approach to save some computing power and introduce some zen.

With the Sentry lambda layer, you don't need to import Sentry (const Sentry = require('@sentry/serverless');). The Sentry lambda layer does this all for you. We will make sure to improve how we communicate this in our docs to make it clearer.

- const Sentry = require('@sentry/serverless');

- Sentry.AWSLambda.init({
-   sampleRate: 0,
- });

- async function main() {
+ exports.handler = function(event, context, callback) {
  console.log(Sentry.VERSION)
  return { statusCode: 200, body: 'pong' };
}

- exports.handler = Sentry.AWSLambda.wrapHandler(main, {
-   ignoreSentryErrors: true,
- });

@dijonkitchen I had the same error before updating. Make sure to use the newest Sentry lambda layer. This fixed it for me.

That would be a lot nicer, however as far as I'm aware without ignoreSentryErrors if we run out of tracing quota the 429 bricks our system. I feedbacked in #4799 that perhaps this should be the default? It's unclear how to pass settings beyond the basic ENV settings (sampling, dsn, release, etc).