Skip to content

ECMAScript Module Support #1952

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
1 task done
astorm opened this issue Jan 28, 2021 · 12 comments · Fixed by #3381
Closed
1 task done

ECMAScript Module Support #1952

astorm opened this issue Jan 28, 2021 · 12 comments · Fixed by #3381
Assignees
Labels
agent-nodejs Make available for APM Agents project planning. kibana
Milestone

Comments

@astorm
Copy link
Contributor

astorm commented Jan 28, 2021

In order to detect when certain methods are called and provide timings for those method calls, the Node.js Agent uses a module named require-in-the-middle. This module "hooks" (monkey-patches, intercepts, etc.) the initial loading of Node.js CommonJS module and allows us to wrap the methods we want with our custom timing and span generation.

Node 15 (and it's LTS support partner, Node 16) mark the support of ECMAScript modules (also known as ES6 modules) as non-experimental. https://nodejs.org/docs/latest-v15.x/api/esm.html#esm_modules_ecmascript_modules

This means third part code can begin using ECMAScript modules (import) in place of CommonJS modules (require). The require-in-the-middle module does not hook these new import/ECMAScript modules. This means any normally instrumented module that uses ECMAScript modules will not be instrumented by the Node.js Agent.

We need to determine the best was to hook these new modules. (perhaps the experimental loaders, perhaps compilation/transpilation, perhaps something else).

This will be a long term effort. If folks have ideas on approaches we'd love to hear about them.

Work Around: In the meantime, users can fallback to importing modules via require. If there's an instrumented module that's providing ECMAScript only modules, or the agent is somehow not working with the new ECMAScript features, we'd like to hear about them.

Todo:

@trentm
Copy link
Member

trentm commented Aug 3, 2022

Relevant to Kibana here: elastic/kibana#106868

@nhe23
Copy link

nhe23 commented Sep 9, 2022

It would be really great to have esm support as it broadly used nowadays. Modern fullstack tooling like vite depends heavily on esm and it is either not possible or really time-consuming to convert all the generated code to cjs again. In my case I'm trying to set up apm with Sveltekit which kind of works, but in many cases I need to start transactions myself because modules cannot be monkey-patched. So I'm really looking forward to this feature.

@trentm trentm added this to the 8.8 milestone Feb 2, 2023
@hjhjdev
Copy link

hjhjdev commented Mar 8, 2023

+1 for node based backend with ts-node or others use .ts directly.

@cdavid15
Copy link

We encountered this problem yesterday. Will this be delivered in 8.8? I can see it was originally considered for 8.1, 8.2 and 8.3 previously so trying to assess whether we hang fire for 8.8 or come up with alternative solutions.

@trentm
Copy link
Member

trentm commented Mar 24, 2023

@cdavid15 No, unfortunately 8.8 won't provide a complete solution for this. Full ESM support is a big change. It will rely on Node.js loader hooks, which are currently experimental and expected to change/break. It will require significant changes to how this agent is doing its auto-instrumentation to most, if not all, supported modules.

We've been using this issue over multiple milestones to do investigatory work. Apologies that that ends up being a bit misleading when we have it on a particular milestone. We're keen to have a solution for instrumenting ESM modules, but don't have an expected or firm target timeline for it yet.

@trentm trentm modified the milestones: 8.8, 8.9 Apr 24, 2023
@trentm trentm self-assigned this May 15, 2023
@cdavid15
Copy link

cdavid15 commented Jun 2, 2023

@trentm I have been keeping an eye on this ticket and latest activity suggests things may be more positive and moving forward now.

Any update on a target timeline even if not fully confirmed for the time being?

@trentm
Copy link
Member

trentm commented Jun 2, 2023

@cdavid15 I hope to have experimental support in a release in the next month or so -- but no promises on a date. ;) You can track my progress here: #3381
So far it is going well.

@cdavid15
Copy link

cdavid15 commented Jun 7, 2023

@trentm that's great news! Will keep a close eye on it!

@trentm trentm mentioned this issue Jun 26, 2023
40 tasks
trentm added a commit that referenced this issue Jun 27, 2023
This adds initial and ECMAScript Module (ESM) support, i.e. `import ...`,
via the `--experimental-loader=elastic-apm-node/loader.mjs` node option.
This instruments a subset of modules -- more will follow in subsequent changes.

Other changes:
- Fixes a fastify instrumentation issue where the exported `fastify.errorCodes`
  was broken by instrumentation (both CJS and ESM).
- Adds a `runTestFixtures` utility that should be useful for running out of
  process instrumentation/agent tests.

Closes: #1952
Refs: #2343
@cdavid15
Copy link

Just upgraded and while following the docs to use the experimental loader I received an error as loader.mjs doesn’t exist int the published 3.48.0 package.

I downloaded the source directly from GitHub and dropped in the missing files and our APM transactions are now picked up again which is great but wanted to flag that I think the published package is invalid (unless I’ve missed something).

Will I create a new issue for this?

@trentm
Copy link
Member

trentm commented Aug 1, 2023

(Sorry for the slow reply. I was away last week.)

as loader.mjs doesn’t exist int the published 3.48.0 package.

@cdavid15 GAH! Thanks for reporting this. I am embarrassed to have missed this. I will get a fix in soon.

@cdavid15
Copy link

cdavid15 commented Aug 2, 2023

(Sorry for the slow reply. I was away last week.)

as loader.mjs doesn’t exist int the published 3.48.0 package.

@cdavid15 GAH! Thanks for reporting this. I am embarrassed to have missed this. I will get a fix in soon.

Perfect thanks @trentm! Will keep an eye out for 3.49.0!

@trentm
Copy link
Member

trentm commented Aug 3, 2023

@cdavid15 3.49.0 is out now. Thanks.

PeterEinberger pushed a commit to fpm-git/apm-agent-nodejs that referenced this issue Aug 20, 2024
This adds initial and ECMAScript Module (ESM) support, i.e. `import ...`,
via the `--experimental-loader=elastic-apm-node/loader.mjs` node option.
This instruments a subset of modules -- more will follow in subsequent changes.

Other changes:
- Fixes a fastify instrumentation issue where the exported `fastify.errorCodes`
  was broken by instrumentation (both CJS and ESM).
- Adds a `runTestFixtures` utility that should be useful for running out of
  process instrumentation/agent tests.

Closes: elastic#1952
Refs: elastic#2343
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agent-nodejs Make available for APM Agents project planning. kibana
Projects
None yet
Development

Successfully merging a pull request may close this issue.

7 participants