Skip to content

Commit f930b26

Browse files
trentmPeterEinberger
trentm
authored and
PeterEinberger
committed
feat: initial ESM support (elastic#3381)
This adds initial and ECMAScript Module (ESM) support, i.e. `import ...`, via the `--experimental-loader=elastic-apm-node/loader.mjs` node option. This instruments a subset of modules -- more will follow in subsequent changes. Other changes: - Fixes a fastify instrumentation issue where the exported `fastify.errorCodes` was broken by instrumentation (both CJS and ESM). - Adds a `runTestFixtures` utility that should be useful for running out of process instrumentation/agent tests. Closes: elastic#1952 Refs: elastic#2343
1 parent 64ad548 commit f930b26

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+2006
-570
lines changed

‎.eslintrc.json

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@
2121
},
2222
"ignorePatterns": [
2323
"/*.example.js", // a pattern for uncommited local dev files to avoid linting
24+
"/*.example.mjs", // a pattern for uncommited local dev files to avoid linting
2425
"/.nyc_output",
2526
"/build",
2627
"node_modules",
@@ -40,6 +41,10 @@
4041
"/test/types/transpile/index.js",
4142
"/test/types/transpile-default/index.js",
4243
"/test_output",
43-
"tmp"
44+
"tmp",
45+
// These files use top-level await, which is *fine* for ESM files but, IIUC,
46+
// not supported by eslint until v8.
47+
"/test/instrumentation/modules/fixtures/use-fastify.mjs",
48+
"/test/instrumentation/modules/http/fixtures/use-dynamic-import.mjs"
4449
]
4550
}

‎.tav.yml

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -115,8 +115,7 @@ pg-old-node:
115115
versions: '4.0.0 || 4.5.7 || 5.2.1 || 6.4.2 || 7.18.2 || 8.0.3 || 8.1.0 || 8.2.2 || 8.3.3 || 8.4.2 || 8.5.1 || 8.6.0 || 8.7.3 || 8.8.0 || 8.9.0 || 8.10.0 || >8.10.0 <9'
116116
node: '<14'
117117
peerDependencies:
118-
- bluebird@^3.0.0
119-
- knex@^0.17.3
118+
- knex@^0.20 # latest knex that supports back to node v8
120119
commands:
121120
- node test/instrumentation/modules/pg/pg.test.js
122121
- node test/instrumentation/modules/pg/knex.test.js
@@ -129,9 +128,6 @@ pg-new-node:
129128
# Maintenance note: This should be updated for newer MAJOR.MINOR releases.
130129
versions: '8.0.3 || 8.1.0 || 8.2.2 || 8.3.3 || 8.4.2 || 8.5.1 || 8.6.0 || 8.7.3 || 8.8.0 || 8.9.0 || 8.10.0 || >8.10.0 <9'
131130
node: '>=14'
132-
peerDependencies:
133-
- bluebird@^3.0.0
134-
- knex@^0.17.3
135131
commands:
136132
- node test/instrumentation/modules/pg/pg.test.js
137133
- node test/instrumentation/modules/pg/knex.test.js

‎CHANGELOG.asciidoc

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,46 @@ Notes:
3131
[[release-notes-3.x]]
3232
=== Node.js Agent version 3.x
3333
34+
35+
==== Unreleased
36+
37+
[float]
38+
===== Breaking changes
39+
40+
[float]
41+
===== Features
42+
43+
* Initial and experimental ECMAScript Module (ESM) support.
44+
With the following invocation the APM agent will now be able to instrument
45+
modules loaded via `import`. (See the https://nodejs.org/api/esm.html#introduction[Node.js introduction to ESM].)
46+
+
47+
[source,bash]
48+
----
49+
node -r elastic-apm-node/start.js \
50+
--experimental-loader=elastic-apm-node/loader.mjs \
51+
server.mjs
52+
53+
# or
54+
55+
NODE_OPTIONS='-r elastic-apm-node/start.js --experimental-loader=elastic-apm-node/loader.mjs'
56+
node server.mjs
57+
----
58+
+
59+
The new usage requirement is the `--experimental-loader=elastic-apm-node/loader.mjs` option.
60+
This initial release only includes support for instrumenting a subset of the
61+
modules listed at <<supported-technologies>>. This set will grow in subsequent
62+
versions. Notably, ESM support does not currently work in node v20 -- only in
63+
recent versions of node v12-v18. ESM support will remain experimental while the
64+
https://nodejs.org/api/esm.html#loaders[Node.js Loaders API] is experimental.
65+
See <<esm>> for full details.
66+
67+
[float]
68+
===== Bug fixes
69+
70+
[float]
71+
===== Chores
72+
73+
3474
[[release-notes-3.47.0]]
3575
==== 3.47.0 - 2023/06/14
3676

‎TESTING.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -103,6 +103,10 @@ To run TAV tests for one or a few modules:
103103

104104
TAV=redis,ioredis npm run test:tav
105105

106+
Or, to run TAV tests for a module in Docker as they are run in CI:
107+
108+
.ci/scripts/test.sh -b "release" -t "ioredis" "18"
109+
106110
TAV tests are run in CI on commits to the "main" branch, as controlled by
107111
"[tav.yml](./.github/workflows/tav.yml)". See the [CI](#ci) section below.
108112
(TODO: TAV tests *will* be runnable on-demand for PRs, but that is awaiting

‎docs/esm.asciidoc

Lines changed: 116 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
ifdef::env-github[]
2+
NOTE: For the best reading experience,
3+
please view this documentation at https://www.elastic.co/guide/en/apm/agent/nodejs/current/esm.html[elastic.co]
4+
endif::[]
5+
6+
[[esm]]
7+
== ECMAScript module support
8+
9+
NOTE: ECMAScript module support is currently incomplete and experimental. It was added in version REPLACEME.
10+
11+
The Elastic APM Node.js agent includes _limited and experimental_ support for auto-instrumentation of https://nodejs.org/api/esm.html#modules-ecmascript-modules[ECMAScript modules] (ESM) -- i.e. modules loaded via the `import ...` statement or the `import(...)` expression. Support is based on the experimental https://nodejs.org/api/esm.html#loaders[Node.js Loaders API], which requires passing the `--experimental-loader` option to node.
12+
13+
As a first example, the APM agent can provide HTTP tracing for the following Express server:
14+
15+
[source,js]
16+
----
17+
// server.mjs
18+
import bodyParser from 'body-parser'
19+
import express from 'express'
20+
21+
const app = express()
22+
app.use(bodyParser.json())
23+
app.get('/hello/:name', function (request, reply) {
24+
reply.send({ hello: request.params.name })
25+
})
26+
27+
app.listen({ port: 3000}, () => {
28+
console.log('Server is listening. Try:\n curl -i http://localhost:3000/hello/grace')
29+
})
30+
----
31+
32+
when invoked as follows:
33+
34+
[source,bash]
35+
----
36+
export ELASTIC_APM_SERVER_URL='https://...apm...cloud.es.io:443'
37+
export ELASTIC_APM_SECRET_TOKEN='...'
38+
node -r elastic-apm-node/start.js \
39+
--experimental-loader=elastic-apm-node/loader.mjs' \
40+
node server.mjs
41+
----
42+
43+
The current ESM support is limited -- only a subset of the modules listed at <<supported-technologies>> are implemented. More will be added in subsequent releases. See below for full details.
44+
45+
The ESM limitations only affects the agent's automatic instrumentation. Other functionality -- such as metrics collection, manual instrumentation and error capture -- still work when using ES modules.
46+
47+
48+
[float]
49+
[[esm-enabling]]
50+
=== Enabling ESM auto-instrumentation
51+
52+
Enabling ESM auto-instrumentation requires starting Node.js with the `--experimental-loader=elastic-apm-node/loader.mjs` option. This can be done by passing the argument on the command line or by setting the https://nodejs.org/api/all.html#all_cli_node_optionsoptions[`NODE_OPTIONS`] environment variable.
53+
54+
[source,bash]
55+
----
56+
node --experimental-loader=elastic-apm-node/loader.mjs server.mjs
57+
58+
# or
59+
60+
NODE_OPTIONS='--experimental-loader=elastic-apm-node/loader.mjs'
61+
node server.mjs
62+
----
63+
64+
As well, the APM agent must also be separately *started* -- for example via `--require=elastic-apm-node/start.js`. See <<starting-the-agent>> for the various ways of starting the APM agent.
65+
66+
67+
[float]
68+
[[esm-compat-node]]
69+
=== Supported Node.js versions
70+
71+
Automatic instrumentation of ES modules is based on the experimental Node.js Loaders API. ESM support in the Elastic APM Node.js agent will remain *experimental* while the Loaders API is experimental.
72+
73+
ESM auto-instrumentation is only supported for Node.js versions that match *`^12.20.0 || ^14.13.1 || ^16.0.0 || ^18.1.0 <20`*. Notably, in the current APM agent version, this _excludes Node.js v20_ because of changes in the Loaders API. The behavior when using `node --experimental-loader=elastic-apm-node/loader.mjs` with earlier Node.js versions is undefined and unsupported.
74+
75+
76+
[float]
77+
[[esm-compat-modules]]
78+
=== Supported modules
79+
80+
Automatic instrumentation of ES modules is currently limited as described here. Note that the supported module version ranges often differ from those for CommonJS (i.e. `require()`) auto-instrumentation.
81+
82+
[options="header"]
83+
|=======================================================================
84+
| Module | Version | Note |
85+
| `@aws-sdk/client-s3` | >=3.15.0 <4 | |
86+
| `express` | ^4.0.0 | |
87+
| `fastify` | >=3.5.0 | |
88+
| `http` | | See <<esm-compat-node>> above. |
89+
| `https` | | See <<esm-compat-node>> above. |
90+
| `ioredis` | >=2 <6 | |
91+
| `knex` | >=0.20.0 <3 | Also, only with pg@8. |
92+
| `pg` | ^8 | |
93+
|=======================================================================
94+
95+
96+
[float]
97+
[[esm-troubleshooting]]
98+
=== Troubleshooting ESM support
99+
100+
If you see an error like the following, then you are attempting to use ESM auto-instrumentation support with too early of a version of Node.js. See <<esm-compat-node>> above.
101+
102+
[source]
103+
----
104+
file:///.../node_modules/import-in-the-middle/hook.mjs:6
105+
import { createHook } from './hook.js'
106+
^^^^^^^^^^
107+
SyntaxError: The requested module './hook.js' is expected to be of type CommonJS, which does not support named exports. CommonJS modules can be imported by importing the default export.
108+
For example:
109+
import pkg from './hook.js';
110+
const { createHook } = pkg;
111+
at ModuleJob._instantiate (internal/modules/esm/module_job.js:98:21)
112+
at async ModuleJob.run (internal/modules/esm/module_job.js:137:5)
113+
at async Loader.import (internal/modules/esm/loader.js:165:24)
114+
at async internal/process/esm_loader.js:57:9
115+
at async Object.loadESM (internal/process/esm_loader.js:67:5)
116+
----

‎docs/index.asciidoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,8 @@ include::./opentracing.asciidoc[]
4040

4141
include::./source-maps.asciidoc[]
4242

43+
include::./esm.asciidoc[]
44+
4345
include::./distributed-tracing.asciidoc[]
4446

4547
include::./message-queues.asciidoc[]

‎docs/supported-technologies.asciidoc

Lines changed: 9 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -24,48 +24,21 @@ The current APM agent version (3.x) works with Node.js versions back to v8.6. We
2424

2525
[float]
2626
[[compatibility-esm]]
27-
=== ES Modules
27+
=== ECMAScript Modules (ESM)
2828

29-
The Elastic APM Node.js agent does _not yet support automatically instrumenting
30-
https://nodejs.org/api/esm.html#modules-ecmascript-modules[ECMAScript module imports (ESM)]_, i.e. modules that are loaded via `import ...` statements. It currently only instruments https://nodejs.org/api/modules.html#modules-commonjs-modules[CommonJS module imports], i.e. modules loaded via `require(...)`.
29+
Beginning with version REPLACEME, the Elastic APM Node.js agent includes
30+
_limited and experimental_ support for instrumenting
31+
https://nodejs.org/api/esm.html#modules-ecmascript-modules[ECMAScript module imports],
32+
i.e. modules that are loaded via `import ...` statements and `import('...')` (dynamic import).
33+
See the <<esm>> document for details.
3134

32-
For example, in the following code the `http` module *is* instrumented:
33-
34-
[source,js]
35-
----
36-
require('elastic-apm-node').start(/* ... */);
37-
38-
const http = require('http'); // CommonJS require
39-
const server = http.createServer((req, res) => {
40-
res.statusCode = 200;
41-
res.end('pong');
42-
});
43-
server.listen(3000);
44-
----
45-
46-
But in the following code the `http` module *is not* instrumented:
47-
48-
[source,js]
49-
----
50-
import apm from 'elastic-apm-node/start';
51-
52-
import http from 'http'; // ESM import
53-
const server = http.createServer((req, res) => {
54-
res.statusCode = 200;
55-
res.end('pong');
56-
});
57-
server.listen(3000);
58-
----
59-
60-
However, if you are using TypeScript or JavaScript that is _compiled/translated/transpiled to CommonJS-using JavaScript_ via tools like Babel, Webpack, esbuild, etc., then using `import ...` in your source code is fine. To ensure your compiler is generating JS that uses CommonJS imports, use the following settings:
35+
Note: If you are using TypeScript or JavaScript that is _compiled/translated/transpiled to CommonJS-using JavaScript_ via tools like Babel, Webpack, esbuild, etc., then using `import ...` in your source code is fine. To ensure your compiler is generating JS that uses CommonJS imports, use the following settings:
6136

6237
- For TypeScript, use https://www.typescriptlang.org/tsconfig#module[`"module": "commonjs"` in your "tsconfig.json"] (a https://github.com/tsconfig/bases/blob/main/bases/node16.json[complete tsconfig.json example]).
6338
- For Babel, use https://babeljs.io/docs/en/babel-preset-env#modules[`"modules": "commonjs"` in your Babel config] (https://github.com/elastic/apm-agent-nodejs/blob/main/test/babel/.babelrc[for example]).
6439
- For Webpack, use `target: 'node', externalsPresets: { node: true }` in your "webpack.config.js".
6540
- For esbuild, use `--platform=node --target=node...` options to `esbuild` (https://github.com/elastic/apm-agent-nodejs/blob/main/examples/esbuild/package.json#L7[for example]).
6641

67-
The ESM limitation only affects the agent's automatic instrumentation. Other functionality -- such as metrics collection, manual instrumentation and error capture -- still works when using ES modules. Support for ES modules is planned for a future version of the APM agent.
68-
6942

7043
[float]
7144
[[elastic-stack-compatibility]]
@@ -168,7 +141,7 @@ so those should be supported as well.
168141

169142
[float]
170143
[[compatibility-better-stack-traces]]
171-
==== Better Stack Traces
144+
=== Better Stack Traces
172145

173146
The APM agent <<span-stack-trace-min-duration,can be configured>> to capture
174147
span stack traces, to show where in your code a span (e.g. for a database query)
@@ -194,7 +167,7 @@ please create a new topic in the https://discuss.elastic.co/c/apm[Elastic APM di
194167

195168
[float]
196169
[[compatibility-continuity]]
197-
==== Continuity
170+
=== Continuity
198171

199172
The Elastic APM agent monitors async operations in your Node.js application to maintain awareness of which request is the active request at any given time.
200173
Certain modules can interfere with this monitoring if not handled properly.

‎lib/instrumentation/http-shared.js

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -181,8 +181,8 @@ function getSafeHost (req) {
181181
exports.traceOutgoingRequest = function (agent, moduleName, method) {
182182
var ins = agent._instrumentation
183183

184-
return function (orig) {
185-
return function (input, options, cb) {
184+
return function wrapHttpRequest (orig) {
185+
return function wrappedHttpRequest (input, options, cb) {
186186
const parentRunContext = ins.currRunContext()
187187
var span = ins.createSpan(null, 'external', 'http', { exitSpan: true })
188188
var id = span && span.transaction.id

0 commit comments

Comments
 (0)