Skip to content

tests: stop testing 'express-graphql' instrumentation #3304

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
May 3, 2023
Merged

tests: stop testing 'express-graphql' instrumentation #3304

merged 2 commits into from
May 3, 2023

Conversation

trentm
Copy link
Member

@trentm trentm commented May 1, 2023

https://www.npmjs.com/package/express-graphql is deprecated.
https://github.com/graphql/express-graphql is archived.
The latest release, [email protected] was 2022-11-19.

Support for instrumenting this will remain, but we will stop testing it.
A motivation for dropping this is that the preinstall: npm uninstall express-graphql
parts of the graphql TAV testing could be significantly contributing to the
graphql TAV tests being among the slowest; similar for apollo-server-express
TAV tests.

@trentm
tests: stop testing 'express-graphql' instrumentation
ee39bf6 
https://www.npmjs.com/package/express-graphql is deprecated.
https://github.com/graphql/express-graphql is archived.
The latest release, [email protected] was 2022-11-19.

Support for instrumenting this will remain, but we will stop testing it.
A motivation for dropping this is that the `preinstall: npm uninstall express-graphql`
parts of the `graphql` TAV testing could be significantly contributing to the
graphql TAV tests being among the slowest; similar for `apollo-server-express`
TAV tests.
@trentm trentm self-assigned this May 1, 2023
@github-actions github-actions bot added the agent-nodejs Make available for APM Agents project planning. label May 1, 2023
@trentm trentm requested a review from david-luna May 1, 2023 18:55
@trentm
Copy link
Member Author

trentm commented May 1, 2023

Some timing info running the graphql TAV tests locally on my dev machine:

# before
TAV=graphql ./node_modules/.bin/tav --quiet  394.87s user 81.70s system 109% cpu 7:15.02 total

# after
TAV=graphql ./node_modules/.bin/tav --quiet  213.31s user 47.51s system 90% cpu 4:47.81 total

So, yes this is a large savings.
(Seperately we could also reduce the number of graphql versions tested by
just testing the latest patch in each minor release.)

@trentm
Copy link
Member Author

trentm commented May 1, 2023

Slow CI TAV jobs before this change 
% ./dev-utils/ci-tav-slow-jobs.sh
49 0m49s test-tav (10, undici)
51 0m51s test-tav (10, next)
53 0m53s test-tav (8, express-graphql)
55 0m55s test-tav (8, next)
56 0m56s test-tav (16, express-queue)
58 0m58s test-tav (19, express-queue)
60 1m00s test-tav (19, restify)
61 1m01s test-tav (18, restify)
68 1m08s test-tav (12, next)
70 1m10s test-tav (10, express-queue)
72 1m12s test-tav (19, pug)
73 1m13s test-tav (14, express-queue)
73 1m13s test-tav (8, undici)
74 1m14s test-tav (18, pug)
76 1m16s test-tav (18, express-queue)
82 1m22s test-tav (18, memcached)
83 1m23s test-tav (8, express-queue)
84 1m24s test-tav (12, express-queue)
85 1m25s test-tav (16, @koa/router)
85 1m25s test-tav (16, memcached)
85 1m25s test-tav (16, pug)
90 1m30s test-tav (19, memcached)
96 1m36s test-tav (8, pug)
99 1m39s test-tav (14, memcached)
100 1m40s test-tav (18, @koa/router)
102 1m42s test-tav (18, @elastic/elasticsearch-canary)
104 1m44s test-tav (10, pug)
104 1m44s test-tav (16, @elastic/elasticsearch-canary)
105 1m45s test-tav (10, @elastic/elasticsearch-canary)
105 1m45s test-tav (19, @hapi/hapi)
105 1m45s test-tav (19, @koa/router)
106 1m46s test-tav (10, memcached)
107 1m47s test-tav (12, memcached)
107 1m47s test-tav (8, @elastic/elasticsearch-canary)
108 1m48s test-tav (12, pug)
108 1m48s test-tav (19, @elastic/elasticsearch-canary)
108 1m48s test-tav (19, express-graphql)
110 1m50s test-tav (16, express-graphql)
111 1m51s test-tav (10, @hapi/hapi)
111 1m51s test-tav (12, @elastic/elasticsearch-canary)
114 1m54s test-tav (16, @hapi/hapi)
116 1m56s test-tav (18, express-graphql)
119 1m59s test-tav (8, @hapi/hapi)
121 2m01s test-tav (18, @hapi/hapi)
123 2m03s test-tav (16, koa-router)
127 2m07s test-tav (14, @elastic/elasticsearch-canary)
127 2m07s test-tav (19, koa-router)
129 2m09s test-tav (14, pug)
138 2m18s test-tav (8, memcached)
150 2m30s test-tav (19, finalhandler)
152 2m32s test-tav (16, finalhandler)
156 2m36s test-tav (14, express-graphql)
162 2m42s test-tav (16, generic-pool)
167 2m47s test-tav (8, @koa/router)
168 2m48s test-tav (18, koa-router)
173 2m53s test-tav (16, mimic-response)
176 2m56s test-tav (18, mimic-response)
178 2m58s test-tav (18, cassandra-driver)
178 2m58s test-tav (18, generic-pool)
181 3m01s test-tav (10, mimic-response)
181 3m01s test-tav (12, express-graphql)
183 3m03s test-tav (18, finalhandler)
184 3m04s test-tav (12, @koa/router)
189 3m09s test-tav (19, mimic-response)
190 3m10s test-tav (19, cassandra-driver)
194 3m14s test-tav (14, @koa/router)
194 3m14s test-tav (14, mimic-response)
198 3m18s test-tav (12, mimic-response)
198 3m18s test-tav (8, mimic-response)
209 3m29s test-tav (16, cassandra-driver)
216 3m36s test-tav (10, express-graphql)
219 3m39s test-tav (10, koa-router)
221 3m41s test-tav (8, @elastic/elasticsearch)
226 3m46s test-tav (19, generic-pool)
233 3m53s test-tav (8, koa-router)
234 3m54s test-tav (16, mysql2)
235 3m55s test-tav (19, handlebars)
238 3m58s test-tav (16, redis)
243 4m03s test-tav (10, @koa/router)
243 4m03s test-tav (16, elasticsearch)
250 4m10s test-tav (10, finalhandler)
250 4m10s test-tav (18, handlebars)
251 4m11s test-tav (16, mysql)
253 4m13s test-tav (16, undici)
256 4m16s test-tav (18, mysql)
257 4m17s test-tav (12, koa-router)
257 4m17s test-tav (16, ioredis)
257 4m17s test-tav (8, finalhandler)
264 4m24s test-tav (16, mongodb)
266 4m26s test-tav (12, finalhandler)
266 4m26s test-tav (16, ws)
268 4m28s test-tav (16, @elastic/elasticsearch)
268 4m28s test-tav (19, mysql)
268 4m28s test-tav (19, ws)
270 4m30s test-tav (18, elasticsearch)
275 4m35s test-tav (18, ioredis)
278 4m38s test-tav (14, finalhandler)
284 4m44s test-tav (18, @elastic/elasticsearch)
286 4m46s test-tav (14, koa-router)
289 4m49s test-tav (18, mysql2)
289 4m49s test-tav (19, elasticsearch)
291 4m51s test-tav (19, ioredis)
297 4m57s test-tav (16, handlebars)
298 4m58s test-tav (18, undici)
302 5m02s test-tav (10, cassandra-driver)
307 5m07s test-tav (16, restify)
308 5m08s test-tav (16, pg)
310 5m10s test-tav (12, @hapi/hapi)
314 5m14s test-tav (18, ws)
315 5m15s test-tav (19, mysql2)
315 5m15s test-tav (19, undici)
317 5m17s test-tav (16, got)
320 5m20s test-tav (19, @elastic/elasticsearch)
322 5m22s test-tav (8, cassandra-driver)
330 5m30s test-tav (8, mongodb)
333 5m33s test-tav (18, redis)
333 5m33s test-tav (19, mongodb)
335 5m35s test-tav (18, mongodb)
340 5m40s test-tav (10, @elastic/elasticsearch)
343 5m43s test-tav (18, got)
344 5m44s test-tav (14, @hapi/hapi)
344 5m44s test-tav (18, pg)
350 5m50s test-tav (14, cassandra-driver)
353 5m53s test-tav (10, mongodb)
355 5m55s test-tav (19, got)
356 5m56s test-tav (10, ioredis)
357 5m57s test-tav (19, pg)
357 5m57s test-tav (8, generic-pool)
360 6m00s test-tav (14, generic-pool)
364 6m04s test-tav (12, @elastic/elasticsearch)
370 6m10s test-tav (8, ioredis)
376 6m16s test-tav (10, mysql2)
376 6m16s test-tav (19, mongodb-core)
384 6m24s test-tav (10, generic-pool)
386 6m26s test-tav (16, mongodb-core)
387 6m27s test-tav (19, express)
396 6m36s test-tav (18, mongodb-core)
400 6m40s test-tav (19, redis)
417 6m57s test-tav (18, apollo-server-express)
421 7m01s test-tav (16, apollo-server-express)
423 7m03s test-tav (10, restify)
424 7m04s test-tav (14, @elastic/elasticsearch)
425 7m05s test-tav (14, pg)
435 7m15s test-tav (8, restify)
440 7m20s test-tav (12, redis)
445 7m25s test-tav (16, fastify)
450 7m30s test-tav (12, mysql)
453 7m33s test-tav (14, redis)
457 7m37s test-tav (10, elasticsearch)
460 7m40s test-tav (12, cassandra-driver)
461 7m41s test-tav (19, fastify)
463 7m43s test-tav (8, redis)
465 7m45s test-tav (14, restify)
466 7m46s test-tav (16, express)
466 7m46s test-tav (19, bluebird)
476 7m56s test-tav (14, undici)
476 7m56s test-tav (8, elasticsearch)
477 7m57s test-tav (18, bluebird)
487 8m07s test-tav (14, mysql2)
491 8m11s test-tav (14, ioredis)
497 8m17s test-tav (8, mysql)
501 8m21s test-tav (14, mysql)
502 8m22s test-tav (8, got)
503 8m23s test-tav (8, mysql2)
508 8m28s test-tav (10, ws)
513 8m33s test-tav (16, tedious)
514 8m34s test-tav (10, mysql)
518 8m38s test-tav (16, graphql)
519 8m39s test-tav (12, elasticsearch)
520 8m40s test-tav (12, restify)
522 8m42s test-tav (18, fastify)
526 8m46s test-tav (18, graphql)
530 8m50s test-tav (8, ws)
538 8m58s test-tav (10, handlebars)
543 9m03s test-tav (10, redis)
545 9m05s test-tav (19, tedious)
547 9m07s test-tav (14, handlebars)
556 9m16s test-tav (19, apollo-server-express)
556 9m16s test-tav (8, handlebars)
559 9m19s test-tav (18, express)
565 9m25s test-tav (16, bluebird)
572 9m32s test-tav (12, generic-pool)
590 9m50s test-tav (12, mysql2)
598 9m58s test-tav (18, tedious)
599 9m59s test-tav (12, mongodb)
602 10m02s test-tav (12, undici)
608 10m08s test-tav (14, elasticsearch)
622 10m22s test-tav (12, ioredis)
681 11m21s test-tav (12, ws)
706 11m46s test-tav (8, tedious)
710 11m50s test-tav (19, graphql)
720 12m00s test-tav (8, fastify)
723 12m03s test-tav (19, aws-sdk)
736 12m16s test-tav (14, mongodb)
738 12m18s test-tav (14, got)
744 12m24s test-tav (16, aws-sdk)
752 12m32s test-tav (10, got)
769 12m49s test-tav (12, handlebars)
770 12m50s test-tav (10, express)
788 13m08s test-tav (12, fastify)
794 13m14s test-tav (12, got)
796 13m16s test-tav (14, ws)
826 13m46s test-tav (16, knex)
839 13m59s test-tav (18, aws-sdk)
849 14m09s test-tav (8, mongodb-core)
858 14m18s test-tav (8, apollo-server-express)
873 14m33s test-tav (10, bluebird)
882 14m42s test-tav (18, knex)
900 15m00s test-tav (12, express)
901 15m01s test-tav (8, knex)
916 15m16s test-tav (19, next)
920 15m20s test-tav (12, apollo-server-express)
922 15m22s test-tav (14, apollo-server-express)
928 15m28s test-tav (14, bluebird)
929 15m29s test-tav (8, bluebird)
931 15m31s test-tav (19, knex)
980 16m20s test-tav (10, tedious)
980 16m20s test-tav (12, mongodb-core)
983 16m23s test-tav (10, apollo-server-express)
989 16m29s test-tav (16, next)
999 16m39s test-tav (12, bluebird)
1005 16m45s test-tav (14, mongodb-core)
1010 16m50s test-tav (14, fastify)
1019 16m59s test-tav (14, aws-sdk)
1022 17m02s test-tav (18, next)
1078 17m58s test-tav (12, aws-sdk)
1087 18m07s test-tav (8, express)
1111 18m31s test-tav (14, express)
1118 18m38s test-tav (10, aws-sdk)
1125 18m45s test-tav (12, tedious)
1129 18m49s test-tav (10, fastify)
1199 19m59s test-tav (8, aws-sdk)
1203 20m03s test-tav (12, pg)
1221 20m21s test-tav (8, pg)
1248 20m48s test-tav (10, mongodb-core)
1256 20m56s test-tav (14, next)
1307 21m47s test-tav (12, knex)
1307 21m47s test-tav (14, knex)
1323 22m03s test-tav (10, pg)
1386 23m06s test-tav (12, graphql)
1431 23m51s test-tav (14, tedious)
1496 24m56s test-tav (10, graphql)
1508 25m08s test-tav (8, graphql)
1757 29m17s test-tav (14, graphql)
1794 29m54s test-tav (10, knex)

TODO: It may be interesting to compare this to a CI TAV run on main once this PR goes in.

@trentm trentm merged commit c90d10d into main May 3, 2023
@trentm trentm deleted the trentm/stop-testing-express-graphql branch May 3, 2023 20:55
@trentm
Copy link
Member Author

trentm commented May 4, 2023

TODO: It may be interesting to compare this to a CI TAV run on main once this PR goes in.

Graphql-related TAV job times before:

53 0m53s test-tav (8, express-graphql)
108 1m48s test-tav (19, express-graphql)
110 1m50s test-tav (16, express-graphql)
116 1m56s test-tav (18, express-graphql)
156 2m36s test-tav (14, express-graphql)
181 3m01s test-tav (12, express-graphql)
216 3m36s test-tav (10, express-graphql)
518 8m38s test-tav (16, graphql)
526 8m46s test-tav (18, graphql)
710 11m50s test-tav (19, graphql)
1386 23m06s test-tav (12, graphql)
1496 24m56s test-tav (10, graphql)
1508 25m08s test-tav (8, graphql)
1757 29m17s test-tav (14, graphql)

and after:

346 5m46s test-tav (16, graphql)
373 6m13s test-tav (20, graphql)
465 7m45s test-tav (18, graphql)
692 11m32s test-tav (8, graphql)
846 14m06s test-tav (10, graphql)
951 15m51s test-tav (12, graphql)
1021 17m01s test-tav (14, graphql)

This is a pretty big improvement.

v1v added a commit to v1v/apm-agent-nodejs that referenced this pull request May 8, 2023
@v1v
Merge branch 'main' of github.com:elastic/apm-agent-nodejs into featu…
a9130f7 
…re/support-specific-modules

* 'main' of github.com:elastic/apm-agent-nodejs: (54 commits)
  chore: fix dev-utils/ci-tav-slow-jobs.sh (elastic#3319)
  test: reduce TAV test matrix for slowest jobs (elastic#3321)
  chore: sync package-lock so 'npm ci' can work (elastic#3318)
  docs: document `useElasticTraceparentHeader` config var (elastic#3316)
  chore, test: test driver improvements (elastic#3293)
  test: drop node 14 from RC tests now that it is EOL (elastic#3315)
  test: fix running fastify.test.js with node v8 (elastic#3317)
  feat: add @apollo/server@4 support (elastic#3203)
  chore: update nvm (elastic#3309)
  tests: stop testing 'express-graphql' instrumentation (elastic#3304)
  chore: fix bitrot.js dev util for recent changes (elastic#3308)
  test: restore testing of Azure Functions on node >=18.x (elastic#3307)
  fix: support Lambda instrumentation for `contextManager: 'patch'`; refactor Lambda tests (elastic#3305)
  test: fix fastify TAV test failures (elastic#3314)
  test: fix @aws-sdk/client-s3 TAV test failures (elastic#3312)
  feat: add instrumentation for aws-sdk S3 client (elastic#3287)
  feat(fastify): add captureBody support (elastic#2681)
  feat: mysql2@3 support (elastic#3301)
  chore(deps): bump @opentelemetry/exporter-prometheus from 0.37.0 to 0.38.0 in /test/opentelemetry-metrics/fixtures (elastic#3295)
  chore(deps-dev): bump fastify from 4.16.3 to 4.17.0 (elastic#3296)
  ...
trentm added a commit that referenced this pull request May 16, 2023
@trentm
tests: stop testing 'express-graphql' instrumentation (#3304)
c6aacce 
https://www.npmjs.com/package/express-graphql is deprecated.
https://github.com/graphql/express-graphql is archived.
The latest release, [email protected] was 2022-11-19.

Support for instrumenting this will remain, but we will stop testing it.
A motivation for dropping this is that the `preinstall: npm uninstall express-graphql`
parts of the `graphql` TAV testing could be significantly contributing to the
graphql TAV tests being among the slowest; similar for `apollo-server-express`
TAV tests.
PeterEinberger pushed a commit to fpm-git/apm-agent-nodejs that referenced this pull request Aug 20, 2024
@trentm @PeterEinberger
tests: stop testing 'express-graphql' instrumentation (elastic#3304)
b7a416a 
https://www.npmjs.com/package/express-graphql is deprecated.
https://github.com/graphql/express-graphql is archived.
The latest release, [email protected] was 2022-11-19.

Support for instrumenting this will remain, but we will stop testing it.
A motivation for dropping this is that the `preinstall: npm uninstall express-graphql`
parts of the `graphql` TAV testing could be significantly contributing to the
graphql TAV tests being among the slowest; similar for `apollo-server-express`
TAV tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@david-luna david-luna david-luna approved these changes

Assignees

@trentm trentm

Labels
agent-nodejs Make available for APM Agents project planning.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@trentm @david-luna