How to work with web workers (use a web worker IN cli project)

[jbgarr]

Bug Report or Feature Request (mark with an x)

- [ ] bug report -> please search issues before submitting
- [X] feature request

Versions.

@angular/cli: 1.0.0
node: 6.9.1
os: darwin x64 (macOS Sierra 10.12.3)

Is there a way to import and use a web worker in a component using angular-cli? I'm trying to incorporate a web worker that I wrote into my project, but I'm not having any luck. The web worker worked fine in my project before attempting to convert it to use the CLI (previous version was based on System.js). I'm trying to import my web worker code like this:

import * as MyWorker from 'assets/workers/dropdown.worker.js';

and then I'm trying to create my actual worker instance like this:

let blob = new Blob(['(this.onmessage=', MyWorker.toString(), ')'], { type: "text/javascript" });
let dropdownWorker = new Worker((<any>window).URL.createObjectURL(blob));

The above does not work though, and if I log MyWorker right after I try to import it I see it's just an empty object instead of a function (which is what it is in my other app). Any help on this would be much appreciated!

UPDATE

I just want to clarify what I'm trying to do: I want to use an existing web worker IN my Angular app, not run my whole app IN a web worker. I know there is separate issue already that relates to the trying to run an entire app in a web worker. These are similar, but very different issues.

[bmayen]

Check out https://stackoverflow.com/questions/43276044/angular-cli-generated-app-with-web-workers

Would be really great to get this support by default!

[jbgarr]

Thanks a lot for the reply! I had seen that answer on Stackoverflow, but unfortunately they are trying to actually run their whole Angular app IN a web worker. I am trying to use an existing web worker in my app, so it's a little different. I appreciate the help though. Hopefully someone else can chime in here.

[maxime1992]

There's an opened issue here too : #2305

[jbgarr]

@maxime1992 I appreciate your reply. That's a similar, but different issue. It's related to the Stackoverflow question that @bmayen linked to as well. Both of those deal with trying to run an entire Angular app IN a web worker, while I just want to use and run an existing web worker in my app.

[bmayen]

@jbgarr, just curious, why do you want to take this approach? The Angular team has done some amazing work supporting running the entire app in a worker. You might meet with less friction if your use case could allow for that approach instead.

[shlangus]

We were using an inline worker-loader, but now it does not work for us (see #4773). So now we are using file-loader.
It's a little uncomfortable since you have to build a worker before building an app, but it works.

[jbgarr]

@bmayen I have a section of my app that is potential performance bottleneck so I built a custom web worker to handle the heavy lifting on another thread, but I don't necessarily need/want to run my whole app in a worker. I built the original web worker for my app before converting over to the Angular CLI to build/bundle my app and I was hoping I could just use it as is without custom tailoring my build process.

@shlangus I had seen issue #4773 and was a bit disappointed. With your new approach are you having to use ng eject and build out some custom handling of your worker? If so, would you mind sharing whatever custom code you had to add in order to get it working? I was really hoping to avoid ng eject, but it may be required if I want to reuse my web worker as is. Thanks for any insight you can share!

[shlangus]

@jbgarr First of all you need to configure build process for your worker. To do that you can create a separate webpack config such as following. It uses ts-loader so you should install it too.

const config = {
  entry: './src/worker/worker.ts',
  output: {
    filename: './src/worker/index.js'
  },
  resolve: {
    extensions: ['.ts', '.tsx', '.js']
  },
  module: {
    loaders: [
      {test: /\.tsx?$/, loader: 'ts-loader'}
    ]
  }
};

module.exports = config;

Now you are able to run webpack --config path_to_worker_config. It makes sense to add this command as "prestart" script.

Once you have done it you need to get ability to include your worker into your angular app.
You should add "node" in types array in your tsconfig.app.json, then you can just require your worker:

const MyWorker = require('file-loader?name=worker.[hash:20].[ext]!../worker/index.js');
...
worker = new Worker(MyWorker);

This solution doesn't require to perform an ejecting but it also doesn't watch for changes in the worker.

[jbgarr]

@shlangus Thanks a lot for the explanation! I'll have to give that a try when I've got some time. The general idea makes sense. Thanks again, I appreciate it.

[filipesilva]

Heya, I added some information on how to currently use webworkers in #2305. I know there's been some conversation here but that's the original issue, and it's better to concentrate discussion in a single one.

[jbgarr]

@filipesilva Thanks for the info and I had seen that issue already, but I think there is a core difference between what I'm asking/trying to do VS what that issue touches on. I am trying to use an existing web worker IN my Angular app, while that issue focuses on running an entire Angular app IN a web worker. I believe these are similar, but very different issues. Can you consider reopening this?

[maxime1992]

@filipesilva in the other issue you talked about service-worker and not web-worker :

Unless it's a typo, maybe you might reopen this (+cc @jbgarr comment)

[filipesilva]

@jbgarr @maxime1992 you're absolutely right, reopening.

[naveedahmed1]

@filipesilva can the steps mentioned in this post be integrated in cli?

https://medium.com/@enriqueoriol/angular-with-web-workers-step-by-step-dc11d5872135

Repo: https://github.com/kaikcreator/angular-cli-web-worker

Demo: https://kaikcreator.github.io/angular-cli-web-worker/

[filipesilva]

@naveedahmed1 Hm... I don't think we can do it in a seamless way, no. It would require app level switches to control the build behaviour.

What could perhaps be done in the interim is provide platformWorkerAppDynamic in @ngtools/webpack.

I can also say we're looking at supporting different platforms in the future (e.g. platform-server) and so platform-worker would also make sense.

[naveedahmed1]

@filipesilva glad to know about support for other platforms in future versions.

Meanwhile, what if the changes in app.module.ts and main.ts are left for the user while the cli:

• Could generate workerLoader.ts (it could be commented and user can uncomment it when required)
• Allow a switch to specify webworker as platform which makes appropriate changes to webpack build process

So, that if a user wants to add support for webworker, they just make changes to app.module.ts and main.ts and specify platform as webworker in build command.

Would it be possible?

[filipesilva]

@naveedahmed1 it would be possible, but it's not a very good idea from the CLI standpoint. It's better to properly provide platform support and a proper API for it, than just trying to get it to work ASAP.

For the moment, the eject based guide seems to be the best option.

[bmayen]

There are two issues being discussed in this thread. OP is asking about having individual webworkers from the context of an app running on the UI thread. Others are discussing what is being requested in #2305 , which is what the eject-based option attempts to solve.

@filipesilva, in general, I think the #2305 approach is most beneficial to the community as this is the official Angular way. Any chance we could get a prioritization of this feature or an idea of the timeline? Would be very helpful for planning when we can fully integrate CLI into our workflow... which we're desperate to do ASAP :)

[filipesilva]

@bmayen I don't have a timeline for it, no. It's something to be considered after platform-server lands, but even then it might not be a 1.x feature.

[naveedahmed1]

@bmayen for individual web workers there is a plugin which can be used https://github.com/haochi/angular2-web-worker

[bmayen]

Thanks @naveedahmed1. May still be of interest to the OP. I'm actually more concerned with #2305, personally. Going to hop out of this thread in favor of that one.

[jbgarr]

@naveedahmed1 Thanks for the plugin link. I'll definitely take a look.

[filipesilva]

Let me know if https://github.com/haochi/angular2-web-worker addresses your needs. The importScripts bit looks like it would work pretty well with a file in assets/.

[naveedahmed1]

@filipesilva should the eject based approach work with Lazyloading?

[dirkluijk]

This is my workaround.

Step 1. Add a custom webpack build to Angular CLI

Since Angular 6 you can use the Angular CLI for custom webpack builds.

Add to angular.json:

  "projects": {
    "your-worker-name": {
      // ...
      "architect": {
        // ...
        "build": {
          "builder": "@angular-devkit/build-webpack:webpack",
          "options": {
            "webpackConfig": "webpack.config.js"
          }
        }
      }

Create a simple webpack config, which builds the worker script to your dist directory, e.g.:

const path = require('path');

const config = {
  entry: './src/your-worker.js',
  mode: 'production',
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'your-worker.js'
  }
};

module.exports = config;

Step 2. include the generated worker script in your Angular app

In angular.json, add to your Angular project:

"scripts": [
  { "input": "dist/your-worker.js", "lazy": true }
]

Step 3: make sure to build your worker script first...

Just run:
$ ng run your-worker-name:build

And/or modify your package.json to run before ng serve and ng build:

"scripts": {
    "ng": "ng",
    "start": "ng run your-worker-name:build && ng serve",
    "build": "ng run your-worker-name:build && ng build",
    "test": "ng test",
    "lint": "ng lint",
    "e2e": "ng e2e"
  }

Step 4: profit!

export class AppComponent {
  ngOnInit(): void {
    const worker = new Worker('your-worker.js');
  }
}

Example

[michaeljota]

I will double check this, but I think that by using Webpack loaders, you'll be fine using Typescript files for the source of the worker.

[dirkluijk]

@michaeljota I did't try that yet. But I guess then you also need to include a typescript loader, as you have a custom webpack config. I didn't want to maintain a huge webpack config. ;)

[jerryorta-dev]

@dirkluijk That is a great solution and something I will use. As an experiment, I wonder how other new Angular 6 CLI features may enhance the architecture -- such as generating the worker code as another app or lib? That may over-engineer this solution for your app. It's just a thought for larger apps.

[sladiri]

@dirkluijk Thank you for your example. I do not see why the scripts array inside angular.json should contain the file? I write the compile worker.js into the assets folder, but maybe that is an anti-pattern in Angular? I can load it fine without its addition to the scripts array.

I have Typescript source, and while it does work, the compiler complains about main.ts accessing the document. That is weird and I could not get rid of the error, since the main.worker.ts does not import anything and does not accesss the DOM. Below is my TS config, where I removed DOM from the libs array.

This is from my project https://github.com/sladiri/web-worker-angular-v7

// webpack.base.config.js
const path = require("path");

module.exports = ({ production = false, ie11 = false }) => ({
  mode: production ? "production" : "development",
  entry: "./src/main.worker.ts",
  // entry: ["../src/polyfills.ts", "./worker/main.worker.ts"],
  output: {
    path: path.resolve(__dirname, "assets", `webworkers${ie11 ? "-ie11" : ""}`),
    filename: "main.worker.js",
  },
  resolve: { extensions: [".ts"] },
  module: {
    rules: [
      {
        test: /\.ts$/,
        loader: "ts-loader",
        options: {
          compilerOptions: {
            baseUrl: "./",
            outDir: `./dist/out-tsc/worker${ie11 ? "-ie11" : ""}`,
            sourceMap: true,
            declaration: false,
            module: "esnext",
            moduleResolution: "node",
            emitDecoratorMetadata: true,
            experimentalDecorators: true,
            target: ie11 ? "es5" : "es2015",
            typeRoots: ["node_modules/@types"],
            lib: ["es2018", "esnext.asynciterable", "webworker"],
          },
        },
      },
    ],
  },
});

[chrillewoodz]

There is actually a package for this:

https://www.npmjs.com/package/ngx-web-worker

I've tried it and it works as far as I can tell.

[mboughaba]

@dirkluijk great and quick solution 👏👏👏

I am wondering if you tried with contenthash or do you have any thought about it

[kiranjholla]

I have tried implementing a version of the solution given by @jerryorta-dev into an Angular 6 library. I used the worker-loader! Webpack Loader hoping to get the Worker as part of a reusable functionality I am packing into an Angular Library.

That approach just wouldn't work. Finally I found this: #5885 (comment). Angular Libraries are packaged using ngpackgr which does not use Webpack and hence there is no support for loaders.

What's more, they also discuss the possibility of Angular CLI moving away from Webpack completely. That means that there would be no supported means to deploy standalone web-workers in Angular CLI. All discussion tends to point to the fact that the entire Angular Application can be run in a web-worker. However, that approach wouldn't work if we need to use some libraries (example: D3) which need access to the DOM.

[michaeljota]

@kiranjholla What that comments points out is that actually Angular CLI has moved from a building system in the past (From SystemJS to Webpack) and the CLI is all about abstract the build job in a way that the developer can have a consisten behavior.

The solution @dirkluijk post in his comment should work, as long as Angular implement Webpack.

However, I really doubt that they are moving to another building system any time soon. Even if they did, as this is now, I'm sure this would still work, as you are just using another builder task.

[dirkluijk]

Exactly. When Angular will replace webpack for another tool, the CLI will still support it by builders like @angular-devkit/build-webpack.

[kiranjholla]

@michaeljota @dirkluijk That's a fair point. Maybe I misinterpreted the comment and jumped the gun.

However, I like the solution that uses worker-loader because of output hashing. The worker.js file is also hashed and I don't need to worry about manual cache-busting. With ng-packagr, I can no longer use worker-loader to load my worker scripts.

Are you aware of any automated way that I could include a output hash in the worker.js file while importing it using the new Worker('hash.worker.js') approach? Any pointers?

Thanks.

[mboughaba]

@kiranjholla

I already asked this question few comments above... #5885 (comment)

Also tried on StackOverflow and angular/cli but without luck.

That said, considering that from what I am understanding you have 2 builds, there would be no way to reference the hashed file in the previous file.
#12582

In case you find a solution don't hesitate to give feedback here

[shlangus]

@kiranjholla @mboughaba Have you tried #5885 (comment)?

[mboughaba]

@shlangus Thanks very much for the suggestion! I managed to get it working using your method
This is how I require my file

const StorageWorker: any = require('file-loader?name=storage.[hash:20].[ext]!../../../../dist/storage.js');
// [...]
// ctor
   this.worker = new Worker(StorageWorker);

On the config side, I have the following

const config = {
  entry: {
    storage: './src/worker/storage.ts'
  },
  mode: 'production',
  watch: false,
  module: {
    rules: [
      {
        test: /\.ts$/,
        loader: 'awesome-typescript-loader',
        query: {
          configFileName: './src/tsconfig.worker.json'
        }
      }
    ]
  },
  output: {
    path: path.resolve(__dirname, 'dist-worker'),
    filename: '[name].[contenthash].js'
  }
};

There is a lot of magic, but hey it works!!

I spoke too fast 😞 ... This was loading the storage.js which was left over in the dist folder... As soon as I run the prod build which uses CleanWebpackPlugin.

[julkue]

You might be interested in using inline web workers to avoid the overhead of having to create a separate Webpack configuration with Typescript etc.:
https://www.html5rocks.com/en/tutorials/workers/basics/#toc-inlineworkers

[mhosman]

It could be great to create a clean repository with a full working example of an Angular 7 app with Lazy-Loading, running entirely in a web-worker.

[dirkluijk]

@mhosman This issue is about using web workers IN an Angular app. Not running Angular itself in a worker.

For the latter, see https://legacy-to-the-edge.com/2018/05/01/platform-webworker-in-angular/ (I just googled for it)

[mhosman]

Thanks @dirkluijk, my problem is that when I'm loading modules, loaders and the UI itself is stucked for a couple of seconds. I need a way to run the logic in the background in order to get the animations only working in the main thread.

[filipesilva]

There's ongoing work to add https://github.com/googlechromelabs/worker-plugin in #12575.

[mhosman]

One of the problems is when you load modules and material2 animations get stuck.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}