RangeError: Invalid string length

[mischkl]

We are experiencing Invalid string errors which crop up with the html-webpack-plugin as the culprit, but only started happening after we added the hard-source-webpack-plugin to our build. See jantimon/html-webpack-plugin#855 for more details.

It occurred to me that perhaps the ordering of the plugins might have something to do with this? If I put hard-source-webpack-plugin in the beginning of the plugin array could this possibly solve the issues? If so it would be great if this could be documented.

[mzgoddard]

RangeError: Invalid string length comes from any time a string that is REALLY big is created. Most of the ones I've seen are trying to create strings larger than 130MBs in size. hard-source doesn't need strings that big. This has been coming about due to different parts of webpack data that regularly make trees of data that in memory reuse objects but as JSON need to each be a unique copy. Adding that all up you get a giant string and the error.

Changing the order of the plugins shouldn't have an effect.

We need to track down that part of the recorded webpack data that is being stored incorrectly. Its really hard to find that stuff in a vacuum as many of the instances of this have been specific npm dependencies that make use of some feature to a large enough order to create the bug.

I see you found #205. Are you getting the [hard-source:serial] Unable to freeze module error logs? If so can you post the file paths it cannot freeze?

Additionally if you run the build with

{
  apply: function(compiler) {
    compiler.plugin('hard-source-log', function(message) {
      console[message.level].call(console, message);
    });
  }
}

in your plugins list you may have more data you can drop here to help.

[mischkl]

Hi,
Thanks for your help. We are not getting the [hard-source:serial] Unable to freeze module error.
I'll try adding the logging plugin as you suggest and see if it sheds more light on the problem. ;)

[xiaohuoni]

@mischkl Hello, have you repaired the problem?

[mischkl]

Unfortunately not, we ended up just disabling the HardSourcePlugin.

[mzgoddard]

@mischkl @xiaohuoni 0.6.2 (released March 5th) was one of the solutions to this class of problem. Specific causes we have seen have often been groups of objects with lots of relations to each other and incorrectly stood up by hard-source that caused those relations to increase drastically. It is the hardest class of problem to solve without tracking down specific npm dependencies (like the apollo framework) that can create these edge cases.

A method I've though of more recently to narrow down the cause is to build a file with webpack with all of the require('node_modules-package') commented out and toggling them on, until a build reaches this error.

[xiaohuoni]

@mzgoddard Thank you, when can you fix it?

[mzgoddard]

@xiaohuoni That's hard to say. Currently I need more information to fix it. This is a common symptom of the sophisticated nature of webpack, html-webpack-plugin and hard-source-webpack-plugin working together. webpack and these two plugins work together in plenty of projects. Its the addition of a 4th, 5th or other number of dependencies producing this error. I need to know the other dependencies that are part of the issue. Those dependencies can be dependencies of dependencies of dependencies.

I hope to be adding checks to help hunt these down but that is pretty vague too in how and where those tests are needed without slowing down hard-source or webpack.

[xiaohuoni]

@mzgoddard Thanks！

[xiaohuoni]

@mzgoddard At the moment this problem only appears on Windows, but it does well on the Mac. You can think about it in this direction.