Static site optimisations

[Rich-Harris]

When using sapper export, there are some interesting opportunities for optimisations that don't really make sense when pages are being generated on-demand. For example, I just learned about subfont by @Munter, which can generate an efficient subset of the fonts a given page uses.

A few ideas off the top of my head:

• Subfont
• Minify HTML
• Further optimise CSS
• Compress SVG assets
• Compress other images
• Probably some other Assetgraph-related optimisations

[Munter]

Looks like https://github.com/sveltejs/sapper/blob/master/src/cli/export.ts#L64-L93 can be replaced by assetgraph crawling the pages and https://github.com/sveltejs/sapper/blob/master/src/cli/export.ts#L47-L62 can be removed, since an assetgraph-based optimisation would require to contain the entire graph in memory at the same time.

So in essensce assetgraph would just scrape the page in its entirety from the temporary server, then run a bunch of optimisations on the entire set before writing it to disc.

Of course this all hinges in sapper outputting standard compliant enough code for assetgraph to pick up in its static analysis

Ping @papandreou

[Rich-Harris]

Now we're talking!

Of course this all hinges in sapper outputting standard compliant enough code for assetgraph to pick up in its static analysis

What are some of the gotchas to be aware of?

[Rich-Harris]

(Also: are there plans to integrate Subfont into Assetgraph? Or would that be a separate step?)

[Munter]

What are some of the gotchas to be aware of?

In most cases it's things like unbalanced html, syntax errors in css that browsers compensate for, missing files etc. All stuff that you'd want to be aware of.

We haven't built in support for the latest js features like ES modules and such. But since this is a compile output I assume your existing pipeline has already build browser compatible output. Things like custom runtime module loaders might not be picked up

[Rich-Harris]

It's currently using webpack, with code-splitting — we plan to also support Rollup soon, which can output ESM (for browsers with native module support) or System.js modules.

In most cases it's things like unbalanced html, syntax errors in css that browsers compensate for, missing files etc. All stuff that you'd want to be aware of.

So it bails if it encounters something invalid? I think that's fine (Svelte won't compile if your CSS is bad anyway, so the only way that would happen is if there was a broken assets/styles.css file or something — as long as it doesn't just fail silently!

[Munter]

So it bails if it encounters something invalid? I think that's fine (Svelte won't compile if your CSS is bad anyway, so the only way that would happen is if there was a broken assets/styles.css file or something — as long as it doesn't just fail silently!

We might already pick up webpacks custom loader syntax, since we had that need before. Obviously modules are on the list of things that need to be implemented, since browsers are getting there now.

Silent failures can happen in the situation where something that describes a reference to an asset is not implemented in assetgraph. Then it simply won't see it, and the graph will not contain the target asset. Any other types of errors are either handles and you get an information even, or there is a warning level event if we are able to proceed. If a complete error happens the pipeline throws, and we usually set it to throw on warning level events as well, because usually they are errors on the web page you want to fix

[papandreou]

are there plans to integrate Subfont into Assetgraph?

It is already integrated. Actually the guts of it are maintained in the assetgraph repo itself, due to be split out as a plugin when we get around to it :)

[0gust1]

I can see situations where it can be dangerous (to subset webfonts according to the rendered content at build time).

Your pipeline subsets the webfont(s) according to the glyphs presents on the rendered page(s)

• if you have content loaded dynamically or conditionnally, which can change arbitrarily.
• if you work on a generic product (an international e-commerce platform, my current mission), which can be overriden, by country's team (adding custom scripts, third-party resources, CSS overrides, external A/B testing libs etc...)

=> You can end up in production with pages where glyphs are missing.
In our case, we implemented subsetting using characters lists (one list by country), it's less optimal, but more secure/resilient.

I know that the use of export is limited to the cases where "two different users, visiting the website at the same time, should have the same webpage". It includes session/authentication controlled content (obviously), but also personalization engines, external third party content, libraries and utilities.

We're seriously considering sapper/svelte for our use case, and even if we have dynamic content, and I still would like to have the ability to put entire pages in the CDN.

PS :
We're looking into sapper/svelte because it's almost the only "quality and performance first" tool of this class (and I'm a firm believer of compilers and misbeliever of frameworks ;-) )

[Rich-Harris]

Yeah, I think this stuff probably needs to be opt-in, rather than the default — something like

npx sapper export --optimize

or

npx sapper export --optimize=minifyhtml,subfont

[Munter]

@0gust1 Subfont has guards built in for missing glyphs. First of all it will warn you about glyphs on the page that are missing in the original font during build, and it also keeps the original font in the fallback chain. So if the subset doesn't contain the glyph, the original font will be loaded.

When you have a page that is static, or partially static while more data is fetched, that means the subset will be able to cover the static part and ensure a really quick first paint including web fonts for those parts. Then the original font is loaded when dynamic content with new glyphs pop in.

[ooloth]

Automatic image optimization would be amazing!

My favourite thing about Gatsby is the hours of manual work gatsby-image saves by automatically resizing, optimizing, lazy-loading your images (from a single, HQ version) and generating optimized <picture> markup that includes ideal srcset + sizes attributes.

The fact that all this happens automatically has huge benefits for devs and users, so I’d be excited by a Svelte/Sapper equivalent!

After styles and animations (which Svelte handles beautifully), images seem like a worthy candidate for first-class framework support to help ensure devs don’t accidentally slow down Svelte/Sapper apps that would otherwise be speedy.

[matyunya]

@ooloth Working on this: https://svelte-image.matyunya.now.sh/

[thomasmorice-zz]

@matyunya This is amazing! keep us updated

[swyxio]

i am experimenting with making a proper static site generator atop sapper export and also noticed that its pretty slow. it times out even at like 5 pages (i'm building about 100) and i dont know if it's taking advantage of parallelization. having the puppeteer crawl links is nice, but the tradeoff seems to be speed. for a static site generator usecase where you are likely to manually control a site index and have many leaves with no undiscovered links, it probably is not the right tradeoff compared to just having an upfront manifest of pages.

[khrome83]

@sw-yx - #869 fixes some of the slow down in sapper export. Specifically sapper crawls pages as FAST as possible, leave a io issue. My co-worker fixed this by respecting concurrent and using a slightly different model.

We were able to generate about 30k pages in 12 minutes. Our plan currently is to use #859 to split the sections of the site that is exported.

I would love if there was a faster way, but running the server and crawling seems fairly fast.

Any routes we can't navigate too, like /404 we will be passing to --entry. We are trying to not have many pages that don't have a link from the html based sitemap.

Our site is also backed by a headless CMS, so we also have a way to pull down all "pages" of content and get the list of unique routes.

[khrome83]

ohh @sw-yx - our plan for #856 if it gets accepted is to have a lambda on AWS listen to a webhook when our CMS published content to rebuild pages associated with that content. This allows us to avoid rebuilding 10s of thousands of files.

[khrome83]

For my personal site (not the same as I talk about above), I exported everything. Then on the exported files, I ran subfont by @Munter. It worked really well and happy with the output.

[swyxio]

ive been watching yoursite. its good stuff.

ssg has also come along since i last wrote. thanks to neal’s help and some elbow grease on my part it is now a zero config sapper overlay with a nice data pipeline. i reckon i just need to add in an incremental build thing and then i can release.

[hugosenari]

My letters to Santa Claus:

1. Option to exclude all component JS (for pure static html);
2. Option to run something (ie: posthtml, postcss) with exported files;
3. Initial rendering remove all nodes from server side rendering and then render them again #911 (more expecifically);
4. Find some way to reduce JS size using html;

Since I'm newbie in rollup, to solve 1 and 2 I'm using gulpfile. 3 and 4 are more svelte related, but was addressed to Santa :-)

[jthegedus]

#1046 should be on this list

[benmccann]

SvelteKit has adapters, which is probably how this should be supported for anyone that's interested in such features. I'm going to close this given that SvelteKit is the future

[jthegedus]

Are we going to be able to compose Adapters? Because having every adapter implement things processes for base64 image inlining seems a bit much for maintainers.

[benmccann]

There's a number of things mentioned in this issue and they each might have different recommended implementations. Image inlining sounds like something the build system would need to do, so probably Vite would need to be involved in this case. Vite shares Rollup's API, so I imagine that you could use @rollup/plugin-url for that