Building Performant Web Apps with Rust, WebAssembly, and Webpack

In my previous post I described how we chose our frontend stack of Rust, WebAssembly, and WebGL - what we call our "dream stack." Unfortunately, between the dream and the reality lies the shadow. We had a lot of work to do to get this stack deployed correctly, and even more to create a fast and reliable developer experience.

While this stack is supported by all major browsers there remains a dearth of material about the practical deployment of a WebAssembly application. There are plenty of high-minded articles about the promise of WebGL and WebAssembly (Wasm) and plenty of highly technical specs from Mozilla describing how they should be supported. But there's little material in between the ideas and the metal.

Since these types of articles are often the most useful to practical engineers, we're planning to fill this gap at Canvas.

WebAssembly (Wasm)

Entire books have been written about WebAssembly. For this article, suffice it to say that WebAssembly is a low-level language that can be run natively by all modern browsers. This allows you to deploy fast, lean, memory-efficient code. You do not write WebAssembly directly; instead, you code in Rust or C++ and compile this into WebAssembly.

wasm-pack

wasm-pack handles compilation from Rust into Wasm (the C++ equivalent is emscripten). The output of wasm-pack is a .wasm binary that can be run by browsers.

However, your Javascript application can’t use this binary directly since the compiled Rust code doesn’t implement the module export syntax required to import the code into your Javascript application.

So, in addition to translating the code, wasm-pack adds some boilerplate Javascript that allows you to import your WebAssembly code like any other package. This is where using WebAssembly can become confusing, since we run into the confusing world of bundlers and the ongoing transition/blood feud between the CommonJS and ES Module standards. Before moving ahead, we need some brief background on these two topics to understand how a package gets imported in the browser.

CommonJS and ES Module

To grossly simplify, CommonJS (CJS) and ES Modules (ESM) are two syntaxes for using modules in Javascript. For a fuller explanation I recommend this concise history. CJS is the legacy standard; ESM is the new standard. They look similar, but functionally are very different. Here's the syntax for CJS:

// util.js
module.exports.sum = (x, y) => x + y;

// main.js
const {sum} = require('./util.js');
console.log(sum(2, 4));

And here's the equivalent for ESM:

// util.js
export const sum = (x, y) => x + y;

// main.js
import {sum} from './util.js'
console.log(sum(2, 4));

Of the many functional differences between the two, one of the most important to us is that top-level await is available in ESM modules but not CJS modules. This allows ESM modules to make asynchronous calls when the application is being initialized, for example downloading a remote package to import. On the other hand, CJS is restricted to using only synchronous calls at the top-level. This means that any loading that needs to happen asynchronously must happen in your application code at runtime.

Looking ahead a bit, WebAssembly requires that we load our binary into memory and then initialize the module. Initialization here means adding functions exported from the wasm binary to the exports of the module. This is what tells your Javascript application how to actually call the Wasm code.

Since fetching the binary is asynchronous, using ESM we can import and initialize WebAssembly as part of the build, while using CJS cannot. Instead we must do a fetch-and-initialize step in our application code.

Importantly, ESM modules are only (mostly) usable in the browser with a bundler. So we’ll take one more brief detour to explain bundlers before moving on.

Bundlers

For a full explanation of bundlers I recommend this great explanation. Bundlers are the magic that takes the languages we love to work with – Typescript and modern Javascript syntaxes – and transpiles them into plain old Javascript that can be understood by most browsers.

One important step in this process is bringing all the external Javascript resources – like npm packages or local external packages – and merging them into a single unified Javascript file that can be downloaded and initialized without making further network requests.

Since our WebAssembly application is built as a local package, this must be handled by the bundler as well.

wasm-pack build

We now know enough to understand the wasm-pack build command which provides two targets of interest to us: --target bundler and --target web. Naturally, bundler builds a package intended to be used with a bundler like Webpack (and really only Webpack), while web builds a package that can be used included in any web browser. But what does this mean practically?

Critically, bundler produces as ESM module while web produces a CJS module. This means that the bundler build is able to perform the asynchronous fetch of the wasm binary and then perform the follow-up initialization at build time and include the initialized Javascript code in the bundle.

This allows you to use the familiar import syntax to import your WebAssembly package like you would any other. You do not have to worry about the fetch or initialize step in your code.

Using WasmPackPlugin, your webpack.config.js will look like:

plugins: [
  new WasmPackPlugin({
	extraArgs: '--target bundler', // default, included for clarity
    ...
  }),

And using in your application:

import {fib} from '../../rust/pkg/fib_bg.wasm'

console.log(fib(20));

On the other handle, the web package cannot make the asynchronous fetch and then perform initialization itself. Instead, your application must request the wasm file and then call the provided initialization function before usage.

Your webpack.config.js using --target web :

plugins: [
  new WasmPackPlugin({
    extraArgs: '--target web',
    ...
  }),

Using in your application:

import("../rust/pkg").then(module => {
  console.log(module.fib(20));
});

// Or using React

import init, {fib} from '../../rust/pkg/fib_bg.wasm'

React.useEffect(() => {
  async function loadWasm() {
    await init();
    console.log(fib(20));
  }
  loadWasm();
}

Since we were already using a bundler our choice seemed obvious. The syntax is much cleaner for bundler. Additionally, when using web there will be a delay between when the page loads and when the WebAssembly application finishes initialization and starts. At Canvas we initially used bundler for these reasons.

However, we've recently switched to using --target web for one major reason: including the fetch and initialization in the build step made the developer experience SLOW. We were experiencing build times of around 90 seconds. web does not perform this step during build, and this reduced our build time to under 30 seconds.

I'm not clear why this needs to be the case. The WebAssembly application takes around a second to initialize in our web application using web , so I'm not sure why it seemingly 60 seconds during build. My guesses are that bundling our large (~30MB) wasm binary is expensive for WebPack, or there's some costly discrepancy in the initialization code between the two modes.

asyncWebAssembly vs syncWebAssembly

If you do use --target bundler then one more bit of Webpack configuration is necessary. These are the somewhat confusing syncWebAssembly and asyncWebAssembly flags which make  reference to a Mozilla spec and ultimately to this useful page. sync is the old deprecated version, while async is the newly specced version. The new proposal would turn WebAssembly packages into asynchronous ES Modules, moving the initialization that has to happen after asynchronous loading into the WebAssembly package itself. This means bundlers wouldn't have to worry about calling initiateStreaming - simply importing would do the trick.

As best as I can tell, using asyncWebAssembly with WasmPackPlugin currently breaks the cleaner import syntax I described above unless you use a bootstrap file as your Webpack entry point:

// webpack.js
...
entry: {
  wasm: ['./src/bootstrap.js'],
},
experiments: {
  asyncWebAssembly: true,
}
...
// src/bootstrap.js
import('./index.tsx').catch(e =>
  console.error('Error importing:', e),
);

This triggers Webpack to do the importing asynchronously. For now the deprecated syncWebAssembly still works correctly.

webpack compiled successfully 😌

While getting things to compile initially takes some hacking, the experience once you have things working is a pleasure. Rust compiled to WebAssembly is fast, has low memory usage, and has a comprehensive type system that lets you write performant and bullet-proof code. It has easy access to the DOM and even good interoperability with TypeScript.

If you'd like to work on cutting-edge technology building a frontend for the modern data stack, join us!

Citations

https://github.com/WebAssembly/esm-integration/tree/main/proposals/esm-integration

https://github.com/webpack/webpack/issues/11347

https://developer.mozilla.org/en-US/docs/WebAssembly/Loading_and_runninghttps://pencilflip.medium.com/using-es-modules-with-commonjs-modules-with-webpack-2cb6821a8b99

https://www.simplethread.com/javascript-modules-and-code-bundling-explained/