22 May 2018, 19:37

Examining Web Worker Performance

Recently I’ve been writing about Web Workers and various options developers have for leveraging them in their applications. For those unfamiliar, Web Workers allow you to create a separate thread for execution in a web browser. This is powerful as it allows work to be done off the main thread which is responsible for rendering and responding to user events. Over recent times we’ve seen a growth in Web Worker libraries such as greenlet, Workerize and Comlink to name a few.

One thing that’s been swirling around in my head is the question of what is the tradeoff of using a Web Worker? They are great because we can leave the main thread for rendering and responding to user interactions, but at what cost? The purpose of this post is to examine empirically where Web Workers make sense and where they might improve an application.

Benchmarking Web Workers

I set out trying to benchmark the performance of Web Workers within the browser. This data was collected based on code I wrote which manifested as a hosted app which can be found here. All the performance numbers specified are on my Dell XPS, Intel Core i7-4500 CPU @ 1.80GHz, 8GB of RAM, running Xubuntu. References to Chrome are version 66 and for Firefox version 59. To preface there is some possibility that numbers are slightly skewed due to garbage collection which is automated by the browser.

Web Worker Performance

At a high level creation and termination of Web Workers is relatively cost free depending on your tolerance for main thread worker:

  • Creation normally comes in at sub 1ms on Chrome
  • Termination is sub 0.5ms on Chrome

The real cost of Web Workers comes from the transfer of a data from the main thread to the Web Worker (worker.postMessage) and the return of data to the main thread (postMessage - onmessage).

This graph reflects this cost. We can see that increased data transfer sizes result in increased transfer times. More usefully we can deduce:

  • Sending an object of 1000 keys or less via postMessage comes in sub-millisecond, and 10,000 is ~2.5ms on Chrome
  • Over this we have more noticeable transfer costs to the worker; 100,000 comes in at ~35ms and 1,000,000 at ~550ms again on Chrome
  • onmessage timings are fairly comparable to this, although coming in slightly higher

Greenlet Performance

There has been an open issue on Jason Miller’s greenlet library for a while now which asks about the performance implications of using the library. As such I extended my research to also explore the library.

Overall Greenlet performance is slower than inlined Web Workers when you combine posting to and from the worker thread. This comes to ~850ms vs ~1700ms (i.e. around double) in Chrome at the 1,000,0000 key level but is slightly less pronounced in Firefox at ~1500ms vs ~2300ms. It’s difficult to deduce why this is the case and may have something to do with the ES6+ to ES5 transpilation process in Webpack or some other factor that I am unaware of (please feel free to let me know if you have an idea!). Overall, however, it’s a substantially easier abstraction for developers to deal with, so this needs to be taken into consideration. The main takeaways for people interested in using greenlet in anger are:

  • Objects with sub 10,000 entries greenlet appear to be sub 50ms for Chrome and Firefox
  • Increase fairly linear after that point (~150ms for 100,000 vs ~1500ms for 1,000,000)

Data Transfer Using JSON.stringify and JSON.parse

Using stringify and parse appears to yield fairly comparable results on Chrome but generally performs better than passing raw objects on Firefox. As such I would recommend having a look for yourself at the demo here to make your own conclusions or test with your own data.

Browser Differences

On average Chrome outperforms Firefox especially under heavier data transfers. Interestingly it performs substantially better at postMessage back to the main thread from the worker by up to a factor of three, although I am unsure as to why. I would love to hear more about how this works on Safari, Edge and other browsers. Again here JSON.stringify and JSON.parse might behave differently on these browsers.

Transferables

Transferables behave more or less as expected with a near constant transfer cost; transfers of all sizes to and from the worker were sub 10ms.

The underlying idea here you can transfer values of type ArrayBuffer, MessagePort, ImageBitmap comparatively cheaply, which is a going to be a large performance boost if you’re using Web Workers, especially if your data is any considerable size. For example, you might transfer geometries as a Float32Array for speedier transfer.

Talking Points

This is the point at which we look at ways of making decisions around when to use Web Workers. Ultimately this is not a simple question but we can make some inferences from the data collected.

Smaller Workloads and Render Blocking

Objects of sub 50,0000 entries (or equivalent complexity) are on average in Chrome going to be less than 16ms to execute a postMessage and shouldn’t have too much noticeable effect on render performance for the user (i.e. there is some possibility that a frame render or two is skipped). However, overall using a Web Worker in a worse case in this situation will add up to ~50ms of overall processing time on top of the work the worker actually has to do. The trade-off is not blocking the main thread with heavy work, but taking a little extra time for the results to come back.

Large Workloads

Transfering over 100,000 entry object (or equivalent) is most likely going to have a noticeable blockage on the main thread because of the cost of postMessage. A recommendation could be to batch up heavy work into multiple postMessages so that the chance of frame rendering being blocked at any point is substantially reduced. You could even spin up a pool of workers and implement prioritisation strategies (see the fibrelite library I worked on for inspiration). Furthermore it may be worth considering if the data can be turned into Transferables which have a fairly minimal constant cost which could in turn be a massive performance boost.

The Trade Off

Ultimately here we are trading off transfer time to and from the Web Worker in exchange for preventing long render blocking tasks in the main thread. You may make overall times to results longer but prevent poor user experience in the process, for example a janky input or scroll experience.

If you can avoid your object transfers to and from the worker being render blocking you can move complex processing over to a Web Worker with the only cost of being the transfer times. We have shown for simple objects (1000 keys or less) should be sub-millisecond.

Final Thoughts

Hopefully this data and commentary has helped explore the cost and benefits of Web Workers. Overall, we have shown how they can be a big win in the right situations. Blocking the main thread with heavy work is never going to be great for user experience, so we can make use of Web Workers here to prevent that, especially when transfer times are low (smaller data loads). For larger loads it might be worth batching work to prevent extensive blocking postMessages.

Although Web Workers are very useful, they do have a cost; the transfer times increases the overall time to work being finished, and they can add complexity to a code base. For some cases, this tradeoff might be undesired. In these situations it might be worth exploring using requestAnimationFrame and requestIdleCallback with batched workloads on the main thread to keep rendering and user interactions fluid.

It’s also worth concluding with the idea that Web Workers can be used for things other than simply running long running tasks however. David East recently wrote an article about wrapping the Firebase JavaScript SDK into a Web Worker which means the importing and parsing of the SDK is handled in the worker, leaving the main thread able to handle user input, reducing the First Input Delay (FID).

Lastly I think there is some strong potential for Web Workers to become core elements of some web frameworks. We’ve seen this jump recently in the start of React Native DOM by Vincent Riemer which tries to move work off the main thread into worker threads leaving the main thread for rendering and handling user input. Time will tell if this takes off as an approach!

23 Apr 2018, 19:37

The Rise of JavaScript Scheduling

What is scheduling?

At a high level, scheduling can be thought of as a way of splitting up work and allocating it to be completed by a compute resource. The work can be processed at some point in the future, in an order, and by a given resource that the scheduler specifies.

For example, if we had a set of tasks of equal importance, lets say calculating billing and distributing invoices for users of your Software as a Service platform, we could use a scheduler to distribute that work in a way that was evenly distributing processing time to each task. If the tasks were of differing importance, we could allow the scheduler to prioritise work, ensuring more processing time was allocated to performing higher prioritised tasks. For instance, you might prioritise the work loads of paying customers over free users.

At a more granular level there are many different approaches of varying complexity to scheduling work. Some examples of mainstream scheduling strategies include: first in first out, earliest deadline first and round robin. The wiki for this is actually really strong, so I’ll defer to that for an in depth explanation.

Why schedule work in JavaScript?

Let us consider why a scheduler may be of value in JavaScript. In web development we have the case that:

“By default the browser uses a single thread to run all the JavaScript in your page as well as to perform layout, reflows, and garbage collection. This means that long-running JavaScript functions can block the thread, leading to an unresponsive page and a bad user experience”. - MDN

There are ways to offload work onto other threads via Web Workers, but these have limitations such as not being able to access the DOM, and having to copy data from the main thread and back again (RIP SharedArrayBuffers). In some ways the single threaded-ness of JavaScript is useful; we would have a complex overhead of managing multiple race conditions of threads trying to access the DOM.

Fundamentally, the JavaScript thread of the browser works by way of the event loop, which cycles round executing queued work to be performed. At a high level we have three major types of work that the the event loop processes:

  • Tasks - Event handlers, setTimeouts, setInterval, etc
  • Microtasks - MutationObserver callbacks, Promises - these get executed whenever the JavaScript call stack empties
  • Rendering steps - requestAnimationFrame queued work, Style, Layout, Paint

Knowing this helps us write and/or understand a well planned scheduler in conjunction with our own code. We can break up long running tasks into smaller tasks and interleave them with other work, for example performing layouts and repaints in an efficient manner.

The specifics of the event loop are much better left to others; specifically I would recommend Jake Archibald’s talk at JSConf Asia 2018 which is a superb elucidation on the subject (he also has a blog post).

User experience and scheduling

A recurring problematic theme in web applications is the the idea of jank; low frame rates and interactivity for end users. Having talked a bit about the event loop, how might we leverage that understanding to better improve our sites user experience? One prime example is Wilson Page’s fastdom which was one of the first schedulers I came across in late 2013. The core premise is that it’s possible to batch up DOM reads and writes, and then schedule them using requestAnimationFrame for noticeably smoother animations. requestAnimationFrame allows developers to schedule DOM updates right before the browser performs the next render cycle (style, layout, paint, composite). This prevents work being done mid frame causing it to miss the frame as shown in the following diagram (thanks Google). Clever stuff!

This approach can noticeably improve user experience for DOM heavy sites. However, fastdom as a library is predominately about preventing layout thrashing, and generic work scheduling is outside of its scope. Furthermore,requestAnimationFrame is arguably not an appropriate tool in and of itself to handle generic work; each request will run in it’s intended frame and does not distribute them across multiple frames.

Adoption by frameworks

Within the past few of years we’ve seen an increased interest in scheduling work by frameworks. The single threaded nature of JavaScript, and the plethora of tasks that need to be completed to allow users to navigate a modern web application pose an interesting challenge - especially for framework developers. We may wish to be animating elements whilst also accepting user input, and sending off that input to a server. There is a possibility that user input and associated handling could cause our frames to take too long to render, resulting in a rough user experience.

Arguably one of the earlier stage examples of scheduling is the AngularJS (1) digest cycle. AngularJS came out in 2010, and it had its own built in event cycle (a scheduler of sorts) for handling its notorious two way data-binding system. An overview diagram can be seen below. The digest cycle checks for changes between the view data model and the DOM, and then re-renders after the cycle to reflect those changes. Certain elements of the cycle, by the documentations own omission, could be considered problematic, for example setTimeout(0) can be janky as the browser repaints after every event.

Angular, AngularJS’s successor has a different approach to change detection and updates which can be read about here. It also does some interesting things with asynchronous execution contexts (zones), allowing for a smarter way of doing operations like updating the DOM, error handling and debugging.

Vue.js takes the approach of batching DOM updates asynchronously. There is no ‘digest cycle’ as per AngularJS as Vue.js encourages a data driven approach. Here the batched operations are then flushed out to the DOM on a given tick cycle. The queue internally uses Promise.then and MessageChannel, with a fall back to setTimeout(fn, 0) if those aren’t available.

Other frameworks have explored a holistic view of how to handle the stream of user interactions, network requests, data flow and rendering. A prime and recent example of this is React. React Fiber, fully implemented in React 16, uses scheduling to improve perceived performance and responsiveness in complex web applications). In the world of the web not all computational work may be of equal importance to a user. For example typing and receiving immediate feedback may be a more critical interaction than having a dashboard receiving external data and updating instantaneously. The React team has done a lot work to it’s reconciliation algorithm to prioritise work in a way that is conducive to a pleasant user experience. This is done fundamentally by scheduling different updates at different priorities. Traditionally, all updates were treated synchronously, with no prioritisation. React’s reconciliation phase was uninterruptible, which could lead to low framerates for complex work loads. Here is an example of the Chrome profiler showing it taking ~750ms to render a frame (the green bar):

The way in which React Fiber attempts to keep consistent framerates is via what they have dubbed time slicing. The process uses requestIdleCallback to defer low priority work to the browser’s idle times. Fiber also estimates the number of milliseconds of time remaining in the current idle stage, and when this elapses stops work to give the browser time to render a frame. For deeper explanation, check out Giamir Buoncristiani’s post, the React Fiber Architecture README by Andrew Clark and also Dan Abramov’s talk at this years Iceland JS Conf. In Dan’s demonstration you can clearly see the difference between the synchronous and asynchronous work patterns:



Another project that has (very recently) been leveraging scheduling is StencilJS. For those of you who aren’t familiar, StencilJS is a modern web framework that takes popular features from many recent frameworks, and combines this with compilation down to native Web Components. Manu Mtz-Almeida of the StencilJS team has recently been pushing commits for scheduling into the core framework, demoing some of his recent work on Twitter:



You can see that the end goal is similar here to what React is doing with Fiber; try and keep the browser rendering at 60fps for a smooth user experience, whilst still completing non-rendering work in reasonable time-frames. Hopefully we’ll be seeing more from the Stencil team in the future!

The building blocks of scheduling in the browser

There are many ways we could build a scheduler in JavaScript. Traditionally we might have implemented it using JavaScript and browser APIs such as:

  • setTimeout - Execute a given function at some point in the future
  • setInterval - Execute a given function on some recurring schedule (every x milliseconds)
  • Promises - An object representing the eventual success or failure of asynchronous operation

With ever improving browser standards we also have some interesting additional browser APIs that might help us to write smarter schedulers:

  • requestIdleCallback - Schedule work to be done during the browsers idle periods. Supports deadlines.
  • performance.now - Granular timing API for modern browsers
  • async/await - Allows developers to write easier and cleaner asynchronous code, making it feel more synchronous

As well these, there is the previously mentioned requestAnimationFrame for visual changes. Interestingly React was originally relying on requestIdleCallback for Fiber, but now they’ve written their own polyfill for this (at time of writing). Indeed there is no single way to write a scheduler, and you could use a myriad of these features to create one. Getting this right appears to be a relatively tricky endeavour; in the words of Bertalan Mikolos “timing is a delicate thing, and slight mistakes can cause some very strange bugs.”.

Personally, the strongest example so far I’ve seen of a generic purpose scheduler is Justin Fagnani’s queue-scheduler a framework agnostic JavaScript scheduler. Here he uses async/await performance.now(), requestIdleCallback and requestAnimationFrame to allow developers to schedule work. It’s worth examining the source code to see how these are used (FYI: it’s written in TypeScript).

Final thoughts

Overtime we have seen a more progressive approach towards scheduling. Modern schedulers such as those in React and StencilJS have been written in a way that keeps end users at their heart, keeping frame rates and interactivity high. It is fair to say that with React (arguably the most popular JavaScript framework in modern applications) having taken scheduling to the core of it’s architecture, we have seen scheduling become mainstream for web developers. We also see API compatible libraries such as Preact looking to follow suit.

With teams like StencilJS following suit with their user centric scheduler, there is strong evidence to suggest that smarter scheduling may become commonplace across many approaches to building web applications. I haven’t seen much work done with Web Workers and scheduling, but feel this could be a strong contender for future work as inline Web Worker libraries have become more popular and Web Workers are non blocking on the main thread. I think this is especially true for long running tasks, see for example a little demo I did making a library called Fibrelite for offloading processing to an inline Web Worker.

03 Apr 2018, 19:37

Cancelling Requests with Abortable Fetch

There are often times in a web application where you need to send a request for the latest user input or interaction. Some examples might be a autocomplete or zooming in and out a map. Let’s think about each of these examples for a moment. Firstly autocomplete; every time we type (or maybe less if we were to debounce) we might send out a request. If the user input changes the old requests might become irrelevant as we keep typing (i.e. ‘java’ and ‘javascript’). That’s potentially a lot of redundant requests before we get to what we’re interested in!

Now the web map case; we’re zooming and panning around the map. As we zoom in and out, we are no longer interested in the tiles from the previous zoom levels. Again, lots of requests might be pending for redundant data.

Taking the first example, let’s set the scene by looking at some naive code about how we might implement an autocomplete. For the purpose of this article we will be using the more modern fetch rather than XMLHttpRequest for making a network request. Here’s the code:


    autocompleteInput.addEventListener('keydown', function() {

        const url = "https://api.example.com/autocomplete"

        fetch(url)
            .then((response) => {
                // Do something with the response
                updateAutocompleteMenu()
            })
            .catch((error) => {
                // Something went wrong
                handleAutocompleteError(error);
            })

    });


The problem in this case is that each one of these requests will complete, even if it is no longer relevant. We could implement some extra logic in the updateAutocompeleteMenu to prevent unnecessary code execution but this won’t actually stop the request. It’s also worth noting here that browsers have a limit of outgoing requests which means that they queue requests once that limit is hit (although that limit varies by browser).

Abortable Fetch

A new browser technology that we can leverage to solve the aforementioned issue is Abortable Fetch. Abortable fetch relies on a browser specification for AbortController. The controller has a property called signal which we can pass to our fetch as an option (also named signal), and then use this at our later convenience to cancel the request with the controllers abort method.

An example might look a little like this:


    const url = "https://api.example.com/autocomplete"
    let controller;
    let signal;

    autocompleteInput.addEventListener('keyup', () => {

        if (controller !== undefined) {
            // Cancel the previous request
            controller.abort();
        }

        // Feature detect
        if ("AbortController" in window) {
            controller = new AbortController;
            signal = controller.signal;
        }

        // Pass the signal to the fetch request
        fetch(url, {signal})
            .then((response) => {
                // Do something with the response
                updateAutocompleteMenu()
            })
            .catch((error) => {
                // Something went wrong
                handleAutocompleteError(error);
            })
        });
    
    });

Here we do feature detection to determine if we can use AbortController (it’s supported in Edge, Firefox, Opera and coming in Chrome 66!). We also determine if a controller has already been created, and if so we call controller.abort() which will cancel the previous request. You can also use the same signal in multiple fetches to cancel multiple fetches at once.

A little demo

I’ve created a small demo showing how to use Abortable Fetch, loosely based on the idea of the autocomplete idea (without any of the implementation details!). What happens is every time you type it makes a network request. If you make a new keystroke before the old request has completed it will abort the previous fetch. It looks a little something like this in practice:

You can check the code out here.

Thinking beyond fetch

Perhaps the coolest part about AbortController is it has been designed to be a generic mechanism for aborting asynchronous tasks. It is part of the WHATWG specification, meaning it is DOM specification rather than a language (ECMAScript) specification, but for frontend development this is still a useful feature. You could leverage it as a cleaner async control flow mechanism for times you implement asynchronous tasks (i.e. when using Promises). Feel free to take a look at Bram Van Damme super article for a more detailed example of what I’m talking about.