Guide for Experiments

This document aggregates information on topics that commonly arise when developing a new experiment for Thunderbird 78. For a complete documentation on each individual topic, refer to the linked articles.

Thunderbird does contain a few useful features related to experiments whose documentation is no longer generated. Especially if you're writing an experiment with complex interactions between the WebExtension and your experiment, it may be helpful to read the documentation blocks within resource://gre/modules/ExtensionCommon.jsm and possibly other modules in the same source code folder.

Designing the API

When porting an add-on, it is easy to think about everything in the context of your existing add-on. But that view is likely limiting your options: when you encounter a feature that cannot get implemented with existing WebExtension APIs, it may be helpful to first think about alternative ways to present similar functionality – maybe it is possible to implement the same high-level feature in a different way?

If not, you should first read through the documentation of some buit-in APIs, this document and some linked documentation to get a feel about how existing APIs encapsulate common problems (i.e. listener registration through events, excessive use of Promise, ...) and what limitations your API will have to live with (i.e. potential complexity when passing functions or raw DOM elements).

Afterwards, think about (hypothetical or real) add-ons that would have similar needs and try to write APIs that would be useful for a wide range of add-ons:

  • A good API provides a single feature with as little complexity as possible (smaller is usually better).

    Example: an API adding both a menu item and a toolbar item for a single callback function might be useful, but it would be better to split it in two separate operations and combine them in the WebExtension.

  • A good API can be used in a generic way and is not tied to your add-on's logic or functionality.

    Example: an API to add a menu item that will perform a fixed action might solve the specific issue you're working on, but it would be better to permit WebExtension code to dynamically specify the action to perform.

  • A good API adheres to the conventions of WebExtension APIs.

    Example: a registerListener function might be a good idea in isolation, but it would be better to use events (that also simplifies the implementation!).

Depending on your time constraints, experience, and the concrete feature you're trying to write an API for it is not necessarily reasonable to satisfy all three criteria, they are just a guideline to aim for.

Building the Structure

Once you have a draft for your API, you can start to build the experiment. Experiments consist of three parts, which are registered through manifest.json:

  1. A schema describes the API that can be accessed by the WebExtension part of the add-on.

  2. A parent implementation implements the API in Thunderbird's main process. All features that were available to a bootstrapped legacy extension can be used here.

  3. A child implementation implements the API in the content process(es). This permits more complex interactions with WebExtension code and potentially improves performance, at the cost of not being able to access the main process.

Either parent or child implementation may be omitted. Full examples for a simple function with parent and child implementations and events add-ons can listen for are available in the Firefox source documentation.

Technically speaking, Thunderbird 78 is not actually using multiple processes (yet). However, the APIs were designed with multiple processes in mind and enforce at least some constraints as if the parts were in different processes.

In most cases, you can start by formalizing your API draft into a schema and adding a parent implementation using one of the linked articles as base. Add or switch to a child implementation if you have performance considerations or need to pass more complex data (see below).

Check out the experiment generator, which creates all the needed files.

Avoid declaring global variables in either implementation of your experiment, as that can cause collisions with other experiments loaded. Instead declare them as members of your API (example).

Managing your Experiment's lifecycle

Experiments do not have a dedicated lifecycle. Whenever some part of your WebExtension attempts to use the Experiment's API, the experiment is loaded into that context. As there can be multiple contexts at the same time, an Experiment may be loaded multiple times in parallel. This means that you cannot directly observe when the add-on is loaded, nor when it is unloaded. This is especially problematic if you load JSMs or manage native resources, but also affects all other experiments due to Thunderbird's caching behavior:

All experiments must invalidate all caches after finishing all other unloading tasks:

const { Services } = ChromeUtils.import("resource://gre/modules/Services.jsm");
Services.obs.notifyObservers(null, "startupcache-invalidate", null);

Failure to invalidate caches may cause parts of the experiment to be cached across updates of the Add-on, even if they are changed in the update.

There are multiple approaches on how to manage your experiment's lifecycle effectively:

Restricting (parts of) your Experiment to the background page

The simplest option is to only use your experiment in the background page, and use context.callOnClose() to perform unloading tasks in your parent implementation.

A slightly more elaborate strategy is to do all loading in an dedicated API function that is called once from the background page only (usually called init()) and to register the close handler in that particular method.

As long as you don't load resources from a client implementation or after the background page unloaded, both options are reasonably safe.

Observing the background page's context

Instead of using some protocol the background page must follow, you can also directly register a close handler for the background page's context (independent of whether you're loaded there or not) using

const backgroundContext = Array.from(context.extension.views).find(
view => view.viewType === "background");
backgroundContext.callOnClose(/* ... */);

This method will, of course, also only work if your add-on has a background page.

Counting contexts

Of course you can also count in how many contexts the experiment is loaded and unload once that number reaches zero. This option is probably the most stable one, but properly implementing reference counting in a future-proof way is not as simple as it sounds. It is thus recommended to go with one of the other options for now (adding a background page if necessary).

Passing data to / from an WebExtension

In general, you can always pass simple data structures as function parameters and return values of an API. Thunderbird will automatically adapt them using the structured clone algorithm, so you do not need to worry about them.

If you want to pass more complex data structures, especially functions or instances of custom classes, you can do so form a child implementation. There is a big caveat, though: the experiment's scripts are privileged relative to WebExtension scripts, which causes their scopes to be disjunct:

  • Accessing data that belongs to the WebExtension from an Experiment: the code in the experiment gains xray vision, permitting it to directly access the chrome implementation of the given object. Usually, you don't need to worry about that and things work out just fine – but if they don't, you can opt-out via Components.utils.waiveXrays().

  • Accessing data that belongs to an Experiment from a WebExtension: it is not possible to directly access chrome-scoped objects from a WebExtension (but it can hold references on it).

    There are two options to work around that: either the experiment clones the object into the unprivileged scope of the WebExtension or it directly constructs an unprivileged object.

    The first option usually boils down to invoking Components.utils.cloneInto() or a related function with context.cloneScope as target scope. A notable exception is returning data from async API functions or wrapping Promises via context.wrapPromise(), which causes automatic cloning of the result (unless the result is wrapped into a ExtensionCommon.NoCloneSpreadArgs).

    The second option is using the constructors in context.cloneScope directly from the experiment. Their results can be used from the WebExtension without further cloning.

Common pitfall: async functions return a Promise in the scope of the function, so you need to wrap such functions before cloning them into a WebExtension scope.

Structuring experiment code

If your experiment is so complex that it does not reasonably fit into a single source file, you can use JavaScript modules just like in legacy extensions with some additional boilerplate:

const { ExtensionParent } = ChromeUtils.import(
"resource://gre/modules/ExtensionParent.jsm");
const extension = ExtensionParent.GlobalManager.getExtension(
"insert-your-extension-id-here@example.com");
const { /* ... exported symbols ... */ } =
ChromeUtils.import(extension.rootURI.resolve("path/to/module.jsm"));
// when unloading: (safe to call even if the import is conditional / elsewhere!)
Components.utils.unload(extension.rootURI.resolve("path/to/module.jsm"));

Do not use moz-extension://*URLs obtained by extension.getURL() in experiments, but instead use extension.rootURI.resolve() to get the raw file://* or jar://* URL. Some calls may have issues with these raw URLs as well (e.g. new ChromeWorker()). In that case you need to manually register a chrome://* URL, as shown in our WindowListener API example extension, and always use that URL when referring to the JSM.

Using different URLs for the same JSM will cause multiple instances of the same JSM to be loaded. These instances will NOT share the same scope and need to get unloaded separately.

The global scope of a JSM is cleared on unload. Unloading JSMs that are still used is only safe if they don't use the global scope and don't store any state information.

Accessing WebExtensions directly from an experiment

Experiments should be standalone and have no dependencies to any WebExtension components. If possible, design your experiments as if they were a feature of Thunderbird that does not know about your add-on, and keep the scope as narrow as possible. If you design your APIs correctly, you will not need to access the WebExtension part of your add-on.

In some rare cases, it may be possible that that is not feasible to follow this recommended practice for the current update cycle. In that case, it is possible to tear down the separation between the experiment and the WebExtension. That can permit you to treat an experiment as if it were a part of the WebExtension, except for the restrictions regarding passing data outlined above.

In a child implementation, you can directly access the real WebExtension scope via Components.utils.waiveXrays(context.cloneScope).

Outside of a child implementation, or if you need the scope of the background page in particular, you can extract the background page's scope from an extension object:

const webextScope = Array.from(extension.views).find(
view => view.viewType === "background").xulBrowser.contentWindow
.wrappedJSObject;

This hack only works because Thunderbird is internally not (yet) using multiple processes. Again, it is highly recommended to design your APIs in a way that these interactions are not necessary as it is likely that this technique will stop working in future versions of Thunderbird.