Experimenting with the proposed Cross-Origin Storage API in Transformers.js

(This is a guest post by Developer Relations Engineer Thomas Steiner from the Chrome team at Google.)

Transformers.js provides Web developers with a simple way to use the power of transformers in their Web apps through task-specific pipelines. To run inference in the browser, developers create an instance of pipeline() and specify a task they want to use the pipeline for. As a concrete example, the following snippet shows how to set up an automatic speech recognition (ASR) pipeline.

You will notice in the source code that I specified Xenova/whisper-tiny.en as the model, which is a very decent choice for common English automatic speech recognition tasks. In fact, it's even the default model according to the Transformers.js default model resolution , as per the linked excerpt .

The proposed Cross-Origin Storage (COS) API introduces a dedicated navigator.crossOriginStorage interface through which web apps can store and retrieve large files across origin boundaries, identified not by a URL, but by a cryptographic hash.

That last point about cryptographic hashes is key. Because COS identifies files by their hash rather than by their URL or origin, the same ort-wasm-simd-threaded.asyncify.wasm Wasm runtime you downloaded while visiting https://googlechrome.github.io is recognized as identical to the one https://rawcdn.rawgit.net is about to request, no matter where either of the two origins fetched it from. See the following code snippet that illustrates the basic flow.

The API is deliberately shaped after the File System Standard 's FileSystemDirectoryHandle.getFileHandle() you likely are familiar with from the Origin Private File System (OPFS) API. The hash parameter plays the same role as the name parameter in OPFS: uniquely identifying a resource. The options.create flag works the same way: absent or false for read-only access, true when you intend to write.

This turns out to be doubly useful in the Transformers.js scenario. Today, after downloading model weights, most apps have no practical way to verify that the CDN served the right bytes. With COS, every file in the store is implicitly verified on write, no matter where it came from, the official Hugging Face CDN or a random site's self-hosted mirror.

Going back to the toy examples from before: the ort-wasm-simd-threaded.asyncify.wasm runtime weighs in at 4,733 kB and is shared by every Transformers.js-powered app regardless of which AI model it uses. With COS, the first app to load it downloads it once and stores it under its SHA-256 hash with origins: '*' . Every subsequent app, whether on https://googlechrome.github.io , on https://rawcdn.rawgit.net , or any other origin, finds it in COS immediately. The 177 MB of duplicate Whisper model weights? Same story: Xenova/whisper-tiny.en gets downloaded once, recognized by hash the second time around, and served from COS in milliseconds. And of course, the same also happens for Xenova/distilbert-base-uncased-finetuned-sst-2-english .

One important rule: visibility can be upgraded but never downgraded. If a file is already globally available, a later attempt to store it with a restricted origins list is silently ignored. This prevents a malicious actor from re-storing a public resource and narrowing its availability. The reverse is possible: a file initially stored with a restricted origins list can later be made more permissive. Any site, not just the original storer, can call getFileHandle() for the same hash (hashes are not a secret) with create: true and a broader origins value, and given the browser verifies the hash matches, the resource becomes available to the wider audience from that point on. Note that the upgrading site must still write the full file through the returned handle. This requirement exists to prevent sites from exploiting the upgrade path as a side-channel to detect whether a particular file was already stored in COS.

When you run this example in the browser , Transformers.js automatically takes care of downloading and caching the relevant model resources and Wasm files. The following screenshot shows the Chrome DevTools Cache storage section after visiting the app. When you reload the page, the resources are served from the Cache API , and the model returns results almost instantly.

However, Xenova/whisper-tiny.en being a popular model (and, as mentioned before, even being the ASR default model in Transformers.js), you can well imagine that more than just one app that you visit would use it. To simulate this situation, here's the same example app from before, but served from a different origin . When you visit this different origin app, rather than being usable almost instantly, the browser instead has to download and cache all the model resources again, even if they're byte-by-byte the same as before. Even in this toy example, this adds up to 177 MB of duplicate download and storage, as you can examine in the Storage section of the Chrome DevTools Application panel . You can imagine that this quickly adds up.

But it gets worse. Let's add a second pipeline to the toy example: sentiment analysis. Sentiment analysis by default uses the Xenova/distilbert-base-uncased-finetuned-sst-2-english model. By not specifying the model, Transformers.js' default model resolution automatically picks it for you.