Storage APIs: Downloading Files for Offline Access – David Bushell – Freelance Web Design (UK)

By admin

Monday 2 Oct 2023
DATE

In my ongoing adventure to build a personal media web app I keep finding new web APIs to make it more awesome. I’ve adopted the Internationalization API for natural sorting and relative dates and the Media Session API for native integration.

Next I’m improving my use of file storage APIs to download and cache audio files for offline playback. A feature that has long been a pain to implement.


Turtle
ORG

is a little demo project I built to understand the new APIs.

Keep reading to learn more!


Safari
PERSON

is back!

Previously my media app was using IndexedDB to store files. IndexedDB is not really designed for that nor it is a pleasant API to use. It was however the best method I found. Except for the time

Safari
PERSON

14 broke IndexedDB.

I use an iPhone so I’m stuck with

Safari
PERSON

(WebKit) which has historically lagged behind on web standards support. Well not long after I had implemented an IndexedDB store,

Safari 15.2
ORG

added some support for the

StorageManager
PRODUCT

API. Now

Safari
PERSON

17 has improved support and removed the

1 GB
QUANTITY

limit:

Now, the origin quota is calculated based on total disk space. This means an origin generally gets a much higher limit, and users will no longer receive permission prompts in

Safari
GPE

.

The Storage APIs and

OPFS
ORG

(origin private file system) give websites access to file storage. This is perfect for my app to download and store audio (usually MP3 between 100–1000 MB).

Safari
PERSON

on my

iPhone
ORG

reports over

38
CARDINAL

GB quota via

navigator.storage.estimate
ORG

() .

Safari
PERSON


17
CARDINAL

on macOS gives me over 76 GB. Firefox limits me to

10 GB
QUANTITY

.

Getting a list of files is as simple as:

const dir = await navigator . storage . getDirectory ( ) ; for await ( const [ key , value ] of dir . entries ( ) ) { }

No more IndexedDB transactions and events! You will not be missed.

Stream Download Progress

Reading and writing files is almost as easy. You can even stream a file directly to storage and track download progress. I’ve whipped up a demo project. Find it at GitHub,

CodePen
ORG

, and deployed at turtle.deno.dev.


Safari
PERSON

doesn’t support createWritable for a writable stream (streams are seriously cool). It does support

createSyncAccessHandle
PERSON

which is only available inside a

Web Worker
ORG

context (to avoid blocking the main thread). I created a worker that accepts a URL message and posts back download progress.

A reduced example of the worker code:

self .

addEventListener
PERSON

( ‘message’ , async ( ev ) => { download ( ev . data . url ) ; } ) ; const download = async ( url ) => { const response = await fetch ( url ) ; const reader = response . body . getReader ( ) ; const length = Number . parseInt ( response . headers . get ( ‘content-length’ ) ) ; const root = await navigator . storage . getDirectory ( ) ; const handle = await root . getFileHandle ( ‘example’ , { create : true } ) ; const writer = await handle .

createSyncAccessHandle
PERSON

( ) ; let read = 0 ; while ( true ) { const { done , value } = await reader . read ( ) ; if ( done ) break ; writer . write ( value ) ; read += value . length ; self . postMessage ( { url , read , length } ) ; } writer . close ( ) ; self . postMessage ( { url , length } ) ; } ;

This was possible with

IndexedDB
NORP

before but you have to temporary store the chunks in memory before writing a

Blob
PERSON

to the database.

My example code is missing all the error handling.

One
CARDINAL

lazy approach would be to wrap the entire download function in a try/catch statement and post back an error message.

A neat trick is to use an

AbortController
ORG

that allows you to cancel the download.

try { const controller = new

AbortController
ORG

( ) ; const response = await fetch ( url , { signal : controller . signal } ) ; } catch ( err ) { } controller . abort ( ) ;

Good stuff!

The only bad thing about these APIs is their names. There is the “Web Storage API”, aka “Storage”, which is the key/value local & session storage.

The “Storage API
ORG

” and

the “File System
ORG

API” which seem to overlap.

The “Storage Standard
ORG

” which attempts to consolidates them all into

one
CARDINAL

API to rule them all, including the

IndexedDB
NORP

and

Cache
ORG

APIs, I think.

Check out my demo at turtle.deno.dev to see it in action.

You can find my media player meSonic² on GitHub (beware: work in progress!)