Fetch API

The Emscripten Fetch API allows native code to transfer files via XHR (HTTP GET, PUT, POST) from remote servers, and to persist the downloaded files locally in browser’s IndexedDB storage, so that they can be reaccessed locally on subsequent page visits. The Fetch API is callable from multiple threads, and the network requests can be run either synchronously or asynchronously as desired.

Note

In order to use the Fetch API, you would need to compile your code with -sFETCH.

Introduction

The use of the Fetch API is quick to illustrate via an example. The following application downloads a file from a web server asynchronously to memory inside the application heap.

#include <stdio.h>
#include <string.h>
#include <emscripten/fetch.h>

void downloadSucceeded(emscripten_fetch_t *fetch) {
  printf("Finished downloading %llu bytes from URL %s.\n", fetch->numBytes, fetch->url);
  // The data is now available at fetch->data[0] through fetch->data[fetch->numBytes-1];
  emscripten_fetch_close(fetch); // Free data associated with the fetch.
}

void downloadFailed(emscripten_fetch_t *fetch) {
  printf("Downloading %s failed, HTTP failure status code: %d.\n", fetch->url, fetch->status);
  emscripten_fetch_close(fetch); // Also free data on failure.
}

int main() {
  emscripten_fetch_attr_t attr;
  emscripten_fetch_attr_init(&attr);
  strcpy(attr.requestMethod, "GET");
  attr.attributes = EMSCRIPTEN_FETCH_LOAD_TO_MEMORY;
  attr.onsuccess = downloadSucceeded;
  attr.onerror = downloadFailed;
  emscripten_fetch(&attr, "myfile.dat");
}

If a relative pathname is specified to a call to emscripten_fetch, like in the above example, the XHR is performed relative to the href (URL) of the current page. Passing a fully qualified absolute URL allows downloading files across domains, however these are subject to HTTP access control (CORS) rules.

By default the Fetch API runs asynchronously, which means that the emscripten_fetch() function call returns immediately and the operation will continue to occur on the background. When the operation finishes, either the success or the failure callback will be invoked.

Persisting data

The XHR requests issued by the Fetch API are subject to the usual browser caching behavior. These caches are transient (temporary) so there is no guarantee that the data will persist in the cache for a given period of time. Additionally, if the files are somewhat large (multiple megabytes), browsers typically don’t cache the downloads at all.

To enable a more explicit control for persisting the downloaded files, the Fetch API interacts with the browser’s IndexedDB API, which can load and store large data files that are available on subsequent visits to the page. To enable IndexedDB storage, pass the EMSCRIPTEN_FETCH_PERSIST_FILE flag in the fetch attributes:

int main() {
  emscripten_fetch_attr_t attr;
  emscripten_fetch_attr_init(&attr);
  ...
  attr.attributes = EMSCRIPTEN_FETCH_LOAD_TO_MEMORY | EMSCRIPTEN_FETCH_PERSIST_FILE;
  ...
  emscripten_fetch(&attr, "myfile.dat");
}

For a full example, see the file test/fetch/test_fetch_persist.c.

Persisting data bytes from memory

Sometimes it is useful to persist a range of bytes from application memory to IndexedDB (without having to perform any XHRs). This is possible with the Emscripten Fetch API by passing the special HTTP action verb “EM_IDB_STORE” to the Emscripten Fetch operation.

void success(emscripten_fetch_t *fetch) {
  printf("IDB store succeeded.\n");
  emscripten_fetch_close(fetch);
}

void failure(emscripten_fetch_t *fetch) {
  printf("IDB store failed.\n");
  emscripten_fetch_close(fetch);
}

void persistFileToIndexedDB(const char *outputFilename, uint8_t *data, size_t numBytes) {
  emscripten_fetch_attr_t attr;
  emscripten_fetch_attr_init(&attr);
  strcpy(attr.requestMethod, "EM_IDB_STORE");
  attr.attributes = EMSCRIPTEN_FETCH_REPLACE | EMSCRIPTEN_FETCH_PERSIST_FILE;
  attr.requestData = (char *)data;
  attr.requestDataSize = numBytes;
  attr.onsuccess = success;
  attr.onerror = failure;
  emscripten_fetch(&attr, outputFilename);
}