LERC

v4.2.0Imaging

LERC 4.2.0, limited-error raster compression, packaged by crossbind as @crossbind/port-lerc and one package per target. Only a variant that is actually on npm beta is listed as published.

npm install @crossbind/port-lerc-wasm@beta

npm ↗Port recipe and licence files ↗Upstream source 4.2.0 ↗

LIVE · 3 APPS · RUNS IN THIS TAB

LERC in your browser: rasters stored to the error you choose

LERC, from Esri, stores rasters such as elevation so that no value moves by more than the error you pick; ArcGIS elevation services send their tiles in it, and GDAL's GeoTIFF and MRF drivers read and write it. Esri's own lerc package on npm and loaders.gl only decode it. These apps run Esri's C++ library, compiled by crossbind: they encode a terrain at six error budgets and check every height, open LERC tiles, and compare LERC with the browser's gzip. The first run downloads 0.9 MB of WebAssembly once; every app on this page shares it, and nothing is uploaded.

APP 01

Shrink an elevation model to the error you can accept

LERC stores a raster so that no value moves by more than the error you choose. Here a 512 × 512 elevation model is encoded at six budgets, from lossless to 5 m, then every height is decoded and compared with the original. Esri's own lerc package on npm and loaders.gl only decode it; this is Esri's C++ library, compiled by crossbind, encoding in your tab.

TERRAIN

The terrain is generated in the module from integer noise, so the sizes are the same on every machine.

Compress the terrain to see how size and error trade off, budget by budget.

SHOW THE CODE

src/support/grid.h

// src/support/grid.h (excerpt)
// LERC quantizes in double precision and rounds back to float32, which can
// overshoot maxError by half a float32 step: ask for one step less.
const double step = double(std::nextafter(largest, INFINITY)) - largest;
const double bound = maxError > step ? maxError - step : 0;
unsigned int size = 0;
check(lerc_computeCompressedSize(values.data(), kFloat, 1, width, height, 1,
                                 masks, valid, bound, &size), "sizing");
std::string blob(size, '\0');
unsigned int written = 0;
check(lerc_encode(values.data(), kFloat, 1, width, height, 1, masks, valid, bound,
                  reinterpret_cast<unsigned char*>(&blob[0]), size, &written), "encoding");

main.js

const m = await initNative();
const lab = await new m.TerrainLab(0);   // 512 x 512 float32 heights, 30 m cells
await m.FS.mkdirTree('/memfs/lercapps');
 
const packed = JSON.parse(await lab.compress(0.01, '/memfs/lercapps/t.lerc'));
// packed.bytes 464,085: 2.26x smaller than the 1,048,576 B of float32
const back = JSON.parse(await lab.decompress('/memfs/lercapps/t.lerc', '/memfs/lercapps/t.f32'));
// back.maxError 0.009521484375: every height within 1 cm

APP 02

Look inside a LERC tile before you decode it

ArcGIS elevation services send each tile as a LERC blob. Open one to read its header without decoding: size, data type, bands, how many pixels hold a value, the value range and the error it was stored with. Then decode a band and draw it. Legacy Lerc1 tiles, the kind ArcGIS Terrain 3D serves, open too.

Open your own .lerc

The sample is written by the module: the lowland from the first app as an island, the sea below 56 m left out through LERC's validity mask. Your own file stays in this tab: it is mounted into the module's in-memory filesystem, never uploaded.

Open the sample, or a .lerc tile of your own, to read its header and draw a band.

SHOW THE CODE

src/native/lerc_inspector.h

// src/native/lerc_inspector.h (excerpt)
unsigned int info[11] = {};  // version, type, depth, width, height, bands, valid pixels, ...
double range[3] = {};        // zMin, zMax, the largest error the encoder allowed
check(lerc_getBlobInfo(blob, size, info, range, 11, 3), "reading the header");
// [min, max] for every band and value per pixel, still without decoding
lerc_getDataRanges(blob, size, depth, bands, mins.data(), maxs.data());
// then any data type decodes to double, with the validity mask
lerc_decodeToDouble_4D(blob, size, masks, valid.data(), depth, width, height, bands,
                       values.data(), usesNoData.data(), noData.data());

main.js

const m = await initNative();
const [path] = await m.autoMountFiles([file], await m.getRandomPath('/memfs'));   // a .lerc from <input type=file>
 
const info = JSON.parse(await m.LercInspector.inspect(path));
// { codec: 'Lerc2 v6', type: 'float32', width: 512, height: 512,
//   bands: 1, validPixels: 171406, maxZErrorUsed: 0.00999..., ... }
const band = JSON.parse(await m.LercInspector.decodeBand(path, 0, '/memfs/b.f32', '/memfs/b.mask'));
const values = new Float32Array((await m.getFileBytes('/memfs/b.f32')).buffer);

APP 03

Where LERC beats gzip, and where it does not

gzip sees a float32 raster as bytes; LERC sees the values. Four 512 × 512 rasters go through this browser's own gzip and through LERC, losslessly and within a stated tolerance. Terrain and measurements come out smaller with LERC; a map of whole-number classes is where gzip wins.

Each raster is 1,048,576 bytes of float32, generated in the module. The gzip bars come from this browser's CompressionStream and can differ a little between browsers; the LERC numbers are the same everywhere.

Compress the four rasters to compare gzip and LERC on each.

SHOW THE CODE

src/native/dataset_lab.h

// src/native/dataset_lab.h (excerpt): LERC within maxError, decoded back and compared
const std::string blob = grid::encode(values, 512, 512, nullptr, maxError);
const std::vector<float> back = grid::decode(blob);
const bool identical = std::memcmp(back.data(), values.data(), values.size() * 4) == 0;

main.js

const m = await initNative();
const lab = await new m.DatasetLab();
await lab.write(1, '/memfs/temps.f32');   // air temperature, read to 0.01
const raw = await m.getFileBytes('/memfs/temps.f32');
const gzip = new CompressionStream('gzip');
const gzipped = await new Response(new Blob([raw]).stream().pipeThrough(gzip)).arrayBuffer();
 
JSON.parse(await lab.compress(1, 0));       // lossless: 252,904 B
JSON.parse(await lab.compress(1, 0.005));   // 235,281 B, largest error 0.0000019

Usage

The calls most LERC code makes, each a small C++ header crossbind binds and the JavaScript that uses it. Every example runs here in WebAssembly and prints what the site build checked; the same headers and calls work on Android and iOS.

Each example also has a JavaScript only tab: the same task with no C++ file, calling LERC's own headers from @crossbind/port-lerc directly. All 4 work that way.

Compress heights to within 1 cm

LERC's main job: lerc_encode stores a float raster so that no value moves by more than the error you allow, and lerc_decode reads it back. The example checks the bound on every one of the 65,536 heights.

src/native/lerc_codec.h

#pragma once
 
#include <Lerc_c_api.h>
 
#include <algorithm>
#include <cmath>
#include <stdexcept>
#include <string>
#include <vector>
 
// One band of float32 heights, row by row from the top left, in and out of LERC. Bytes cross the
// binding as a byte string: one UTF-16 code unit (0-255) per byte.
class LercCodec {
public:
    static std::string version() {
        return std::to_string(LERC_VERSION_MAJOR) + "." + std::to_string(LERC_VERSION_MINOR) + "." + std::to_string(LERC_VERSION_PATCH);
    }
 
    // Every decoded height stays within maxError of the original; 0 keeps every bit.
    static std::u16string encode(const std::u16string& heights, int width, int height, double maxError) {
        const std::vector<float> values = toFloats(heights, width, height);
        const double bound = boundFor(values, maxError);
        unsigned int size = 0;
        check(lerc_computeCompressedSize(values.data(), kFloat, 1, width, height, 1, 0, nullptr, bound, &size), "sizing");
        std::vector<unsigned char> blob(size);
        unsigned int written = 0;
        check(lerc_encode(values.data(), kFloat, 1, width, height, 1, 0, nullptr, bound, blob.data(), size, &written), "encoding");
        return toUnits(blob.data(), written);
    }
 
    static std::u16string decode(const std::u16string& blob) {
        const std::vector<unsigned char> bytes = fromUnits(blob);
        const auto size = static_cast<unsigned int>(bytes.size());
        unsigned int info[11] = {};
        double range[3] = {};
        check(lerc_getBlobInfo(bytes.data(), size, info, range, 11, 3), "reading the header");
        const int width = static_cast<int>(info[3]);
        const int height = static_cast<int>(info[4]);
        if (info[1] != kFloat || info[2] != 1 || info[5] != 1) throw std::invalid_argument("this codec reads one band of float32");
        if (info[6] != info[3] * info[4]) throw std::invalid_argument("this blob has missing pixels: decode it with its mask");
        std::vector<float> values(static_cast<size_t>(width) * height);
        check(lerc_decode(bytes.data(), size, 0, nullptr, 1, width, height, 1, kFloat, values.data()), "decoding");
        return toUnits(reinterpret_cast<const unsigned char*>(values.data()), values.size() * sizeof(float));
    }
 
private:
    static constexpr unsigned int kFloat = 6;  // dt_float in Lerc_types.h
 
    // LERC quantizes in double precision and rounds back to float32, which can overshoot maxError by
    // half a float32 step (31 µm at 1,000 m). Like Esri's own sample, ask for a little less: one
    // float32 step at the largest height.
    static double boundFor(const std::vector<float>& values, double maxError) {
        if (maxError <= 0) return 0;
        float largest = 0;
        for (const float value : values) largest = std::max(largest, std::fabs(value));
        const double step = static_cast<double>(std::nextafter(largest, INFINITY)) - largest;
        return maxError > step ? maxError - step : 0;
    }
 
    static void check(lerc_status status, const char* step) {
        static const char* const names[] = {"ok", "failed", "wrong parameter", "buffer too small", "NaN", "uses noData", "dimensions too large"};
        if (status != 0) throw std::runtime_error(std::string("LERC failed ") + step + ": " + (status < 7 ? names[status] : "unknown error"));
    }
 
    static std::vector<float> toFloats(const std::u16string& units, int width, int height) {
        if (width <= 0 || height <= 0 || units.size() != static_cast<size_t>(width) * height * sizeof(float)) {
            throw std::invalid_argument("expected width * height float32 values");
        }
        std::vector<float> values(static_cast<size_t>(width) * height);
        auto* bytes = reinterpret_cast<unsigned char*>(values.data());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return values;
    }
 
    static std::vector<unsigned char> fromUnits(const std::u16string& units) {
        std::vector<unsigned char> bytes(units.size());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return bytes;
    }
 
    static std::u16string toUnits(const unsigned char* bytes, size_t size) {
        std::u16string units(size, u'\0');
        for (size_t i = 0; i < size; ++i) units[i] = bytes[i];
        return units;
    }
};

main.js

import { initNative, LercCodec } from './native/lerc_codec.h';
 
await initNative();
let seed = 42;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const heights = new Float32Array(width * height); // metres, row by row from the top left
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) {
        const dx = x - 128;
        const dy = y - 128;
        heights[y * width + x] = 1500 - (dx * dx + dy * dy) / 70 + random(1000) / 1000; // a hill, rough to 1 m
    }
}
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const toBytes = (text) => Uint8Array.from(text, (unit) => unit.charCodeAt(0));
 
const blob = await LercCodec.encode(toText(new Uint8Array(heights.buffer)), width, height, 0.01);
const decoded = new Float32Array(toBytes(await LercCodec.decode(blob)).buffer);
let worst = 0;
for (let i = 0; i < heights.length; i += 1) worst = Math.max(worst, Math.abs(decoded[i] - heights[i]));
console.log(`LERC ${await LercCodec.version()}: ${heights.byteLength} B of float32 heights -> ${blob.length} B`);
console.log(`largest error ${worst.toFixed(4)} m, within 1 cm: ${worst <= 0.01}`);

PRINTSfirst run downloads 0.9 MB

LERC 4.2.0: 262144 B of float32 heights -> 94775 B
largest error 0.0099 m, within 1 cm: true

The same lerc_computeCompressedSize, lerc_encode and lerc_decode calls the C++ codec makes. JavaScript now allocates every buffer, copies the float32 bytes in and out as one character per byte, reads the unsigned int blob size back with readNumberAt and works out the error bound, one float32 step under 1 cm, itself. LERC has its version only as the LERC_VERSION_MAJOR, LERC_VERSION_MINOR and LERC_VERSION_PATCH macros, which the binding does not export, so the first line has no version.

main.js

import {
    initNative, lerc_computeCompressedSize, lerc_encode, lerc_decode, allocBuffer, writeBytes,
    readBytes, readNumberAt,
} from '@crossbind/port-lerc/Lerc_c_api.h';
 
await initNative();
let seed = 42;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const heights = new Float32Array(width * height); // metres, row by row from the top left
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) {
        const dx = x - 128;
        const dy = y - 128;
        heights[y * width + x] = 1500 - (dx * dx + dy * dy) / 70 + random(1000) / 1000; // a hill, rough to 1 m
    }
}
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const toBytes = (text) => Uint8Array.from(text, (unit) => unit.charCodeAt(0));
const check = (status) => {
    if (status !== 0) throw new Error(`LERC failed with status ${status}`);
};
const FLOAT = 6; // dt_float in Lerc_types.h
 
// LERC rounds back to float32 and can overshoot maxZErr by half a float32 step: ask for one step less.
const largest = heights.reduce((max, value) => Math.max(max, Math.abs(value)), 0);
const maxZErr = 0.01 - 2 ** (Math.floor(Math.log2(largest)) - 23);
const data = await allocBuffer(heights.byteLength);
await writeBytes(data, toText(new Uint8Array(heights.buffer)));
const size = await allocBuffer(4); // the unsigned int both calls write back
check(await lerc_computeCompressedSize(data, FLOAT, 1, width, height, 1, 0, null, maxZErr, size));
const capacity = await readNumberAt(size, 0, 'uint32');
const blob = await allocBuffer(capacity);
check(await lerc_encode(data, FLOAT, 1, width, height, 1, 0, null, maxZErr, blob, capacity, size));
const blobSize = await readNumberAt(size, 0, 'uint32');
 
const values = await allocBuffer(heights.byteLength);
check(await lerc_decode(blob, blobSize, 0, null, 1, width, height, 1, FLOAT, values));
const decoded = new Float32Array(toBytes(await readBytes(values, heights.byteLength)).buffer);
let worst = 0;
for (let i = 0; i < heights.length; i += 1) worst = Math.max(worst, Math.abs(decoded[i] - heights[i]));
console.log(`LERC: ${heights.byteLength} B of float32 heights -> ${blobSize} B`);
console.log(`largest error ${worst.toFixed(4)} m, within 1 cm: ${worst <= 0.01}`);

PRINTSfirst run downloads 0.8 MB

LERC: 262144 B of float32 heights -> 94775 B
largest error 0.0099 m, within 1 cm: true

Read a blob's header without decoding it

lerc_getBlobInfo answers from the header alone: size, data type, bands, valid pixels, the value range and the largest error the encoder allowed. A tile viewer uses it to size its buffers, or to skip an empty tile, before decoding. The blob comes from the codec in the first example, which asks for one float32 step less than 10 cm.

src/native/lerc_blob_info.h

#pragma once
 
#include <Lerc_c_api.h>
 
#include <cstdio>
#include <stdexcept>
#include <string>
 
// What a LERC blob holds, read from its header without decoding a pixel. Bytes cross the binding as
// a byte string: one UTF-16 code unit (0-255) per byte.
class LercBlobInfo {
public:
    // {"codec","width","height","depth","bands","type","validPixels","masks","blobSize","zMin","zMax","maxZErrorUsed"}
    static std::string read(const std::u16string& blob) {
        std::string bytes(blob.size(), '\0');
        for (size_t i = 0; i < blob.size(); ++i) {
            if (blob[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<char>(blob[i]);
        }
        unsigned int info[11] = {};  // version, type, depth, width, height, bands, valid pixels, blob size, masks, depth, noData bands
        double range[3] = {};        // zMin, zMax, and the largest error the encoder allowed
        const lerc_status status = lerc_getBlobInfo(reinterpret_cast<const unsigned char*>(bytes.data()), static_cast<unsigned int>(bytes.size()), info, range, 11, 3);
        if (status != 0) throw std::runtime_error("not a LERC blob (status " + std::to_string(status) + ")");
        static const char* const types[] = {"int8", "uint8", "int16", "uint16", "int32", "uint32", "float32", "float64"};
        const std::string codec = info[0] == 0 ? "Lerc1" : "Lerc2 v" + std::to_string(info[0]);
        return "{\"codec\":\"" + codec + "\",\"width\":" + std::to_string(info[3]) + ",\"height\":" + std::to_string(info[4]) + ",\"depth\":" + std::to_string(info[2]) +
               ",\"bands\":" + std::to_string(info[5]) + ",\"type\":\"" + (info[1] < 8 ? types[info[1]] : "unknown") + "\",\"validPixels\":" + std::to_string(info[6]) +
               ",\"masks\":" + std::to_string(info[8]) + ",\"blobSize\":" + std::to_string(info[7]) + ",\"zMin\":" + number(range[0]) + ",\"zMax\":" + number(range[1]) +
               ",\"maxZErrorUsed\":" + number(range[2]) + "}";
    }
 
private:
    // 17 significant digits read back as the same double.
    static std::string number(double value) {
        char text[40];
        std::snprintf(text, sizeof text, "%.17g", value);
        return text;
    }
};

main.js

import { initNative, LercBlobInfo } from './native/lerc_blob_info.h';
import { LercCodec } from './native/lerc_codec.h';
 
await initNative();
let seed = 42;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const heights = new Float32Array(width * height);
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) {
        const dx = x - 128;
        const dy = y - 128;
        heights[y * width + x] = 1500 - (dx * dx + dy * dy) / 70 + random(1000) / 1000;
    }
}
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const blob = await LercCodec.encode(toText(new Uint8Array(heights.buffer)), width, height, 0.1);
 
const info = JSON.parse(await LercBlobInfo.read(blob));
console.log(`${info.codec}: ${info.width}x${info.height}, ${info.bands} band of ${info.type}, ${info.validPixels} valid pixels, ${info.blobSize} B`);
console.log(`heights ${info.zMin.toFixed(3)} to ${info.zMax.toFixed(3)} m, stored within ${info.maxZErrorUsed.toFixed(6)} m`);

PRINTSfirst run downloads 0.9 MB

Lerc2 v6: 256x256, 1 band of float32, 65536 valid pixels, 67591 B
heights 1032.268 to 1500.945 m, stored within 0.099878 m

What the C++ wrapper packed into JSON, JavaScript reads straight from the two arrays lerc_getBlobInfo fills: an unsigned int[11] and a double[3] it allocates itself and reads one number at a time with readNumberAt, in the order of InfoArrOrder and DataRangeArrOrder in Lerc_types.h. Both lines print as in the C++ version.

main.js

import {
    initNative, lerc_computeCompressedSize, lerc_encode, lerc_getBlobInfo, allocBuffer, writeBytes,
    readNumberAt,
} from '@crossbind/port-lerc/Lerc_c_api.h';
 
await initNative();
let seed = 42;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const heights = new Float32Array(width * height);
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) {
        const dx = x - 128;
        const dy = y - 128;
        heights[y * width + x] = 1500 - (dx * dx + dy * dy) / 70 + random(1000) / 1000;
    }
}
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const check = (status) => {
    if (status !== 0) throw new Error(`LERC failed with status ${status}`);
};
const FLOAT = 6; // dt_float in Lerc_types.h
 
// Encoded as in the first example, within 10 cm less one float32 step.
const largest = heights.reduce((max, value) => Math.max(max, Math.abs(value)), 0);
const maxZErr = 0.1 - 2 ** (Math.floor(Math.log2(largest)) - 23);
const data = await allocBuffer(heights.byteLength);
await writeBytes(data, toText(new Uint8Array(heights.buffer)));
const size = await allocBuffer(4);
check(await lerc_computeCompressedSize(data, FLOAT, 1, width, height, 1, 0, null, maxZErr, size));
const capacity = await readNumberAt(size, 0, 'uint32');
const blob = await allocBuffer(capacity);
check(await lerc_encode(data, FLOAT, 1, width, height, 1, 0, null, maxZErr, blob, capacity, size));
 
const infoArray = await allocBuffer(11 * 4); // unsigned int[11], in the order of InfoArrOrder in Lerc_types.h
const rangeArray = await allocBuffer(3 * 8); // double[3]: zMin, zMax, maxZErrUsed
check(await lerc_getBlobInfo(blob, await readNumberAt(size, 0, 'uint32'), infoArray, rangeArray, 11, 3));
const info = [];
for (let i = 0; i < 11; i += 1) info.push(await readNumberAt(infoArray, i, 'uint32'));
const range = [];
for (let i = 0; i < 3; i += 1) range.push(await readNumberAt(rangeArray, i, 'float64'));
const [version, dataType, , columns, rows, bands, validPixels, blobSize] = info;
const [zMin, zMax, maxZErrUsed] = range;
const types = ['int8', 'uint8', 'int16', 'uint16', 'int32', 'uint32', 'float32', 'float64'];
console.log(`${version === 0 ? 'Lerc1' : `Lerc2 v${version}`}: ${columns}x${rows}, ${bands} band of ${types[dataType]}, ${validPixels} valid pixels, ${blobSize} B`);
console.log(`heights ${zMin.toFixed(3)} to ${zMax.toFixed(3)} m, stored within ${maxZErrUsed.toFixed(6)} m`);

PRINTSfirst run downloads 0.8 MB

Lerc2 v6: 256x256, 1 band of float32, 65536 valid pixels, 67591 B
heights 1032.268 to 1500.945 m, stored within 0.099878 m

Keep integer data exact

Class maps and sensor counts must not change at all. With a maxZError of 0, LERC stores integers exactly (it raises 0 to 0.5, which rounds back to the same whole number) and float32 bit for bit. Here, a 12-bit sensor band in 16-bit pixels.

src/native/lerc_lossless.h

#pragma once
 
#include <Lerc_c_api.h>
 
#include <stdexcept>
#include <string>
#include <vector>
 
// Lossless LERC for any of its eight data types, bands one after another, each row by row. A
// maxZError of 0 keeps every value: LERC raises it to 0.5 for integers, which rounds back to the
// same whole number, and keeps float32 and float64 bit for bit. Bytes cross the binding as a byte
// string: one UTF-16 code unit (0-255) per byte.
class LercLossless {
public:
    // type: "int8", "uint8", "int16", "uint16", "int32", "uint32", "float32" or "float64".
    static std::u16string encode(const std::u16string& values, const std::string& type, int width, int height, int bands) {
        const unsigned int dataType = typeCode(type);
        if (width <= 0 || height <= 0 || bands <= 0 || values.size() != static_cast<size_t>(width) * height * bands * kSizes[dataType]) {
            throw std::invalid_argument("expected width * height * bands values of " + type);
        }
        const std::vector<unsigned char> data = fromUnits(values);
        unsigned int size = 0;
        check(lerc_computeCompressedSize(data.data(), dataType, 1, width, height, bands, 0, nullptr, 0.0, &size), "sizing");
        std::vector<unsigned char> blob(size);
        unsigned int written = 0;
        check(lerc_encode(data.data(), dataType, 1, width, height, bands, 0, nullptr, 0.0, blob.data(), size, &written), "encoding");
        return toUnits(blob.data(), written);
    }
 
    // The values in the blob's own type, band after band.
    static std::u16string decode(const std::u16string& blob) {
        const std::vector<unsigned char> bytes = fromUnits(blob);
        const auto size = static_cast<unsigned int>(bytes.size());
        unsigned int info[11] = {};
        double range[3] = {};
        check(lerc_getBlobInfo(bytes.data(), size, info, range, 11, 3), "reading the header");
        const unsigned int dataType = info[1];
        const int depth = static_cast<int>(info[2]);
        const int width = static_cast<int>(info[3]);
        const int height = static_cast<int>(info[4]);
        const int bands = static_cast<int>(info[5]);
        if (dataType > 7) throw std::invalid_argument("unknown LERC data type");
        if (info[6] != info[3] * info[4]) throw std::invalid_argument("this blob has missing pixels: decode it with its mask");
        std::vector<unsigned char> values(static_cast<size_t>(depth) * width * height * bands * kSizes[dataType]);
        check(lerc_decode(bytes.data(), size, 0, nullptr, depth, width, height, bands, dataType, values.data()), "decoding");
        return toUnits(values.data(), values.size());
    }
 
private:
    static constexpr size_t kSizes[] = {1, 1, 2, 2, 4, 4, 4, 8};  // bytes per value, in Lerc_types.h order
 
    static unsigned int typeCode(const std::string& type) {
        static const char* const names[] = {"int8", "uint8", "int16", "uint16", "int32", "uint32", "float32", "float64"};
        for (unsigned int code = 0; code < 8; ++code) {
            if (type == names[code]) return code;
        }
        throw std::invalid_argument("unknown type " + type);
    }
 
    static void check(lerc_status status, const char* step) {
        static const char* const names[] = {"ok", "failed", "wrong parameter", "buffer too small", "NaN", "uses noData", "dimensions too large"};
        if (status != 0) throw std::runtime_error(std::string("LERC failed ") + step + ": " + (status < 7 ? names[status] : "unknown error"));
    }
 
    static std::vector<unsigned char> fromUnits(const std::u16string& units) {
        std::vector<unsigned char> bytes(units.size());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return bytes;
    }
 
    static std::u16string toUnits(const unsigned char* bytes, size_t size) {
        std::u16string units(size, u'\0');
        for (size_t i = 0; i < size; ++i) units[i] = bytes[i];
        return units;
    }
};

main.js

import { initNative, LercLossless } from './native/lerc_lossless.h';
 
await initNative();
let seed = 7;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const band = new Uint16Array(width * height); // a gradient plus sensor noise, 12-bit values
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) band[y * width + x] = 1800 + ((x + y) >> 1) + random(64);
}
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const toBytes = (text) => Uint8Array.from(text, (unit) => unit.charCodeAt(0));
 
const blob = await LercLossless.encode(toText(new Uint8Array(band.buffer)), 'uint16', width, height, 1);
const back = new Uint16Array(toBytes(await LercLossless.decode(blob)).buffer);
console.log(`uint16, ${width}x${height}: ${band.byteLength} B -> ${blob.length} B`);
console.log(`identical: ${back.length === band.length && back.every((value, i) => value === band[i])}`);

PRINTSfirst run downloads 0.9 MB

uint16, 256x256: 131072 B -> 59091 B
identical: true

A maxZErr of 0 through the same three calls, with data type 3 (dt_ushort), and the 16-bit values cross as their bytes. The C++ decoder read the type and the size from the blob header so it could decode any blob; this one already knows them and passes them straight to lerc_decode.

main.js

import {
    initNative, lerc_computeCompressedSize, lerc_encode, lerc_decode, allocBuffer, writeBytes,
    readBytes, readNumberAt,
} from '@crossbind/port-lerc/Lerc_c_api.h';
 
await initNative();
let seed = 7;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const band = new Uint16Array(width * height); // a gradient plus sensor noise, 12-bit values
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) band[y * width + x] = 1800 + ((x + y) >> 1) + random(64);
}
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const toBytes = (text) => Uint8Array.from(text, (unit) => unit.charCodeAt(0));
const check = (status) => {
    if (status !== 0) throw new Error(`LERC failed with status ${status}`);
};
const USHORT = 3; // dt_ushort in Lerc_types.h
 
const data = await allocBuffer(band.byteLength);
await writeBytes(data, toText(new Uint8Array(band.buffer)));
const size = await allocBuffer(4);
check(await lerc_computeCompressedSize(data, USHORT, 1, width, height, 1, 0, null, 0, size)); // maxZErr 0: lossless
const capacity = await readNumberAt(size, 0, 'uint32');
const blob = await allocBuffer(capacity);
check(await lerc_encode(data, USHORT, 1, width, height, 1, 0, null, 0, blob, capacity, size));
const blobSize = await readNumberAt(size, 0, 'uint32');
 
const values = await allocBuffer(band.byteLength);
check(await lerc_decode(blob, blobSize, 0, null, 1, width, height, 1, USHORT, values));
const back = new Uint16Array(toBytes(await readBytes(values, band.byteLength)).buffer);
console.log(`uint16, ${width}x${height}: ${band.byteLength} B -> ${blobSize} B`);
console.log(`identical: ${back.length === band.length && back.every((value, i) => value === band[i])}`);

PRINTSfirst run downloads 0.8 MB

uint16, 256x256: 131072 B -> 59091 B
identical: true

Leave out pixels that have no data

Rasters mark gaps with a NoData value such as -9999. Stored as a height, it stretches the value range of every block it lands in. Passed as the validity mask (pValidBytes in lerc_encode and lerc_decode), it costs at most a bit per pixel, and decoding hands the mask back.

src/native/lerc_nodata.h

#pragma once
 
#include <Lerc_c_api.h>
 
#include <algorithm>
#include <cmath>
#include <stdexcept>
#include <string>
#include <vector>
 
// One band of float32 heights with gaps. Pixels equal to noData (NaN too, when noData is NaN) go into
// LERC's validity mask, a bit per pixel, instead of into the values; decoding puts noData back. Bytes
// cross the binding as a byte string: one UTF-16 code unit (0-255) per byte.
class LercNoData {
public:
    static std::u16string encode(const std::u16string& heights, int width, int height, double noData, double maxError) {
        const std::vector<float> values = toFloats(heights, width, height);
        std::vector<unsigned char> valid(values.size());
        float largest = 0;
        for (size_t i = 0; i < values.size(); ++i) {
            valid[i] = missing(values[i], noData) ? 0 : 1;
            if (valid[i]) largest = std::max(largest, std::fabs(values[i]));
        }
        // LERC rounds back to float32, which can overshoot maxError by half a float32 step: ask for one step less.
        const double step = static_cast<double>(std::nextafter(largest, INFINITY)) - largest;
        const double bound = maxError > step ? maxError - step : 0;
        unsigned int size = 0;
        check(lerc_computeCompressedSize(values.data(), kFloat, 1, width, height, 1, 1, valid.data(), bound, &size), "sizing");
        std::vector<unsigned char> blob(size);
        unsigned int written = 0;
        check(lerc_encode(values.data(), kFloat, 1, width, height, 1, 1, valid.data(), bound, blob.data(), size, &written), "encoding");
        return toUnits(blob.data(), written);
    }
 
    static std::u16string decode(const std::u16string& blob, double noData) {
        const std::vector<unsigned char> bytes = fromUnits(blob);
        const auto size = static_cast<unsigned int>(bytes.size());
        unsigned int info[11] = {};
        double range[3] = {};
        check(lerc_getBlobInfo(bytes.data(), size, info, range, 11, 3), "reading the header");
        const int width = static_cast<int>(info[3]);
        const int height = static_cast<int>(info[4]);
        if (info[1] != kFloat || info[2] != 1 || info[5] != 1) throw std::invalid_argument("this reads one band of float32");
        std::vector<float> values(static_cast<size_t>(width) * height);
        std::vector<unsigned char> valid(values.size());
        check(lerc_decode(bytes.data(), size, 1, valid.data(), 1, width, height, 1, kFloat, values.data()), "decoding");
        for (size_t i = 0; i < values.size(); ++i) {
            if (!valid[i]) values[i] = static_cast<float>(noData);
        }
        return toUnits(reinterpret_cast<const unsigned char*>(values.data()), values.size() * sizeof(float));
    }
 
private:
    static constexpr unsigned int kFloat = 6;  // dt_float in Lerc_types.h
 
    static bool missing(float value, double noData) { return std::isnan(noData) ? std::isnan(value) : value == static_cast<float>(noData); }
 
    static void check(lerc_status status, const char* step) {
        static const char* const names[] = {"ok", "failed", "wrong parameter", "buffer too small", "NaN", "uses noData", "dimensions too large"};
        if (status != 0) throw std::runtime_error(std::string("LERC failed ") + step + ": " + (status < 7 ? names[status] : "unknown error"));
    }
 
    static std::vector<float> toFloats(const std::u16string& units, int width, int height) {
        if (width <= 0 || height <= 0 || units.size() != static_cast<size_t>(width) * height * sizeof(float)) {
            throw std::invalid_argument("expected width * height float32 values");
        }
        std::vector<float> values(static_cast<size_t>(width) * height);
        auto* bytes = reinterpret_cast<unsigned char*>(values.data());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return values;
    }
 
    static std::vector<unsigned char> fromUnits(const std::u16string& units) {
        std::vector<unsigned char> bytes(units.size());
        for (size_t i = 0; i < units.size(); ++i) {
            if (units[i] > 0xFF) throw std::invalid_argument("not a byte string");
            bytes[i] = static_cast<unsigned char>(units[i]);
        }
        return bytes;
    }
 
    static std::u16string toUnits(const unsigned char* bytes, size_t size) {
        std::u16string units(size, u'\0');
        for (size_t i = 0; i < size; ++i) units[i] = bytes[i];
        return units;
    }
};

main.js

import { initNative, LercNoData } from './native/lerc_nodata.h';
import { LercCodec } from './native/lerc_codec.h';
 
await initNative();
let seed = 42;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const heights = new Float32Array(width * height);
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) {
        const dx = x - 128;
        const dy = y - 128;
        heights[y * width + x] = 1500 - (dx * dx + dy * dy) / 70 + random(1000) / 1000;
    }
}
for (let i = 0; i < heights.length; i += 1) if (random(50) === 0) heights[i] = -9999; // the survey missed 2% of its points
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const toBytes = (text) => Uint8Array.from(text, (unit) => unit.charCodeAt(0));
const text = toText(new Uint8Array(heights.buffer));
 
const asHeight = await LercCodec.encode(text, width, height, 0.01); // the codec from the first example
const blob = await LercNoData.encode(text, width, height, -9999, 0.01);
const back = new Float32Array(toBytes(await LercNoData.decode(blob, -9999)).buffer);
let gaps = 0;
let worst = 0;
for (let i = 0; i < heights.length; i += 1) {
    if (heights[i] === -9999) gaps += back[i] === -9999 ? 1 : 0;
    else worst = Math.max(worst, Math.abs(back[i] - heights[i]));
}
console.log(`-9999 stored as a height: ${asHeight.length} B`);
console.log(`-9999 as missing pixels: ${blob.length} B`);
console.log(`${gaps} gaps come back as -9999, largest error elsewhere ${worst.toFixed(4)} m`);

PRINTSfirst run downloads 0.9 MB

-9999 stored as a height: 147759 B
-9999 as missing pixels: 98308 B
1273 gaps come back as -9999, largest error elsewhere 0.0099 m

JavaScript builds the validity mask itself, a byte per pixel written with writeBytes, and lerc_decode fills a second one on the way back. Missing pixels decode as 0, so putting -9999 back is a JavaScript loop, and the error bound, one float32 step under 1 cm at the largest value each encode stores, is worked out in JavaScript too.

main.js

import {
    initNative, lerc_computeCompressedSize, lerc_encode, lerc_decode, allocBuffer, writeBytes,
    readBytes, readNumberAt,
} from '@crossbind/port-lerc/Lerc_c_api.h';
 
await initNative();
let seed = 42;
const random = (n) => (seed = (seed * 48271) % 2147483647) % n;
const width = 256;
const height = 256;
const heights = new Float32Array(width * height);
for (let y = 0; y < height; y += 1) {
    for (let x = 0; x < width; x += 1) {
        const dx = x - 128;
        const dy = y - 128;
        heights[y * width + x] = 1500 - (dx * dx + dy * dy) / 70 + random(1000) / 1000;
    }
}
for (let i = 0; i < heights.length; i += 1) if (random(50) === 0) heights[i] = -9999; // the survey missed 2% of its points
const toText = (bytes) => Array.from(bytes, (byte) => String.fromCharCode(byte)).join('');
const toBytes = (text) => Uint8Array.from(text, (unit) => unit.charCodeAt(0));
const check = (status) => {
    if (status !== 0) throw new Error(`LERC failed with status ${status}`);
};
const FLOAT = 6; // dt_float in Lerc_types.h
 
const data = await allocBuffer(heights.byteLength);
await writeBytes(data, toText(new Uint8Array(heights.buffer)));
const valid = await allocBuffer(heights.length); // the mask: a byte per pixel, 1 valid, 0 missing
await writeBytes(valid, toText(Uint8Array.from(heights, (value) => (value === -9999 ? 0 : 1))));
const size = await allocBuffer(4);
// LERC rounds back to float32 and can overshoot maxZErr by half a float32 step: ask for one step
// less at the largest value it stores.
const encode = async (nMasks, mask, largest) => {
    const maxZErr = 0.01 - 2 ** (Math.floor(Math.log2(largest)) - 23);
    check(await lerc_computeCompressedSize(data, FLOAT, 1, width, height, 1, nMasks, mask, maxZErr, size));
    const capacity = await readNumberAt(size, 0, 'uint32');
    const blob = await allocBuffer(capacity);
    check(await lerc_encode(data, FLOAT, 1, width, height, 1, nMasks, mask, maxZErr, blob, capacity, size));
    return [blob, await readNumberAt(size, 0, 'uint32')];
};
const largestOf = (values) => values.reduce((max, value) => Math.max(max, Math.abs(value)), 0);
const [, asHeightSize] = await encode(0, null, largestOf(heights));
const [blob, blobSize] = await encode(1, valid, largestOf(heights.filter((value) => value !== -9999)));
 
const values = await allocBuffer(heights.byteLength);
const mask = await allocBuffer(heights.length);
check(await lerc_decode(blob, blobSize, 1, mask, 1, width, height, 1, FLOAT, values));
const back = new Float32Array(toBytes(await readBytes(values, heights.byteLength)).buffer);
const validBack = toBytes(await readBytes(mask, heights.length));
let gaps = 0;
let worst = 0;
for (let i = 0; i < heights.length; i += 1) {
    if (validBack[i] === 0) back[i] = -9999; // missing pixels decode as 0: put the NoData value back
    if (heights[i] === -9999) gaps += back[i] === -9999 ? 1 : 0;
    else worst = Math.max(worst, Math.abs(back[i] - heights[i]));
}
console.log(`-9999 stored as a height: ${asHeightSize} B`);
console.log(`-9999 as missing pixels: ${blobSize} B`);
console.log(`${gaps} gaps come back as -9999, largest error elsewhere ${worst.toFixed(4)} m`);

PRINTSfirst run downloads 0.8 MB

-9999 stored as a height: 147759 B
-9999 as missing pixels: 98308 B
1273 gaps come back as -9999, largest error elsewhere 0.0099 m

Add it to your project

One package per platform: install the ones you build for and list each in crossbind.config.js; crossbind compiles only the one that matches the build target. Your C++ goes in src/native, next to the headers it binds. Libraries explains the whole flow.

shell

npm install @crossbind/port-lerc-wasm@beta

crossbind.config.js

import lercWasm from '@crossbind/port-lerc-wasm/crossbind.config.js';
 
export default {
    dependencies: [lercWasm],
    paths: { config: import.meta.url },
};

Platforms

Platform	Runs in	Builds	Page
WebAssembly	browsers, Node.js and edge runtimes	`wasm32`, single-threaded and multi-threaded	LERC for WebAssembly
Android	React Native apps on Android	`arm64-v8a` devices and the `x86_64` emulator	LERC for Android
iOS	React Native apps on iOS	`arm64` devices and simulators	LERC for iOS
macOS	native Node.js addons and Electron on macOS	`arm64` and `x64`, macOS 11 or later	LERC for macOS
Linux	native Node.js addons on Linux	`x64` and `arm64`, glibc 2.28 or later	LERC for Linux
Windows	native Node.js addons on Windows	`x64` and `arm64`, Windows 10 or later	LERC for Windows
WASI	command-line programs under wasmtime	`wasm32-wasip3`, single-threaded	LERC for WASI

Packages

Target	Package	npm `beta`
Meta package	`@crossbind/port-lerc`	`2.0.0-beta.62`
Web and Node.js	`@crossbind/port-lerc-wasm`	`2.0.0-beta.62`
WASI library	`@crossbind/port-lerc-wasi`	`2.0.0-beta.62`
Android	`@crossbind/port-lerc-android`	`2.0.0-beta.62`
iOS	`@crossbind/port-lerc-ios`	`2.0.0-beta.62`
macOS	`@crossbind/port-lerc-darwin`	`2.0.0-beta.62`
Linux	`@crossbind/port-lerc-linux`	`2.0.0-beta.62`
Linux (musl)	`@crossbind/port-lerc-linuxmusl`	not published
Windows	`@crossbind/port-lerc-win32`	`2.0.0-beta.62`

Licence

npm license field of @crossbind/port-lerc: Apache-2.0.
The licence files that ship with the package, and the port recipe, are in the port directory.

Facts on this page come from the port manifests in the repository and from what npm served on beta when the site was built. See the Libraries guide for the full consumer flow.

MORE LIBRARIES

cURL Expat GDAL GEOS GeoTIFF iconv libjpeg-turbo libTIFF OpenSSL PROJ SpatiaLite SQLite WebP zlib Zstandard