Ignacio Castaño

The Spark codecs are often bandwidth-bound. The codec usually reads an uncompressed RGBA8 texture and outputs a block-compressed texture that, depending on the format, is 2–8 times smaller. Reading the input is what dominates bandwidth. When bandwidth-bound, execution stalls waiting for input data, while the GPU tries to hide that latency by overlapping loads for some blocks with computation on others.

Mobile GPUs render one tile at a time and keep its data in on-chip memory to save bandwidth and power. Wouldn’t it be great if our codecs could also use this tile memory as their input? That’s exactly what Metal tile shaders allow us to do.

Continue reading →

I’m excited to announce the release of Spark 1.4.

Spark is a set of real-time GPU texture codecs that produce results competitive with offline encoders while running thousands of times faster, on every GPU from budget phones to high-end PCs and consoles.

The highlight of this release is HDR. Spark 1.4 adds two new HDR codecs: BC6H for desktop and consoles and ASTC HDR for mobile, and both deliver quality that matches or exceeds the best offline encoders at real-time speeds.

Real-time HDR compression opens up a new range of applications:

• Light probes and reflection maps that are re-rendered as the scene changes and compressed on the fly.
• Dynamic lightmaps for time-of-day cycles and destructible environments.
• Dynamic skies or any other HDR render target whose contents can be cached across frames.
• HDR images decoded at runtime, from EXR, AVIF or JPEG/R, without the need for an offline pipeline.

What’s new in Spark 1.4:

• New HDR codecs: BC6H and ASTC HDR. With offline quality at real-time performance.
• BC1 Ultra: a new quality level for BC1 that surpasses the perceptual quality of offline encoders at a fraction of their cost.
• BC7 NRM: a new BC7 codec for packed material maps that store normals, roughness and metalness in a single texture.
• ASTC 5×5: a new block size that fills the gap between our 4×4 and 6×6 codecs.
• Broader compatibility: PowerVR Series D (Pixel 10), Vulkan 1.0 devices, improved GLES mediump support, and improved PlayStation®5 support.

This release also includes a new Getting Started guide and Performance Guidelines, many new examples, improved regression testing, bug fixes, and many other small quality and performance improvements.

It’s exciting to see Spark gaining momentum and shipping in a wide range of games and applications. Check out our front page for the latest additions to our client list.

Continue reading →

Today I’m releasing Spark for Unity, a Unity package that brings the Spark texture codecs to the Unity engine. It’s available now on GitHub and it’s free for projects with lifetime revenue under $100K.

Unity’s built-in runtime texture compression is CPU-bound, limited in format support, and often too slow for real-time use cases. Spark for Unity brings GPU-native texture compression to Unity, making it practical to compress textures at runtime in real-time.

Texture2D compressed = Spark.EncodeTexture(source, SparkFormat.RGB);

That single line takes an existing texture and produces a standard Texture2D in a GPU-native format (BC7 or ASTC depending on the device), ready to assign to a material. It runs entirely on the GPU, at least an order of magnitude faster than Unity’s Texture2D.Compress, and produces higher quality results.

The package works across all Unity render pipelines (URP, HDRP, and Built-in) without modification. It’s been tested on Metal (macOS, iOS), Vulkan (Android, Windows), OpenGL ES 3.1 (Android), and Direct3D (Windows), on Unity versions 6.3 to 6.6.

If you want to see it running, the demo app is available for iOS, Android, macOS, and Windows.

Continue reading →

You would think that loading images in the browser is a solved problem. After all, websites are basically composed of text and images.

Turns out that loading images from JavaScript to use in WebGL/WebGPU applications is not nearly as simple or well supported as it should be. There are several APIs for it, and the right choice depends on which browser you are targeting. They each have different performance characteristics, and some have unexpected behaviors or outright bugs.

This is surprising considering that loading images is one of the most fundamental things a browser does.

Continue reading →

The Apple Lossy compression format was introduced in the A15 and M2 chipsets (which share the same GPU generation) and it enables sampling and rendering to textures with a 1:2 compression ratio. The format is transparent to the application: the GPU handles compression and decompression automatically, and the block layout is never exposed through the API.

A few weeks ago I wrote about it in Hardware Image Compression and gave a brief overview of how it’s enabled at the API level and how the results compare to other hardware compression implementations. In this blog post I’ll take a deeper look at the underlying block format and how the hardware decoder is implemented.

Unlike traditional GPU compression formats, the lossy format is not documented. There’s no spec that you can read to understand how the format works. In order to do that I employed the same technique I described in Writing to Compressed Textures in Metal.

Continue reading →

In my previous post Writing to Compressed Textures in Metal, I showed how Metal heaps can be used to alias buffers and textures to inspect their actual memory layout. Something that caught my attention was how much memory is wasted with non-power-of-two textures. On many devices the texture dimensions get rounded up to the next power of two, and you end up paying for padding memory you never use.

You may think that non power of two textures do not have much use outside of render targets, but it turns out that they are much more prevalent than what most people realize. When allocating ASTC textures with power-of-two pixel dimensions the resulting textures often have non-power-of-two block dimensions, and it’s the block dimensions, not the pixel dimensions, that determine how much memory the texture occupies.

How bad is it in practice? I ran a test on several mobile devices to find out. In the tables below I compared the nominal bpp, that is, the exact memory footprint with no alignment padding, against the actual bpp considering the alignment.

Continue reading →

About a year ago I wrote GPU Texture Compression Everywhere, a post in which, among other things, I lamented that Metal did not have support for writing to compressed textures.

Unlike Vulkan or D3D12, Metal doesn’t support resource casting. There’s no way to write to a compressed texture through an uncompressed view. The only way we can do that is by using a blit operation, so we need to output our results to a temporary buffer and then copy the contents of the buffer to the texture. This requires a temporary memory allocation that needs to be managed, and if the buffer is reused for multiple uploads, hazard tracking may add some synchronization overhead.

I requested support for this feature to Apple when I started working on Spark, more than 3 years ago. Since then not much progress has been made, and support is still not available, so I decided it was time to take matters into my own hands.

Continue reading →

One of the things I’ve always lamented about hardware image formats is the slow pace of innovation. Developers were usually unwilling to ship textures in a new format unless that format was widely available. That is, the format had to be supported in the majority of the hardware they were targeting, and must be supported across all vendors.

For example, even though ATI introduced the 3Dc formats in 2004 with the Radeon X800 (R420) and exposed them through D3D9 extensions, in practice their use did not become widespread when Direct3D 10 standardized them as BC4 and BC5 in 2007, but only when Direct3D 10 became the minimum hardware requirement.

Crysis was the first major game which shipped with BC5 textures, but most games were not willing to have such a steep hardware requirement until many years later. To avoid these adoption delays, the BC6 & BC7 formats were designed in collaboration between ATI and NVIDIA for inclusion in Direct3D 11.

Hardware development cycles are already long, and for a new format to gain adoption it needs to be proposed for standardization, which often makes the process even longer.

This is one of the reasons why I find real-time texture compression so exciting. When the encoder runs in real-time it’s a lot easier to introduce new hardware formats, because adopting a new format no longer requires waiting for content to be created targeting it.

Continue reading →

I’m excited to announce spark.js 0.1, now with WebGL support!

spark.js has been evolving since I released it last summer. Since then, the WebGPU ecosystem has matured considerably. WebGPU is now more stable and widely supported across browsers and platforms. However, users kept telling me the same thing: even though targeting WebGPU is practical today, most teams have codebases that still rely on WebGL, and that made adoption difficult. For that reason I committed to adding WebGL support.

This felt like the right moment to bump the version number to 0.1 and signal that spark.js is production ready, not just experimental. That said, I expect the API to continue evolving based on the features developers need and the friction points they encounter.

Continue reading →

One of the advantages of runtime texture compression is that mipmaps can also be computed at runtime and do not need to be transmitted. In contrast, offline codecs must precompute mipmaps so they can be compressed and transmitted alongside the base image.

Each mip level requires one quarter of the memory of its parent, so a full mipmap chain increases the total texture size by approximately 33%. In principle, offline codecs could attempt to exploit correlation between mip levels, but in practice they rarely do. In fact, rate-distortion optimized codecs often exceed the theoretical 33% overhead, because lower-resolution mip levels tend to have higher entropy and compress less efficiently.

By generating mipmaps on the device, we eliminate the need to transmit them entirely, reducing the total transmitted texture data by 25%. This is a dramatic reduction by image codec standards, and can be achieved without any loss of quality. The tradeoff is that mipmaps must be generated at runtime, but fortunately this is a task that GPUs excel at.

Continue reading →