The other day I was looking at this clustered shading WebGPU demo and was surprised by the jarring texture compression artifacts:
The glTF model using KTX ETC1S textures is very compact, but is it worth the dramatic quality reduction?
Continue reading →Normal maps are one of the most widely used texture types in real-time rendering, but they’re also a bit unusual. They don’t represent color, rely on geometric assumptions, and small encoding or decoding details can lead to subtle artifacts.
This article takes a practical look at how normal maps are commonly compressed today, the tradeoffs involved, and a few pitfalls that are easy to overlook. We’ll look at these details and see how they are handled in practice in the context of spark.js and web 3d development.
Continue reading →This is a supplement to Don McCurdy’s excellent article, Choosing texture formats for WebGL and WebGPU applications. Don covers the tradeoffs between modern image formats like WebP and AVIF, and GPU optimized container formats like KTX2, with practical guidance on when each makes sense.
Since that article was published, a new option has emerged: spark.js, a real time GPU texture compression library for WebGPU, which makes it possible to ship textures as standard web images while still ending up with efficient block compressed textures in GPU memory.
Prior the arrival of spark.js, developers wanting to load textures in their applications effectively had three options:
three.js r180 was released last week and among the many improvements, the one that I’m most excited about is a series of changes by Don McCurdy that enable the use textures encoded with spark.js in three.js:
Support for ExternalTexture objects in the WebGPU backend allows you to wrap a GPUTexture object, such as the one produced by Spark.encodeTexture and use it directly in three.js. This makes it straightforward to work with Spark-encoded textures.
Here’s a brief example:
// Load and encode texture using spark:
const gpuTexture = await spark.encodeTexture(textureUrl, { srgb: true, flipY: true });
// Wrap the GPUTexture for three.js
const externalTex = new THREE.ExternalTexture(gpuTexture);
// Use as any other texture:
const material = new THREE.MeshBasicMaterial({ map: externalTex });With this feature in place, I began testing spark.js in more complex scenarios. The first thing I wanted was a GLTF viewer example. With some help from Don McCurdy I was able to get something running quickly, and after after a bit of polish, I’m very happy with the results. It requires very minimal changes to existing code.
Continue reading →I’m excited to announce the release of Spark 1.3 and the launch of spark.js, our new JavaScript API for real-time GPU texture compression on the web.
The highlight of this release is full WebGPU support. Spark’s real-time codecs are now available as WGSL compute shaders that run natively in modern browsers. To make it easy to use Spark on the web, we’re also introducing spark.js, a separate JavaScript package that wraps a subset of the codecs with a simple API. spark.js is free for non-commercial use and available under an affordable license for commercial projects.
What’s new in Spark 1.3:
This release also includes new examples, bug fixes, and continued improvements based on user feedback.
I’m thrilled to welcome Infinite Flight and Netflix as new Spark licensees, and deeply grateful to HypeHype for their continued support.
I’m excited to announce that Spark 1.2 is finally out!
Spark is the go-to solution for real-time GPU texture compression, offering high-performance codecs that deliver high-quality results at real-time speeds. It enables developers to efficiently compress textures directly on the GPU, reducing memory usage and improving rendering performance across a wide range of platforms.
With Spark 1.2, I’ve taken things even further with major optimizations, new compression formats, and expanded platform support:
f16 vectorization of all codecs and improved performance on all devices, but especially Bifrost GPUs, and a much faster EAC codec.For these release notes, I’m trying a slightly different format. Instead of just sharing the highlights on social media, I’m using my blog to provide a more personal and in depth update. I’m afraid I will have to keep some of the specifics under wraps, so I won’t be sharing as much implementation details as in my typical blog posts, but I hope I can offer enough insight to satisfy my regular audience. If you are interested, read on!
Continue reading →When I joined NVIDIA in 2005, one of my main goals was to work on texture and mesh processing tools. The NVIDIA Texture Tools were widely used, and Cem Cebenoyan, Sim Dietrich and Clint Brewer had been doing interesting work on mesh processing and optimization (nvtristrip, nvmeshmender). That was exactly the kind of work I wanted to be involved in.
However, the priorities of the tools team were different, and I ended up working on FX Composer instead. I wasn’t particularly excited about that, so in 2006, I switched to the Developer Technology group.
At the time, NVIDIA and ATI were competing for dominance in the GPU market. While we had a solid market share, our real goal was to grow the overall market. Expanding the “pie” rather than just our slice of it. If you imagine a gamer with a fixed budget, we wanted him to allocate more of that budget to the GPU rather than the CPU. One way to achieve this was by encouraging developers to shift workloads from the CPU to the GPU.
This push was part of the broader GPGPU movement. CUDA had just been released, but it had no integration with graphics APIs, and compute shaders didn’t exist yet. One of the workloads that caught our attention was GPU texture compression. Under the pretext of harnessing the GPU, I found my way back to working on texture compression.
Another idea gaining traction at the time was runtime texture compression.
In April 2004, Farbrausch released .kkrieger, a first-person shooter that packed all its content into just 96 KB by using procedural generation for levels, models, and textures, but it wasn’t until late into the development of fr-041: debris in 2007 that they started using runtime DXT compression to reduce GPU memory usage and improve performance.
Around the same time, Allegorithmic was developing ProFX, the predecessor to Substance Designer, a middleware for real-time procedural texturing. ProFX also included a fast DXT encoder, allowing procedural textures to be converted into GPU-friendly formats at load time.
Simon Brown was working on PlayStation Home, Sony’s 3D social virtual world where players could create and customize their avatars. To support this, he wrote a fast DXT encoder optimized for the PS3’s SPUs, demonstrating the potential of offloading texture compression to parallel processors.
John Carmack had been talking about the megatexture technology for a while, but in 2006, when Jan Paul van Waveren published the details of their Real-Time DXT Compression implementation on the Intel Software Network, we at NVIDIA saw a potential problem: if Rage ended up CPU-limited, it could push gamers toward CPU upgrades rather than GPUs. That made real-time texture compression in the GPU a strategic priority for us.
Continue reading →I would like to share some details about the tools that I’ve built to develop the Spark codecs.
When I started working on Spark I had no idea how much time and effort I would be investing on this project. I started with very few tools and helpers, with the goal of obtaining results quickly. While I was able to create a proof of concept in a few days, over time it became clear that I would need better tools in order to maintain that fast development pace.
The first tool I built was Spark Report, a command-line application that automates codec testing. It runs the codecs on diverse image sets, calculates error metrics, generates detailed reports, compares results with other codecs, and tracks performance over time. With Spark Report, I could confidently iterate on the codecs, knowing that any changes I made wouldn’t introduce regressions and would improve results across a wide range of images.
In addition to detecting regressions and doing comparative analysis against other codecs, I also needed a tool to dig deeper, understand the behavior of the codecs and the consequences of the changes. Something that I learned early is that you cannot trust error metrics too much, and that there’s no alternative to visual inspection, so I also built Spark View, a tool to view the output of the codecs.
There’s always been some tension between codec development, tools, and quality of life improvements, but in retrospect I think that every effort on better tools has paid off handsomely, and if anything I’d say I’ve delayed that work too much. In my defense, it was hard to justify the work without knowing the full scope of the project. Would I be working on Spark for a few months? Or a few years? Initially I did not know if it would be worth building a commercial product around it, but as things progressed, the scope of the project grew, adding more formats, more codecs, more platforms, and the depth of the codecs and the complexity of the optimizations increased.
Looking back, I wish I had prioritized tool development earlier, but hindsight always feels clearer in retrospect.
Continue reading →I’m excited to announce that I’m starting my own business to research and develop graphics and game technologies, with a focus on the texture and mesh processing pipelines.
My first product is a real-time ASTC encoder that is orders of magnitude faster than existing offline compressors. It targets a small subset of the available encoding space, but achieves competitive quality through carefully crafted algorithms and creative optimizations.
In addition to that I’m exploring middleware products and applications that advance the state of the art in the areas of RDO texture compression, mesh processing algorithms such as simplification and parameterization, and alternative representations for rendering and physical simulation.
For inquiries, contact me at: castano@ludicon.com
The NVIDIA Texture Tools (NVTT) had the highest quality BC1 encoder that was openly available, but the rest of the code was mediocre: The remaining encoders had not received much attention, the CUDA code paths were not maintained, and the accompanying image processing and serialization code was conventional. There was too much code that was not particularly interesting, it was a hassle to build, and required too much maintenance.
This compelled me to package the BC1 compressor independently as a single header library:
https://github.com/castano/icbc
While doing that I also took the opportunity to revisit the encoder and change the way it was vectorized. I wanted to write about that, but before getting into the details let’s overview how a BC1 compressor works.
Continue reading →