Voxel rendering on mobile devices

Nikolai Shalimov, Lead Software Engineer, EPAM Systems

1. VOXEL RENDERING on mobile devices

2. GOALS Mobile devices Interactive (realtime rotation and slicing) Real world data set (MRI and other scanners) Smooth

3. INPUT DATA DICOM packed MRI scanner output 217 x 181 x 181 (>7 millions) points of intensity values Same amount of masks (layers) data Table with masks descriptions (up to 255 values)

4. APPROACHES Uses a large number of very small graphic objects One small object (point in our case) is one quad, i.e. two triangles on four vertices 217 x 181 x 181 x 4 = 28’436’548 vertices. Too much, even for the newest iPad Simple set of texels with three dimensions Not supported on target platforms Complicated technique of model ray casting to generate 2D projection Heavy usage of CPU resources PARTICLE SYSTEM 3D TEXTURE RAY CASTING VOLUME RENDERING

5. ALTERNATIVE APPROACH. GOALS Reduce geometry size (number of vertices) Reduce CPU usage It should be easy to implement Scalable for other features, like slicing and colored layering

6. SOLUTION Generate sets of axis aligned textures for each axis (217 + 181 + 181 = 579 quads = 2316 vertices) Each texture can have up to 4 channels (RGBA), so we need to use them as much as possible We have to use dirty tricks and magic

7. TEXTURES AND GEOMETRY GENERATION Need to generate quads and textures for each of axis Trivial task, especially for geometry. Three simple loops We have 4 channels at our disposal, so use them all! First channel is for intensity of MRI points Second channel is for indexed layer coloring Third and fourth channels are for slicing

8. 1 st ROUND RESULTS “Nice” solid rendered brain Indexed dynamic layer coloring. For recoloring of brain section we just need to change one texel in palette texture Axis-dependent slicing could be set with cutoff material parameter, that works with third and fourth channels 2’316 draw calls and 7’109’137 fragment shader calls. It’s still quite a lot for a target device. Time for tricks!

9. OPTIMIZATION. PART 1 “Batch! Batch! Batch!” For reducing draw calls we need to minimize material switches. Now we have 2’316 materials, each is for every slice. For example Z-axis has 181 textures with size 217x181. Overwhelming majority of mobile devices can support textures of size 2048 x 2048. It’s enough for group small slices in two atlases. 181 => 2! Not bad! Now we have only 6 materials and potentially 6 draw calls

10. OPTIMIZATION. PART 2 It all depends on your point of view We don’t need to render all of quads sets for any particular position of camera Depending on the position of the camera, we will turn on or off those sets dynamically According to our experience only one or two sets at the moment could be shown without visible changes 2-4 draw calls. We cheated ! You almost didn’t notice anything!

11. OPTIMIZATION. PART 3 Save your fillrate Our quads contains of huge amount of useless transparent areas around actual data We need to refuse quads and use more suitable custom geometry for each slice of brain

12. RESULT [email protected]

13. THE END THANK YOU [email protected]