At Studio Meije, we believe Gaussian Splatting will be an important part of the future of Apple platforms.
Since the launch of the first Vision Pro, projects like MetalSplatter have made it possible to load Gaussian splat files on Apple devices. That work is very useful, but it is based on a Compositor Services path.
We wanted to explore something different: a RealityKit path.
Today, you cannot mix a Compositor Services renderer with a standard RealityKit rendering flow. So we wanted to see how far Gaussian Splatting could go inside RealityKit itself.
Why we built GSKit
GSKit started as a technical experiment, not as a finished product.
We wanted to answer one simple question: can Gaussian splats be rendered through a RealityKit-native pipeline?
We are releasing GSKit on GitHub because this kind of exploration is more useful when other developers can inspect it, run it, and build on it.
How the RealityKit path works
Our approach was to stay as close as possible to the standard RealityKit pipeline.
We take splat data from binary .ply files, decode the position, size, orientation, color, and opacity of each splat, and convert those splats into small camera-facing quads. RealityKit can then manage that geometry as a LowLevelMesh.
The full cloud is rendered through a single CustomMaterial. A custom surface shader gives each quad the soft blended look that makes Gaussian splats feel volumetric rather than flat.
The goal was never to build a separate renderer around Compositor Services. It was to see how much of the work could stay inside RealityKit while still keeping the visual character of splatting.
The hard part: transparency
The most difficult part was transparency.
Gaussian splats need to be drawn in roughly back-to-front order, otherwise blending breaks. That order changes every time the camera moves, so it cannot be solved once and forgotten.
To handle this, we built a GPU sorting path in Metal with a radix sort. On each frame, we calculate the depth of the visible splats relative to the camera, sort their indices on the GPU, and write the reordered index list back into the mesh before rendering.
We also added a few practical controls to keep the experiment usable, including visibility culling, downsampling, and an adaptive render budget that can reduce how many splats are drawn when performance drops.
What we learned
The good news is that Gaussian Splatting can be explored through a RealityKit-native path, with CustomMaterial for shading and Metal compute for sorting.
As visionOS developers, it is frustrating that this approach is not possible on Vision Pro today because CustomMaterial is not supported on visionOS.
We hope future RealityKit updates will open this path beyond macOS.
