Developer Builds Playable FPS Using Gaussian Splats in Browser

Key Takeaways

• A free browser-based FPS game now demonstrates Gaussian splats can work for interactive media
• Gaussian splats create photorealistic 3D environments from photos or video at modest rendering cost
• The technology has moved from VFX (Superman, A$AP Rocky music videos) into real-time applications

London-based developer Iakov Sumygin has built something that shouldn't quite work: a playable first-person shooter that uses Gaussian splats for its environment. You can load it up right now in your browser and walk around a photorealistic 3D world. The gameplay is basic. The tech behind it is not.

Gaussian splats have been used for visual effects in the 2024 Superman movie and in A$AP Rocky music videos. Until now, the technology was considered unsuitable for interactive applications because it captures only how things look, not their physical properties. This browser game challenges that assumption.

What Are Gaussian Splats?

Think of voxels, the volume pixels used in games like Minecraft. Gaussian splats are similar in concept but far more flexible. They don't exist on a fixed grid. Each splat has variable size, density, position, shape, opacity, and color. The result is 3D environments with photorealistic visual quality at a modest rendering cost.

Creating a splat scene (sometimes called a "3DGS" scene) starts with taking many photos or videos of a real space. A process called Structure-from-Motion interprets this footage as a 3D scene and creates a sparse point cloud. Scene optimization, often AI-assisted, then adjusts each Gaussian's properties to match the input images.

From 1990s Research to 2024 Hollywood

The core idea dates back to Lee Westover's work in the early 1990s. The modern version stems from a 2023 paper by French researchers that refined the technique. Since then, adoption has accelerated.

A$AP Rocky's music video "HELICOPTER" uses 4D Gaussian Splatting, which bakes a temporal element into the capture. This allows moving objects to be captured and replayed, not just static scenes. The Superman movie used splats for complex visual effects that would have been far more expensive with traditional methods.

The Interactive Problem

Here's the challenge Sumygin tackled: Gaussian splats capture visual data only. They don't know that a wall is solid or that a floor should support the player. The technology was built for viewing, not interacting.

Traditional game engines need collision meshes, physics data, and navigation meshes to make a world playable. A splat scene has none of these. It's a collection of semi-transparent colored blobs that happen to look like a real place when viewed from the right angle.

Making Splats Playable

Sumygin built this project using PlayCanvas, a browser-based game engine. The technical challenge was creating collision geometry from visual-only data. The specifics of his approach aren't fully documented, but the result works: you can walk through a photorealistic environment, and the walls actually stop you.

The game runs entirely in your browser. No download required. The rendering performance is surprisingly smooth given that Gaussian splatting was designed for offline viewing, not real-time interaction. This suggests the technology has matured faster than expected.

Current Limitations

The game is basic for a reason. Splat scenes can show visual artifacts when viewed from angles not well-represented in the source photos. Collision detection requires manual work since the splat data doesn't inherently define solid surfaces. And while rendering is efficient for static views, interactive applications add complexity.

Still, the proof of concept works. You can play it right now. That's more than anyone expected from a technology that was considered view-only until recently.

What This Means for Development

Game developers spend enormous resources creating realistic environments. Gaussian splats offer a shortcut: scan a real location with a phone camera, process the footage, and get a photorealistic 3D environment. The visual quality exceeds what most studios can achieve with manual modeling.

The limitation has been interactivity. Sumygin's project shows that limitation isn't absolute. With some additional work to define collision and physics, splat scenes can become playable spaces. Whether this approach scales to full games remains to be seen.

Frequently Asked Questions

What are Gaussian splats in 3D rendering?

Gaussian splats are a data type for representing 3D worlds with photorealistic quality. Created from photos or video of real spaces, they use variable-sized, semi-transparent colored elements to recreate scenes with high visual fidelity at modest rendering cost.

Can you play games using Gaussian splatting?

Yes, as demonstrated by this browser-based FPS. While Gaussian splats capture only visual data (not physics or collision), developers can add these layers to create interactive experiences.

How do you create Gaussian splat scenes?

Take many photos or videos of a real space, run Structure-from-Motion to create a point cloud, then use AI-assisted optimization to adjust each Gaussian's position, shape, opacity, and color to match the source imagery.

What software supports Gaussian splatting for games?

PlayCanvas, the browser-based game engine used in this demo, supports Gaussian splats. Other 3D tools and game engines are adding support as the technology matures.

Is 4D Gaussian Splatting different from regular splats?

Yes. 4D Gaussian Splatting adds a temporal element, allowing capture of moving objects. A$AP Rocky's HELICOPTER music video used this technique to include motion in the scene.

Source: Latest from Tom's Hardware