Researchers have developed a new approach for capturing 3D models using neural radiance fields.
For a long time, 3D scanning was done by either photogrammetry or probing through light beams or patterns. Then, around a year ago, a new approach emerged called Neural Radiance Fields, or “NeRF”. It’s an AI technique that is quite different.
While NeRFs require a set of images much like photogrammetry, the processing is entirely different. A neural network is trained to predict the colour and opacity at any 3D point, as seen from any direction. When rendering, you cast virtual rays through the scene, sample points along each ray, and integrate the predicted colours/densities to figure out what the camera would see — producing a realistic new image.
Incredibly, this allows you to generate views of a scene that you didn’t capture in the input photos, with photorealistic results.
This is an amazing technique that I have been using occasionally with excellent results. You can try this yourself using the Luma AI smartphone app.
But there’s one problem: it requires the subject to remain motionless. That’s fine for scanning a part, sculpture, or cooperative human, but not so much for a speeding car or airplane flying by.
Now, researchers at Chung-Ang University in Korea have developed a new method they call “MoBluRF”. It’s a framework that creates sharp 4D (dynamic 3D) reconstructions from blurry videos captured on everyday handheld devices.
Previously, blurry videos, particularly with blurred backgrounds while the camera follows a moving subject, were of insufficient quality to train NeRFs.
Assistant Professor Jihyong Oh explained:
“Our framework is capable of reconstructing sharp 4D scenes and enabling NVS from blurry monocular videos using motion decomposition, while avoiding mask supervision, significantly advancing the NeRF field.”
MoBluRF works in two stages. First comes Base Ray Initialization (BRI), where the system roughly reconstructs the 3D scene from the blurry input and uses that to refine its “base rays.” This is important because simply taking the rays straight from the blurry images can lead to major inaccuracies. BRI essentially gives MoBluRF a better starting point.
Once those base rays are locked in, MoBluRF moves on to Motion Decomposition-based Deblurring (MDD). This stage uses something called Incremental Latent Sharp-ray Prediction (ILSP) to peel apart the blur step by step. It separates the blur caused by camera motion from that caused by objects moving within the scene — a clever trick that leads to much sharper results.
On top of this, MoBluRF brings in two fresh loss functions to train the system more effectively: one that can tell static from moving areas without needing motion masks, and another that improves the accuracy of object geometry. Both tackle problems that have tripped up previous deblurring methods.
This is a notable development, as it suggests that common smartphones, drones, and consumer cameras can capture much sharper, more immersive 3D content without specialized equipment.
A powerful 3D scanner in everyone’s pocket.
Via IEEE
