What is the purpose of converting RGB images to spatial photos?

Converting RGB images to spatial photos is useful for various computer vision and 3D modeling applications. It enables the creation of depth maps, stereo images, and HEIC files for use with Apple Vision Pro.

How does it work?

Spatial Media Converter can be used to convert JPG or PNG images into a variety of downstream spatial formats:

1. It first converts from the RGB image to a depth map using a monocular depth estimation network. You can then visualize that depth network in a three.js viewer where we project the image onto a 3D mesh.
2. Next it converts the RGB-D image into a stereo image by computing the estimated disparity between the left and right eyes. The result of this step can be viewed as an anaglyph or a left and right eye stereo image.
3. Finally, we we convert the stereo image to a HEIC file which is compatible with Apple Vision Pro.

What depth estimation models are supported?

We support the following monocular depth estimation models:

• Depth Anything Small
• Depth Anything Base
• dpt-hybrid-midas
• dpt-large