FlowMap: High-Quality Camera Poses, Intrinsics, and Depth via Gradient Descent
End-to-end differentiable method for precise camera pose estimation, depth mapping, and intrinsics calculation from videos
FlowMap introduces a method for 3D scene reconstruction from a sequence of images. It leverages a feed-forward neural network architecture that re-parameterizes depth, pose, and camera intrinsics to accelerate convergence and improve reconstruction quality. By optimizing these re-parameterizations in a single forward pass, FlowMap efficiently estimates poses and intrinsics, crucial for generalizable pre-training settings.
One key aspect of FlowMap is the incorporation of point tracking in addition to optical flow computation. Point tracks enable accurate point correspondence across multiple frames, reducing drift in longer sequences such as object-centric 360° scenes. This integration of point tracking enhances the view synthesis performance of FlowMap, indicating potential benefits from further advancements in point tracking methods.
Re-parameterizations in the optimization process is also of key importance. Comparisons between direct free-variable optimization and FlowMap's re-parameterized approach demonstrate significantly better reconstruction results and faster convergence with the latter. Specifically, the re-parameterization of focal length, along with depth and pose, plays a critical role in achieving high-quality reconstructions efficiently.
FlowMap's compatibility of loss formulation with conventional correspondence methods emphasizes the flexibility and adaptability of its approach. By combining re-parameterizations, point tracking, and efficient optimization strategies, FlowMap presents a promising solution for robust and accurate 3D scene reconstruction from image sequences.
Comments
None