Moving Object Segmentation: All You Need Is SAM (and Flow)
The paper explores motion segmentation in videos using the Segment Anything model (SAM) combined with optical flow, showcasing significant improvements in performance over existing approaches, particularly in single and multi-object benchmarks.
Two innovative models, FlowI-SAM and FlowP-SAM, have been introduced in the paper to address the task of motion segmentation in videos. These models aim to identify and segment moving objects, leveraging the Segment Anything Model (SAM) framework.
FlowI-SAM focuses on utilizing optical flow inputs to capture motion information for object segmentation. It preserves the architecture of the original SAM model and is directly finetuned with optical flow inputs. FlowP-SAM, on the other hand, introduces a flow prompt generator and a finetuned segmentation module to improve segmentation accuracy. This model progressively adds new components to the vanilla SAM checkpoint, enhancing its performance. Both models are trained on synthetic datasets and then finetuned on real datasets such as DAVIS sequences.
Comments
None