SAM 3

Meta just dropped SAM 3 this week. It's a computer vision model that can find and segment objects in images and videos using plain text—type "red baseball cap" and it'll automatically highlight every one in the scene. Previous versions only worked with clicks and boxes.

They trained it on 270K unique concepts, which is 50x more than what other models use. According to Meta's tests, it's twice as good as competing systems at recognizing and segmenting objects from text descriptions.

They're already using it in Instagram's video editor for adding effects to specific things, and in Facebook Marketplace so you can see how furniture would look in your actual room before buying. The model's open source and you can test it on their playground. They also released SAM 3D alongside it for turning 2D images into 3D objects.