Custom 3D Object Tracking in CameraKit on Web?

swissDAO

I would like to train my own model to track 3D objects in Web with the Camerakit sdk in javascript.

Where can I start? By 3D tracking I mean 6D tracking, not just 2D; have look on the video:

stevenxu

@swissDAO Thanks for posting on the forum. Submitted a ticket for our eng to look into this. Hang tight! (4771)

Michael Mase

Hi @swissDAO! Custom neural networks/ML models can be imported into Lens Studio for use in your Lens features. At this time, Lens Studio does not offer the ability to train additional 3D models. The following tracking is available for use today:

• Face
• Hand
• Hand Joints
• Shoulder
• Upper Body
• Full Body
• Cat
• Dog
• Cats and Dogs

More information about custom ML Components can be found here: https://docs.snap.com/lens-studio/references/guides/lens-features/machine-learning/ml-component/ml-component-overview

swissDAO

@Michael Mase thx for your response - do you have an overview of the plan, if and when this will be possible?

There are two companies doing this already for the web:
https://www.reddit.com/r/augmentedreality/comments/z22d7m/features_of_our_new_3d_object_tracking_from/