Body Depth Texture - Multiple People/Crowd Support
Hi all,
Is there a way to occlude objects using the camera depth texture for multiple people at the same time? Currently it seems that only a single person depth texture is supported. A following question would be: Is it possible to have this feature through an ML Component? (And of course, everything I mentioned here should be supported in CameraKit integrations)
(tagging @stevenxu for convenience based on our latest talk)
Thank you!
Best Answer
-
Hello @onurk
Depending on your use case there are a couple of approaches you could take.
For a scenario where there are a people moving through the frame while AR content remains in the background, this could be achieved through a Segmentation Texture. You could set the segmentation type to body and click the inverted checkbox. Then you can use this as the Mask Texture for your camera which captures your AR content. This will make it so only objects which are not covered by a person are drawn but ignores depth so even if the person is far away and the object is close, it will not be drawn.
For a more interactive option, I'd recommend looking at the Body Mesh template or just add Full Body Mesh to your objects panel. From there, import the Body Mesh Occluder material from the Asset Library and assign that as the body mesh material. That allows you to occlude objects behind the people in frame but draw content in front as needed:
You can also create your own ML models and use those if that will be the best optimized for your experience. If your question is more related to depth textures specifically, we can also look into more options there.
2
Answers
-
Thank you @ForumChris for the detailed answer! We're checking out all the possibilities you mentioned. I guess the closest one would be having a custom ML Model utilizing depth texture.
Thank you for the update @stevenxu Looking forward to hearing from you.0
