How can I create a face inset with hair segmentation?

Subtropic Studios
Subtropic Studios Posts: 2
in Getting Started #1

I am looking to create a mask that includes face inset and hair segmentation, similar to attached examples. I know there is a way to create this mask, but I'd like for it to be pinned to certain areas in the screen so that its only detecting face + hair.

Examples:
https://t.snapchat.com/JUBt1rU8
https://t.snapchat.com/mFnkmp9T
https://t.snapchat.com/1GtBuTkA

Best Answer

  • Bakari Mustafa
    Bakari Mustafa Posts: 178 🔥🔥🔥
    #2 Answer ✓

    To create a mask that combines both of these features, you will need to use some SnapML models and components.

    One possible way to do this is:

    1. Add a SnapML component to your scene and import a model that can segment the user’s hair. You can find some examples of such models on Fritz AI or Lens List.
    2. Add a Camera Texture component to capture the camera input and connect it to the SnapML component’s input texture.
    3. Add an Unlit material and set its base color texture to the camera texture. Set its opacity texture to the SnapML component’s output texture. This will create a transparent mask of the user’s hair.
    4. Add another Unlit material and set its base color texture to black. Set its opacity texture to the same SnapML component’s output texture, but invert it by selecting Invert Mask in the Inspector panel. This will create a black background for the user’s hair.
    5. Add two Screen Image components and assign them each one of the Unlit materials. Adjust their size and position on the screen as you like.
    6. Add another SnapML component and import a model that can segment the user’s face. You can use one of Lens Studio’s built-in models for this purpose.
    7. Connect another Camera Texture component to this SnapML component’s input texture.
    8. Add another Unlit material and set its base color texture to black. Set its opacity texture to this SnapML component’s output texture, but invert it by selecting Invert Mask in the Inspector panel. This will create a black background for the user’s face.
    9. Add another Screen Image component and assign it this Unlit material. Adjust its size and position on top of one of the hair masks. This will create a face inset with hair segmentation.

    Quick links with useful info for you to unpack

    https://medium.com/@austin_32493/custom-face-mask-segmentation-for-lens-studio-with-fritz-ai-video-transcript-609d17351ec

    https://blog.lenslist.co/2021/03/09/snapml-in-lens-studio-an-overview-of-ml-model-types-and-use-cases/

    https://support.lensstudio.snapchat.com/hc/en-us/community/posts/360075256791-Can-I-do-a-head-and-shoulders-inset-instead-of-a-face-inset-

    https://www.youtube.com/watch?v=EfsbKQWk5tA

Categories