Heads up! We're sunsetting this forum to streamline our community space.

join us on Discord for support, updates, and all things Snap AR! See you there!
Sign up for our live Camera Kit office hour on July 17 @ 9:30am (PT) / 4:30pm (GMT)

[Camera-kit for Web] Frame processing is slow

Tony
Tony Posts: 5
in General #1

Hi guys,

Test device : Iphone13pro, iOS16+, Safari/chrome
Camera kit : Web sdk
Lens : footwear try on, try-on, garment transfer.

I am developing a lens for web app using camera-kit. I found frame processing is very slow ( i.e. camera feed is getting frozen) when a specific lens is loaded compare to others. I tested three lens template to compare CPU load. Among lens templates provided in lens studio, "footwear try on" and "try-on" are ok, but "garment transfer" is running extremely slow ( camera feed is almost frozen.) I think that lens computation time using camera-kit for web depends on which type of lens is loaded. Does any body fix this problem?

Thank you.

Comments

  • Michael Mase
    Michael Mase Posts: 66 🔥🔥
    #2

    Hi @Tony! Yes, as you suggested, Lens performance in Camera Kit (all platforms) is dependent on the Lens and device being used. In regards to the Web SDK specifically, these issues may be exasperated in certain cases since all processing has to occur through the browser rendering pipeline. This is good feedback nonetheless and we will see what we can do to improve Web SDK performance for these templates.

  • Tony
    Tony Posts: 5
    #3

    Thank you for reply. Maybe, since garment transfer template runs two ML algorithms - segmentation and bodytracking, the webgl rendering takes more time than other templates. Hope to find a way to work around this issue soon!

  • Marcos_Santos
    Marcos_Santos Posts: 1
    #4

    Hi
    Do you have any idea how can I increase the performance for the Garment Transfer?
    Is there a way we can modify the code? Or implement it in some way?

    Another question, is there a way we can communicate the lens with js application ?

  • stevenxu
    stevenxu Posts: 612 👻
    #5

    @Marcos_Santos Thanks for posting on the forum. I created a ticket for our eng to look into this.

    Regarding your last question, I want to say this feature is not yet available... but will let our eng confirm when they get a chance. Hang tight

  • Mikalai
    Mikalai Posts: 6
    #6

    @Marcos_Santos

    Another question, is there a way we can communicate the lens with js application ?

    Yes, that is doable through Remote API: https://docs.snap.com/camera-kit/guides/web-customization/remote-api#launch-data (don't be confused with the name, it can be used for local communications as well, but your custom Remote API spec has to be registered)

Categories