How do I use my own animation for Bitmoji on PlayCanvas?

Kotone
Kotone Posts: 15 🔥

hello. I am a Japanese developer.
Currently, we are developing SnapMini using Bitmoji on PlayCanvas.
I made my own animation in Maya while looking at the Bitmoji document, but the correct animation data (GLB format) is not created even if I import it into PlayCanvas.
Even if I output GLB format data in Maya, I could not use it in PlayCanvas.
I've been doing a lot of research, but I don't understand why I can't use my own animation.
I would like you to tell me. Thank you.

Best Answers

  • Kotone
    Kotone Posts: 15 🔥
    edited July 2022 #2 Answer ✓

    I was able to apply my own animation to Bitmoji by following the steps below. Is this the correct way?
    Do you have a better idea?

    ① Select "snap_bake.py> PlayCanvas-Mobile" in Maya and execute Export.

    ② Exported FBX data to "BitmojiForGames-master \ Documentation \ AnimationToolset \ converters"
    Open it with "FBX2glTF-windows-x64.exe" in. (https://github.com/Bitmoji/BitmojiForGames)

    ③ Open the output GLTF data in Blender

    ④ Output in GLB format with Blender

    The animation created by the above method did not contain face animation data. Is there any solution here?

  • yaustar
    yaustar Posts: 6
    edited August 2022 #3 Answer ✓

    There's two parts here:

    Using the mac version of the convertor, I created a binary glb with the following command:

    ./FBX2glTF -i sidewalk_L_000.fbx -j sidewalk_L_000.json -b
    

    (Use ./FBX2glTF -h for all the commands)

    This gave me a file sidewalk_L_000.glb with the face animations flips inside it.

    I then went to test it in a fork of the 3D Bitmoji Library project https://playcanvas.com/project/964811/overview/kotone-bitmoji-animation

    And found out that the timings for the faces to change where at frame times that was longer than the animation.

    The animation is about 1.3 secs long but the you can see from the JSON data in the animation:

    "extras": {
        "facial_swaps": {
            "expressions": [
                "idle",
                "idle-blink",
                "idle"
            ],
            "frame_rate": 30.0,
            "times": [
                0.0,
                1.9666666984558105,
                2.133333444595337
            ],
            "appliesTo": [
                "face",
                {
                    "geometryName": "C_head_GEO",
                    "prefix": "face",
                    "postfix": "texture",
                    "swapper": "texture"
                },
                "beard",
                {
                    "geometryName": "C_beard_GEO",
                    "prefix": "C_beard",
                    "postfix": "GEO",
                    "swapper": "geometry"
                }
            ]
        }
    }
    

    The times are at 0.0, 1.96 and 2.13 which are times beyond the time of the walking animation.

Answers

  • stevenxu
    stevenxu Posts: 612 👻

    @Kotone That's great news! Thanks for sharing that you got it working. Let us know if you have any other questions. We're here to help!

  • Kotone
    Kotone Posts: 15 🔥

    @stevenxu
    I didn't realize I was getting the answer, so I edited the post I added. excuse me.

    The GLB data created by my own method does not include face animation. Any ideas to solve this?
    Thank you.

  • stevenxu
    stevenxu Posts: 612 👻

    @Kotone Ah my apologies. I'll ask our Bitmoji team tomorrow and get back to you as soon as we can!

  • Kotone
    Kotone Posts: 15 🔥

    @stevenxu Waiting for an answer, thank you very much!

  • yaustar
    yaustar Posts: 6
    edited July 2022 #8

    @Kotone, just to check, are you using the PlayCanvas 3D Bitmoji Library as that has extra logic for working with face animation on Bitmoji mobile animations? https://docs.snap.com/minis/playcanvas/3d-bitmoji

    If you are, are you using the mobile instructions for the face animations? https://github.com/Bitmoji/BitmojiForGames/blob/master/Documentation/AnimationToolset/README.md#using-the-bitmoji-rig

  • Kotone
    Kotone Posts: 15 🔥

    @yaustar
    Yes, I use PlayCanvas' Bitmoji Library for development.
    Do face animations require special processing?
    I've read the docs many times, but I'm reading them while translating them into Japanese, so I may not understand the docs correctly.

    The animation for Bitmoji uses Bitmoji's mobile model data distributed on Github.
    Playcanvas also uses Bitmoji for mobile.
    When creating an animation in Maya and outputting the animation, "snap_bake.py" is also used for output.

  • yaustar
    yaustar Posts: 6
    edited July 2022 #10

    Ok, this looks like a tooling issue with the Bitmoji tools as you are using the PlayCanvas Bitmoji Library (which supports the face animations for mobile rigs).

    The face animations for mobile should added via this method in Maya

    @stevenxu Can you ask the Bitmoji team to help out in this thread please?

  • Kotone
    Kotone Posts: 15 🔥

    @yaustar
    Yes, Maya does something like a video.
    Therefore, when previewed on Maya, the normal Face animation will play.

    Face animations cannot be applied only on PlayCanvas.
    thank you.

  • yaustar
    yaustar Posts: 6

    Can you upload the GLB or post a link for the animation that you create so that we can have a closer look please?

  • stevenxu
    stevenxu Posts: 612 👻

    @Kotone You may need to bake the face swap data in the glTF extras fields, but please follow @yaustar request for us to fully understand the issue. ty!

    Can you upload the GLB or post a link for the animation that you create so that we can have a closer look please?

  • Kotone
    Kotone Posts: 15 🔥

    @yaustar
    I uploaded GLB data and Maya data to the link destination, and FBX data and Json data output from Maya using the Bitmoji plug-in.
    https://drive.google.com/drive/folders/1gxORtMkEgwyB2jbrcCx6PPPpzDlwwl0A?usp=sharing

    GLB data is created as described in my previous post.

    please confirm.

  • yaustar
    yaustar Posts: 6
    edited July 2022 #15

    Looking at the GLB, the face animations aren't included in it (you can see the JSON data when opening the GLB in a text editor).

    I don't know offhand how to add the face animations into the GLB as part of the Maya export process and @stevenxu will need to get the Bitmoji team involved as they wrote the tools

  • Kotone
    Kotone Posts: 15 🔥

    @yaustar
    yes. Probably, when I output from the plug-in in Maya, it is divided into FBX and Json files, but I think that it is because the Json file contains face animation and FBX does not contain face animation.

    @stevenxu
    We would appreciate it if you could support us.

  • yaustar
    yaustar Posts: 6

    I spoke with the Bitmoji team and there's a tool that takes the FBX and JSON and converts it to the GLB

    https://github.com/Bitmoji/BitmojiForGames/tree/master/Documentation/AnimationToolset/converters

  • Kotone
    Kotone Posts: 15 🔥

    @yaustar
    Thank you.
    But reading the JSON with the converter doesn't output anything.
    Both JSON and FBX cannot be read by the converter.
    The GLTF data that is output when only FBX is read is the one I am currently using.
    How to use my converter is as per my post on July 14th.

    The explanation of the converter can only be found when using Unity.
    Is there a better way for Playcanvas?