Pass glb 3d model through api to lens

SlideUp

Hey!

With the new api that allows us to pass arbitrary data from app to camera kit lens, is it possible to pass a 3D model and instantiate it inside lens?

Today there’s a gltf asset used to download a remote Bitmoji in lens studio. Can we use something similar to load an arbitrary 3D model ?

Thanks

stevenxu

@SlideUp said:
Hey!

With the new api that allows us to pass arbitrary data from app to camera kit lens, is it possible to pass a 3D model and instantiate it inside lens?

Today there’s a gltf asset used to download a remote Bitmoji in lens studio. Can we use something similar to load an arbitrary 3D model ?

Thanks

Yes this can be achieved through the loadResourceAs APIs as outlined here. The payload will need to only include the GLTF asset and you would replace the loadResourceAsImageTexture method with the loadResourceAsGltfAsset method

stevenxu

@SlideUp Great Q. Checking with team and will get back to ya!

SlideUp

Awesome! thanks!!

Eric G

@stevenxu I've been working on getting a working example of loading a GLTF asset using remote API. Are there any samples available?

stevenxu

@Eric G Thanks for posting on the forum! Hmm.. let me double check with the team if there is one. Are you just trying to get it to work in Lens Studio right now? or on web or mobile?

Eric G

@stevenxu I'm working on getting this to work in the iOS Camera Sample. I have found some samples of glb files and something is now showing but it is just white model so not sure what is missing now.

Setup

Objects
-- Camera
-- -- Model Container (empty scene object)

Resources
-- load_script
-- Remote Model Material (empty material used for tryInstantiate)

Example 3D Model

https://github.com/KhronosGroup/glTF-Sample-Models/tree/master/2.0/Avocado/glTF-Binary
-- I am using the binary glb file here.

Code to Display

// @input SceneObject modelContainer
// @input Asset.Material emptyMaterialHolder

function updateAsset(gltfAsset) {
if (gltfAsset !== null) {
var component = script.modelContainer.copySceneObject(script.modelContainer);
var model = gltfAsset.tryInstantiate(component, script.emptyMaterialHolder);
model.getTransform().setLocalScale(vec3.one().uniformScale(script.size));
}
}

stevenxu

@Eric G Thanks! Team is aware and will get back to you as soon as they can.

ForumChris

@Eric G just to confirm, you now have the model coming in but the baked in materials/ textures are not? If you use the built in GLTF material it should render correctly:

Eric G

@ForumChris I have not used the GLTF material, I'll give that a shot and report back!

Eric G

@ForumChris @stevenxu Using the GLTF specific material did the trick, thank you so much for the help!

Eric G

Last issue, is there any way to get size of these models and scale to fit within visual space or bound to fit within a box?