Face Mesh Using User's Face Texture


Is there a way to use the user’s face from the camera feed to apply to the face mesh? I would like to use this to distort and morph the user’s face.



There’s no existing way of doing this. One way I can imagine it could be approached is:

  1. Get the screen space position of the face tracking anchor using Camera.WorldToScreenPoint
  2. Defining a box around this point and copy the pixels from ZapparCameraBackground.GetCameraTexture to your Texture2D.
  3. Update the main texture of your FaceMaterial with your new texture.

It might need a bit of tweaking of box size to avoid the background from face texture, but you can be creative with that.

Hope that helps.