New "reset_3d" Layer Mode (the missing one!) so we can do per-object/group masking


#1

I’ve only spent a few days playing with ZapStudio so it’s possible I’ve missed something in the docs, or this may just not be possible. But it’d be super powerful if it was.

During rendering a depthBuffer is used to track the depth of previously rasterised objects, pixel by pixel, so occlusion can be performed correctly. It’s how we can do that portal masking effect: you drop a Square Mask at the start of your hierarchy, set it to render nothing to the screen (Alpha = 0) but still have it write to the depthBuffer, so even though it’s invisible it stops anything behind if from being rendered.

Once something’s been written to the depthBuffer, though, it’s there for ever (well, until the frame is redrawn), and regardless of render order, you’ll never be able to have a 3D object render behind it.

If we had a way to clear that depthBuffer at a certain point in the render order then it’d open up loads of new and exciting effects:

  • you could mask groups of objects individually
  • multiple objects could occupy the same 3D space, but depending on which mask your phone was “looking through”, different ones could appear
  • you could add 3D overlays (not to be confused with the “overlay” layer mode) over previously drawn objects, to create holographic display effects
  • but mostly: once you’ve created a masked portal, you could then reset the depthBuffer and add elements around and behind the portal - something impossible at the moment.

For example - my first experiment with a portal:

44

scrappy image target:
33

It’s a load of 3D gubbins masked with the Square Mask. It’s an idea for a postcard.

Say I want to add some sparkles coming out from behind the card - I can’t do it, because once the first mask has written to the depth buffer it’s there for good, so anything I try and put there gets occluded:

Nosing around behind the scenes, there seem to be two flags that get set depending on an object’s LayerMode: depthTest (which determines whether to test the buffer before writing pixels) and fillDepthBuffer (which determines whether to write the new pixels’ depths to the buffer).

We can currently choose 3 out of the 4 possible permutations:
overlay: test = 0; fill = 0;
test_3d: test = 1; fill = 0;
full_3d: test = 1; fill = 1.

We need that missing permutation. Maybe call it reset_3d:
reset_3d: test = 0; fill = 1. Put another way: don’t test the depth buffer, but do write the new pxel depths to it.

We could apply it to a large plane set in the distance, and it’d effectively reset the buffer, stopping any existing masks and objects from occluding new objects. Then we could start over - render new masks and objects to the colour and depth buffers freely.

It’d be an advanced tool: it would be up to us to ensure the render order made sense so occlusion happened correctly for the particular scene and camera orientation, but it’d open up a whole new set of visual possibilities; allow for some clever and eyecatching compositing tricks.

Anyway, that’s been my TED talk; if you’ve got this far, thank you :slight_smile:


#2

Hi @howiemnet,

I have good news for you! This is one of those areas where we have already got a solution implemented but it isn’t yet documented, and isn’t rolled out on all of our runtime platforms. If you’re targeting WebAR though, both our mainline https://web.zappar.com and the beta-branch https://beta.zappar.app support this feature.

There’s a new property on node called renderLayer. This can be set to an integer or null, and is null by default. It is inherited down from parents, and updated whenever a non-null value for it is found.

When rendering, this acts as the highest level sorting on draw calls. So everything in render layer 0 (the default) will be drawn in hierarchy order, followed by everything in render layer 1. The depth buffer is cleared between each render layer.

What this means in practice is it’s quite flexible and easy to use - you can just set the renderLayer on a group to 1, for example, and now everything under that group in the hierarchy will be rendered after other objects in the scene, and with a clear depth buffer to start.

As it’s not documented yet it’s also not in Studio’s typescript definitions, nor in the properties panel, so you’ll need to set it from a script and cast it to any to keep typescript happy, and as I say you’ll need to be careful to ensure that you’re using a runtime with support for the feature.

In the end, it boils down to this:

const RenderLayerGroup = symbol.nodes.RenderLayerGroup;
(RenderLayerGroup as any).renderLayer(1);

You can pass null too to revert to the default behaviour of inheriting the value from the parent.

Have fun!


#3

Wow - that’s wonderful :slight_smile: Can’t wait to have a play with it. Thanks Simon!

EDIT: it works! Quick (and silly) proof of concept - I promise I’ll use this new knowledge for more sensible things - https://www.youtube.com/watch?v=yECZg4sPf74


#4

Nice demo! Can’t actually figure out how you’re got that set up :slight_smile:


#5

Now that’s the response I’m after :wink:

This renderLayer thing really does open up a world of possibilities. Watch this space…