First Look - New and Improved World Tracking for WebAR

Hi everyone,

We’ve been hard at work on our new and improved World Tracking for WebAR implementation for a while now, and I’m really pleased to be able to share a first look now with our great forum community, just in time to see in the New Year!

This new implementation removes a number of the big limitations of our previous Instant World Tracking approach - you can now look away from the placement position and the content will continue to track, and we have also removed the requirement for a textured horizontal surface to place content.

Here’s a video of this first look build in action on a typically cloudy and wet winter’s day at the glourious British seaside :slight_smile:

You can try it out yourself by scanning the QR Code, or visiting this link on your smartphone.
eoPGozEaEVb6n

The source for this demo is available in the zip below, which also contains the standalone build of the Universal AR SDK for Three.js with the updated World Tracking implementation. This is exposed via the same InstantWorldTracker API as previously - simply set the pose of the anchor with a camera-relative position (and orientation option), and the library will aim to keep the content anchored at that position in the world as the user moves. In the demo, we reset this every frame during “Placement” mode, and then just stop calling the setAnchorPoseFromCameraOffset function once the user has placed the content. Other UX choices are possible with the same API.

RobotWorldTrackingDemo.zip (3.8 MB)

We’re sharing this now as it is in a state where we believe it is useful and it has a number of clear advantages over the previous implementation. There are still some caveats: it works best in static, well-textured environments, with smooth camera motion - moving the device position without rotating is best at the start (much like the general side-to-side motion that works best for initializing native ARKit and ARCore experiences). This implementation does also make portal-type experiences possible but as they often encourage fast rotational motion they are likely to suffer more from scale drift than outside-in content where the user is primarily focussing on content in front of them.

Zappar is officially closed this week for the holidays, so we’ll have much more to say about this early in 2022. One of the first tasks will be making this implementation available across more of our Universal AR SDKs and distribution types (such as making beta libraries available via npm), and looking at an update to the ZapWorks Studio WebAR sites too.

I’d like to thank the awesome Zappar team for all their work to make this first release possible. This is just the first milestone on our roadmap to a more complete World Tracking for WebAR implementation. We’ve got performance and quality improvements to come early in 2022, we’ll be working on specific improvements for portal-type experiences, and we’ll also be looking at API additions now we have the required groundwork in place for supporting other use cases such as multiple anchors and extended tracking (combining World Tracking and Image Tracking, for example).

Wishing everyone all the best for the New Year - we’re really excited by what we’ve got in store for 2022!

10 Likes

@simon this looks awesome! I have 2 questions:

  1. Is this already available on Studio? If not, when will it be?
  2. Are you guys using WebXR for this, or is it proprietary technology? What’s the minimum required iOS version?

Thanks guys. I hope you had a merry christmas and I wish you a happy new year! This is a great gift!

How about a particle system for Studio now? Hahaha.

1 Like

Hi @marks, thanks!

This will be used as a new implementation for the Instant Tracking API in Studio, so the plan is we’ll just update beta.zappar.app (and the lite-branded arweb.app) to include this implementation. We’ll likely do that in January, and then any Studio Instant Tracking content will benefit from the improvements mentioned above.

The biggest current reason not to switch immediately is the binary size impact - the web assembly is now 1.3MB compressed (vs 800KB) and 3.7MB uncompressed (vs 2.2 MB), so it will have an impact on download time and startup time. Later in the beta process we’ll look in more detail at opportunities for reducing the binary size impact.

With Studio WebAR sites the runtime is shared by all content, whereas in Universal AR each project can choose which version of the runtime libraries to use. We expect to keep iterating pretty rapidly on this implementation, so perhaps in the early days we can make it opt-in with a query string parameter or something on the Studio WebAR sites too.

This isn’t WebXR - it’s our proprietary computer vision code (in WebAssembly) using data from the standard web APIs for accessing cameras and motion sensors. So the required versions are the same as for the current libraries; effectively iOS 11.3 and up for Safari and 14.3 and up for other WKWebView browsers - full details here: https://docs.zap.works/universal-ar/javascript/compatibility/

3 Likes

@simon I sent an email called “new instant tracking urgent question” could you take a look? Thanks!

1 Like

Happy New Year everyone!

@marks - I don’t usually see the emails to support, but the learning and support team have forwarded it on to me. I’ll share the points arising here so everyone can see them. The quotes below aren’t straight from your email but I think I cover all the main points.

When can we start building Studio projects using this?

We have now discussed internally, and our rollout plan for the Studio WebAR builds is to update the bleeding-edge staging.zappar.app site (rather than the beta / lite-branded sites for now). That will make it usable for testing without the downsides mentioned above whilst we’re at this “early beta” stage. For more “permanent” deployments we will be able to deploy the same version to a custom Studio WebAR site.

There’s still some work needed to get the Studio WebAR site build updated to use the new implementation. It’s not something we can commit to a specific timeframe for at this point. As I mentioned, we’d hope to make something available this month but we’re not able to guarantee that.

Can we opt-in a project [I assume you mean a Studio project] to use this implementation before a wider roll-out?

Not on a per-project level. The runtime is part of the WebAR site for Studio content - so we have a different version on beta.zappar.app vs web.zappar.com, for example. So the level at which we can deploy this is at the site level, which means that a custom WebAR site could also be updated as mentioned above.

Yay, Portals!

As I mentioned in the first post, portal-type experiences aren’t brilliant at the moment. They’re technically possible, but will get better with future updates to the implementation. At the moment I’d definitely recommend testing to see if it meets your expectations before going that route.

In general I’m not a massive fan of portals from a UX perspective - you need a lot of floor area to make the “walk through” mechanic work, and space tends to be pretty limited in many indoor spaces (especially in homes). That said, we recognise that portal experiences are still in-demand, they are cool in the right spaces (outdoor or very large floorspace) and they do also make for good videos. For all those reasons we are committed to improving how well they work with this new implementation.

Finally, the Learning and Support Team are an awesome resource for all our “official” APIs and tools, but whilst we’re in this early beta phase we’re still heavily iterating on the implementation and so it doesn’t come with our usual high level of support. Though I can’t provide urgent responses I will keep an eye on replies to this thread and attempt to answer things on a best-effort basis here.

5 Likes

How’s the update for studio coming along?

Hi there,
Just jumping in on this thread to see when the tracking will be available in a WebAR Environment, are we any closer? We’ve got some awesome projects to come.

Thanks,
Sean

Hi @simon ,
Is there any update on this feature being available as a predefined template in Studio?

Regards,
Sean

1 Like