We’ve been working on a new tracking type for the Zappar platform over the last few months which we’re calling “Instant Tracking”. I’m excited to be able to share an early preview of the technology with the ZapWorks community.
So, what is Instant Tracking?
Instant Tracking sits between orientation-only 360 experiences and full world tracking. Instant Tracking allows content to be placed on a horizontal surface in the world (no specific markers or image targets required) and maintains the position of that content in the world as the user moves.
What’s the difference between World Tracking and Instant Tracking?
A full world tracking solution aims to build up a complex understanding of all of the surfaces in the world and the exact motion of the device, with everything measured in absolute real-world units such as metres. ARCore and ARKit offer this type of full world tracking, and you can make use of these in Zappar experiences through our World Tracking API (which is implemented using ARCore and ARKit internally).
In contrast, Instant Tracking doesn’t build such a complex model of the world, but instead focuses on keeping the position of the content consistent from one camera image to the next.
This inherently simpler approach has a number of advantages:
A world tracking approach working in real-world scale requires some device motion to determine the absolute scale of things in the camera view, and it often takes a noticeable few seconds until those estimates become stable and content can be placed without jitter. Likewise device motion is required to detect surfaces, and surfaces must be detected before content can be placed on them. Instant Tracking allows placing content straight away, no camera motion required.
Wide Device Support
Instant Tracking requires a device with a gyroscope, but doesn’t need accurate device-specific calibration outside of this - any device with a gyroscope should work. This is especially important on Android where ARCore only supports around 200 devices out of the more than 10,000 distinct device models with access to Google Play.
Works in WebAR
Zappar Instant Tracking is our own implementation that doesn’t rely on ARCore or ARKit. This allows Instant Tracking to be used anywhere Zappar content is supported, including both native app runtimes and WebAR deployments. For WebAR, Zappar Instant Tracking targets the mobile browsers that are already in widespread use - no unusual browsers or support for upcoming web standards is required.
Instant Tracking’s simpler approach means there is less computation required to update the position of the content in each frame. This enables high frame rate experiences even on lower-end devices or in WebAR.
Sounds awesome! Any downsides?
Instant Tracking works best when the anchor point in the world used to initially place the content remains in the camera view throughout the experience - it’s great for things like placing characters in a photo feature and simple visualization of 3D objects for example.
Experiences that involve the user looking away from the initial placement point are not well-suited to our initial implementation of Instant Tracking. For example we would not recommend using Instant Tracking to build an experience where a user explores a virtual environment by freely walking around a large area.
Instant Tracking also aims to maintain relative scale and position as the user moves, but does not know the absolute scale of the world. Using “relative scale” is a key reason Instant Tracking can offer instant and stable placement before any device motion, but means it is not suitable for experiences where correct real-world scale is a critical part of the experience.
When can I use it?
Intrepid ZapWorks users can experiment with an open beta of our Instant Tracking implementation for WebAR right now! There are more technical details and a sample ZapWorks Studio project over on this thread. During the beta you’ll need to view any projects making use of Instant Tracking through our beta WebAR site at https://beta.zappar.app.
Over the coming weeks we’ll be refining and finalizing the first full release of the Instant Tracking API and implementation, and then rolling out support for Instant Tracking in both our Zappar native apps and our main https://web.zappar.com WebAR page.
If you’ve got a WebAR-only campaign ready to roll before then, we can deploy a Custom WebAR Site to support your launch with the current pre-release implementation. Get in touch if you want to talk about that option.
What’s the future roadmap?
Instant Tracking emerged from the Zappar R&D team’s work towards a full World Tracking solution for all of our supported platforms. As we continue working towards that goal we will also be folding in further improvements to Instant Tracking.
These improvements will address some of the use cases that are not well-supported by the initial Instant Tracking implementation - for example adding the ability to continue to update the content position when the original anchor point is no longer in view, and relaxing the requirement for placement on horizontal surfaces.
You won’t need to make any changes to your Instant Tracking content to benefit from these future updates, and you won’t even need to republish - they will just be improvements to the underlying implementation of the simple Instant Tracking placement API.
We’ll also be looking at how we can combine the instant placement benefit of Instant Tracking with the robust World Tracking provided by ARKit and ARCore on supported devices and platforms.
So you’ve got all that good stuff to look forward to in future, but for now please have a play with Instant Tracking and let us know what you think!