Get Started with Curved Tracking in Universal AR

[Francesca]

Hi everybody!

We’re so pleased to announce the release of our newest tracking type, Curved Tracking.

Curved surface tracking expands upon Zappar’s existing image tracking (which was previously only designed for flat surfaces), opening up a world of new opportunities. This is especially true when it comes to connected packaging, allowing you to create interactive, branded experiences on a range of curved surfaces including cans, cups or bottles.

You can view a quick showcase of curved tracking in action below :

This post details how to get started with the beta version of curved tracking in a Universal AR SDK project.

A quick note before you get started with this tracking type - this functionality is currently in beta. This means that we are working on it to make improvements. The beta version is not recommended for live projects. Please consider this carefully; and rigorously test your projects across multiple devices and browsers if you do decide to use this version.

---

What makes a good curved target image?

• Lots of unique high-contrast texture
• Good coverage over the object

• Repetitive patterns
• Largely see-through or translucent labels
• Fancy finished, e.g very shiny or colour changing

Training a curved image using the ZapWorks CLI

To begin with curved image tracking, you’ll need to install and train up a curved target with the ZapWorks CLI.

Currently, this is the only way to train up a curved target. We’ll be bringing updates to our Universal AR for Unity SDK & ZapWorks Studio soon to enable curved image training directly in each tool.

To install and use the ZapWorks CLI, first make sure you have Node.js installed, then run the following command in a terminal on your computer:

npm i -g @zappar/zapworks-cli@beta

Once installed, you can use the train function to create a .zpt from your PNG / JPG image, as seen in the example below:

zapworks train YOURFILENAME.jpg/png --circumference=..cm --input-height=..cm

In order for the image training algorithm to understand the curvature of your object, please specify exactly one of:

–radius=Xcm
–circumference=Xcm

For the algorithm to understand the real-world size of your target, specify exactly one of:

–input-height=Xcm
–input-width=Xcm
–dpi=X

When one of input-height, input-width or dpi are specified during training, the coordinate system of your anchor will be in units of meters. If no physical size is provided, the coordinate system will be +1 Y at the top of the image and -1 Y at the bottom of the image as usual.

(Image above showing example command usage)

Using the .zpt file in your Universal AR SDK Project

Once you’ve trained your target image, you’ll have a .zpt file in the chosen directory. To use the .zpt file in your Universal AR SDK project, simply follow the usual image tracking implementation according to your chosen SDK.

You will need to use at least version 2.0.0 of your chosen SDK to use the new Curved Tracking type. To find out how to do so, click here.

Click the drop downs below to learn how to include your curved target image in your Universal AR SDK project.

Implementing Image Tracking: Universal AR Web-Based SDKS

Universal AR SDK for A-Frame
Universal AR SDK for Three.js
Universal AR SDK for Babylon.js
Universal AR SDK for React Three.js

Implementing Image Tracking: Universal AR for Unity & PlayCanvas

We’re working really hard to develop a curved preview image for the Unity editor and will update this post when it’s ready!

Universal AR SDK for Unity
Universal AR SDK for PlayCanvas

(Image above showing package.json for beta branch usage in a Universal AR for Three.js project)

(Image above showing example image tracker usage with Universal AR for Three.js)

Previewing

To preview your experience, simply test as you ordinarily would by following the Universal AR Testing guide, or the README instructions if you are cloning our Bootstrap Projects or Example Projects.

Publishing

To publish your experience, do so as you ordinarily would by following the Universal AR Hosting guide.

---

We hope you have fun experimenting and we’re delighted to see what you create!

Team ZapWorks

[lucalilli]

Hi, I love this new feature…
In Studio may i know which is the best way to stream a video on a curved fbx?
I have modified a Videplayer and succeeded in streaming a video in the curved surface but the video is wrapping the surface the wrong way. Which is the method to rotate the texture on the surface? Maybe there’s a best way to achieve this?
regards
Luca

[r.liechti]

Hi, in UAR or in the studio? i have a running version with three.js, something like this

video = document.getElementById( htmlVideoID ) // get your html 5 video
var videoTexture = new THREE.VideoTexture( video );
videoTexture.minFilter = THREE.LinearFilter;
videoTexture.format = THREE.RGBFormat;

var movieMaterial = new THREE.MeshBasicMaterial( map: videoTexture );

var geometry = new THREE.PlaneBufferGeometry( 1, 1);
var videoMesh = new THREE.Mesh( geometry, movieMaterial );
videoMesh.scale.setScalar(0.0075);

---

function if video ended:

HTML Audio/Video DOM Reference

htmlVideoElement.onended = function() {
videoEnded();

};

---

HTML video element:

on your html video element, use: webkit-playsinline="" playsinline="" and display none, remove the “loop”

[lucalilli]

Hi i’m working in a Studio project… thank you so much
Luca

[lucalilli]

I think it has to do with the UV of the Fbx…
Luca

[George]

Hi Luca,

This thread is specifically about curved tracking in Universal AR. Could you please pop your Studio question in this thread and I’ll take a look:

@George

We’re happy to announce the release of our newest tracking type, Curved Tracking.

George

[stevesanerd]

Hay @George,

How’s that Curved tracking coming for Unity??

Steve

[r.liechti]

hello everyone, is this feature also available for the standalone bootstrap version?

i use (also tested with v2.0.0-beta.8)

    <script src="https://unpkg.com/three@0.137.0/build/three.min.js"></script>
    <script src="https://libs.zappar.com/zappar-threejs/2.0.0-beta.6/zappar-threejs.js"></script>

after load the chrome dev console show:

zappar-threejs.js:1 Zappar for ThreeJS v2.0.0-beta.6
zappar-threejs.js:1 Zappar JS v2.0.0-beta.3
zappar-threejs.js:1 Zappar CV v2.0.0-beta.3

for the image training i use:
zapworks train IMAGE.png --cirumference=182mm --input-height=78mm

“name”: “@zappar/zapworks-cli”,
“version”: “2.0.0-beta.1”,

"@zappar/imagetraining": "^3.0.0-beta.5",

for the Geometry:
new THREE.CylinderGeometry( 0.65, 0.65, 1.78, 32, 2, true);
But the Cylinder and other elements arenot in the correct size.

Is there a function to check if the .zpt file is detected as curved target and the three units are in meters?

[stevesanerd]

Hay @George,

It’s been 3 months how’s that Unity Curved tracking coming???
I have have some venders that are waiting on this!!!

Steve

[George]

Hi @stevesanerd!

Curved tracking in Unity is just around the corner now. Sorry for the delay, our engineers have been working tirelessly to get an update out.

It’s slightly more complex compared to the other SDKs as Unity can deploy to native apps as well as WebGL and we want to support deployment to all platforms from the start.

I’ll be sure to update / post here as soon as the new Unity build including curved tracking is deployed. It shouldn’t be too much longer!

George

[stevesanerd]

Thanks @George
Can’t wait!!

Steve

[rendermouse]

We are also eagerly awaiting Curved tracking in Unity.

Andy

[stevesanerd]

a month later …

Steve

[ndl0901]

Hi, can i do this on zappar studio?

[stevesanerd]

yes, and just about all UARs
Just not Unity

Steve

[ndl0901]

How to do it? i can only make a flat surface image tracking

[ndl0901]

nvm i found this post Get Started with Curved Tracking in ZapWorks Studio said that i need to train the image separately

[stevesanerd]

Yes you have to use the ZapWorks CLI to make the tracking image.

Steve

[lc.contreras]

Hello, do you know when is going to be Curved Tracking available for Unity?

Luis