Get Started with Curved Tracking in Universal AR

Hi everybody!

We’re so pleased to announce the release of our newest tracking type, Curved Tracking.

Curved surface tracking expands upon Zappar’s existing image tracking (which was previously only designed for flat surfaces), opening up a world of new opportunities. This is especially true when it comes to connected packaging, allowing you to create interactive, branded experiences on a range of curved surfaces including cans, cups or bottles.

You can view a quick showcase of curved tracking in action below :champagne::

This post details how to get started with the beta version of curved tracking in a Universal AR SDK project.


:sparkle: What makes a good curved target image?

:+1:

  • Lots of unique high-contrast texture
  • Good coverage over the object

:-1:

  • Repetitive patterns
  • Largely see-through or translucent labels
  • Fancy finished, e.g very shiny or colour changing

:sparkle: Training a curved image using the ZapWorks CLI

To begin with curved image tracking, you’ll need to install and train up a curved target with the ZapWorks CLI.

To install and use the ZapWorks CLI, first make sure you have Node.js installed, then run the following command in a terminal on your computer:

npm i -g @zappar/zapworks-cli@beta

Once installed, you can use the train function to create a .zpt from your PNG / JPG image, as seen in the example below:

zapworks train YOURFILENAME.jpg/png --circumference=..cm --input-height=..cm

In order for the image training algorithm to understand the curvature of your object, please specify exactly one of:

–radius=Xcm
–circumference=Xcm

For the algorithm to understand the real-world size of your target, specify exactly one of:

–input-height=Xcm
–input-width=Xcm
–dpi=X

train-curved-image-cli
(Image above showing example command usage)

:sparkle: Using the .zpt file in your Universal AR SDK Project

Once you’ve trained your target image, you’ll have a .zpt file in the chosen directory. To use the .zpt file in your Universal AR SDK project, simply follow the usual image tracking implementation according to your chosen SDK.

Click the drop downs below to learn how to include your curved target image in your Universal AR SDK project.

Implementing Image Tracking: Universal AR Web-Based SDKS

Universal AR SDK for A-Frame
Universal AR SDK for Three.js
Universal AR SDK for Babylon.js
Universal AR SDK for React Three.js

Implementing Image Tracking: Universal AR for Unity & PlayCanvas

Universal AR SDK for Unity
Universal AR SDK for PlayCanvas


(Image above showing package.json for beta branch usage in a Universal AR for Three.js project)

implement-curved-image
(Image above showing example image tracker usage with Universal AR for Three.js)

:sparkle: Previewing

To preview your experience, simply test as you ordinarily would by following the Universal AR Testing guide, or the README instructions if you are cloning our Bootstrap Projects or Example Projects.

:sparkle: Publishing

To publish your experience, do so as you ordinarily would by following the Universal AR Hosting guide.


We hope you have fun experimenting and we’re delighted to see what you create!

Team ZapWorks :zap:

3 Likes

Hi, I love this new feature…
In Studio may i know which is the best way to stream a video on a curved fbx?
I have modified a Videplayer and succeeded in streaming a video in the curved surface but the video is wrapping the surface the wrong way. Which is the method to rotate the texture on the surface? Maybe there’s a best way to achieve this?
regards
Luca

Hi, in UAR or in the studio? i have a running version with three.js, something like this

video = document.getElementById( htmlVideoID ) // get your html 5 video
var videoTexture = new THREE.VideoTexture( video );
videoTexture.minFilter = THREE.LinearFilter;
videoTexture.format = THREE.RGBFormat;

var movieMaterial = new THREE.MeshBasicMaterial( map: videoTexture );

var geometry = new THREE.PlaneBufferGeometry( 1, 1);
var videoMesh = new THREE.Mesh( geometry, movieMaterial );
videoMesh.scale.setScalar(0.0075);


function if video ended:

htmlVideoElement.onended = function() {
videoEnded();

};


HTML video element:

on your html video element, use: webkit-playsinline="" playsinline="" and display none, remove the “loop”

Hi i’m working in a Studio project… thank you so much
Luca

I think it has to do with the UV of the Fbx…
Luca

Hi Luca,

This thread is specifically about curved tracking in Universal AR. Could you please pop your Studio question in this thread and I’ll take a look:

George

1 Like

Hay @George,

How’s that Curved tracking coming for Unity??

Steve

1 Like

hello everyone, is this feature also available for the standalone bootstrap version?

i use (also tested with v2.0.0-beta.8)

    <script src="https://unpkg.com/three@0.137.0/build/three.min.js"></script>
    <script src="https://libs.zappar.com/zappar-threejs/2.0.0-beta.6/zappar-threejs.js"></script>

after load the chrome dev console show:

zappar-threejs.js:1 Zappar for ThreeJS v2.0.0-beta.6
zappar-threejs.js:1 Zappar JS v2.0.0-beta.3
zappar-threejs.js:1 Zappar CV v2.0.0-beta.3

for the image training i use:
zapworks train IMAGE.png --cirumference=182mm --input-height=78mm

“name”: “@zappar/zapworks-cli”,
“version”: “2.0.0-beta.1”,

"@zappar/imagetraining": "^3.0.0-beta.5",

for the Geometry:
new THREE.CylinderGeometry( 0.65, 0.65, 1.78, 32, 2, true);
But the Cylinder and other elements arenot in the correct size.

Is there a function to check if the .zpt file is detected as curved target and the three units are in meters?

Hay @George,

It’s been 3 months how’s that Unity Curved tracking coming???
I have have some venders that are waiting on this!!!

Steve

1 Like

Hi @stevesanerd!

Curved tracking in Unity is just around the corner now. Sorry for the delay, our engineers have been working tirelessly to get an update out.

It’s slightly more complex compared to the other SDKs as Unity can deploy to native apps as well as WebGL and we want to support deployment to all platforms from the start.

I’ll be sure to update / post here as soon as the new Unity build including curved tracking is deployed. It shouldn’t be too much longer!

George

1 Like

Thanks @George
Can’t wait!!

Steve

1 Like