High quality head tracking for £5.50 or $6 [ArUco+OpenTrack]

How to create a very smooth headtracker only using a webcam and a piece of paper

Demonstration

Video showing in-game tracking:
https://www.youtube.com/watch?v=tDTN6lcLi80

Video showing Opentrack tracking a pink toy:
https://www.youtube.com/watch?v=hlLjPzOAhn0

What you need to buy/make

Ps3 eye: $6 or £5.50 (The ps3 eye is ideal as it can record a good resolution at 75 fps. You can buy the camera new on amazon. Any webcam should work, but the Ps3 eye gives very smooth tracking)

[Optional?]CL eye driver: $3 or £2.30 (You need a driver to use the Ps3 eye on a PC. I have heard that there is other free alternatives, so feel free to try those)

[Optional] Ps3 eye clip: About $2 or £1.50 (To be honest you can just use blue tack, like me :P )

ArUco paper marker: This is what your camera will track. I will show you how to create one in the next section

Total: £5.50 to £9.30 or $6 to $11

Making the ArUco paper marker

Generator website[terpconnect.umd.edu]
Use this website to generate an ArUco marker. Set Marker Id to a random number between 0 and 1023.
For Marker size and marker padding, I recommend 75mm and 20mm respectively. A large padding is important because otherwise the camera can lose line of sight to the part of the marker, resulting in unexpected pauses whilst tracking.

Once you have generated a marker, right click on the web page and select print. Do not save/print the image itself as that will print at an incorrect size.

Cut out the marker (including the padding) and use either sticky tape or glue to attatch it to something that won't flop – cardboard works well.

Now you need to find a way to attatch the marker to your head. I slit my marker into the back of a cap, which holds it in place nicely. You could also try using a headband, headlamp etc.

Setting up the software

CL eye driver installation

Run the CL eye driver or alternative installer
You can use the included application to preview the camera

Setting up Opentrack

Opentrack is the head tracking software we will use. Download it for free here: https://github.com/opentrack/opentrack/releases

Run the installer and launch Opentrack

Step 1: Select "aruco – paper marker tracker" as your input. Click the hammer icon on the side to configure it.
Set the following:

• Camera name to PS3Eye
  
• Frames per second to 75
  
• Resolution to 640x480

Step 2: Select "freetrack 2.0 Enhanced" as your output

Step 3: Select "EWMA" as your filter and click the hammer.
These settings work the best for me, but feel free to tweak them as you wish:

• Min = 8.43%
  
• Max = 32.19%
  
• Curve = 23.23%

Step 4: Open Options>Output and map Roll, X, Y, Z to Disabled. You can turn them on if you would like, but I find them best turned off.

Step 5: Open Options>Shortcuts and bind the Center and Toggle functions to the same key. I recommend PauseBreak or ScrollLock(you can use a joystick button if you want). When in-game, press this button to pause headtracking – when unpausing, make sure your head is in a centred position.

Step 6: It is important to mount your Ps3 eye to the top of your monitor – use the clip or bluetack, etc. Tilt the camera so it can see ALL of the paper marker, even when you are moving your head

Start tracking!

Press the start button to start tracking your paper marker. Opentrack should show a red square around your paper marker. Move your head and the square will move accordingly. Elite Dangerous(or another game) will start mirroring your head movements immediately. You may notice moving your head moves your character's head too much/little in game. If so, you need to configure your curves/mappings. If you find the character's movement is choppy/un-smooth, read the section "Making tracking smoother" at the end of the guide.

Configuring mapping/curves

Open the "Mapping" tab in opentrack.
For your Yaw tab, I recommend a straight line from x=0,y=0° to x=90°,y=180°
For your Yaw tab, I recommend a straight line from x=0,y=0° to x=80°,y=180°
Adjust the curves/mappings as you like

Making tracking smoother

On your first run, headtracking will probably not be smooth as in the demonstration video(I will say it again: this method of tracking can be very smooth!). Here are several ways to reduce choppiness:

Toggle Calibration: Stop tracking(press the stop button) and open up the aruco configuration again. Enable "Toggle calibration" and start tracking again. Stop and start the tracking a few more times for good measure. It should be noticably smoother now.

Tilt your ArUco: In your cap/hat/strap, try tilting/pitching your paper marker towards the ceiling. This can make a surprising difference.

FPS: If you cannot run the game at a steady 60 frames per second or more, try reducing your graphics settings and/or upgrade your PC

Give the filter time: Give Opentrack up to 60 seconds after each start to smoothen out .

Decrease the FOV on your Ps3eye: On the Ps3 eye itself, there is a switch/knob on the front - turn it to the left to record at a lower Field Of View.

Turn on the light: Even though ArUco tracking works well in most lighting, being in complete darkness is probably not a good idea.

Position the camera: Open opentrack and try moving your head. Make sure the camera has a red line around the marker at all times - if it doesn't, you need to tilt your camera so it does

Future runs

The next time you want to play Elite Dangerous, launch opentrack and click start(with your paper marker on, of course). You can do this whilst the game is already running

Important links

Marker generator[terpconnect.umd.edu]
Opentrack aruco documentation[github.com]
Opentrack wiki[github.com]
Ps3 eye (UK)[amzn.eu]
Ps3 eye (USA) [a.co]
CL eye driver[codelaboratories.com]

My user's experience

From a user's perspective, there is very low(barely noticable) latency and the tracking overall feels incredibly smooth. This headtracking solution is not only great value because it's cheap: it's great value because it is excellent quality.




Thanks for reading :D
As always, if you liked, don't forget to like and favourite!

[The Rat King]

in what universe can you get a ps eye for $6

[Zemar]

can someone explain to me why head is not working left or right in ets2? i did everything right bu i just cant make good preset or configuration, anyone help pls

[RC-3141]

Link to the generator is broken now, this seems to work though https://chev.me/arucogen/

[6FINGERDEATHPUNCH]

does this work on linux?

[warrat106]

"FPS: If you cannot run the game at a steady 60 frames per second or more, try reducing your graphics settings and/or upgrade your PC"

if i had money to do this, why would i bother with the rest of this guide xD

[RC-3141]

It works! This is pretty neat, but needs just a little bit of tinkering to apply to your camera setup if you didn't drop $5 on a new one.

[webpoet]

For anyone looking at this now and finding the link to the marker generator not working (like me) just use this one: https://chev.me/arucogen/
On the page set the dictionary to "Original ArUco". At least that is what the OpenTracker documentation advises to do.

[ciggy rat]

thank you man

[IceBox]

I have TrackIR but had to write a comment. This is one of the most useful, money saving posts on Steam, I have ever seen. Of course I gave a Thumbs up and also thank you, for your effort and detailed guide.

[Micael Jarniac]

Interesting fact, this can also be used with pretty much any camera, as well as using a smartphone as a camera. Sure, there's probably some added latency, and frame rate might not be as good, but so far I've managed to get it to work with DroidCam and an ArUco target I literally drew on a piece of paper.