Moondust: Creating Content for Knuckles EV2

Developer notes and lessons learned from creating content for Knuckles EV2.

Introduction

As of now, the full source files for Moondust are available on github.[github.com] All original assets are available to explore, pick apart, and even use in your own games!

Knuckles EV2 is the cutting edge of VR input. Soon, hundreds of developers will get their hands on developer kits and begin experimenting, but for the last few months that task has been left up to us. During the development of Knuckles EV2, we created a demo we call Moondust - both as a way to validate design decisions, and as a sandbox to test potential game mechanics with the new inputs.

Knuckles EV2 developers, you can add the Moondust Demo to your Library here. Look for SteamVR Knuckles Tech Demos in Library > Games.

I'd like to start out by saying that VR is already a step above regular video games: it incorporates the physical motions of the player in addition to their button presses. With Knuckles EV2, it’s yet another step in this direction, with a constant stream of high-fidelity hand pose and force data coming in that has to be interpreted. While it's great to get all this data, it can be really complicated to deal with as a developer. With that in mind, we've already done a lot of the heavy lifting for you with the grab mode in the new SteamVR Input System. Then with the SteamVR Unity Plugin we've given you an interaction system that puts this mode to use and helps with throwing as well.

Moondust takes place in the Portal universe. With moon-based conversion gel stores running dangerously low, Aperture looks to the stars for a new source of the valuable portal conductor. Take control of a giant robot on the moon, spearheading the Aperture Lunar Resources Initiative. This demo utilizes all of the new inputs of Knuckles EV2 and showcases a few new game mechanics enabled by the new controller.

• Crush moon rocks to a fine powder with your powerful hands, taking advantage of the high-fidelity force sensors on Knuckles EV2
  
• Practice driving a moon buggy, utilizing the joystick and the rest of the new inputs of Knuckles EV2
  
• Assemble a space station in orbit, with fine interactions driven by all 10 fully tracked fingers
  
• Enjoy some target practice in the throwing range, with the most natural throwing VR can offer

Let's get started and look at all the sections of this demo (and our experiences developing them) in detail.

** Please note that this content was developed for Knuckles EV2 and will not work with other controllers. **

Rock Crushing

In this scene, you are a vital part of the conversion gel production pipeline. Moon rocks harvested from underground arriving on a conveyor belt must be crushed into the highest quality moondust, then deposited in the Conversion Gel Converter. This scene demonstrates finger tracking, pickup, and crushing objects using a combination of cap sense and force sensors.



One of the first things I wanted to make when I heard Knuckles would have force sensors was a rock crushing scene. Ideally as a gigantic robot. Initially, we forced the player to squeeze the rock for an extended period of time to crush it. It turns out this is just fatiguing and makes it feel like the force sensor is unresponsive, so we changed it to a instantaneous threshold for crushing. The force sensors have an incredible range and very little latency, so fully taking advantage of this does make more sense.

Fine tuning the threshold was important as well: too low and some people accidentally crushed the rocks, too high and people couldn't squeeze hard enough. Or, in one case, someone actually broke a prototype controller by squeezing with all their strength while playtesting the scene. Whoops.

With instant response, the right threshold, visual and haptic feedback (glowing and shaking), there's a satisfying physicality to crushing these rocks in VR.

BuggyBuddy™

BuggyBuddy is an experimental line of lunar rovers currently in the R&D phase. Take them out for a test drive in this lunar arena. Just don't leave the test site, as a Material Emancipation Grill has been installed around the perimeter. This scene demonstrates natural tool use (pick up and hold to "equip"), as well as thumbstick, track button, and face button functionality.



The buggy scene was a nightmare to create. We went through at least 20 control schemes, mapping and remapping the buttons. Every playtester liked something different. We started out with the joystick controlling the global direction of the buggy, top down style. About 50% of people loved this, and 50% hated it and couldn't steer to save their life. Eventually (2 days ago), we settled on what we have now. The trigger, which used to be the brake, is now the throttle. The joystick, once steering in absolutes, is now relative to the buggy. We began with something imitating video games and established third person controls, but we ended up with something identical to most real-world RC car control schemes. Food for thought?



But hey, with the new binding system, players can rebind inputs in your games to anything they want. That's a big step forward.

You can also teleport by clicking the touchpad (as long as you're not holding anything). With the physical click gone, this is done with the force sensors and a strategic haptic tap to imitate a button. This works quite well, and definitely holds up to the old touchpad clicks on Vive controllers.

This scene was created to utilize all of the more "traditional" inputs adorning the head of Knuckles EV2. There are a few to choose from, but you shouldn't feel pressured to use them all. Something to try could be making virtual controllers that only have the inputs you're using, such as the buggy remotes with their missing touchpad. Despite the focus on traditional input, finger tracking is still utilized to provide an incredibly natural equip/drop mechanic for the buggy remotes. You just have to grab on to the controller, and hold it as long as you want before dropping it. It's not fatiguing as first generation controller solutions of constantly holding a grip button or pulling a trigger. With high fidelity cap sense and force sensors this feels perfectly natural.

Space Station Construction

Head up to orbit to assemble a space station from modular pieces. This scene demonstrates fine motor interactions with Knuckles EV2. Items can be picked up with a full grip or a pinch grip (touch trackpad and touch trigger).


This scene is an experiment in fine interaction, with small objects and precise placement. The Knuckles pinch grip is best shown off here, though regular grip grabbing can be used too. Constructing this scene was all about fine-tuning the interaction code and stabilizing physics.

It's not groundbreaking gameplay, but Knuckles EV2 is a new step in natural and precise VR interaction and this scene shows that off.

Throwing Range

We've included an infinite pile of grenades to toss in the throwing range, as well as squishy turret toys. The grenades can be primed with a squeeze (using the force sensors), then thrown at your leisure any time within the next four seconds. The squishy turrets show the fidelity of the analog force values in both the track button and in the grip, and are just extremely fun. They can be thrown at targets as well, of course.


Throwing is an eternal problem in VR. Everything's moving fast, there are tons of variables, and tiny differences in release times can result in wild inaccuracies. Now, with Knuckles EV2, we introduce even more variables! The capsense for finger poses and force sense for grip strength can be utilized to make throwing more accurate than ever, but they also increase the complexity of the release calculations. Fortunately, we did all the work for you.

Using a dynamic combination of capacitive and force sensors, we calculate accurate throw timing without any buttons having to be touched. Looking backwards and forwards in time, we find a reasonable direction and magnitude for the throw. If you're interested in the details, we talk more about our throwing solution in the article here. Until we have exact joint tracking and can emulate the feel of objects in game throwing will never be perfect, but we think this is a significant improvement over previous algorithms.


The throwing range is a benchmark for throwing mechanics in VR. As we improve our algorithms we will continue to share with you, and I hope you share with us any developments you make.

Conclusion

Moondust was created on a limited schedule, and is not perfect. It is, however, the best current demonstration of all the features of Knuckles EV2. Now that it's out there in the world, it's your turn. Take the ideas presented here and expand on them. Prove us right, prove us wrong, build on our ideas and learn from our mistakes. You are at the forefront of VR input, so get out there and make some great games.

Things that we learned:

• + Hand presence is nice!
  
• + More buttons and traditional inputs are nice too. The more the merrier in my opinion.
  
• + Interactions can feel really good if done right
  
• - Interactions can feel really bad if not done right.

Developing for Knuckles would probably be a lot more challenging if not for the updates to SteamVR Input. Most of the hard work and algorithms we developed while building Moondust has been built into the input system for all other VR developers to take advantage of. Otherwise we might end up with each developer reinventing the wheel (and have a bunch of inconsistent wheels of questionable integrity).

Finally, remember to think outside the box. This is a step forward from previous-gen VR input, and a huge step from traditional videogame input. Experimentation and risks are the only way forward.

We're excited to see what you make!

[kratijun]

can i use the controler without vr headet?

[Nicholas]

@Zulubo, is 2019 the year of the VR games?

[Zulubo]

Hey Klow, they were replaced so we could release the source without copyright. Now devs can reuse (almost) all assets freely.

[dvdimagen]

Hi. I am sorry for my ignorance but I am trying this Moondust project with the HTC Vive and its first Controllers( not the Knuckles ones) in Unity and these controllers do not appear in any example, for it, Could anybody help me to fix it? Thanks

[Klow]

Is there a reason why the Aperture and Black Mesa logos on the rocket and buggies have been replaced by generic moon logos in the source files?

[Nicholas]

Where can I download?

[spatial_hair]

Link broken. Direct download?

[JamesButlin]

I personally found the throwing more cumbersome than traditional trigger throwing, I guess it might need either some getting used to or a simple slider for throwing detection. I also found it incredibly hard to squeeze the rocks/prime grenades on one controller but not the other, guessing that's a hardware problem though of course.

[nccvoyager]

@Timerider @Feralinx
You don't remember what happened last time?
Earth losing a whole bunch of atmosphere through the relatively small hole?

[Mr. Rawbawto]

helo pls send these to me I am not a dveloper but I want it also send a vive tank u