NewtonVR: Physics-based interaction system for Unity

[Snickersnacks]

Hey all,

Keith, Nick and I over at Tomorrow Today Labs recently released our in-house interaction system. We built it as a foundational part of the unannounced Vive game we're currently making. Our goal is to provide and maintain this system as a free resource to VR devs, game jams, etc.

Looking for feedback from the dev community, or if you end up using it, feel free to let us know what's working or what changes you end up making. Especially if something is feeling just plain wrong. :)

** Here's the main website: http://www.newtonvr.com **

Here's a write-up Keith did about features & integration: http://vrinflux.com/newtonvr-a-free-physics-based-interaction-system-for-unity-and-the-htc-vive/

Nick wrote a quick article about the challenges we were trying to overcome with the design of this system: http://www.vrinflux.com/newton-vr-physics-based-interaction-on-the-vive/

And here's the Github link: https://github.com/TomorrowTodayLabs/NewtonVR/

Thanks for checking it out!

[Mechabit]

Awesome resource, also nice write ups. Will save me a lot of effort XD

['Rito]

Thanks for building such a comprehensive set of interactions!

Curious as to a few things about this:

First: Why are you setting the rigidbody velocity instead of using Rigidbody.MovePostion() and Rigidbody.MoveRotation()?

It seems to me that the former method will always lag at least one FixedUpdate() behind the controller's actual position and rotation, which is noticeable even at small velocities and really quite far away from the correct position for big, quick motions.

Second: Is there a reason you haven't disabled gravity on held objects?

EDIT:
It seems haptic pulses get stuck if an object is taken from a controller by the other, or if an object is detached because of distance while the controller is inside another interactable object. The dropping controller immediately activates phys-controller mode, and the pulses remain until the next grip pressdown.

[Markwar]

Hi YM,

Thanks checking out system out and giving the feedback :)

First: By setting Velocity and AngularVelocity instead of MovePosition() and MoveRotation(), we save ourselves the need to calculate the velocities later when interaction with an object has ended, and say, the player is throwing the item. Trying to throw an item moved with MovePosition/Rotation() causes the item dropping straight down.

I definitelty agree the lag is noticeable. I'd like to try out a scaling multiplier, so that velocity increases more exponentially with PositionDelta.

Second: By modifying velocity directly, we're already ignoring Rigidbody mass, and relatedly, gravity. Turning off gravity at the start of an interation, and turning it back on at the end, really doesn't do much. I could imagine problems if acceleration due to gravity is applied by the engine AFTER we modify velocity but before the frame ends. We'll keep an eye out for those.

Haptic issue: Fixed :) You'll still notice haptic feedback if a ghost controller is inside a held object, which is what we intend. Thanks for pointing that out!

-Nick

[Markwar]

Addendum: Another reason we don't set position absolutely is because that doesn't take colliders into account, e.g. the player could pass an object they've picked up through a wall.

['Rito]

That's true, Rigidbody.position = blah doesn't; but Rigidbody.MovePosition( blah ) does.

I modified your code to use Rigidbody.MovePosition() and initially it seems like a flawless solution. But when compared side by side to updating the velocity, interactions with other rigidbodies is a TON better when updating velocity. MovePosition() made for very jerky interactions, if you picked up the bottom box in a stack with MovePosition(), the other boxes wouldn't move nicely but with the velocity method you could carry a stack.


You ask for things we change/build on, so here's what I've done so far:

Made a class that added a 'total turns' to NVRInteractableRotator so I not only know the current angle, but how far it's been turned in either direction.

I made another that added 'preferred angles' to NVRInteractableRotator so that I could specify that the dial had n sides and when released it should move to the nearest one

I made a spring pull version of NVRSlider that adds/removes a spring component based on hand interaction so it works like a bathroom cord light switch.

These were relatively simple additions and I'm happy for them to just inherit from yours, the one other change I did was to alter NVRInteractable.DropDistance to a smaller value. It'd be nice if you exposed this value either per-item, or put it in NVRPlayer so that it can be adjusted from the camera prefab.

Finally I've noticed the physical controller mode's rotation is waaaaayyyy off from what it should be. It seems to rotate around the puck, when the rotation I'm doing with my hand pivots around my wrist. I've not used this mode much yet so maybe I'll have more detailed comments later.

[railboy]

This looks really clean and it sounds like you've fixed a LOT of the problems I've been having with physics based stuff. Thanks for sharing - I'll be checking it out today!

[railboy]

We've just swapped out all of our Hand / Draggable scripts for NewtonVR's NVRHand / NVRInteractableItem in Felt Tip Circus and so far we love it. I'm uploading a new build.

We didn't have to change much - the game has several items that are frozen in place until you grab them, and some which remain frozen after you let them go, so this preserved that functionality:

public abstract class NVRInteractable : MonoBehaviour
{
	public bool AutoDisableKinematic = true;
	public bool AutoEnableKinematic = false;
	...

        public virtual void BeginInteraction(NVRHand hand)
        {
		if (AutoDisableKinematic) {
			Rigidbody.isKinematic = false;
		}
		AttachedHand = hand;
        }

        public virtual void EndInteraction()
        {
		AttachedHand = null;
		ClosestHeldPoint = Vector3.zero;
		if (AutoEnableKinematic) {
			Rigidbody.isKinematic = true;
		}
        }

}

Otherwise everything you did was a near-match to our techniques, only cleaner - throwing objects feels a lot more natural and balancing objects isn't prone to wacky behavior when the controllers jitter for a frame or two.

Really nice work! (I'll probably have more feedback after we've played with it for more than two hours.)

[Keith]

Glad you guys are enjoying it!

@YM,
Those sound like cool changes. If you want to post code or submit a github pull request we could integrate them into the example scene.
We'll make that change to dropdistance and make it a public member. So you can set it on an instance level.
The rotation is at a kinda weird point. It rotates from the pivot point of the controller, which is the tracking hat. I'm not sure I have a good solution for how to change this cleanly. If I change the pivot point of the physical controller it won't match the pivot point of the ghost. I kinda like having the hand.position correspond to the tip of the controller so I don't really want to change both.

@railboy
I hadn't thought of using kinematic items. We've got one in the test scene but it's real annoying :P I can imagine scenarios where it makes sense though so I'll go ahead and get that added in.

#edit: added

[Keith]

Just made a somewhat critical change so external velocity changes are accounted for now. Previously they were overwritten as we were just setting RigidBody.velocity with no regard to it's previous value. Now we try and move towards our target velocity (and angular velocity). Change is on github.

Also, with some modifications to the magic numbers this project totally works at normal unity scale. The demo project is scaled up by a factor of 10 but that's not required, physx just seems to work better at that scale.

#edit: A good example of this is to pick up the rocket and then pull the lever. The rocket will fly out of your hand. Whereas previously I had to set the force added to the rocket to an absurdly high number so it'd launch.

[Keith]

Added NVRAttachJoint and NVRAttachPoint. This lets you kind of slot items into place. I added an example in the scene so you can slot the green cubes in and they'll pull themselves into place. Now that I think about it the surrounding walls are kind of unnecessary, I just put them in as a visual aide of where to put the piece.

[shole]

I recently used NewtonVR in our GGJ16 entry "Good Morning".
Info & download: http://globalgamejam.org/2016/games/good-morning-1
Full playthrough video here https://www.youtube.com/watch?v=Djv3qlj4itI
And thought I'd offer some thoughts on my experience with NewtonVR...
-It became clear early on in testing that most people expect the controller to always be physical to hit things, so I switched to always in the physical state - except when grabbing stuff because that caused things to explode.
-The shape of the dk1 track hat is really ugly and doesn't really feel like it describes anything, so I hid it and switched it to a sphere to more indicate its purpose as a grab point.
-Some players still expected that they could grab things with the side, with the location where the side grip buttons are. I didn't have time to do this, but it would make sense to have the grab trigger be a capsule that was a few cm off the entire upper half of the controller surface so things could be intuitively grabbed regardless of position.
-There's a bug with the target orientation in physical form when pointing "south" and the controller does a full 360.
-Rotation in physical mode is also really slow. I had to tweak the magic value a bit to not be so noticable.
That said, NewtonVR is clearly a starting point I will be using for any VR physical interactions I will do in the future.
Please keep developing this, and thanks for making it available.

[Keith]

Thanks for the feedback shole. I fixed that bug with the rotation just now. I can't seem to figure out why the rotation won't speed up though. Will keep looking into it. The dk2 track hat is fancier looking but still not descriptive in a "you can pick this up kind of way". Probably not going to fix that.

I also scaled everything back to 1. I'm thinking the extra precision gained by the scale wasn't worth the issues with gravity, shadows, draw distance, etc. Not sold on that 100% yet though.

[Keith]

Added support for the Vive Pre.

Interesting note, you can use both the dk1 controllers and the pre controllers at the same time. I connected the old receiver to the back of the headset with the included cable to get better tracking.

[Keith]

Got the old dk1 controllers and the new pre controllers working with NewtonVR. Turns out they both work at the same time: http://gfycat.com/BrightMildIndianspinyloach

[Eiseno]

did anyone have crash issue after stop game with latest unity patch ?