Use Chromakey

[camus]

How does this setting work? For example, if the chromakey color is green and I'm playing a game with a green skybox, will a camera box show through just the skybox? Or, would it be anything that appears green in the HMD camera(s) shows through to the game?

[Ray]

Originally posted by camus:

Or, would it be anything that appears green in the HMD camera(s) shows through to the game?

Yes, anything that is the right shade of green in real life, and is captured by the camera(s), will be transparent in VR and show the game world.
(It's in development and currently only works on Vive.)

[camus]

Thanks, that makes sense. I'm using an HP Reverb G2 so I wasn't sure what it would do with the grayscale video.

Any plans to do virtual chromakey like that example though?

[Ray]

Originally posted by camus:

Any plans to do virtual chromakey like that example though?

Which example are our referring to?

[camus]

Originally posted by Ray:

Originally posted by camus:

Any plans to do virtual chromakey like that example though?

Which example are our referring to?

Playing a game with a green skybox or background that you could see camera boxes on. Basically something like an AR game. Right now the closest thing is setting camera box max opacity to 20-30%, making it very large, then using attach on the HMD tracking.

Edit: My description is probably confusing because it seems like Reality Mixer already does this but what I mean is, rather than using camera boxes to show real objects in the game world I want to show game objects in the real world.

[Ray]

OK, I understand. Trouble is that the app cannot read the colors of the pixels that are produced by the game. So it has no way of detecting any green game pixels on which it would place the camera image.

Another issue is that the app uses 'overlays', which are always rendered on top of the game world. Even if a game object would be closer than a camera box, the box would cover and hide the game object. Indeed, if you give the box a max opacity of 30%, you can see both the box and the game object.

BTW, instead of the very large HMD-attached box, you could use a very large Cockpit Box. It has the camera images on the inside of the box. You'd have to tweak it by editing the .xml settings file, so that it shows the camera image on all six (inside) sides.
Or use several boxes that you place around you, on the edge of the play area. (But still issue #2 would apply.)

[camus]

That's a shame, I might try the cockpit box idea though. It seems like just attaching a big box to my face is enough though.

[cam2trappy]

Originally posted by Ray:

Originally posted by camus:

Or, would it be anything that appears green in the HMD camera(s) shows through to the game?

Yes, anything that is the right shade of green in real life, and is captured by the camera(s), will be transparent in VR and show the game world.
(It's in development and currently only works on Vive.)

Would this be coming to quest 3 with the passthrough in the near future?

[Ray]

Originally posted by cam2trappy:

Would this be coming to quest 3 with the passthrough in the near future?

Quest 3 is supported by the app, when used in combination with Virtual Desktop on the Quest. You need to select a solid color to use for the boxes (green, blue, white, black, or custom color) and set the same color in Virtual Desktop (on the Quest) as passthrough color.

For the chromakey/greenscreen feature the app needs to have access to the camera pixels, which Meta doesn't allow. The only party that could make chromakeying on the Quest is Meta itself.

[FlowGadget]

Originally posted by Ray:

Originally posted by cam2trappy:

Would this be coming to quest 3 with the passthrough in the near future?

Quest 3 is supported by the app, when used in combination with Virtual Desktop on the Quest. You need to select a solid color to use for the boxes (green, blue, white, black, or custom color) and set the same color in Virtual Desktop (on the Quest) as passthrough color.

For the chromakey/greenscreen feature the app needs to have access to the camera pixels, which Meta doesn't allow. The only party that could make chromakeying on the Quest is Meta itself.

I got the app for that chrmakey/greenscreen with quest 3, VD, and mixed reality but i couldnt find the setting the make the boxes a solid color. Im pretty sure i looked through the entire menu but couldn't find a color picker for the boxes. Is there something im missing?

[Ray]

Originally posted by FlowGadget:

I got the app for that chrmakey/greenscreen with quest 3, VD, and mixed reality but i couldnt find the setting the make the boxes a solid color. Im pretty sure i looked through the entire menu but couldn't find a color picker for the boxes. Is there something im missing?

Quest support was in beta, but is now supported in the normal, public branch as well. On the Camera Setup page there's a toggle for the Solid Color. You can select the color there as well.

(Chromakey/greenscreen on the Quest will only ever work if Meta does that itself, or if Meta allows direct access to the camera feed.)