Dev Diary - Procedural Sky

[ttcrose]

Hello LEGO® Worlds fans! I’m Emilio, Junior Render Programmer at Tt Games. I’ve only been here for nine months, but I’m working hard to raise the visual bar of the LEGO® Video Games. I’m going to talk about the procedural sky in LEGO® Worlds, but first I’d like to lay down some basics and discuss why doing skies in a dynamic environment is tricky. To begin, requirements you’d expect are:

• Believable (not necessarily realistic, because we could potentially want an alien sky)
  
• Dynamic (day and night)
  
• Influences lighting

Some different solutions have been proposed and used over the years.

Skybox

A skybox is a hemisphere textured on the inside to the color of the sky. This solution has been used at TT for many years and is standard industry practice, however it comes with downsides.

• You need big textures to cover the hemisphere and they need to be high quality. Think 2048x512 pixels big, which is about ~600kB of compressed video memory that’s not being used for something else.
  
  
• Not dynamic and no night time. There are workarounds like having several textures and blending between them, but it doesn’t always look nice and every additional texture is more memory.
  
  
• Fog coming from the sky has to be approximated using another method and tweaked so that it matches the sky. Object lighting is not affected either.
  
  
• Prone to z-fighting, the effect where polygons flicker when they’re far away in very big levels.

http://steamcommunity.com/sharedfiles/filedetails/?id=558852276&fileuploadsuccess=1

Realistic formula

Another approach uses very accurate formulae to simulate the scattering effects of the atmosphere (the kind of math you would study in a physics course at university). It is very precise and realistic, but expensive for some of the consoles (and lower end PC hardware).

http://steamcommunity.com/sharedfiles/filedetails/?id=558853088&fileuploadsuccess=1

 
Our approach

Daytime fitted formula

Some very smart people research hard problems and provide affordable solutions for videogames. In this case we have the Hosek-Wilkie model, which takes the formulae for the realistic approach and simplifies it. Not all possible sky conditions can be represented, and the model is limited in the fog information available to the game, but it’s a good solution. It looks like this:

F(θ,γ)=(1+Ae^(B/(cosθ+0.01)) )·(C+De^Eγ+Fcos^2 γ+G·χ(H,γ)+I·√cosθ)

Don’t worry if it looks intimidating, it was for me too! The important thing here is that the function depends on θ and γ, two angles which express the direction I’m looking at. Imagine yourself looking at the sky; there is a range of directions you can see. For example, you can say “I’m looking at 30 degrees up and 20 degrees right, and my field of vision is 90 degrees horizontally and 45 degrees vertically.” I need to compute the above formula for all possible angles in my vision, so for every pixel I have a direction, and for every direction, a pair of angles to compute the formula and get a color.
The letters A, B, C, D, E, F, G, H, and I are constants that change the sky conditions. Several variables like the position of the sun or the sky turbidity (a measure of how clear or murky the sky is) affect these variables, and that is the way we can get different skies.

http://steamcommunity.com/sharedfiles/filedetails/?id=558869046&fileuploadsuccess=1

http://steamcommunity.com/sharedfiles/filedetails/?id=558869674&fileuploadsuccess=1

Here we have a sunset with a clear sky and a sunset with a murky sky. Notice how the sun is in the same position but the aureola around it is very different.

Fog

Fog doesn’t come naturally from this model of the sky. If you look below at the left image, you’ll see the sky in action, but notice there is no fog added to the image, and it doesn’t look natural. To the right is the same image with fog. So where do we get it from?

http://steamcommunity.com/sharedfiles/filedetails/?id=558854335&fileuploadsuccess=1

http://steamcommunity.com/sharedfiles/filedetails/?id=558854596&fileuploadsuccess=1

Fog is added to distant objects because the light that comes to you gets scattered by the particles in the atmosphere. It is the same reason why the sky is blue, sunsets are red/orange and clouds are white. They’re all part of the same kind of interactions.
Notice how I just said fog is the same interaction as the sky. So why not reuse the sky formula as an approximation to get the fog color, and then add more or less of that color depending on how far the object is from us? That’s exactly what we do! It’s not totally correct physically but gives good results and looks nice. The image below contains a top-down texture that encodes this information. When objects are rendered into the screen, they look to this texture to get their fog value, based on their distance to the camera, height above the ground and direction. To the left, the fog color, and right, the fog density.


http://steamcommunity.com/sharedfiles/filedetails/?id=558854993&fileuploadsuccess=1

http://steamcommunity.com/sharedfiles/filedetails/?id=558855296&fileuploadsuccess=1

Nighttime

For nighttime we want a star field and other celestial bodies, such as planets, nebulae, constellations, etc. Currently in LEGO Worlds we don’t display any celestial bodies other than the sun and the moon, but be sure that they will eventually appear.
First we render the star field in the same pass as the sky. There is a parameter that depends on the angle of the sun and determines whether it’s night or day. The more the sun approaches the horizon, the more night it becomes, which means we start to blend in the stars and blend out the day sky. During the twilight transition, you can see how a part of the horizon still has some faint light coming from the sky, while the stars start to appear.

http://steamcommunity.com/sharedfiles/filedetails/?id=558855586&fileuploadsuccess=1



The randomness of the stars is also a trick. Remember how we said using a big texture for the sky takes up a lot of memory? What if we were to have a repeating tiled texture? That would give us many stars, but the pattern would be obvious. What if we had two textures? Then we could hide the pattern by giving them different repeating frequencies. What if we were to rotate the texture randomly to disguise it even further? Essentially, that’s what we do, using a texture with two channels (red and green) used as star distributions. Below is a sample texture. A small detail that was included in the system, as well as to have a few red and blue stars, just like real skies have on very clear night skies.

http://steamcommunity.com/sharedfiles/filedetails/?id=558855868&fileuploadsuccess=1

Next we render the celestial bodies, e.g. the sun and the moon, nebulae, constellations, planets and any other space object. These are effectively flat textured squares wrapped around a sphere. There is not much mystery to them, except perhaps for the fact that they are rendered on top of the sky! So how do we tell it to be behind the day sky and only appear at night? What we do is run the same sky formula we saw earlier and display a blend of both colors using the same variable that decides whether it’s day or night. Notice how the planet gets fogged as the atmosphere is in front of it.


http://steamcommunity.com/sharedfiles/filedetails/?id=558856177&fileuploadsuccess=1

http://steamcommunity.com/sharedfiles/filedetails/?id=558856354&fileuploadsuccess=1



Lighting

For the global lighting in LEGO Worlds we use the light coming from the sky. This gives us realistic and dynamic lighting for all the objects in the game. We compute the lighting every frame by sweeping the sky and getting its color in all possible directions. Then we encode it in something called Spherical Harmonics. What this does is “adds up” all possible lighting colors and compresses them into a series of numbers, which later the objects in the game can process to retrieve the lighting coming from the sky. There are only two sources of light in the game, the sky and the sun. Everything that is not lit by the sun would be totally black so the sky light is essential for the game to look correct.

Future Development

This system is, of course, a work in progress, and I’d like to give you some insight into what further developments we have planned for it.

• Sky variation: This feature is arguably the most interesting, since currently the sky is a realistic model, but we’d like to be able to modify it and give it properties it currently doesn’t have. This would naturally affect the fog, the lighting, the color of the sun, etc., giving very interesting results.
  
  
• Dynamic sky reflections: Dynamic reflections give a huge sense of depth, especially in a game where all the components are so shiny and reflective. Currently we use a static cubemap to get these reflections, and while part of it will remain static, the part that reflects the sky can be dynamic. This is especially visible at night where all the bricks reflect a lot of light when they shouldn’t.
  
  
• Celestial bodies: We’d like to add planets, nebulae, constellations, moons, etc., to the worlds (all, of course, built out of LEGO!). We’re planning to design a system that places them randomly so your island has a unique feel to it!
  
  
• Dynamic Exposure: This has to do with how we light the game during nighttime. Currently we pump up the lighting artificially so you can see things, but wouldn’t it be great if we didn’t have to do that and a dynamic system could take care of brightening up the scene if it feels that it’s too dark? That’s what we aim to do. This also would apply to the sky, we could brighten it up like real skies do, and the system would adapt and darken the scene a bit.
  
  
  Thanks!
  Emilio

[Fappy-さま]

ohhh :D

[ttcrose]

Hi folks,

Obviously I'm not Emilio, but he's written this up for you to understand some of the in's and out's of the new Sky. Let us know if you have any questions/comments, I'm pretty sure he'll be reading over them!

[☂]

awesome work!

[FrozenZeroLight]

nice explanation! :)

[Mysteli]

So whether it is a cube or hemisphere it is called a skybox? I like to call the hemispheres skybubble. =P

[HeroSci]

I'll have to read this more carefully, but does this mean you will be able to generate infinite diversity in skies? And will you also be able to have thing like planets flowting around that you would be able to go to in the near future, or something dynamic like that.

Also it looks like there are some mountaints in one of the pictures, will you be able to use the sky box to create terrain as far as the eye can see?

[SpaceSentinel]

This is a great Dev Diary!

[HeroSci]

Love that nebual and planet, I do something like that with my art.

[CommanderRed]

Is there a plan for the moon to ever have phases? I think it would add some interest to that key aspect of the night.

[DanBo]

Originally posted by CommanderRed:

Is there a plan for the moon to ever have phases? I think it would add some interest to that key aspect of the night.

what we really need is a rocket that allows us to fly to the moon

[HeroSci]

I think this will make a path to space gameplay, if we can simply render everything in space using this tech we could have solar systems and accurate galexies to look at from a planet or even if you are going into space.

10,000+

This could really be great, keep working on it.

[BaronLuke]

Great diary!

[Keeden]

now i have a use for my telescope / observatory now http://steamcommunity.com/sharedfiles/filedetails/?id=559315210 http://steamcommunity.com/sharedfiles/filedetails/?id=559315413
i love all i see of Celestial bodies. now we need way to fly in outer space fun as hell

[VaLiuM]

E = mc²

[Leveled_up]

The first thought to pop into my head is: "This is a Lego game...why do we need a realistic sky?" Shouldn't the sky look Legoish? Plus, it would give the program a chance to use resources elsewhere.