The Savegame situation

[Daynab]

As of Revision 32, most savegame issues should be fixed.
We're trying to knock them all out, so please report if you still get some problem related to saving or loading.

[Owen Snowdon]

yerrrr

[LadyAth]

But more importantly, will we be able to load these saved games too? :)

[Tleno]

Postat inițial de LadyAth:

But more importantly, will we be able to load these saved games too? :)

I think you already can, just that it doesn't really load it back as it's supposed to be. Erm, judging by blog that is.

[Nicholas Vining]

Yeah, the problem right now definitely the loading. We have a lot of people whose heads are detached from their neck stumps on loads, broken buildings, and a bunch of UI that gets garbage information. Otherwise, it's not too bad!

[eagleFMJ]

Once the save/load feature is in the game and working correctly I will gladly buy the game.

[Panic Fire]

Once multiplayer gets released I will deffinilty buy this game.

[Pareod]

Not trying to be sarcastic in any way, but how hard is implementing a save/load feature? I am a game enthusiast, and want to look into game design in my future, and I've never really considered the simple act of saving and loading a game, and all of the possible bugs attached to that.

[Nicholas Vining]

We need to take every single object in the renderer, the game engine, and the scripting layer, mark up every element of them to indicate what it is and how we should save it, stream them to disk in a ZIP file, and then unload them from disk to repeat the process. It's probably the most aggravating thing in the world. :)

[[BLA]Unmortal]

im gone wait on that save option first before im gone play but i did already buy youre product i had fun watching it and the game feels funny already with out that i played it but like i said im gone wait on the save game option. stil youtube had some funny gameplay vids that acctuly nailed down the purchase of youre game. =)

[pho]

No offense but... It sounds that you do this for the first time and just realized how to program a save/load function and just explaining the basics to us. Of cause you need to take every single object and save it to a file. if this is already a problem which took you over a month with your best men in team taking care of it, i'm really scared about the real challenges that you are facing later in your development.

[Tikigod]

Postat inițial de pho:

No offense but... It sounds that you do this for the first time and just realized how to program a save/load function and just explaining the basics to us. Of cause you need to take every single object and save it to a file. if this is already a problem which took you over a month with your best men in team taking care of it, i'm really scared about the real challenges that you are facing later in your development.

Translation of how it reads:

I know I'm being a rude ♥♥♥♥ but I think I know more than you.

Why are you answering that persons question when they asked what's so time consuming about a stable save/load system alongside keeping development of the game itself ongoing?

I mean come on, clearly all you've been doing for over months is assigning everyone to work on the save/load functionality, it's not like there's a engine that's needed a lot of tuning and improving nor a game to develop!

And how hard can it be saving all the objects to a file! That's obvious stuff! Everyone knows you need to save all the objects to a file!

No offense, of course....;)

[Nicholas Vining]

Tough crowd!

pho: The issue is not, of course, saving every object in the file. (The previous commentor asked what the complexity was, so I answered.) And it's not even necessarily the case - most FPS games save a very minimal subset of what they need to from state to state, precisely because saving every object in the game is such a pain in the neck. (Shadowrun Returns did a checkpoint save for precisely this reason, which is why their saves are a little wonky.)

So, in case anybody's interested, here's the deal with what took/is taking so long for the saves:

1. Objects may reference other objects. If you save both objects, you need to make sure that, on load, you match the newly created version of the object with anything that expects it to point to it. For Dungeons of Dredmor, our previous title, this was a huge pain in the neck, but pretty much okay - Dredmor levels are pretty flat and generally don't tend to keep many references to other objects around. For Clockwork Empires, everything references everything else. We implemented what is known as a pointer-chasing scheme in order to ensure that we can save, and load, everything in the right order. Until recently, the data structures involved in tracking all of this had a rather nasty tendency to break the 2-gigabyte address limit that Clockwork Empires suffers from under Windows because it's a 32-bit application. We didn't find that out until about half way through adding all the stuff to saves.

2. Our saves are not totally binary - they are plain text (specifically XML) stored in ZIP files, along with binary files for some things (like terrain) that are stored as flat arrays. Dredmor used binary saves, and it was a terrible idea; if one byte is off, everything explodes. This necessitated writing a lot of code to work with ZIP files, streaming them to disk and so forth.

3. Clockwork Empires has three basic systems, which we will call "game", "renderer", and "script." Because of how we do multithreaded things, a given object (say a cabbage) has three different representations - how it's seen by the scripting language, how it's seen by the jobs system/pathfinding/networking ("game") and how it's seen by the renderer. This gives us three times as much stuff to mark up with save/load information, and about three times as many systems to make work. The benefits outweigh the negatives of this approach - the game actually runs deterministically in its own thread at about one tenth the speed of the renderer, and lets it know what's going on as it reaches decisions. The renderer can then focus on rendering the game as fast as possible. If we didn't do this, we couldn't do things like the replay system (which we use for debugging) or the networking (which depends on being deterministic for making the whole thing work - since we can't send the entire game state over the network, we send only the commands that players input, and let the determinism ensure that all player commands are carried out in the same order on everybody's computer. Fairly standard stuff for strategy games and RTS games.)

However, it does complicate saving and loading.

The script state of a cabbage lives in Lua, and everything else lives in C++ - so we end up writing a little Lua script for each cabbage that we execute in order to recreate it on load. (Believe it or not, this is the recommended way of doing things. Lua is weird.)

4. We also ended up doing a lot of nice stuff based on our experiences from Dredmor, where I basically just slapped things together furiously - we have nice little functions for everything, and a bunch of stuff to make saves stable and more better. Doing all of that infrastructure building up front took a certain amount of time before we could even begin writing saves, as opposed to just "bashing through everything in memory and dumping it to disk in great steaming chunks" (read: what we did for Dredmor, never again!)

In hindsight - yes, absolutely we should have had saves in our engine framework from day zero. This didn't happen for a number of reasons - mainly getting other things in the engine that needed to be there from day zero. :) Anyhow, the actual system to do the loading and saving is working, and all that is outstanding is a certain amount of bug-chasing and marking up the portions of the renderer code base that talk to the user interface. I have a big pot of coffee and I'm plugging away at finishing it. At this point the top men have gotten it done, and it's all just me.

I am pleased we didn't make any of the same mistakes that I made with the Dredmor save game system; however, we made up for this by making new ones. :) Anyhow, hopefully this status update is at least interesting and answers some questions. Back to work!

[Tikigod]

Awesome post Nicholas.

Hopefully it will help create some more appreciation over that it's not as straight forward as "Save all the objects to file". :)

Thanks for the write up explaination.

[Doktor Sleepless]

Thanks Nicholas. I love when devs take the time to explain stuff in detail. Keep up the good work. And get some rest soon!

[pho]

You can't be to careful these days about early access games. I'm sorry if this sounded offensiv but that wasn't my intension. But I saw so many games fail in this type of citybuilding genre. And I never ever wan't something like an other -towns-building game with a similar name to happen again. Thats just my biggest fear here so thank you for explaining the tough stuff in the detail. And thank you for keeping up the good work. I'm happy to invest real time in this when the save(and load) functions work.