Low FPS, Low CPU Usage, Low RAM Usage - How to fix?

[John Cena]

I have ~200 dwarfs in my fortress. I have disabled temperature. I did branch mine 3 layers but I walled them off after I got what I wanted. I do have around 7000 stone lying around and 10000+ other items. Disabling temperature seems to have a minor impact on FPS.

FPS is around 30-40. CPU usage is around 30-40% with occasional spikes to 70% on the most used core. Ram usage is around 1 GB.

How can I improve performance? Why isn't DF using more of my CPU or RAM? Do I need to change something in the options?

[rome of oxtrot]

about the only thing you can do to improve performance is to get a CPU with more L3 cache

that said, 30-40 FPS for a fort with 200 dorfs is quite good. DF is not a first person shooter. it is not simply reasonable to expect a simulation game like DF to maintain a TPS of 60-100 ticks per second. if you want your games that maintain such high tick rate (despite there being no real reason why they shoud), play something computationally simpler

[John Cena]

Originally posted by rome of oxtrot:

about the only thing you can do to improve performance is to get a CPU with more L3 cache

that said, 30-40 FPS for a fort with 200 dorfs is quite good. DF is not a first person shooter. it is not simply reasonable to expect a simulation game like DF to maintain a TPS of 60-100 ticks per second. if you want your games that maintain such high tick rate (despite there being no real reason why they shoud), play something computationally simpler

thanks. is there anything I can do to get more utilization of the CPU (9700k)? I would expect a much higher usage if FPS has decreased substantially.

[ryansDeViL]

I don't think low performance is always a cause of hardware. I think some games it's more because of the way things are coded and how complicated the in-game functions are. With Dwarf fortress steam edition, that was actually what had me fall out of playing it, over rimworld. The unsteady performance I have on it on my laptop makes things just feel chunky.

I haven't given it a go in awhile, as i'm waiting for the adventure mode still, but i'm hoping the performance has improved some since last year

[Jeff]

Getting better performance is usually more about planning out your world than changing the settings. The size of your world has a large impact on your fps, so generating a smaller one with fewer civilizations with improve your fps, as fewer things are being simulated in the background. The size of your embark can also impact fps. Dwarf Fortress is a very old game, and because of that it doesn't have the same level of optimization you would expect from recent titles, so you probably won't see it using 100% of your system resources. I do know some optimizations are planned for upcoming updates so you can look forward to that. It's also important to know that this isn't exactly a game that lends itself to large population counts. Every dwarf has a number of different things that are tracked like mood, personal opinion, and family ties, so having large amounts of dwarves only compounds this. If you've never tried it, I recommend playing a fortress with a lower population around 60.

[Perenti]

I find that binding DF and dfhack to a subset of cores that share a cache _seems_ to help marginally. This is using the linux cpuset facility.

But any difference is marginal at best once you've huge stocks and stuff at 200+ pop

[Zee~]

To answer one of your questions, DF doesn't use more than one core, and will never use more than one core, so it will always use a relatively small amount of CPU even when it's maxing itself out. This is not a GPU game, this is a CPU game built on a very old style of single-threaded programming languages and assumptions. Changing this would require a complete recoding of about 75% of the core game. The steam version comes with some minor optimizations compared to previous versions that lessens some of the burden that background world developments put on your CPU, but it's not going to fundamentally change.

[Fletch]

Originally posted by Zee~:

To answer one of your questions, DF doesn't use more than one core, and will never use more than one core, so it will always use a relatively small amount of CPU even when it's maxing itself out. This is not a GPU game, this is a CPU game built on a very old style of single-threaded programming languages and assumptions. Changing this would require a complete recoding of about 75% of the core game. The steam version comes with some minor optimizations compared to previous versions that lessens some of the burden that background world developments put on your CPU, but it's not going to fundamentally change.

Incorrect my friend.

Game Settings --> Game --> Enable multithreading (experimental!) --> Yes

[John Cena]

Somehow I am able to get ~80% avg usage on a single core now.

This is a gigantic improvement in performance as I'm hitting 70-80 FPS with 200 dwarves.

The only thing I did was after starting the game up today was I disabled/re-enabled experimental multi-threading and set real-time priority in the task manager for Dwarf Fortress. I have done these things previously but haven't gotten good results until today so I'm not sure what's different.

[rome of oxtrot]

Originally posted by Jeff:

generating a smaller one with fewer civilizations with improve your fps, as fewer things are being simulated in the background

the actual issue here is that a large number of historical figures causes a quadratic slowdown in frequently-run code, and so limiting the number of historical figures has a substantial impact on tick processing times. larger world -> more civilizations -> more historical figures

the experimental multiprocessing option accelerates the particular set of calculations that are impacted here, but the real problem is that historical figures are stored, in sorted order, in a linear vector, which means looking one up by id requires a binary search, O(lg n) where n is the number of historical figures. parallelizing these searches (which is what the "experimental multiprocessing" option does) only clonks this down by a constant factor (the number of available cores). a much greater speedup would be obtained by replacing the linear vector with a hash table, which has O(alpha(n)) lookup performance, which is O(1) for all practical purposes. this would make performance of this particular frequently-executed code essentially insensitive to the number of historical figures

in general, there are lots and lots of places where dwarf fortress uses linear vectors for data structures, imposing O(n) costs for operations that would be O(1) with a more wisely chosen data structure. even toady recognized this, but his solution was to maintain the lists in sorted order and hand-roll a binary division search, which is certainly an improvement, but still not good even to get a passing grade in Data Structures 101. there's a reason people use hash tables. and don't even get me started on his home-baked priority queues... i have to wonder how many of the issues with irregularities in labor assignment are potentially due to flaws in that implementation

[Darzan]

Children of those days :P

I do not think that 30-40 FPS is anywhere close to bad. There have always been spikes - I suppose it requires complete rewrite of code to get rid of that.

CPU usage of 30-40% is completely normal on single-process games. I kind hope they can go beyond that at some point.

RAM usage of 1 GB is very nice! It all depends how big area your castle and your world have. Keep in your mind that most now-days games overuse RAM like 10x times or more. Usually it is due bad programming skill or libraries but sometimes they do it deliberately to make game look more impressive.

[Fletch]

Originally posted by Luke Skywalker:

Somehow I am able to get ~80% avg usage on a single core now.

This is a gigantic improvement in performance as I'm hitting 70-80 FPS with 200 dwarves.

The only thing I did was after starting the game up today was I disabled/re-enabled experimental multi-threading and set real-time priority in the task manager for Dwarf Fortress. I have done these things previously but haven't gotten good results until today so I'm not sure what's different.

Weird, wonder if there is a problem reading/writing that setting (the multithreading option) where it is not respected on restarts? If you have the option enabled, and getting 70-80 FPS, then when you restart the game do you get the same FPS or does it behave like multithreading is off again (requiring you to toggle the setting to get good performance again).

[Darzan]

Multithreading does not always involve multiple processes. It is quite possible to make multithread work on single process - I think it is what game does.

There is also processor based multithreading which is generally enabled on most processors unless you change it - it is generally good to disable it on gamers PC-s as it lower your maximum power what process can have. However it could help you with some stupid programs which try to grab all power to themselves regardless if they need it or not.

Ideally I would like game to run on multiple processes - 3-4 would be good enough and do data sharing through memory sharing instead of pipes/files which are way slower. Also 2D game could theoretically also make good use of graphic processor and memory for lot of extra power.

[John Cena]

Originally posted by Darzan:

Children of those days :P

I do not think that 30-40 FPS is anywhere close to bad. There have always been spikes - I suppose it requires complete rewrite of code to get rid of that.

CPU usage of 30-40% is completely normal on single-process games. I kind hope they can go beyond that at some point.

RAM usage of 1 GB is very nice! It all depends how big area your castle and your world have. Keep in your mind that most now-days games overuse RAM like 10x times or more. Usually it is due bad programming skill or libraries but sometimes they do it deliberately to make game look more impressive.

It's the average CPU usage on a single core, the one that is being used the most. The other cores are around 10-20%.

[Darzan]

Well, lot of PC-s have 4 cores. When one process uses 100% of time on one core it is 25% of total processing power. Rest of 5-15% comes from OS, daemons/services, steam, discord and possibly browser in background.

One process can maximum use one core power. That why multiple processes are much better. However it requires game to solve data transfer between processes which needs extra level of complexity in game code.

[rome of oxtrot]

Originally posted by Darzan:

One process can maximum use one core power.

The 1980s called, they would like their scheduling model back. We stopped using this threading model ("heavyweight threads") in the early 1990s (this here[www.usenix.org] is a very nice paper on this topic from 1992).

In the modern world, processes can (and routinely do) contain a pretty much any number of threads, each of which can be individually scheduled on a logical processor and thus can easily utilize more than one processor core.