4GB to 43GB? How does Steam compress games that much?

[Cyber Akuma]

I recently got Backfirewall from a bundle, I remember playing the demo of that a while ago and loved it.

I was shocked at first that the game is apparently 34GB, I had no idea it could take up that much space when far more visually complex games I have took much less space. And then I was shocked again when I saw that Steam only needed to download 4GB and the installed that into 34GB of data. How can it possibly compress 34GB of data into a mere 4GB? That's a compression ratio of about 85%! The only type of data I know that can compress that well is ASCII text, not anything digital like executable code, graphics, sound, etc. In the same bundle I also got The Ascent and that game is 19 gigs installed and the download was 17gigs.

[sfnhltb]

Maybe it is using uncompressed graphics and/or sound formats for some reason? Graphics and sounds only don't compress well because there are more commonly already compressed, not because they are graphics and sound files - which by their nature are usually fairly repetitive and have pretty high compression ratios even when compressing losslessly, especially when compressed using an algorithm that understands the data rather than general purpose compression algorithms.

[Cyber Akuma]

Originally posted by sfnhltb:

Graphics and sounds only don't compress well because there are more commonly already compressed, not because they are graphics and sound files - which by their nature are usually fairly repetitive and have pretty high compression ratios even when compressing losslessly, especially when compressed using an algorithm that understands the data rather than general purpose compression algorithms.

Yeah I know, generally they have already had compression, and usually lossy compression at that, applied to them. So compressing them again, and lossless this time, does not do much generally.

Even lossless though they tend to not compress well because there isn't enough instances of repetitive data to compress.

I tried looking through the game's files and I saw a lot of "sharedassets(number).assets.resS" files that were each several gigs, which makes me believe there is a lot of repetitive data between those files. That's rather annoying to see, that the game is probably closer t around 4GB of data but is wasting 34GB because of redundant data everywhere. I assume modern game SDKs would detect and optimize that. No wonder games are becoming absurdly huge these days.

I recall another game I had which was a badly done port of some mobile game that basically downloaded all of it's data from Steam when you installed it.... then when you launched it basically ignored all that data and re-downloaded it all on it's own in again... but that eventually got patched at least.

[SpiffingExaggeration42]

They don't bother detecting this stuff any more.

[Steve]

This thread was quite old before the recent post, so we're locking it to prevent confusion.