Acetyl (Banned) Jan 2, 2022 @ 1:09pm
Game compression
I use 7-zip to compress game files (quite often 50 - 70% size reduction, some cases of 90+%), add them to a "compressed" list, and then disable automatic updates for those games. It would be ideal if this process could be automated and the steam client would track the compression state along with the other aspects indicating whether a game is installed and usable, so it won't try to update, automatically decompresses if you try to launch it, etc. "A "Compress game" or "Archive" option could be added to the client. This would do all of the aforementioned and automatically return the game to its uncompressed state if you try to launch.

Edit for clarification. The game will not be stored compressed by default, nor will it be decompressed and recompressed with any file changes every time you launch it. The idea is if you go into the properties page and click Archive, it will be compressed. If you launch it while compressed, it will be decompressed until you decide to compress it again.

Steam already has an archival system, but it uses poor compression and is set to split the file into CD or DVD size pieces. This could be modernized in several ways.

A list of compression results will be maintained here
Last edited by Acetyl; Aug 4, 2022 @ 3:00pm

Something went wrong while displaying this content. Refresh

Error Reference: Community_9734361_
Loading CSS chunk 7561 failed.
(error: https://community.cloudflare.steamstatic.com/public/css/applications/community/communityawardsapp.css?contenthash=789dd1fbdb6c6b5c773d)
< 1 2 3 >
Showing 1-15 of 33 comments
Gwarsbane Jan 2, 2022 @ 1:45pm 
Or just get a bigger hard drive....

If you have such a bad system that you have to compress games to save space then the act of compressing and uncompressing a game is going to take a LONG time.

Then when you uncompress, you are going to have to install any patches that have happened anyway to be able to play with other people. Which means you will just have to recompress the game again.

And when you uncompress a game to play it, it means you are going to have to have a lot of free room to put that game in anyway.


In some cases the largest games are already compressed in a way that allows them to still be played. Take Payday 2 for example. At one point people were complaining that it was 50+ gigs in size. So they turned on compression and got the game down to 35 gigs. Payday 2 is now up to over 76 gigs in size, and thats compressed in a playable state.

But people also complain at how long patches take because the game files have to be uncompressed, which requires hard drive space and time...

So ya the simplest suggestion is get a bigger hard drive. Last year I bought a 6tb western digital black drive for 200 bucks Canadian. I don't have all my games installed but I still have lots of room left to do so even though I broke up the drive into 2 evenish partitions.



Also Steam already has a backup system which I think compresses a game, though its not playable anymore till you uncompress it, no idea how well it works as I have never used it. I'd rather remove games if I need the space than just compress games, but I also don't have to worry about traveling and not having my games with me or don't have to worry about download costs as others do.
DC-GS Jan 2, 2022 @ 2:03pm 
Hey, I know a way to achieve 100% compression rate!
Deinstall the game. Reinstall it if you want to play. Depending on your internet connection this may be even faster than compressing and uncompressing the game.

If you really want the games kept installed, like Gwarsbane said, buy another drive to store the games on.
Acetyl (Banned) Jan 2, 2022 @ 2:05pm 
Originally posted by Gwarsbane:
Or just get a bigger hard drive....
The point is compressing files you rarely use, but don't want to have to redownload. This is a matter of efficiency and independent of how much storage space you have. Realistically you don't need everything ready to go at any given moment.

Originally posted by Gwarsbane:
the act of compressing and uncompressing a game is going to take a LONG time.
It doesn't take that long. Far longer to re-download.

Originally posted by Gwarsbane:
Then when you uncompress, you are going to have to install any patches that have happened anyway to be able to play with other people. Which means you will just have to recompress the game again.
I think the way it ought to work is launching a compressed game would decompress it and thus bring it out of the compressed state, then patch it and whatever else needs to happen. Launching the game implies you'll be using it more than once in the near future, so the compressed version could just be deleted by dfefault, or if steam wanted to be more clever and didn't use solid compression, it could keep the archive and update it with changed files if the user goes to recompress.

Originally posted by Gwarsbane:
And when you uncompress a game to play it, it means you are going to have to have a lot of free room to put that game in anyway.
Not necessarily. You could have just enough, or, you could extract it to another drive (eg an SSD).

Originally posted by Gwarsbane:
In some cases the largest games are already compressed in a way that allows them to still be played.
Well yeah, devs can utilize all kinds of streaming and on the fly (lossless) compression for their assets. Sometimes those files can be further compressed by eg LZMA2, sometimes not. Steam could use more specialized compression but I doubt they want to devote dev time to developing their own archive format when they could just throw a 7z implementation in. Regardless, these are things we handle on our own end. No need to rely on the dev.

Originally posted by Gwarsbane:
But people also complain at how long patches take because the game files have to be uncompressed, which requires hard drive space and time...
Compression wouldn't be automatic, the user would set the game to archived.

Originally posted by Gwarsbane:
So ya the simplest suggestion is get a bigger hard drive. Last year I bought a 6tb western digital black drive for 200 bucks Canadian. I don't have all my games installed but I still have lots of room left to do so even though I broke up the drive into 2 evenish partitions.
I have several Tb free, doesn't mean you waste space though. These are just common sense solutions that save on bandwidth and storage space.


Originally posted by Gwarsbane:
Also Steam already has a backup system which I think compresses a game, though its not playable anymore till you uncompress it, no idea how well it works as I have never used it.
Not very well. I tried it and don't recall what compression scheme it uses if any, but by default it wants to break the game up into CD or DVD sized chunks to burn on discs. Very different use case.
Acetyl (Banned) Jan 2, 2022 @ 2:06pm 
Originally posted by DC-GS:
Hey, I know a way to achieve 100% compression rate!
Deinstall the game. Reinstall it if you want to play. Depending on your internet connection this may be even faster than compressing and uncompressing the game.

If you really want the games kept installed, like Gwarsbane said, buy another drive to store the games on.
Or you know, I could just keep doing what works and steam could add it as a proper feature.
Start_Running Jan 2, 2022 @ 2:06pm 
M dude. I can grab a 1TB drive for like $30-$60...
Just put off buying one or two games, and get yourself a new drive.

Teh reason why Valve won't be able to do what you suggest is because quitesimply. It can't recognize the compressed files as your game files. Compression changes the data structture. That's basicallty how i functions.
Originally posted by Acetyl:
Originally posted by DC-GS:
Hey, I know a way to achieve 100% compression rate!
Deinstall the game. Reinstall it if you want to play. Depending on your internet connection this may be even faster than compressing and uncompressing the game.

If you really want the games kept installed, like Gwarsbane said, buy another drive to store the games on.
Or you know, I could just keep doing what works and steam could add it as a proper feature.

Or not because it is such a niche situation that it would be a waste of time to add a feature that is already accomplished by uninstalling unused/infrequently used software.
Acetyl (Banned) Jan 2, 2022 @ 2:08pm 
Originally posted by Start_Running:
M dude. I can grab a 1TB drive for like $30-$60...
Just put off buying one or two games, and get yourself a new drive.

Teh reason why Valve won't be able to do what you suggest is because quitesimply. It can't recognize the compressed files as your game files. Compression changes the data structture. That's basicallty how i functions.
Right, that's why it would mark it as compressed internally. Same as knowing a game is not installed or needs an update.
Acetyl (Banned) Jan 2, 2022 @ 2:10pm 
Originally posted by Damp Wizard Sleeve:
Originally posted by Acetyl:
Or you know, I could just keep doing what works and steam could add it as a proper feature.

Or not because it is such a niche situation that it would be a waste of time to add a feature that is already accomplished by uninstalling unused/infrequently used software.
I think the utility is fairly obvious. It's not all that complicated either.
Crazy Tiger Jan 2, 2022 @ 2:13pm 
Originally posted by Acetyl:
Originally posted by Damp Wizard Sleeve:

Or not because it is such a niche situation that it would be a waste of time to add a feature that is already accomplished by uninstalling unused/infrequently used software.
I think the utility is fairly obvious. It's not all that complicated either.
It's questionable that there is a large enough use case to warrant implementing it, though.
Acetyl (Banned) Jan 2, 2022 @ 2:18pm 
Originally posted by Crazy Tiger:
Originally posted by Acetyl:
I think the utility is fairly obvious. It's not all that complicated either.
It's questionable that there is a large enough use case to warrant implementing it, though.
It's difficult to fully know the reasons or use cases that exist for people until you make a feature available and known to them. You cannot consider (or make use of) what you never thought to think about.
Last edited by Acetyl; Jan 2, 2022 @ 2:19pm
Mad Scientist Jan 2, 2022 @ 2:25pm 
Originally posted by Acetyl:
The point is compressing files you rarely use, but don't want to have to redownload. This is a matter of efficiency and independent of how much storage space you have. Realistically you don't need everything ready to go at any given moment.
Efficiency is also nto wasting immense amounts of time compressing/decompressing files; it's efficient to get a large drive that can handle the demands asked of it. 6TB drives are often on sale as low as $75 USD, so there's very little excuse. Also, if you're doing that on an SSD you're going to kill the SSD.

Originally posted by Acetyl:
Originally posted by Gwarsbane:
the act of compressing and uncompressing a game is going to take a LONG time.
It doesn't take that long. Far longer to re-download.
The best compression takes far longer than most people can download the same thing through Steam.

Originally posted by Gwarsbane:
Then when you uncompress, you are going to have to install any patches that have happened anyway to be able to play with other people. Which means you will just have to recompress the game again.
I think the way it ought to work is launching a compressed game would decompress it and thus bring it out of the compressed state, then patch it and whatever else needs to happen. [/quote]
Your idea is literally designed to kill SSDs for SSD users. This would only work with HDDS, in which being an HDD usually means you have a much larger storage potential.

Originally posted by Acetyl:
Originally posted by Gwarsbane:
So ya the simplest suggestion is get a bigger hard drive. Last year I bought a 6tb western digital black drive for 200 bucks Canadian. I don't have all my games installed but I still have lots of room left to do so even though I broke up the drive into 2 evenish partitions.
I have several Tb free, doesn't mean you waste space though. These are just common sense solutions that save on bandwidth and storage space.
It's a waste of time & space to compress & decompress things, highly inefficient, and since games get updates you'd still have to download the updates to be usable. If anything isn't compressed, it'll see it as damaged and will re-download the entire thing, wasting everything.

Originally posted by Acetyl:
Originally posted by Gwarsbane:
Also Steam already has a backup system which I think compresses a game, though its not playable anymore till you uncompress it, no idea how well it works as I have never used it.
Not very well. I tried it and don't recall what compression scheme it uses if any, but by default it wants to break the game up into CD or DVD sized chunks to burn on discs. Very different use case.
You can compress it into a single piece, there is no point in using CD or DVD compressed chunk backups, that's just a waste of resources.

Big "No." to this awful suggestion. It would have to be entirely disabled for SSDs as well.

Originally posted by Acetyl:
Originally posted by Crazy Tiger:
It's questionable that there is a large enough use case to warrant implementing it, though.
It's difficult to fully know the reasons or use cases that exist for people until you make a feature available and known to them. You cannot consider (or make use of) what you never thought to think about.
No, it's extremely easy to even brainstorm something, have experience with something etc. One does not have to implement something to consider it, as it has to be considered to be made. Steam is a giant, valuable company. They think of things all the time and implement them in Valve Time so they're very aware of what could be done and why. This is a case of "just because you can, doesn't mean you should". Most suggestions come from a self-serving nature where the individual wants that thing, no matter how much of a time waster it is or how illogical compared to what they should realistically be doing.
nullable Jan 2, 2022 @ 2:45pm 
Originally posted by Acetyl:
Originally posted by Damp Wizard Sleeve:

Or not because it is such a niche situation that it would be a waste of time to add a feature that is already accomplished by uninstalling unused/infrequently used software.
I think the utility is fairly obvious. It's not all that complicated either.

Clearly there's utility for you. I'm not sure that utility is as universal as you imagine.

I for one don't have any use for it. It doesn't sound like most users posting here so far have use for it either. That might hint a few things.

After all, some features aren't worth it, even if they're good features, if no one uses them.
Acetyl (Banned) Jan 2, 2022 @ 2:58pm 
Originally posted by Mr. Gentlebot:
Originally posted by Acetyl:
The point is compressing files you rarely use, but don't want to have to redownload. This is a matter of efficiency and independent of how much storage space you have. Realistically you don't need everything ready to go at any given moment.
Efficiency is also nto wasting immense amounts of time compressing/decompressing files; it's efficient to get a large drive that can handle the demands asked of it. 6TB drives are often on sale as low as $75 USD, so there's very little excuse. Also, if you're doing that on an SSD you're going to kill the SSD.
1) No one will be forcing you to compress your games.
2) Compression doesn't take that long.
3) Filling up your case with hard drives to store games you could reduce in size by 30 - 70% is not sensible.
4) Repeatedly transmitting the same information over the internet is not efficient

Originally posted by Mr. Gentlebot:
Originally posted by Acetyl:
It doesn't take that long. Far longer to re-download.
The best compression takes far longer than most people can download the same thing through Steam.
Not so. I'm running some crap single core AM3 opteron and just compressed Dead Space from 7.8 Gigs down to 3.5, took around an hour and I just left it on idle priority in the background. Ram usage was ~4.5 gigs. Settings were:
-mx=9 -myx=9 -ms=5g -mqs=on -mf=on -slp -m0=LZMA2:d=415m:fb=273:mc=1000

Originally posted by Mr. Gentlebot:
Originally posted by Acetyl:
Originally posted by Gwarsbane:
Then when you uncompress, you are going to have to install any patches that have happened anyway to be able to play with other people. Which means you will just have to recompress the game again.
I think the way it ought to work is launching a compressed game would decompress it and thus bring it out of the compressed state, then patch it and whatever else needs to happen.
Your idea is literally designed to kill SSDs for SSD users. This would only work with HDDS, in which being an HDD usually means you have a much larger storage potential.
You're kidding me right. You have people that have their OS installed on an SSD, that's hundreds to thousands of registry and system file reads and writes every second. This is nothing. If you have these files stored on an SSD for some reason ever so slightly adding to its usage (your own doing of course, your problem if you don't know better) all you have are the reads to bring the file in, the writes to write the decompressed output, and then block updates if you make changes to the files. The blockupdates are only generated and written when you go to return the game to the compressed state. An ideal use case would be archiving on a hard drive and decompressing onto an SSD for high performance (especially open world games and things like that). Less demanding stuff could be extracted onto the same drive.

Not really sure what your point is here, it's really reaching.

Originally posted by Mr. Gentlebot:
Originally posted by Acetyl:
Originally posted by Gwarsbane:
So ya the simplest suggestion is get a bigger hard drive. Last year I bought a 6tb western digital black drive for 200 bucks Canadian. I don't have all my games installed but I still have lots of room left to do so even though I broke up the drive into 2 evenish partitions.
I have several Tb free, doesn't mean you waste space though. These are just common sense solutions that save on bandwidth and storage space.
It's a waste of time & space to compress & decompress things, highly inefficient, and since games get updates you'd still have to download the updates to be usable. If anything isn't compressed, it'll see it as damaged and will re-download the entire thing, wasting everything.
Read the OP, this was all covered.

Originally posted by Mr. Gentlebot:
Originally posted by Acetyl:
Originally posted by Gwarsbane:
Also Steam already has a backup system which I think compresses a game, though its not playable anymore till you uncompress it, no idea how well it works as I have never used it.
Not very well. I tried it and don't recall what compression scheme it uses if any, but by default it wants to break the game up into CD or DVD sized chunks to burn on discs. Very different use case.
You can compress it into a single piece, there is no point in using CD or DVD compressed chunk backups, that's just a waste of resources.

Big "No." to this awful suggestion. It would have to be entirely disabled for SSDs as well.
Read the reply you quoted again. This is not my suggestion, this is how their system already works.

Originally posted by Mr. Gentlebot:
Originally posted by Acetyl:
Originally posted by Crazy Tiger:
It's questionable that there is a large enough use case to warrant implementing it, though.
It's difficult to fully know the reasons or use cases that exist for people until you make a feature available and known to them. You cannot consider (or make use of) what you never thought to think about.
No, it's extremely easy to even brainstorm something, have experience with something etc. One does not have to implement something to consider it, as it has to be considered to be made.
This is not what I said. I said (in most cases) you cannot fully know without actually implementing something, and you certainly can't model demand. In some cases the system is simple enough that you can trace out every reasonable permutation and generate a risk:benefit, risk:risk, whatever, but not in all cases or for all aspects. Every a priori model has limitations. It depends.

Originally posted by Mr. Gentlebot:
Steam is a giant, valuable company. They think of things all the time and implement them in Valve Time so they're very aware of what could be done and why. This is a case of "just because you can, doesn't mean you should". Most suggestions come from a self-serving nature where the individual wants that thing, no matter how much of a time waster it is or how illogical compared to what they should realistically be doing.
Are you familiar with the Steam codebase? If not, you don't really have much basis to speculate on what it would take to implement a given feature. I assume it wouldn't be difficult and iirc several LZMA implementations exist with viable licenses, but again, pointless speculation.
Last edited by Acetyl; Jan 2, 2022 @ 3:28pm
Acetyl (Banned) Jan 2, 2022 @ 3:05pm 
Originally posted by Snakub Plissken:
Originally posted by Acetyl:
I think the utility is fairly obvious. It's not all that complicated either.

Clearly there's utility for you. I'm not sure that utility is as universal as you imagine.

I for one don't have any use for it. It doesn't sound like most users posting here so far have use for it either. That might hint a few things.

After all, some features aren't worth it, even if they're good features, if no one uses them.
This is a fairly small and biased sample of a large userbase. Not much can be extrapolated from it alone.

I agree with the lattermost to an extent, though I don't think this would be "bloat" considering they already have a dated and not really useful backup system.
Last edited by Acetyl; Jan 2, 2022 @ 3:07pm
Mad Scientist Jan 2, 2022 @ 3:37pm 
Originally posted by Acetyl:
Originally posted by Damp Wizard Sleeve:
Or not because it is such a niche situation that it would be a waste of time to add a feature that is already accomplished by uninstalling unused/infrequently used software.
I think the utility is fairly obvious. It's not all that complicated either.
"Things users can do without the client if they choose to do so" does not mean Steam needs to implement every whim of every user out there. The most basic fact is if you want something badly enough, and there's tools available; you use them & move on, not ask for something with little benefit to community especially since most people will complain it takes so long if they utilize it.

Originally posted by Acetyl:
1) No one will be forcing you to compress your games.
2) Compression doesn't take that long.
3) Filling up your case with hard drives to store games you could reduce in size by 30 - 70% is not sensible.
4) Repeatedly transmitting the same information over the information is not efficient
High grade compression takes much longer than quick compression, and quick compression is pointless towards your suggestion. It would be faster to download and install through the client. We have 6TB drives for next to nothing, you really have zero excuse especially if you have a larger library. How many games do you own in Steam? How many of them are AAA? How many of them are 5GB or less?

Originally posted by Acetyl:
Originally posted by Mr. Gentlebot:
The best compression takes far longer than most people can download the same thing through Steam.
Not so. I'm running some crap single core AM3 opteron and just compressed Dead Space from 7.8 Gigs down to 3.5, took around an hour and I just left it on idle priority in the background. Ram usage was ~4.5 gigs. Settings were:
-mx=9 -myx=9 -ms=5g -mqs=on -mf=on -slp -m0=LZMA2:d=415m:fb=273:mc=1000
Again, we're talking best compression vs quick compression. Older CPUs will struggle to pack something down to the smallest size possible, and older AMD CPUs will basically red line due to how the old architecture is.

Originally posted by Acetyl:
Originally posted by Mr. Gentlebot:
Your idea is literally designed to kill SSDs for SSD users. This would only work with HDDS, in which being an HDD usually means you have a much larger storage potential.
You're kidding me right. You have people that have their OS installed on an SSD, that's hundreds to thousands of registry and system file reads and writes every second.
Tiny files vs GBs of data each pack, unpack, temporary storage use etc.
That's just an immensely bad faith deflection on your part of two incomparable things for pointing out a serious flaw in this self-serving suggestion.

Originally posted by Acetyl:
Not really sure what your point is here, it's really reaching.
Compress and decompress 25, 50, 75 and 100+GB games. Repeatedly. All day. Demonstrate that there is no such side effect, as I'd rather not chunk a drive similar to when people run performance tests. HDDs would be far better suited than SSDs for this purpose, and new drives are very fast compared to 2003.

Originally posted by Acetyl:
This is not what I said. I said (in most cases) you cannot fully know without actually implementing something, and you certainly can't model demand.
"Implement my suggestion so we can figure it out after" is not a good suggestion. This is Valve we're talking about, not amateur hour, meaning that they can easily know how something would function, be utilized, etc. This isn't a one-man operation of just throwing something at the dart board hoping it ends up a good idea; Valve has the competence to know that this is not whatsoever worth even entertaining, and would be a waste of programmers time.

Originally posted by Acetyl:
Are you familiar with the Steam codebase? If not, you don't really have much basis to speculate on what it would take to implement a given feature. I assume it wouldn't be difficult and iirc several LZMA implementations exist with viable licenses, but again, pointless speculation.
I very much doubt you are, and that's a very poor deflection from the response. Just because Valve can, does not mean they will or should. You're making far more speculation, which is just hypocritical a just another OP that can't stand the immense flaws in a suggestion. It's also beyond silly to download compressed packages only to later compress them again, which users may freely do themselves if they want to back it up for a much later use. After that, all they have to do is uninstall the game so it wont queue, and unpackage when they're ready to update & play.

Nothing is stopping you from manually accomplishing this, the issue with most suggestions is people want Steam to be an everything-client, which mostly comes from users not wanting to put effort into things they want which Steam does not offer, which if they entertained each suggestion the client would be a giant bloated mess rather than essentially a "thin" client with the functionalities desired for an online store & game library client.
< 1 2 3 >
Showing 1-15 of 33 comments
Per page: 1530 50

Date Posted: Jan 2, 2022 @ 1:09pm
Posts: 33