Game compression

[Acetyl]

I use 7-zip to compress game files (quite often 50 - 70% size reduction, some cases of 90+%), add them to a "compressed" list, and then disable automatic updates for those games. It would be ideal if this process could be automated and the steam client would track the compression state along with the other aspects indicating whether a game is installed and usable, so it won't try to update, automatically decompresses if you try to launch it, etc. "A "Compress game" or "Archive" option could be added to the client. This would do all of the aforementioned and automatically return the game to its uncompressed state if you try to launch.

Edit for clarification. The game will not be stored compressed by default, nor will it be decompressed and recompressed with any file changes every time you launch it. The idea is if you go into the properties page and click Archive, it will be compressed. If you launch it while compressed, it will be decompressed until you decide to compress it again.

Steam already has an archival system, but it uses poor compression and is set to split the file into CD or DVD size pieces. This could be modernized in several ways.

A list of compression results will be maintained here

[Acetyl]

I'm sorry, I'm just going to add another reply. I read through this thread again and I just cannot get over the weirdness. I start a thread, even before my edits it was quite clear that all I was doing is running 7-zip on the game directory, then adding it to a category tagging it as compressed, disabling automatic updates, then deleting the folder. That's it. Implicitly, since I'm already doing it, I know how it works. The improvement would be that these steps could be automated and built right into steam and would replace the broken CD/DVD based archival steam already uses.

1st response comes in. "Just get a bigger hard drive". Then a bunch of other stuff, some of which is only tangentially relevant like compression performance on assets which are already compressed (it isn't stated from this angle).

2nd response. Snarky comment saying to just delete games you're not using. Goes on to talk ab out relative speed of decompressing vs redownloading, as if I was born yesterday and have no conception of any of this. Goes on to affirm the others, buy a new drive. Uh, thanks? Like, I never thought of that?

3rd response. Buy another drive again. Goes on to demonstrate they didn't even read my OP, erroneously suggests this is impossible because steam can't recognize the files once they're compressed.

4th response. Reply to my reply. Claims my use is "niche" and speaks as though they're personally managing Valve employee's time, says implementation is a "waste of time". How they're familiar with the steam codebase and know what implementation would entail, anyone's guess. Provides no reasoning on why such use is so obviously "niche".

5th is reasonable. Questions if ultimate use warrants implementation time. Fair enough.

6th response comes in. Writes a long post, many quotes. Tells me to buy a bigger drive (again). Claims the best compression takes far longer than people can redownload. This is actually demonstrably false via Valve's hardware surveys. The average computer, using eg 7-zip's multithreaded LZMA2, would compress quite quickly at high settings. I used:
-mx=9 -myx=9 -ms=5g -mqs=on -mf=on -slp -m0=LZMA2:d=415m:fb=273:mc=1000
These are basically "maximum" settings. I'm using a crap single core AM3 opteron @ 2 GHz or so, and I ran the process with low priority. It doesn't take that long.

He goes on to try to bolster his case (and maximize engagement) by layering on as many charges as possible, implies my idea is practically malicious and "designed to kill SSDs". Goes on to also demonstrate a complete lack of reading comprehension, or simply that he did not bother to read the OP, and says it is inefficient to constantly compress and recompress. Which is not the idea. In subsequent replies, he will never break from this initial, false, interpretation, despite being immediately corrected. Even more hilariously, says steam will detect compressed files as damaged and redownload everything wasting the whole process! I mean what? My OP explicitly was about solving this problem.

Goes on to say what I said steam already does, poor compression then breaking it up into CD or DVD sized chunks... was what I was suggesting, then says it's a horrible idea.

Finishes by condescendingly gloating over me and reveling in his imagined victory within the grandiose fantasy. Goes all in with the characterization, narrativization, and thinly veiled insults to make it clear that he is in fact intellectually AND morally superior. Nice. Well done.

7th reply comes in, a reply to my reply. Flippant and aggressive as usual, no conception of sampling bias or statistics apparently, claims that the very few responses to my thread are indication that the entire billions in the steam userbase don't like my idea, and that maybe I should take the hint.

8th reply, Gentlebot replies to my reply. It doesn't need to be broken down, same pattern of assuming whatever thought he already had must be correct, ignoring my corrections, and continuing to demonstrate he has no idea what he's talking about. Starts to derail by building up an imaginary case based on a strawman and his misinterpretations, any attempt to corect them will be construed as "OP doesn't want a real discussion".

9th reply. Helpful info, but another case of for some reason thinking this entails constant compression and recompression.

-------

I mean look, I've used forums, I know what can go on, but this is really another level. I don't know if they're just young, trolling, terminally online, crazy. I see the exact same pattern play out on most every thread. It's a lot like "sealioning", and they're careful to always maintain composure and the appearance of good faith.... sigh. The internet didn't used to be like this. It's become practically unusuable, public discourse is impossible. Fried attention spans, cluster B behaviors. I don't know.

[Washell]

Originally posted by Acetyl:

I mean look, I've used forums, I know what can go on, but this is really another level. I don't know if they're just young, trolling, terminally online, crazy. I see the exact same pattern play out on most every thread. It's a lot like "sealioning", and they're careful to always maintain composure and the appearance of good faith.... sigh. The internet didn't used to be like this. It's become practically unusuable, public discourse is impossible. Fried attention spans, cluster B behaviors. I don't know.

Well, it's not that we/most of us are opposed to the idea. We just can't see Valve bothering and share that with you so you don't get your hopes up too high. The NTFS compression thing vs 7z is probably more because it's the better solution in conjunction with setting steam to do not update. And if none of us responded at all this thread would be 3 or 4 posts over the last 8 months of you going "Anyone here?", "No-one interested in this?"

Lastly, I don't think you were expecting public discourse. I think you were expecting the masses to cheer you and hoist you onto their shoulders going "Great idea!", "Valve should do this right now!". Well, the masses have large drives and/or fast internet and simply don't care. The poor may not have those, but also don't have scores of games to store. Your idea, while decent in itself, simply doesn't have that large of a use case. And it's getting smaller by the day through dropping hardware prices and cheaper internet services and rural broadband solutions like Starlink.

[Acetyl]

Originally posted by Washell:

Well, it's not that we/most of us are opposed to the idea. We just can't see Valve bothering

Okay.

Originally posted by Washell:

and share that with you so you don't get your hopes up too high.

It's not that big of a deal, it's not high stakes. You don't need to try to manage (your idea of) my emotions.

Originally posted by Washell:

The NTFS compression thing vs 7z is probably more because it's the better solution in conjunction with setting steam to do not update.

It isn't, it's the exact opposite. We don't want universal on the fly compression and decompression, this is an archival proposal.

Originally posted by Washell:

And if none of us responded at all this thread would be 3 or 4 posts over the last 8 months of you going "Anyone here?", "No-one interested in this?"

Lastly, I don't think you were expecting public discourse. I think you were expecting the masses to cheer you and hoist you onto their shoulders going "Great idea!", "Valve should do this right now!".

More characterization, narrativization, framing. I would rather no signal and silence to just noise. The problem with people, and particularly this subforum, is an overreliance on thye adversarial approach. This may be effective in legal scenarios, where you will build and defend your case and try to dismantle your opponent's case (in the eyes of an audience), it is not good for proper and substantive conversation in unstructured public discourse. Most people dump all of their energy (what little they invest anyway) into breaking a case, and none into making it work. I consider these to be convergent (interpolative, deductive, linear) and divergent (extrapolative, lateral, nonlinear/creative) thinking, respectively. When dealing with these and other matters you must combine the two. Failure to do so reflects either environmental conditioning or bad faith, ie, using the conversation as a surrogate for some other (emotional) purpose.

Originally posted by Washell:

Well, the masses have large drives and/or fast internet and simply don't care. The poor may not have those, but also don't have scores of games to store.

False dichotomy.

Originally posted by Washell:

Your idea, while decent in itself, simply doesn't have that large of a use case. And it's getting smaller by the day through dropping hardware prices and cheaper internet services and rural broadband solutions like Starlink.

There are more use cases than this. Underneath, it's just the same old debate about "why should I not just trust "the cloud" and store all my data on someone else's computer?" As stated prior, steam already has an archival system, but it uses poor compression and is set to break the file up to CD or DVD sized chunks, to burn to discs. This can be modernized in several ways. I have updated the OP to reflect this.