Steam Compression is slowing down Download on modern Hardware

[Blank]

I downloaded Apex Legends with "only" 1,5Gbit/s my local CDN server can handle 5Gbit/s esaily. My Ryzen 5 5600 is too slow to decompress it. xz (lzma) is very slow but has high compression rations, but it is worth it? LZ4 or even ZSTD would be better choice, steam would save computing power and the people with faster internet connection can download their games faster without buying the newest high end Processor.

[Satoru]

Steam downloads are limited by

1) your cpu
2) your disk IO
3) your anti virus
4) your isp

Pick one

[William Shakesman]

Originally posted by Satoru:

Steam downloads are limited by

1) your cpu
2) your disk IO
3) your anti virus
4) your isp

Pick one

Did you read OP? This is completely irrelevant to his statement about compression methods. In addition, it is inherently silly to assume Steam can serve infinite bandwidth to any location.

[cSg|mc-Hotsauce]

Originally posted by William Shakesman:

In addition, it is inherently silly to assume Steam can serve infinite bandwidth to any location.

What do you think is the maximum theoretical download speed of Steam's servers?

[aiusepsi]

The OP's point is a good one. Here's some data on compression algorithms:

Algo                 Decompression rate  Ratio (out of 100, lower is better)
lzma 19.00 -9        107 MB/s            22.98
fastlzma2 1.0.1 -10  105 MB/s            22.96
xz 5.2.4 -9          88 MB/s             23.00
zstd 1.4.3 -22       865 MB/s            24.88

Data from here: https://github.com/inikep/lzbench (Note: this data may be quite old, the most recent Zstd version is 1.5.6, and is still under active development, so it may have improved.)

Steam uses the LZMA algorithm for compression, which is represented in this table by the implementations lzma, fastlzma2, and xz. LZMA has a very good compression ratio, 22.96% for fastlzma2. On the other hand, zstd decompresses 8.2 times as fast, and only slightly sacrifices compression ratio, at 24.88%.

The upshot being that if you were downloading a file which was 1 GB uncompressed, you'd download about 19 MB more if it was compressed with zstd rather than LZMA, but it would decompress much faster (or use less CPU if decompression isn't what's bottlenecking).

[Blank]

Originally posted by William Shakesman:

In addition, it is inherently silly to assume Steam can serve infinite bandwidth to any location.

I know that steam cant server infinity bandwidth but im speaking about my local cdn server. I host an cdn server in my house for many PCs but im bottlenecked by the cpu performance sadly because of the poor decompression performance.

[Blank]

Originally posted by cSg|mc-Hotsauce:

Originally posted by William Shakesman:

In addition, it is inherently silly to assume Steam can serve infinite bandwidth to any location.

What do you think is the maximum theoretical download speed of Steam's servers?

:cool_seagull:

https://www.peeringdb.com/net/4782