Reinstate the download RAM cache system

[Sebastian Nielsen]

Before this "Disk Busy" system with the red/green light was implemented, Steam seemed to use a cache system which did cache everything in RAM and then writing it to disk in a steady nice speed, everything simultaniously so it could download and write to disk at the same time. When RAM got full, (if internet speed exceeds disk transfer speed). it did stop download for a while and then flush all to disk, to resume again.
This also was good for a standard rotating disk drive, since it did write everything steadly to disk which means the disk does not have to search for the location to place the files each time, instead it did write everything in one shot.

Now with this "Disk Busy" system, Steam seems to have removed the whole RAM caching system, so it instead writes the download directly to disk, or use a vastly smaller cache, so it results in very poor download speeds and "spikes"...
Like downloading - not downloading - downloading - not downloading and so on in small 5 second spikes.
It seems like steam no longer can write to disk and download simultaniously, so it must switch back and forth all the time, so now it does not matter if your internet speed is higher than disk speed or vice versa...

Why this change? The green light gets red and then it stops downloading, and then light gets green and it start again.
It get choppy, and as you know, a download is slower in the beginning due to network overhead and handshaking, so this system with very choppy downloads means much lower average speed.
This also means each time it stop downloading and start writing to disk, the disk has to search for the correct location, and then it start downloading and stop writing to disk - causing disk to "lose" its place until next chop of game data.

What was wrong with the RAM caching system? Why not let the user decide if this caused problems with RAM getting full for some users? Eg a setting "Use RAM cache for downloading" and then a textbox letting user decide how much RAM should set aside for downloading?

[Scutterman]

Well, I don't know any of the technical details, but the beta release notes for the indicator made it sound like it was purely informational. It allows people to see why a download may be pausing. If the "Disk busy" light is on, you know your system is trying to do something else on the disk. If it isn't, it could be a problem with the network. I haven't seen anything else that would suggest they changed the actual mechinism for downloading data.

Oh, and a handshake happens before the download starts, and has very low overhead. It's pretty much just the client saying "I'm here" and the server saying "I see you", and then the client can go on and request the download.

[Sebastian Nielsen]

Yes, but since the indicator added, or some period before, Steam did do a major change in the download system. It might also be that Steam now considers itself low-priority and cease to download if another application are requesting disk access to give the other application way.

Because before this system was added - eg rollback lets say 1 year back in timeline, the downloads were smooth with no spikes, regardless on how busy the disk was.

[Scutterman]

Steam updates are designed to happen in the background. That makes them low priority, just like (for example) torrenting. If you are doing disk-intensive things in the background and you don't want to stop those things while the download happens, then they are a higher priority to you.