Steam is using the wrong hard drive for caching downloads

[󠀡󠀡]

I have a lot of games installed and have a Steam games folder on several hard drives.

One is one a fast SSD, which is where I put the newest and most demanding games.
I noticed that the recent Cyberpunk 2077 patch was taking a long time to install. Looking at the Task Manager and Activity Monitor I pinpointed the issue: Steam was downloading and installing the patch from one of my slow hard drives, which is maxing out at 100%.

Is there a way to set which steamapps\downloading folder Steam uses?

Removing Steam folders on slow hard drives is not an option.

Edit: I believe I know why this is happening. I had about 55 GB free on the SSD. The way Steam patching seems to work is that it sends a difference between your files and the new version. It sends not just the files that changed, but the difference inside of those files. To re-create the patched games it needs temporary space equal to the size of the game. There wasn't enough space on my SDD for another copy of the game so it used a different folder.

tl;dr 3 gb Cyberpunk 2077 patch needs 60 gb of temporary storage.

[Glimmer]

Originally posted by quaixor:

Is there a way to set which steamapps\downloading folder Steam uses?

Not officially. You could move the download folder to the desired drive and then create a Junction pointing back to the original location
https://www.addictivetips.com/windows-tips/create-delete-a-junction-link-on-windows-10/

[Solaron]

This just happened to me with the new Cyberpunk 2077 1.21 patch. I noticed the patch was taking forever to install after a swift download. Without notifying me, Steam decided to use my slowest storage hard drive in my computer as a cache buffer instead of asking me if I wanted to free up space on my ultra fast Samsung 970 Evo Plus 1TB first.

The patch is taking forever to install as a result, and I don't want to cancel it for fear of corrupting my game. I've never seen the words "Committing" and "Finalizing" for a patch update before and it's taking forever.

[rawWwRrr]

Originally posted by Solaron:

Without notifying me, Steam decided to use my slowest storage hard drive in my computer as a cache buffer instead of asking me if I wanted to free up space on my ultra fast Samsung 970 Evo Plus 1TB first..

First, Steam has never promoted a user for a destination of download cache.

If you're running that lean on space on the SSD, that's a you problem. You should never attempt to completely fill a hard drive. You always need to allow some space, the perfect example for why is exactly what you're trying to perform: an update. The update process is probably patching files, which means a read/write/replace process where you need that extra space.

Your update isn't slow because the files downloaded to your slower spindle drive. It's slow because there isn't sufficient space on the SSD.

[celerun]

Originally posted by rawWwRrr:

Originally posted by Solaron:

Without notifying me, Steam decided to use my slowest storage hard drive in my computer as a cache buffer instead of asking me if I wanted to free up space on my ultra fast Samsung 970 Evo Plus 1TB first..

First, Steam has never promoted a user for a destination of download cache.

If you're running that lean on space on the SSD, that's a you problem. You should never attempt to completely fill a hard drive. You always need to allow some space, the perfect example for why is exactly what you're trying to perform: an update. The update process is probably patching files, which means a read/write/replace process where you need that extra space.

Your update isn't slow because the files downloaded to your slower spindle drive. It's slow because there isn't sufficient space on the SSD.

This is just not true.

I just found this thread from googling this "error" and right now my slowest harddrive is being used to install 40gb worth of DLC for Hitman 2, which is installed on my nvme 1 tb harddrive with currently over 500 GB available.

[crunchyfrog]

Originally posted by celerun:

Originally posted by rawWwRrr:

First, Steam has never promoted a user for a destination of download cache.

If you're running that lean on space on the SSD, that's a you problem. You should never attempt to completely fill a hard drive. You always need to allow some space, the perfect example for why is exactly what you're trying to perform: an update. The update process is probably patching files, which means a read/write/replace process where you need that extra space.

Your update isn't slow because the files downloaded to your slower spindle drive. It's slow because there isn't sufficient space on the SSD.

This is just not true.

I just found this thread from googling this "error" and right now my slowest harddrive is being used to install 40gb worth of DLC for Hitman 2, which is installed on my nvme 1 tb harddrive with currently over 500 GB available.

Actually it IS true.

Steam works like this.

Your installed drive(s)) for Steam should never go over 90% usage or it goes wonky. This can be not downloading, downloading slowly, or stopping and starting or even downloading on the wrong drive.

Fruthermore when you download all games are both encrypted and compressed so any game file you must allow up to THREE times the total file size.

So for example, if you have a 1TB hard drive, and you've used 800GB that's 80% used so you have 100GB (or10%) left. But if you try to download an 80GB game you won't have enough space as you'd need to allow THREE times that space, putting you way beyind the 90% mark.

So you MUST keep your drive clear to these rules or that IS a you problem.

[celerun]

40GB x 3 is 120, not 521GB which is currently available on the NVME destination drive.

The drive it chooses to cache this 40 DLC download (my slow HDD) has even less space available (382GB) but is way bigger in total size (4TB).

It seems that Steam chooses this drive because of total amount of storage capacity . Not storage availability .

If what you say is true, it would indeed utilize the space on the NVME instead of what it is currently doing.

So no, it is just not true.

[󠀡󠀡]

Originally posted by celerun:

40GB x 3 is 120, not 521GB which is currently available on the NVME destination drive.

The drive it chooses to cache this 40 DLC download (my slow HDD) has even less space available (382GB) but is way bigger in total size (4TB).

It seems that Steam chooses this drive because of total amount of storage capacity . Not storage availability .

If what you say is true, it would indeed utilize the space on the NVME instead of what it is currently doing.

So no, it is just not true.

This has happened to me a few times since I've made the post. I believe I have a solution. I tried it just now with a game and it worked, but given that I haven't tested it extensively, proceed at your own risk.

The idea is that we don't want Steam to use our slow drives for temporary downloads. To do this we trick Steam into thinking the folder it's using is on the slow drive while it's actually on the fast drive.

I have three slow drives with Steam folders: Drives D, F and G. For each one:

I made a new folder on my SSD. This will be where the files are downloaded when Steam tries to write to the slow drive. Here is where I put them but you can put them anywhere on the fast drive:

M:\SteamGames\steamapps\tempOnD
M:\SteamGames\steamapps\downloadingOnD

Then delete the Steam temp and downloading folders on your slow drive.
After that open elevated command prompt, cd into the steamapps folder on the slow drive, and make a symlink:

mklink /D downloading "M:\SteamGames\steamapps\downloadingOnD"
mklink /D temp "M:\SteamGames\steamapps\tempOnD"

Do this for each slow drive with a Steam folder. I recommend creating a new set of folders on the SSD for each slow drive.

Just make sure that your SSD has at least double the size of the game you're trying to patch as free space or your game patching/downloading will fail.

Note that while I know this needs to be done with the downloading folder--that's where newly downloaded games land--I'm not sure about the temp folder. I did it with the temp folder as well just in case. I don't want to ever deal with this issue again.

As an added benefit of this workaround, if you have games on the slow drives, they will use the SSD for downloading and patching. This could greatly speed up certain parts of the process since Steam is no longer reading and writing from the same drive (linear writes to HDD can be 10 to 20x faster than random access ones).

[celerun]

Originally posted by quaixor:

Originally posted by celerun:

40GB x 3 is 120, not 521GB which is currently available on the NVME destination drive.

The drive it chooses to cache this 40 DLC download (my slow HDD) has even less space available (382GB) but is way bigger in total size (4TB).

It seems that Steam chooses this drive because of total amount of storage capacity . Not storage availability .

If what you say is true, it would indeed utilize the space on the NVME instead of what it is currently doing.

So no, it is just not true.

This has happened to me a few times since I've made the post. I believe I have a solution. I tried it just now with a game and it worked, but given that I haven't tested it extensively, proceed at your own risk.

The idea is that we don't want Steam to use our slow drives for temporary downloads. To do this we trick Steam into thinking the folder it's using is on the slow drive while it's actually on the fast drive.

I have three slow drives with Steam folders: Drives D, F and G. For each one:

I made a new folder on my SSD. This will be where the files are downloaded when Steam tries to write to the slow drive. Here is where I put them but you can put them anywhere on the fast drive:

M:\SteamGames\steamapps\tempOnD
M:\SteamGames\steamapps\downloadingOnD

Then delete the Steam temp and downloading folders on your slow drive.
After that open elevated command prompt, cd into the steamapps folder on the slow drive, and make a symlink:

mklink /D downloading "M:\SteamGames\steamapps\downloadingOnD"
mklink /D temp "M:\SteamGames\steamapps\tempOnD"

Do this for each slow drive with a Steam folder. I recommend creating a new set of folders on the SSD for each slow drive.

Just make sure that your SSD has at least double the size of the game you're trying to patch as free space or your game patching/downloading will fail.

Note that while I know this needs to be done with the downloading folder--that's where newly downloaded games land--I'm not sure about the temp folder. I did it with the temp folder as well just in case. I don't want to ever deal with this issue again.

As an added benefit of this workaround, if you have games on the slow drives, they will use the SSD for downloading and patching. This could greatly speed up certain parts of the process since Steam is no longer reading and writing from the same drive (linear writes to HDD can be 10 to 20x faster than random access ones).

You sir are a gentleman and a scholar

[crunchyfrog]

Originally posted by celerun:

40GB x 3 is 120, not 521GB which is currently available on the NVME destination drive.

The drive it chooses to cache this 40 DLC download (my slow HDD) has even less space available (382GB) but is way bigger in total size (4TB).

It seems that Steam chooses this drive because of total amount of storage capacity . Not storage availability .

If what you say is true, it would indeed utilize the space on the NVME instead of what it is currently doing.

So no, it is just not true.

What I've said in my description is absolutely true. It's taken directyl from Valve's mouth.

That IS how it works, and the examples I gave of how it goes wonky when you run out of space is also correct too.

[Rockenrooster]

Originally posted by crunchyfrog:

Originally posted by celerun:

40GB x 3 is 120, not 521GB which is currently available on the NVME destination drive.

The drive it chooses to cache this 40 DLC download (my slow HDD) has even less space available (382GB) but is way bigger in total size (4TB).

It seems that Steam chooses this drive because of total amount of storage capacity . Not storage availability .

If what you say is true, it would indeed utilize the space on the NVME instead of what it is currently doing.

So no, it is just not true.

What I've said in my description is absolutely true. It's taken directyl from Valve's mouth.

That IS how it works, and the examples I gave of how it goes wonky when you run out of space is also correct too.

You can be totally correct and wrong at the same time.

There is how it SHOULD work for the end user, and then there is how it ENDS UP working for the end user. Anyone in software development knows this.

We have no idea what Valve's solution is designed around (PC configs), and since there are virtually infinite combinations of PC hardware, there are weird exceptions like this where it doesn't work as intended.

I have the same issue.
When ARK updates, it uses my 1TB HDD that has less free space than my 1TB nvme 4.0 SSD, and takes forever to update. It SHOULD be using my SSD since it has more free space, but it doesn't. However, I also have 4 different drives in my config 1TB SSD, 500GB HDD, 1TB HDD, 4TB HDD. Why wouldn't it use the 4TB HDD??? Its got more than 1TB free and is faster than my 1TB HDD???

Designing software to handle so many different configs I imagine is a complete nightmare lol!!!

[Rockenrooster]

take a look at this: https://steamcommunity.com/app/346110/discussions/0/1748980761793517929/?ctp=2
This might fix it, I just tried it, set my staging folder to 0

[crunchyfrog]

Originally posted by Rockenrooster:

Originally posted by crunchyfrog:

What I've said in my description is absolutely true. It's taken directyl from Valve's mouth.

That IS how it works, and the examples I gave of how it goes wonky when you run out of space is also correct too.

You can be totally correct and wrong at the same time.

There is how it SHOULD work for the end user, and then there is how it ENDS UP working for the end user. Anyone in software development knows this.

We have no idea what Valve's solution is designed around (PC configs), and since there are virtually infinite combinations of PC hardware, there are weird exceptions like this where it doesn't work as intended.

I have the same issue.
When ARK updates, it uses my 1TB HDD that has less free space than my 1TB nvme 4.0 SSD, and takes forever to update. It SHOULD be using my SSD since it has more free space, but it doesn't. However, I also have 4 different drives in my config 1TB SSD, 500GB HDD, 1TB HDD, 4TB HDD. Why wouldn't it use the 4TB HDD??? Its got more than 1TB free and is faster than my 1TB HDD???

Designing software to handle so many different configs I imagine is a complete nightmare lol!!!

Well as I said all I'm doing is repeating what they said. It ain't me that's wrong and right. :)

[Drifter]

Well that sounds like a you problem to me Crunchyfrog.

Can't believe they still don't allow you to preference a hdd for download cache.

[crunchyfrog]

Originally posted by Drifter:

Well that sounds like a you problem to me Crunchyfrog.

Can't believe they still don't allow you to preference a hdd for download cache.

How can it possibly be a you problem when this does in no way apply to me?

I'm advising of how it works.

Where does thsi even remotely say anything about it applying to me? That's not how logic works.

[celerun]

What I've done to "fix this" is to just have my largest HDD function as a storage-only and for (large) games I don't want to keep downloading.
Then I removed the steamlibrary folder, so steam has no affiliation with it.

When I want to play them I just copy it into a steamlibrary on my nvme. This works well with games I want but not always play. Arma3, squad, RDR2 etc, which are all +100GB

This has, of course, fixed the problem with steam using that harddrive at all, but with a small compromise and one I can live with.
It works better for me that way, since I have a lot of games installed and I got really tired of the amount of time it took on every single update..