Disable disk space preallocation for SSD write-back caches

[Vertex]

I would like an option to disable the preallocation of disk space when installing steam games. I use PrimoCache to create an SSD write-back cache for my HDD storage. Everytime I install a large steam game, the steam client floods and empties my cache.

If there's other concurrent write operations happening my system, it slow them down drastically. Unnecessary write operations like this also kill the nand flash cell on my SSD. When the level 1 ram cache is flooded, it immediately goes to the disk cache.

This feature is very outdated and is not relevant anymore for all use cases. Some users such as myself are writing to a 3TB drive and can feel confident there will almost always be space availible. Furthermore, most users do not have a cache on HDDs and it takes a ridiculously long time for large steam games to preallocate the space.

In all honestly, I'd rather the steam client throw an error if the write operation fails. Currently, it's breaking my disk configuration. This preallocation feature doesn't even work properly on my Windows 10 configuration. The task completes before any space is preallocated. My cache swallows the operation before it has a chance to contact the disk! The steam client doesn't even know if there's enough space to install the game.

Lastly, there is absoutely no performance gain for preallcoation on a SSD write-back cache. Overall there is actually a net loss becuase the procressing is handled by the caching algorithm. It takes anywhere from 30-90s longer to install the game. I have no loss of speed during a download. There is always sufficient resources for procressing.

[Levi]

This feature is very outdated and is not relevant anymore for all use cases.

Other than preventing further download issues later on, it allows that space to be reserved.

Giving non-savy users an option to disable pre-allocation, might as well cause chaos for them with all the future download issues.

Without pre-allocation, you could in theory run out of disk space during the patching process because you also decided to download a few 4k movies onto the same drive - which are very large.

Overall, no, i don't want to see pre-allocation deprecated.

[Vertex]

Robin3sk a écrit :

This feature is very outdated and is not relevant anymore for all use cases.

Other than preventing further download issues later on, it allows that space to be reserved.

Giving non-savy users an option to disable pre-allocation, might as well cause chaos for them with all the future download issues.

Without pre-allocation, you could in theory run out of disk space during the patching process because you also decided to download a few 4k movies onto the same drive - which are very large.

Overall, no, i don't want to see pre-allocation deprecated.

:yinyangflip:

That is exactly why Windows has dialog boxes that pop-up when clicking things. The user could be warned when checking the box off, about the pontential side-effects of disabling this feature.

The reality is people are going to be writing steam games to mechanical hard drives for a very long time. There is absoutely no point in pre-allocating with over a terabyte of space left! It takes a very long time to spin up the hard-drive and there's no point with virtually unlimited space left.

Perhaps, the option should be titled "Disable pre-allocation with 500gb of storage space left". I would definately want pre-allocation on my 500GB NVME SSD, but not my 3TB HDD.

[helswrath]

agreed. they need to kill pre-allocation. or give a disable option like the torrent clients do. you can rationalize why they should keep it all day long, but in the end it never outweighs why they should disable it. it kills SSD's, and a lot of people would appreciate shaving 5-10MINS from their installation times. tons of wasted space and complicates network installations so and so forth. ive read several threads on this already, and option to disable wins everytime. especially when it comes to preserving the life of your hardware, and saving time.

[chrcoluk]

OP's points are valid and I support his feature request, I would be amazed if it got implemented tho as new options been added to steam that are not related to marketing are rare now days.

[Plex]

there is a big myth that pre allocation does wear out SSD nand cells, but I don't think so.
All it does it create 0 byte files, its not actually writing big files.
the activity is just faked to make you think its actually writing big files full of zeroes but it does not do that.

Anyways, who buys TLC SSDs anyways? anyone who knows whats good buys SLC SSDs with at least 1TB.

[Vertex]

Plex a écrit :

Anyways, who buys TLC SSDs anyways? anyone who knows whats good buys SLC SSDs with at least 1TB.

Most people don't buy SLCs anymore. I have a 480GB NVME which is a TLC. It has Windows on it for all my software applications and a few games. It's so fast, everything is CPU bound now.

I use Primocache which writes to an SSD cache before hitting the HDD. These operations actually wear out the SSD. The task is cached for write-back, then the cells are erased each time.

[Katze]

So I'm probably not the average user with a 6×3TB Windows Storage Volume with parity, and the best solution would probably FreeBSD or GNU/Linux and VT-d, but seriously, I'm having 5/10 Terrabytes (or Tibibytes, who knows) of free diskspace and preallocation for a 60G game just takes ages.

Make it a command-line switch maybe?
No user can "accidentally" set that, but tech-savvy users can simply set that stuff and be done with it, cross-platform without issues (I guess?).

[est]

this needs to be the top.

[chrcoluk]

Plex a écrit :

there is a big myth that pre allocation does wear out SSD nand cells, but I don't think so.
All it does it create 0 byte files, its not actually writing big files.
the activity is just faked to make you think its actually writing big files full of zeroes but it does not do that.

Anyways, who buys TLC SSDs anyways? anyone who knows whats good buys SLC SSDs with at least 1TB.

you do realise a 0 is still data right?

Ironically earlier this week my expensive fancy 850 pro 512 gig died. As a bonus samsung sent it back claiming its ok. The drive by the media is considered virtually indestructible LOL.

I will probably sell whatever they eventually give me in as a swap. Currently was going to replace it with a 860 pro but given a 860 1tb evo has the same TBW as a 512 gig 860 pro and is the same price I think I will be jumping on to the TLC bandwagon even with my dead MLC drive. But of course things like browser cache, pagefile, and steam preallocate folder will be put on spindle.

[Hannibal]

It should be on by default for those less than pc savy but damn let us who know what we're doing turn it off. Its mind numbingly slow even for small updates. No other service requires it and world hasnt ended.

[Anyone999991]

its been more than an hour of waiting and still hasnt finished (at about 70%).
I have 900GB left on the drive I'm installing it on.
Yes please add option to disable it.

[Rafał]

I fully agree, there should be such option. Bah there should be option to turn ON allocation if there is even ahy reason I would want that in first place.

[Dave]

ᵡᴳḽᴰ::Vertex a écrit :

I would like an option to disable the preallocation of disk space when installing steam games. I use PrimoCache to create an SSD write-back cache for my HDD storage. Everytime I install a large steam game, the steam client floods and empties my cache.

If there's other concurrent write operations happening my system, it slow them down drastically. Unnecessary write operations like this also kill the nand flash cell on my SSD. When the level 1 ram cache is flooded, it immediately goes to the disk cache.

This feature is very outdated and is not relevant anymore for all use cases. Some users such as myself are writing to a 3TB drive and can feel confident there will almost always be space availible. Furthermore, most users do not have a cache on HDDs and it takes a ridiculously long time for large steam games to preallocate the space.

In all honestly, I'd rather the steam client throw an error if the write operation fails. Currently, it's breaking my disk configuration. This preallocation feature doesn't even work properly on my Windows 10 configuration. The task completes before any space is preallocated. My cache swallows the operation before it has a chance to contact the disk! The steam client doesn't even know if there's enough space to install the game.

Lastly, there is absoutely no performance gain for preallcoation on a SSD write-back cache. Overall there is actually a net loss becuase the procressing is handled by the caching algorithm. It takes anywhere from 30-90s longer to install the game. I have no loss of speed during a download. There is always sufficient resources for procressing.

So much misinformation...

Pre-allocation does NOT write data to every block where the game is going. It simply reserves the space in the File Allocation Table. It really doesn't write all that much data to the SSD

And if you're worried about too many writes to a SSD, then you have a terribly cheap SSD. Most SSDs can handle petabytes of data before wearing out.

.. you used to be able to turn pre-allocation off. It used to be an option in steam, but for some dumb reason, they removed it.

And no pre allocation will not "break your disk configuration" That's kind of laughable. I've had SDDs for years, and the pre allocation function doesn't cause any issues whatsoever.

A large enough write cache could swallow the operation, but that data WILL eventually be written, and of course steam knows how much space you have. That's a simple API call to windows.

And actually pre allocation does speed installation up... GREATLY on a HDD, and SLIGHTLY on a SDD. Since the space is pre allocated, that far less write operations needed to the NTFS file system. Pre allocation also limits fragmentation during installation (an issue for HDDs but not SDDs)

[RiO]

Plex a écrit :

there is a big myth that pre allocation does wear out SSD nand cells, but I don't think so.
All it does it create 0 byte files, its not actually writing big files.
the activity is just faked to make you think its actually writing big files full of zeroes but it does not do that.

If you use the proper APIs then pre-allocation essentially just sets aside space and is instant. But as pre-allocation is in fact not instant (and some users even get stuck with weird ~20m wait periods while a game pre-allocates), we can deduce that Valve is doing it in -- not the proper way.

TallBear a écrit :

And actually pre allocation does speed installation up... GREATLY on a HDD, and SLIGHTLY on a SDD. Since the space is pre allocated, that far less write operations needed to the NTFS file system. Pre allocation also limits fragmentation during installation (an issue for HDDs but not SDDs)

A well-sized in-memory write buffer paired with streamed writes to files created with an upfront known file size, performs equally well on both accounts. If not flat out better.
(It's effectively pre-allocating a file and then directly filling it, but doesn't have downloads waiting on the pre-allocation.)

It's one of the few cases where Steam should be allowed to - temporarily - consume in the GB range of RAM memory.

[chrcoluk]

TallBear a écrit :

ᵡᴳḽᴰ::Vertex a écrit :

I would like an option to disable the preallocation of disk space when installing steam games. I use PrimoCache to create an SSD write-back cache for my HDD storage. Everytime I install a large steam game, the steam client floods and empties my cache.

If there's other concurrent write operations happening my system, it slow them down drastically. Unnecessary write operations like this also kill the nand flash cell on my SSD. When the level 1 ram cache is flooded, it immediately goes to the disk cache.

This feature is very outdated and is not relevant anymore for all use cases. Some users such as myself are writing to a 3TB drive and can feel confident there will almost always be space availible. Furthermore, most users do not have a cache on HDDs and it takes a ridiculously long time for large steam games to preallocate the space.

In all honestly, I'd rather the steam client throw an error if the write operation fails. Currently, it's breaking my disk configuration. This preallocation feature doesn't even work properly on my Windows 10 configuration. The task completes before any space is preallocated. My cache swallows the operation before it has a chance to contact the disk! The steam client doesn't even know if there's enough space to install the game.

Lastly, there is absoutely no performance gain for preallcoation on a SSD write-back cache. Overall there is actually a net loss becuase the procressing is handled by the caching algorithm. It takes anywhere from 30-90s longer to install the game. I have no loss of speed during a download. There is always sufficient resources for procressing.

So much misinformation...

Pre-allocation does NOT write data to every block where the game is going. It simply reserves the space in the File Allocation Table. It really doesn't write all that much data to the SSD

And if you're worried about too many writes to a SSD, then you have a terribly cheap SSD. Most SSDs can handle petabytes of data before wearing out.

.. you used to be able to turn pre-allocation off. It used to be an option in steam, but for some dumb reason, they removed it.

And no pre allocation will not "break your disk configuration" That's kind of laughable. I've had SDDs for years, and the pre allocation function doesn't cause any issues whatsoever.

A large enough write cache could swallow the operation, but that data WILL eventually be written, and of course steam knows how much space you have. That's a simple API call to windows.

And actually pre allocation does speed installation up... GREATLY on a HDD, and SLIGHTLY on a SDD. Since the space is pre allocated, that far less write operations needed to the NTFS file system. Pre allocation also limits fragmentation during installation (an issue for HDDs but not SDDs)

if it only reserved in the FAT then it would be a lot quicker, sometimes for me preallocate takes longer than the download. It writes zeroes. So e.g. if the game is 40 gig, it will write 40 gig of zeroes before it even starts downloading.