Unaccessible space?

[Xion]

So I have a 4TB WD Black HDD dedicated to my Steam Library, GameCube ISOs (of games I own), and movies (I also own). A year ago my Steam Library finally grew to a point it's now too large for the 4TB drive. I'm intent on upgrading, but recently I went ahead and flagged the entire drive for compression (it and my PC are very fast, so the performance loss is minimal). 2.8TB of used volume went down to 2.2TB.

Now here's where things get weird. I began redownloading and installing the overflow of my library and after queuing everything up Steam and my HDD reported I should've had nearly 500GB of space left after everything finished. So I left it alone for a couple days to download (a bit more than 1TB throttled to 24Mbps takes a while). After day one I noticed that the free space on my HDD had shrunk by about half. After day two is was clearly going to cap before it finished. I figured some inaccuracy in Steam's volume calculation, but ~40% inaccurate?

So I launched SpaceSniffer and scanned the drive to find a grey zone of "Unaccessible space" totaling nearly 500GB, and no other identifiable information. I then tried other programs to detect the space, but most wouldn't even recognize it existed. I tried searching for possible causes with all (found) potential means of identifying the space failing. I even compared folder volumes as reported in OS and in the various volume graphing programs. Either the "Unaccessible space" was detected (rarely) or not even recognized, and the drive was reporting less than it's factory capacity of 3.6TB effective (the variance of 1000xB marketed vs 1024xB actual; not getting into this wall of text). I then began uninstalling the largest games from my Steam Library in order by volume. Some specific games like Hitman (the newest episodic one), and Quantum Break, but not Deus Ex: Mankind Divided (so it's likely not DX12 related) reduced the volume of the "Unaccessible space" by more than their install size. Prior to compressing the drive I had used SpaceSniffer before to find the games taking up the most space and selectively uninstalled them to free up space for others. There was no "Unaccessible space" volume in those scans.

It stands to reason that this is somehow related to compressing the drive, but why some games are effectively using more than double their install volume baffles me. Why that excess volume is considered "Unaccessible space" just compounds this mystery. Why only some specific games, and not all of them, is the icing on this double-layer cake. Would anyone who reads this happen to legitimately know what is actually going on here?

EDIT: Additional information in reply #3

[TehSpoopyKitteh]

The "inaccessible" space you've seen is very likely the compressed data on the hard disk. The data speed cap is probably because each time your PC has to access the compressed part of the hard disk, it has to decompress it.

[Carlsberg]

Disk compression utilities do not increase the amount of storage space you have, they increase the amount of data the disk can hold, two different things. The compression utility works independantly from the OS and needs room to operate, to compress files before being put to storage and to uncompress files for access and will reserve the space to do so, no mystery.

One thing to consider with that is HDD failure, can you get that information back without the software used to compress them?

[Xion]

Thank you both for your contributions.

The data speed cap is manually imposed. I've set a manual limit in Steam for 24Mbps since I have a 40Mbps service. No caps though. Was an off-topic point to illustrate why it would take me 2+ days to download over 1TB of games.

I am aware compression doesn't increase storage space; nor did I imply it in any context. To suggest otherwise would be an insult to one's intelligence. I know how compression works as implied in the OP. Not the point at all.

I was not using anything but Windows' internal disk compression functionallity. Having mentioned I "flagged the entire drive for compression" implies this explicitly. On top of that the "Unaccessible space" would decrease in size only when I uninstalled specific games; not all - reinforcing that explicit implication. Those games that do reduce the "Unaccessible space" when uninstalled reduce it (adding it to free space) by differing amounts per game; some releasing more than the install volume of the game either with or without compression. Decompression through Windows Drive Compression is done in RAM as needed; first being accessed by the CPU for decompression before loading into RAM. Yes true compression like .zip and .rar would typically need to be decomressed onto disk before being processed (and do get decompressed into temp folders), but that's done on the OS drive unless you've taken the time to move your temp and caching folders.

I've since unchecked "compress this drive to save space" for said 4TB drive. The "Unaccessible space" is now effectively gone as expected. More to the point I just wanted to know why some specific games (since not all do it) when compressed will eat up not only more space than Steam reports, but also more than they would without compression. I get that the compression is what's causing the "Unaccessible space" to appear. What I don't get is why only specific games are generating it under compression, and effectively using more space compressed than uncompressed. That and why are they also falsely reporting their volume to generate this "Unaccessible space"?

EDIT: Case in point would be the afore mentioned Hitman and Deus Ex: Mankind Divided. Hitman uses ~55GB reported space when the drive is compressed, and another 80GB of "Unaccessible space." However it uses only 70GB uncompressed. Deus Ex: Mankind Divided on the other hand uses ~50GB compressed, and no "Unaccessible space" as far as SpaceSniffer can show. Also scaling the same as Hitman with about 65GB installed uncompressed.