Help figurine out non-steam data

[giolove]

So to Start with, I already have disk usage analyzer installed and it has been helpful in the past but now after a while it hasn't been anymore. I have almost 26 GBs of storage taken from non-steam and it isn't the shader and compat data (I already deleted most of the one's that was already there but it didn't make a difference). I would appreciate any help or solutions in what certain data is actually taking up my internal drive as the analyzer application doesn't convey what it what.

[ReBoot]

It's the OS & everything you've stored with OS means.

[nicoper]

Do you have any games that use their own launchers to download game data after you launch them from Steam?
The storage manager in the Steam client only counts data downloaded from Steam, not anything put into the folder afterwards so that's a probable cause.

The OS is, of course, also going to take up a chunk of your storage, and perhaps more than you'd expect since it's A/B partitioned.

[giolove]

Първоначално публикувано от nicoper:

Do you have any games that use their own launchers to download game data after you launch them from Steam?
The storage manager in the Steam client only counts data downloaded from Steam, not anything put into the folder afterwards so that's a probable cause.

The OS is, of course, also going to take up a chunk of your storage, and perhaps more than you'd expect since it's A/B partitioned.

From what I know of not at all (unless God of war 2018 counts), the only game that had a launcher in the game was jedi fallen order and even then I had it uninstalled a year ago now after beating it. One thing I did noticed on the steam deck settings in storage is that I have multiple verisons steam protons installed on my internal drive so maybe that could be taking a bit storage too?? I'm not too familiar with proton so I don't wanna mess with that atm

[WarnerCK]

Първоначално публикувано от giolove:

One thing I did noticed on the steam deck settings in storage is that I have multiple verisons steam protons installed on my internal drive so maybe that could be taking a bit storage too?? I'm not too familiar with proton so I don't wanna mess with that atm

Each version of Proton that you have installed is about 1 GB.

[giolove]

Първоначално публикувано от WarnerCK:

Първоначално публикувано от giolove:

One thing I did noticed on the steam deck settings in storage is that I have multiple verisons steam protons installed on my internal drive so maybe that could be taking a bit storage too?? I'm not too familiar with proton so I don't wanna mess with that atm

Each version of Proton that you have installed is about 1 GB.

Oooh okay gotcha, so it won't even make a difference then if I delete some older versions lol. Gonna try to look deeper into the files and hopefully find the main culprit soon enough. I appreciate some answers and ideas so far!

[nicoper]

Първоначално публикувано от giolove:

Първоначално публикувано от WarnerCK:

Each version of Proton that you have installed is about 1 GB.

Oooh okay gotcha, so it won't even make a difference then if I delete some older versions lol. Gonna try to look deeper into the files and hopefully find the main culprit soon enough. I appreciate some answers and ideas so far!

You also don't really want to delete official Proton versions since specific games might be set by Valve to use particular ones based on their internal testing.
If you've installed Proton-GE there's not much reason to keep older versions of that though.

[giolove]

Aaah okay I see, definitely will check that then to see!

[giolove]

Okay so quick update I might have accidentally found a solution maybe: there's a settings bar for separate games before your start it up and there is a developer section with an option that reads "delete proton files". Is this a another way to also delete shader/cache files from a game if I uninstalled it or can delete junk data that clogs the storage that I can't fully find with the data usage analyzer or should I not mess with it as it does nothing for that front?

[nicoper]

Първоначално публикувано от giolove:

Okay so quick update I might have accidentally found a solution maybe: there's a settings bar for separate games before your start it up and there is a developer section with an option that reads "delete proton files". Is this a another way to also delete shader/cache files from a game if I uninstalled it or can delete junk data that clogs the storage that I can't fully find with the data usage analyzer or should I not mess with it as it does nothing for that front?

I believe that button effectively nukes the game's Proton prefix, aka the compatdata folder. Should be removed automatically if you uninstall a game but I believe there was a bug a couple years ago where that didn't happen so maybe you've got some really old garbage left?
Not sure what the best way to find those would be, maybe compare app id's of all compatdata folders to ones you've got installed? Could be done manually if you don't have a lot of games but otherwise you'd wanna script it to not lose sanity.

[Mahjik]

There is a Decky Loader plugin called "Storage Cleaner":

"Quickly visualize, select and clear shader cache and compatibility data."

[Pepe]

I think Steam has a bug showing whatever non-steam data is. I don't have a SteamDeck, but I run GNU/Linux. I have a fully dedicated disk for Steam.
Steam shows:
- Games ~300 GB
- DLCs ~9 GB
- Workshop ~400 MB
- Non-Steam 50 GB
- Free ~578 GB
- Total 937 GB

* note that GB for Valve is actually GiB (1 GiB = 1024 MiB).

Actual reliable system tools from GNU/Linux show the following:
1. df -h <partition_mount>
- Used ~312GiB
- Free ~579GiB
- Total 938GiB
2. du -sch <partition_mount_if_dedicated_partition_or_steam_library_path>
- Used ~312GiB

The difference between Steam and GNU/Linux tools for free and total is just about 1GiB, and so does the difference in used space (Games+DLCs+Workshop). 1GiB difference in the way they calculate the size is fine (let's say it's within an acceptable margin of error if they don't calculate the right value of size on disk).

The conclusion is there is no 50 GiB of used space anywhere on disk, so it must be some weird bogus math they do, or it means something from Steam that is calculated twice, once in Games, DLCs or Workshop, and then separately in this Non-Steam space.

Shader Pre-Caching is off on my setup, so I don't think it's that. I don't have Proton installed either.

Considering the Used (Games, DLCs or Workshop) space, Free space and Total space displayed by Steam is fine, I suggest just ignoring this bogus "Non-Steam" number.

You should be able to use the same tools from KDE Terminal on SteamDeck going in Desktop mode and see for yourself there's no space occupied with anything else. lsblk and mount can be useful commands to figure out your partitions layout and mount points.

[Pepe]

Mods, please don't close this topic for necroing it! Because of the award I received, I've actually came back to this topic and questioned my answer.

So, it's pretty clear that the occupied space, free space and the total are a match between Steam and the system tools. So everything is fine, right?

What I've missed is the difference between Used Space + Free Space and the Total Space that doesn't add up.

From Steam

Used (300+9+0)  = 309 GB
Free            = 578 GB
Used + Free     = 887 GB ( ! )
Total           = 937 GB ( ! )
! Difference !  =  50 GB

So it seems like the Non-Steam in my Games disk is some kind of phantom space that is not used or free, but somehow is adding up to the total.

Let's see if I get the same result on the system numbers.
From df -h

Used            = 312 GiB
Free            = 579 GiB
Used + Free     = 891 GiB ( ! )
Total           = 938 GiB ( ! )
! Difference !  =  47 GiB

What is this sorcery?!

The sorcery is quite simple magic. It's ext* file system's reserved space:

reserved-blocks-percentage
       ... the percentage of the file system which  may  only
       be allocated by privileged processes.   Reserving some
       number  of  file  system  blocks for use by privileged
       processes is done to avoid file system  fragmentation,
       and  to  allow  system daemons, such as syslogd(8), to
       continue to function  correctly  after  non-privileged
       processes  are prevented from writing to the file sys‐
       tem.  Normally, the  default  percentage  of  reserved
       blocks is 5%.

So, having some reserved free space is clearly beneficial to the system.

I've done a few tests on my system to see if that's right.

$ sudo tune2fs -l <games_partition> | grep -i 'block count'
Block count:              250046720
Reserved block count:     12502336
$ printf 'scale=4; 12502336/250046720\n' | bc 
.0500

# other partitions results...
.0499
.0499

So, it's true then.

Maybe 5% reserved space is a bit too much when you have 1, 2, 4, 6 TB of disk space, or multiple disks, one for system, others for data storage and so on. Maybe...

That being said, I would not recommend changing that on the SteamDeck.

If your really really must, don't wipe out the whole reserved space. Keep it to at least 1% (even though it is possible to set it under 1% with decimal numbers).

# set reserved space to 4%
$ sudo tune2fs -m 4 <games_partition>

# validate
$ printf 'scale=4; %s/%s\n' "$(sudo tune2fs -l <games_partition> | grep -i '^reserved block count' | grep -Eo '[0-9]+')" "$(sudo tune2fs -l <games_partition> | grep -i '^block count' | grep -Eo '[0-9]+')" | bc
.0399

# validate with df -h ...

There is also a way to set the reserved space not as a percentage, but as a fixed size of blocks, but I don't recommend it at all. It's simple math, but it may lead to mistakes. The percentage version is less headaches. Use that one.

If you have a new disk (SSD or SD) and you've just put it in and you want to set the reserved space when formatting the partition, you can do that too.

# format the partition to ext4 setting the reserved-blocks-percentage to 3
$ sudo mkfs.ext4 -m 3 <partition>

After posting this, I thought to do another check in Steam Settings > Storage. I do have another partition for Steam Games, but I've also kept the default /home storage. There, the Non-Steam value looks different. It's huge, way more than the file system reserved space. At a closer look, that's all the files in my /home partition (except Steam's) plus the file system reserved space.

To wrap it up:
0. I was wrong about Valve's math.
1. Non-Steam is everything that is not in Steam library on the partition + File System Reserved Space. Though, I'd say Valve doesn't do the math like that, but more like Total Partition Size - Used by Steam (Games + DLC + Workshop + Media) - Free Space
2. For system responsiveness you should keep the 5% mark of reserved space.
3. If you need to squeeze a little more space out of your storage, I advise doing so in 0.5% decrements: 4.5%, 4%, 3.5%, and so on until 1%. The more you limit the file system reserved space, the more chances for the system responsiveness to decrease in the long run.
4. Maybe Valve could create a "File System Reserved Space", aside "Non-Steam", to make it easier for people to understand where's the missing space.