Possible major bug in storage subsystem

[TooMeeK]

Hello, please review these threads for possible major bug in Steam storage subsystem:
http://steamcommunity.com/app/8190/discussions/0/558751364332665078/
http://steamcommunity.com/app/65740/discussions/0/864973981198247582/
http://steamcommunity.com/app/55230/discussions/0/540736965826805570/
Confirmed affected game list is:
Just Cause 2
Saints Row 2
Saints Row: The Third
Carrier Command: Gaea Mission
all in Steam version. All affected by "sector size" problem.
Moving them around to different volume fixes the problem.

Thank You,
TooMeeK

[TeKraken]

If you want someone from valve to see this you'd be better off opening a support ticket.

[TooMeeK]

I've got no response so far..

[Black Blade]

Originally posted by TooMeeK:

I've got no response so far..

support takes time you will just have to wait for it

[aiusepsi]

That's intensely weird. I'm not sure it's anything to do with Steam, though. Steam just puts the files onto disk, it's the OS's responsibility to abstract away details like sector sizes. And Steam has nothing to do with reading the files from the disk when the game uses them.

I'd be tempted to say driver bug or weird OS bug rather than Steam bug. Wouldn't be surprised if all the affected drives have the same manufacturer.

[TooMeeK]

It seems that Steam doesn't recognize my raid volume correctly or NTFS filesystem on it.
I saw other posts about problems with different games with similar symptoms, without solution.
The point is that.. not all games from Steam are failing during load from this volume.
(That's the way I discovered this issue so late, I was thinking it's my hardware fault with one or 2 games)
And each has different symptom eg.
Saints Row: The Third - 2 intro movies plays fine, then crash (saw this somewhere on forums, no solution)
Saints Row 2 - stuck at LOADING 5% (should load to 100% then menu)
Carrier Command: Gaea Mission - doesn't even start, it creates error log instead in Steam..\Userdata directory with problem pointing to "could not read config" (I saw this also on forum, no solutions)
and of course Just Cause 2 - just logo rolling...

These drives are ST1000DM003 in RAID0, so 4kB sector disks.
However, I also have single 2TB drive with 4kB sectors which run these games perfectly fine (tested).

If this is driver bug then why I can play without problem from single drive attached to same RAID controller?

Because there is still NO ANSWER from support I'm going to make couple tests including remove RAID volume then try starting games from each of these disks (I'll take NTFS info details) and then again make it as RAID 0 and format with default NTFS options.

[TeKraken]

Support will probably point you to the support teams of the devs/publishers of those games as they dont do support for other peoples games.

[TooMeeK]

Dear all,

I have isolated the issue and it can be esily recreated.
All games work without problems from each of these drives in single mode.
However, with array..

To create issue:

1. Go to RAIDXpert management console at:
http://localhost:25902/
2. Create array with any level (checked on mirror and stripe - 0, 1)
3. Set "Sector Size" to "4 KB" or larger, leave other default settings (including round to GB and max size).
Comment: larger sector size is better for large files, which Steam contains a lot.
4. Create default NTFS filesystem on it
5. Copy Steam folder to new volume
6. Start one of affected games

Affected volume will return:
fsutil fsinfo ntfsinfo d:
NTFS Volume Serial Number : 0xSomeLongNumber
NTFS Version : 3.1
LFS Version : 2.0
Number Sectors : 0x000000000e8d47ff
Total Clusters : 0x000000000e8d47ff
Free Clusters : 0x000000000e150743
Total Reserved : 0x0000000000000000
Bytes Per Sector : 4096
Bytes Per Physical Sector : 4096
Bytes Per Cluster : 4096
Bytes Per FileRecord Segment : 4096
Clusters Per FileRecord Segment : 1
Mft Valid Data Length : 0x0000000001000000
Mft Start Lcn : 0x00000000000c0000
Mft2 Start Lcn : 0x0000000000000002
Mft Zone Start : 0x00000000000c0000
Mft Zone End : 0x00000000000cc820
Resource Manager Identifier : SOME-ID

or

fsutil fsinfo ntfsinfo t:
NTFS Volume Serial Number : 0xSomeLongNumber
NTFS Version : 3.1
LFS Version : 2.0
Number Sectors : 0x00000000009ffcff
Total Clusters : 0x00000000004ffe7f
Free Clusters : 0x00000000004fd318
Total Reserved : 0x0000000000000000
Bytes Per Sector : 4096
Bytes Per Physical Sector : 4096
Bytes Per Cluster : 8192
Bytes Per FileRecord Segment : 4096
Clusters Per FileRecord Segment : 0
Mft Valid Data Length : 0x0000000000100000
Mft Start Lcn : 0x0000000000060000
Mft2 Start Lcn : 0x0000000000000001
Mft Zone Start : 0x0000000000060000
Mft Zone End : 0x0000000000066420
Resource Manager Identifier : SOME-ID

And these games no longer work..

Affected games:
Just Cause 2
Saints Row 2
Saints Row: The Third
Carrier Command: Gaea Mission
Affected OS:
Windows 7
Windows 8
and probably more.
Affected RAID controllers:
AMD SB850
and probably more
Affected software:
FreeNAS
iSCSI volume with 4096 or larger Logical Block Size
and probably more
Not affected software:
Windows software RAID

WORAROUND:
Use 512 bytes compatibility sector size for any volume until this issue is fixed by Valve Support.


Hope You enjoyed...

[TooMeeK]

About 2 weeks later..
stable versions:
Apr 23 2014 13:50
API 015
Jun 5 2014 6:28
API v015
and beta:
May 29 2014 9:25
API v015
are still affected.

Since Valve Support has already closed my ticked ETA is unknown.
Hope You guys are not running large arrays for Your games..

Another game has been added to affected list:
Ghost Recon Online (any version)

[aiusepsi]

I'd be pretty willing to bet that it's not a Steam bug.

[TooMeeK]

So test for yourself:
1. download Oracle VirtualBox
2. download FreeNAS
3. deploy FreeNAS
4. share iSCSI disk
5. use sector larger than default 512 bytes
6. connect to iSCSI target
7. copy Steam folder to it (or install fresh new then restore games from backup)

And You will see results (on affected games).
This means multiple array types are affected. And it is Steam engine bug.

[aiusepsi]

I'm not saying not a bug, I'm saying not a Steam bug.

1) Steam doesn't intermediate games accessing their own files in any way. Steam just does the downloading. Games talk straight to the OS to read their files.

2) The entire point of having a filesystem is to abstract away details of the underlying storage hardware. That's why reading a file from a flash drive works the same as a rotational HD works the same as a DVD. If something like sector size causes a problem, it means somebody screwed up implementing the filesystem. If you're only seeing a problem with FreeNAS, perhaps that's the culprit?

I'd love to test for myself, but I only have a Mac to hand, and I think all the games you list are Windows-only.

[TooMeeK]

None of following updates fixed storage problem:
[+] 30 Czerwca - Steam Client Beta Update - June 30
[+] 27 Czerwca - Steam Client Beta Update - June 26th
[+] 25 Czerwca - Steam Client Beta Update - June 25th
[+] 24 Czerwca - Steam Client Beta Update - June 23rd
[+] 20 Czerwca - Steam Client Beta Update - June 20th
[+] 10 Czerwca - Steam Client Beta Update - June 9th
Running beta version Jun 30 2014 API v015 and still cannot play games from 4kB volume, no problem from 512 bytes volume.
Since more people use modern NAS servers at their homes to store games data I assume issue rate will grow soon.
I'll keep posting updates until this issue is solved..

[Τhe Rolling Cheese Wheel]

Originally posted by TooMeeK:

Since Valve Support has already closed my ticked ETA is unknown.
Hope You guys are not running large arrays for Your games..

Good thing most people just run games off a single hard drive with no fancy/rediculous setup.

[MA☝Omgwtfbbqstfu™]

Use verify game data for steam to refind the files. many things seem to break steam library so i can't find the game anymore... I canceled a backup session and that broke two of my game installs once, fixed by revalidating, and it redownloaded some files for some reason...

and yea the os abstracts all that file access, apps lijke steam have no low level access.

[Satoru]

http://support.microsoft.com/kb/2510009
It sounsd more like your RAID array is screwing the translation up especially since you already have a 4K drive that works. ergo its' not a 'steam' problem. its a RAID driver issue.

As otehrs have noted the file-subsystem and the application are abstracted from the drives. The only factor in play here is the RAID sub-system which you already admit that the raid0 config breaks but not the single drive config. Ergo the raid config is breaking the 4k emulation.