How to read BSP and MDL?

[Edward]

Hello.
I'm part of a SFM Animation Studio, and with so many people working on the same DMX file, there are times when not everyone has all the models or textures. So I thought I'd help them out a bit by programming a "DMX Dependency Checker", but it seems like I can't quite read the maps and models properly.

BSP
According to the documentation of a map file, if I want to get a hold of the embedded files in a BSP (pakrat) I should jump to "lump 40" (40*16 bytes after the initial 8), get the offset and size of this "lump", jump to "offset bytes" and just start reading until I reach offset+size bytes.

My goal here is to get the file names so that I know what textures or models are already included in the map. So, I ask "Are the next 5 bytes 50 4b 03 04 0a?" This should tell me that there is an uncompressed ZIP here, and so I jump to about 17 bytes, read the next 4 bytes to get the file size, then the next 4 bytes to get the name length, and store that name for later comparison. Then jump past file and ask for the "50 4b 03 04 0a" again.

Next, I jump to "lump 22" (or one of its 4 copies starting at 22*16 bytes) to get the model names. The "lump index" does not say any size or number of items so I collect the number of items just after the "prps" headers. Then proceed to read the model names, first seeing if they exist in the map from the pak names I stored, then checking to see if they exist in SourceFilmmaker\game.

Next, I jump to "lump 43" (43*16 bytes) which is according to the documentation Texture name data. These seem to be "null terminated strings" containing the path and file name of textures used. I simply read until I reach the size specified in the "lump size". Once I got all the texture names I see if they were found in the map under pak, then see if they exist in SourceFilmmaker\game.

However, it seems as though a couple of textures don't seem to exist, but I have a feeling they are internal texture names. Scanning a scene that uses "arena_granary.bsp", the material "materials\maps\arena_granary\water\water_well_-1947_1534_-391.vmt" is marked as non-existant. Another one that uses "cp_well.bsp" has a good number of these "materials\maps\cp_well\water\water_well_0_1696_-480.vmt", "materials\maps\cp_well\tile\floor_tile_001a_-334_-3750_-40.vmt" and "materials\maps\cp_well\glass\glasswindow001a_-1157_3834_-456.vmt" marked as missing. Did I read the map file too far here?

MDL
According to the documentation of a model file, if I want to find out which textures are used, I should jump 204 bytes into the file, read the number of textures used followed by offset, then the number of directories it should look in and its offset.

However, jumping to these offsets in the file I do not see any text of that nature. I do, however, see these names at the very bottom of the file. So, I proceed to read the file backwards, and at the first occurrence of a non-null byte, start counting: number of directories, number of textures.

However, the texture count in the beginning seems to be off, and it varies from model to model. In one model it would say 19 textures, but there are only 18 entries at the bottom, making me read in a face flex name or bodygroup name. Another would say 16 textures when there are only 14, making me read in 2 face flex names. How do these models actually work?

(programming in a form of VB)

[Jesbus]

Trying to read the prps game lump right now.
It's a shame that there is so little documentation on these file formats

[ZeqMacaw]

Look at Crowbar's source code to learn how to read the various versions of an MDL file.
Look at BSPSource source code for BSP files.

Both tools have their source code on GitHub.