ANNOUNCEMENT: Miasmod - Advanced Configuration & Modding Tool

[DarkStarSword]

I'm announcing the first release of Miasmod, the Miasmata Advanced Configuration & Modding Tool, as well as several sample mods to get you started!

This tool allows you to edit Miasmata'a saves.dat and environment files in arbitrary ways - you can edit the configuration variables that are not exposed through the menu or dig right into the environment file and tweak many aspects of the game itself.

--- Quick Start ---
You can download Miasmod from here:
https://github.com/DarkStarSword/miasmata-fixes/releases/tag/miasmod-v1.2.1

--- Sample Mods ---
There are sample mods available here:
https://github.com/DarkStarSword/miasmata-fixes/releases/download/miasmod-v1.0/miasmata-mods.zip

These mods have not been heavily play tested yet and could very easily ruin your experience playing the game. If you have not yet finished the game I would urge you to stick to just using the unofficial community patch for now, since that only fixes bugs in the original game and does not aim to alter the experience.

To use these mods, drop them in your Miasmata directory (the same place as Miasmata.exe and communitypatch.miasmod*), then run Miasmod. It will detect the mods and automatically create two new files - alocalmod.miasmod and alocalmod.rs5. If you later install/upgrade/remove any mods (or the community patch) you should re-run Miasmod and choose to "synchronise alocalmod.rs5".

* communitypatch.miasmod is not strictly required, but is highly recommended. Grab it from here:
http://steamcommunity.com/app/223510/discussions/0/648812916771880184/


- ailurophobia.miasmod removes the creature. I really really *really* recommend you play with the creature enabled and just learn how to deal with it effectively, but if you absolutely must remove the creature... well there you go, now you can. This has been playtested by my fiance with no creature showing up in her 15 day run (well, it did once with an earlier version of the mod - but after tweaking a few more things it never showed up again).

- cartographobia.miasmod prevents your map from being filled in when picking up notes and removes the effect of using the mental clarity tonic (I didn't remove the tonic itself - it just no longer does anything). You should start a new game for this one since it doesn't clear any map you have already filled in.

- dehydrated.miasmod disables the water jugs and only allows a single drink of water to be carried in the canteen at a time. This may need balance testing.

- insomnia.miasmod disables sleeping in beds and removes the tiredness rate so hopefully you won't die from sleep deprivation. This has not yet been tested for long enough to confirm that you can actually survive.

--- Modding HowTo ---
If you just want to experiment you can work with the "alocalmod.miasmod" file, which is opened by clicking "Open local environment..." or double clicking on it in the list. This will open the editor view of the environment that the game will use (including changes made by any other installed mods).

If you want to get rid of all your local changes, you should delete both alocalmod.miasmod and alocalmod.rs5 then re-run Miasmod (if you only delete one, Miasmod will re-create it from the other).

When you are ready to create an actual mod, click "New Mod..." and give it a name. Miasmod will open the editor view and you can make whatever changes you want. When editing a dedicated mod, the editor will NOT show any changes made by alocalmod.miasmod or other mods - only the changes that the specific mod you are editing will be shown. Whenever you save the mod, Miasmod will ask you if you want to synchronise alocalmod.rs5 - you should choose to synchronise it to make sure your changes are picked up by the game.

--- If You Fix A Bug... ---
Please let me know so I can add it to the unofficial community patch!

--- What are these *.miasmod files? ---
These files track the changes a mod has made compared to whatever it was based on (e.g. communitypatch.rs5/environment). Miasmod creates alocalmod.rs5 by taking a base environment file (the community patch if it is installed, otherwise the vanilla environment) and applying the changes from each installed miasmod file. These effectively contain the same information you see when clicking on "Show mod changes..." in a machine readable JSON encoded format. They can be opened in a text editor (e.g. wordpad) if you need to make manual tweaks.

--- What are these *.rs5mod files? ---
These are just rs5 files with a different extension. Miasmata can crash on launch if it finds extra .rs5 files with certain types of contained files, so these have a different extension to prevent Miasmata crashing if they have been placed in the game directory. Miasmod can be used to merge their contents into main.rs5 to safely use in the game.

--- What is this NOT (yet)? ---
- This is not a generic RS5 archive tool (rs5-extractor.py is - see the next post)
- This is not a map editor (I'd love that too, but it's a hell of a lot of work)
- This does not allow you to add scripted events to the game
- This cannot tweak anything that is hardcoded in the game's executable.

--- Known Issues ---
- Miasmod does not currently detect or warn about conflicts between mods.
- Miasmod does not check if saves.dat has been modified externally (by Miasmata) before saving it, so you should avoid running Miasmata if you have saves.dat open in Miasmod. It does create a timestamped backup of saves.dat though, so you should be able to recover if you accidentally overwrote saves.dat.
- There is currently no way to edit raw binary data fields (the underlying tools can do it if you want to dig into the python scripts - I just haven't exposed any way to edit it through the GUI).
- Expanding "texture_flags" in an environment file is very slow due to the large amount of data under that node, made worse by the Qt Model/View API adding extra overhead (seriously - why can't I just pass it a list of all children at once?) and the fact I'm using a QSortFilterProxyModel to sort and search the tree. Oh, and my code probably sucks too :-p

[DarkStarSword]

--- Working With RS5 Files ---
I've designed Miasmod so you shouldn't have to worry about the rs5 files if you are just editing the game's environment file, but if you do want to extract/create them, there are two options:

1. klightspeed created a utility that can extract many of the files in an rs5 archive and automatically converts them into useful formats. His tool can't repack the extracted files back into an rs5 archive and does not extract any types of files that it doesn't know how to decode (such as the game's instance nodes and terrain files which form the map).
Edit: It appears that this tool has recently been updated to add support for writing rs5 files and extracting other types of files.

It still remains an infinitely useful tool when starting out - especially if you just want to look through the game's textures or 3D models to look up the names of notes, etc. His post is over here:

http://steamcommunity.com/app/223510/discussions/0/828925216426421301/#c846943082463048999

2. My tools are a bit lower level and are not very user friendly (In that they are command line only - if you have used the Unix tar command you can probably figure them out). They can extract every type of file inside the rs5 archives and can create new rs5 archives and modify existing ones, but they do not attempt to convert the contained files to useful formats.

There are some standalone scripts that can work with several of the types of files found in the rs5 archives that I was interested in (environment, instance nodes, images, markers, etc) - but they were written more to help me understand the formats and will need cleaning up before they are ready for general use.

You can find my tools over here (you will need Python 2.7 and some additional dependencies depending on which scripts you try to use), along with the source code to Miasmod and Miaschiev (which come from the same code base):

https://github.com/DarkStarSword/miasmata-fixes

rs5-extractor.py is the main script to extract/pack/repack/add/delete files to an rs5 archive.

--- Notes About RS5 Files ---
Here's a few things I've learned:

- The game will search every rs5 archive in it's directory alphabetically and will use the first environment file it finds. This tool exploits that by naming it's combined rs5 archive "alocalmod.rs5", which should take priority over the communitypatch.rs5 and the game's built in environment.rs5.

- Naming an rs5 archive that contains an environment file alphabetically after "environment.rs5" will cause the game to crash on launch.

- Some files cannot be overridden by placing them in an rs5 file other than main.rs5, and attempting to do so will cause the game to crash on launch. I haven't tried many files yet to work out what is and is not affected, but at least the game's markers file and textures do cause this.

[DarkStarSword]

-- Technical Details --

No one needs to read or understand this post to use Miasmod, I'm providing it to document a few tricks that Miasmod uses under the hood, which may be of interest to some technical people. For everyone else, just ignore this ;-)


The main.rs5 modding support employs a few tricks to support being able to remove & re-order mods, and to restore main.rs5 back to it's vanilla state. The vast majority of people won't need to care about these details, but if anyone was developing their own tool to manipulate rs5 files (or perhaps even the guys at IonFX) they might want to be aware of what I've done.

Firstly, when main.rs5 is modified for the first time the tool adds a file MIASMOD\UNDO which stores the original location of the central directory in the archive and the original filesize, and aside from the pointer to the central directory no writes are ever performed before this point in the archive. That allows the tool to always point the header back to the original central directory and truncate the archive to restore it back to it's vanilla state.

Whenever a mod is added, it's files are listed in both the new central directory as well as under MIASMOD\MODS\xxx.manifest. The tool uses a combination of the original central directory (found via the undo information), the manifest files and the current central directory to determine which parts of the archive are in use and which are free (and everything prior to the undo file is treated as in use) so that it never overwrites some data used by a mod - even if that file has been hidden in the central directory by another mod that provides the same file.

The final trick that is employed is the concept of a mod order stored in MIASMOD\ORDER - this is used to indicate which mods have priority over each other to determine which version of each file to use in the central directory, and this mod order is used to recreate the central directory whenever mods are added or removed.

[Typo]

Thanks a ton man. I haven't started the game yet but I absolutely love seeing community support behind a game and what you've been up to for Miasmata is awesome.

I wish the devs would take notice of your patches and work.

Thanks again!

[Koriar]

Thank you so much for your work!

[Spiderpig]

Looking forward to playing around with these soon. Really hope the Johnson brothers feel encouraged that someone loves their game enough to put all this work into it! (Cuz it's an awesome game.)

[Spiderpig]

Just a long overdue and huge thanks for this - I continued my current playthrough with the cartographobia mod installed and it's been great being able to have the maps in my journal.

Gotta say, filling in the map by triangulation only (apart from the first Draco map) is a real challenge but really cool. I've seen parts of the island I don't think I've ever visited on my first three games. It's been my longest game as well (day 41 and I'm still filling in the SE corner).

I'm considering synthesising two of the agents and then explore for the final agent, and see how aggressive Milo gets. If he gets very aggressive, I may then check how well ailurophobia.miasmod works. ;)

[Arrow]

I really want to try one of these mods! Though when I tried to run Miasmod for the first time it threw this error and won't do anything else:

---------------------------
MiasMod
---------------------------
Unhandled Exception
---------------------------
TypeError: coercing to Unicode: need string or buffer, NoneType found
---------------------------
Traceback (most recent call last):
File "_main_miasmod_.py", line 40, in catch_unhandled_exceptions
File "_main_miasmod_.py", line 545, in synchronise_alocalmod
File "_main_miasmod_.py", line 522, in generate_new_alocalmod
File "_main_miasmod_.py", line 511, in generate_new_alocalmod
File "_main_miasmod_.py", line 411, in generate_env_from_diffs
Typrerror: coercing to Unicode: need string or buffer; NoneType found

Any ideas? :U

[Ap2000]

Basic stuff, but pretty cool.
Just as a (long-term) suggestion; enabling the boat for traveling around and between islands would be pretty cool imo.

[DarkStarSword]

İlk olarak GhostTheZombie tarafından gönderildi:

I really want to try one of these mods! Though when I tried to run Miasmod for the first time it threw this error and won't do anything else:

Thanks for the bug report :-)

Turns out I had an off by one error if MiasMod was run with no mods and it hadn't already created alocalmod. I'll push out a fix tonight (already on github, I just need use my Windows box to package it), in the meantime you can get it to work by installing at least one sample mod before running MiasMod (you can then remove the sample mod and re-sync alocalmod if you don't want it).

[DarkStarSword]

I've released version 1.1 of Miasmod (see the updated top post for the download link). This is mostly a bug fix release, though you can now enable/disable mods from being included in alocalmod.rs5 instead of having to remove them from the Miasmata directory.

-- Miasmod v1.1 Changelog --

New Features:
- Can now enable / disable mods from being included in alocalmod.rs5
- Showing mod changes now shows the contents of what was removed (potentially including whole subtrees)

Bug Fixes:
- Log output and errors to miasmod.log
- Catch unexpected exceptions inside the editor and display an error message (before it was failing silently)
- Use faulthandler python module to catch backtrace when crashing due to accessing invalid pointers (e.g. objects passed to Qt that have since been garbage collected).
- Fix issues creating a new list type
- Warns if detected/selected Miasmata directory does not include at least one .rs5 file (other than main.rs5)
- Fix errors generating alocalmod.rs5 (e.g. on first run) when no mods are installed.
- Fix occasional crashes and the bold dirty indicators not being cleared properly after saving.

[Dolus]

This works with the GOG version too right?

[DarkStarSword]

İlk olarak DeadWire tarafından gönderildi:

This works with the GOG version too right?

I don't see any reason it wouldn't - AFAIK both versions are almost identical (other than steamworks integration), but I don't have the GOG version to try.

Please give it a go and report back :)

[Arrow]

İlk olarak DarkStarSword tarafından gönderildi:

Thanks for the bug report :-)

Turns out I had an off by one error if MiasMod was run with no mods and it hadn't already created alocalmod. I'll push out a fix tonight (already on github, I just need use my Windows box to package it), in the meantime you can get it to work by installing at least one sample mod before running MiasMod (you can then remove the sample mod and re-sync alocalmod if you don't want it).

Ahhh, it works good now! :D Thanks so much for your help (and creating this!)

[Spiderpig]

Just to say, this works fine with my GOG version. :O)

[Jakub Majewski (Quarto)]

Hi,

Great work! I had a quick look at Miasmod yesterday, and it's a really impressive tool. I have a question, though - the one thing I really wanted to do is fix up the visuals a bit by changing the LOD-loading distances - I'm bothered by the way all kinds of stuff pops up so close to the player. I've searched everywhere in Miasmod, and I can't see anything along the lines of LOD distances. Any idea whether they can be changed at all, or are they hardcoded in the EXE?