modding game scripts

It's been said that it's impossible to edit the scripts for Battle Brothers to add new items, traits, etc. No longer! Now the scripts can be edited, and since almost the entire game is written in script, this means almost anything can be modded now. Even if you're not into modding, just being able to read the scripts can grant new insights into the game.

Background

I created a rudimentary mod kit that allows editing the encrypted Squirrel[www.squirrel-lang.org] scripts (.cnut files) within the game. Since almost the entire game is implemented in script, this allows tremendous flexibility for modding. It started when I wanted to know which character backgrounds were best to hire in a "value for money" sense. There was a spreadsheet that the community had painstakingly created over the years based on hundreds of manual tests showing the typical stat ranges of various backgrounds, but it was invalidated by the Beasts & Exploration DLC which rebalanced various parts of the game. I wished I could just take a look at how characters are generated, but like many people trying to peek under the hood I soon saw that the scripts are shipped only in a compiled form, and not only that but they appear to be corrupted or encrypted.

Accessing the files

Before we can start messing with the game data files, we first have to know where they are. If your Battle Brothers installation directory is bbros, then the data files are stored within bbros/data/. Within that directory you'll see files named like data_001.dat. These are actually .zip files, and if you rename it to data_001.zip you'll be able to open it. Images, sound effects, etc., are not encrypted, so you can examine them. (Don't be tempted to modify them inside the .zip file, though. See below for how to make changes.) The scripts are the .cnut files, but if you open them they just look like garbage.

Encryption and decryption

You can skip this section if you like, since knowledge of the details isn't necessary to use the mod kit.

The "corruption" in the compiled scripts is actually an unusual encryption algorithm that encrypts only bits and pieces of the files, making them look like normal compiled scripts at a glance, but preventing them from actually working if you try to run or examine them. After painstakingly reverse engineering parts of the game, I was able to decipher the encryption algorithm. I spent quite a long time discovering how it worked and then a while longer translating the cryptic instructions into C code that I could run. Only after I was done did I realize that one of the constants in the algorithm is well-known, and that the core cipher is XXTEA[en.wikipedia.org].

But like I said, only parts of the file are encrypted. The encryption algorithm is unusual in that the format of a Battle Brothers encrypted .cnut file cannot be described in a stand-alone way since it's intimately tied to the internal implementation details of the Squirrel script engine. Specifically, the Squirrel API calls to parse a compiled script take a reader callback function, which gets called with a buffer and a number of requested bytes, and the function is supposed to read those bytes from the application's storage. Basically, it's a byte-oriented stream interface. Now, where it gets tricky is that the game only decrypts runs of 8 or more bytes requested through that stream interface, and it doesn't keep any state between invocations. If the Squirrel engine changed to break one read into two, or to combine two reads into one, or even just changed the size of any encrypted read, the file format would effectively change. So, to be safe you should use the same version of Squirrel as the game. At the time of writing, this is 3.0.4, but I've also tested with 3.0.7. Also, only multiples of 4 bytes are encrypted, so a 15-byte read has the first 12 bytes encrypted and the last 3 bytes unencrypted.

So, without further ado, here's how you'd go about decrypting a Battle Brothers .cnut file:

// the standard XXTEA block cipher algorithm
#define DELTA 0x9e3779b9
#define MX(p) (((z>>5^y<<2) + (y>>3^z<<4)) ^ ((sum^y) + (key[((p)&3)^e] ^ z)))

void decrypt(uint32_t *values, uint32_t count, const uint32_t key[4])
{
    uint32_t rounds = 6 + 52/count, sum = rounds*DELTA, y = values[0], z;
    do
    {
        uint32_t e = (sum >> 2) & 3;
        for (uint32_t p = count-1; p; p--)
        {
            z = values[p-1];
            y = values
 -= MX(p);
        }
        z = values[count-1];
        y = values[0] -= MX(0);
        sum -= DELTA;
    } while (--rounds);
}

// the key used by Battle Brothers
static uint32_t bbkey[4] = { 238473842, 20047425, 14005, 978629342 };

// the Squirrel reader function
SQInteger read_encrypted(SQUserPointer file, SQUserPointer buffer, SQInteger length)
{
  length = file_read(file, buffer, length); // read the data from the file
  if (length >= 8) decrypt((uint32_t*)buffer, length / sizeof(uint32_t), bbkey);
  return length;
}

...
sq_readclosure(vm, read_encrypted, file);[/code]

To encrypt a file, you can't simply do the analogous approach of using a writer callback function to encrypt all blocks of 8+ bytes, since the Squirrel writer doesn't invoke the callback for the same offsets and lengths as the reader, which is needed for the two to match. (And even if they did match, there's no guarantee that it would stay that way.) So, I implemented encryption using a reader function as well, so you read the file that you want to encrypt, and as a byproduct of reading it the file gets encrypted.

void encrypt(uint32_t *values, uint32_t count, const uint32_t key[4])
{
    uint32_t rounds = 6 + 52/count, sum = 0, y, z = values[count-1];
    do
    {
        sum += DELTA;
        uint32_t e = (sum >> 2) & 3, p;
        for (p = 0; p < count-1; p++)
        {
            y = values[p+1];
            z = values
 += MX(p);
        }
        y = values[0];
        z = values[count-1] += MX(p);
    } while (--rounds);
}

SQInteger read_and_encrypt(SQUserPointer file, SQUserPointer buffer, SQInteger length)
{
  length = file_read(file, buffer, length); // read the data from the file
  if (length >= 8)
  {
    // encrypt the file in place. you could also write the changes to a separate file
    encrypt((uint32_t*)buffer, length / sizeof(uint32_t), bbkey); // encrypt the data
    file_seek(file, -length, SEEK_CUR); // seek back to the start of the block
    file_write(buffer, length); // write the encrypted bytes back to the file
    file_flush(file); // some C file APIs require a flush when switching from write to read

    // decrypt the data again so we don't return garbage to Squirrel
    decrypt((uint32_t*)buffer, length / sizeof(uint32_t), bbkey);
  }

  return length;
}[/code]

Decompiling a script

Now we can decrypt a script file, but it's still in compiled form, so we can't easily edit it. To decompile it, I recommend using NutCracker, but not the Github version[github.com]. I actually don't like that decompiler, because the output is incorrect. It has numerous bugs, some of which I've fixed, and is more verbose than it needs to be. It also doesn't preserve all the information from the .cnut file, so if you decompile a file with NutCracker and then recompile it, the result will not be the same as the original. I've packaged a modified version of Nutcracker in the mod kit which has many bug fixes.

Because of those problems, I wanted to write my own decompiler. I'm sure I can create a better one than NutCracker, but my free time is limited so I probably won't do so unless there's a real demand. The problems above are not fatal, since Battle Brothers is capable of loading plain-text scripts (for the most part) and most scripts decompile in a way that's good enough. A few scripts don't, though, so be careful.

Anyway, here's an example of how to use NutCracker to decompile a script. By default it prints the script to the console, so you simply redirect the output to a file. (The NutCracker author doesn't distribute binaries for the decompiler, but a bug-fixed binary is included in my mod kit.) From a command-line:

nutcracker decrypted.cnut >decrypted.nut

Getting a modified script loaded by the game

Lets say you decompile scripts/items/weapons/knife.cnut, which starts like this:

this.knife <- this.inherit("scripts/items/weapons/weapon", {
	m = {},
	function create()
	{
		this.weapon.create();
		this.m.ID = "weapon.knife";
		this.m.Name = "Knife";
		this.m.Description = "A short knife, not made for combat.";
		this.m.Categories = "Dagger, One-Handed";
    ...

Just for a test, you change "not" to "totally" within the description. Now how to get the modification back into the game? The game will prioritize an encrypted script named "foo.cnut" over a plain-text script named "foo.nut" in the same directory, so if you're adding a new file, the most convenient way is to just use the plain-text script, but if you're changing an existing file, as we are in this example, you'll need to recompile the .nut file using the standard Squirrel compiler (sq.exe) into a .cnut file and then reencrypt it using the mod kit.

You could simply insert the new or modified file into the data_001.dat archive, but that's a bad idea, because your modification may cause file corruption if the game is patched, and any game update would undo your changes. We want the game to load the base data and your modified version. Thankfully, the game uses the PhysFS[www.icculus.org] filesystem abstraction library, and configures it in such a way that the game will load files from subdirectories of the data directory as well as files from the .dat archives. So you could actually place your modified knife.cnut file in bbros/data/scripts/items/weapons/ (where "bbros" is your game's installation directory) and it will take precedence over knife.cnut from the data_001.dat archive. This is good for testing, but is not a good solution for mods in general because it makes it hard to combine multiple mods. (Their files would be all mixed together.)

A better way is to simply package your mod into its own .dat file. Since the .dat files are really .zip files, just create a .zip file with the scripts/items/weapons/knife.cnut file inside and place it in the bbros/data/ directory along with data_001.dat. (It's not necessary to use the .dat extension. The game will also load files named *.zip.) The important thing here is that mods get loaded after the base data, and I believe this happens based on the file name. Since 'm' sorts after 'd', naming the archive "mod_knife.zip" ensures that it gets loaded after the file named "data_001.dat". If order among mods is important (to resolve some conflicts), you could tweak the names to get the right ordering.

Coding guidelines for better mods

Now we can mod the game, but mods created as above will not play well together either with other mods or with future official updates. The reason is that you are overwriting entire files rather than just tweaking the bits you want to change. If an official update changes the file, your copy won't have the update unless you recreate your mod based on the latest official version. And, two mods that update the same file will overwrite each other's changes even if the changes shouldn't conflict. So we need a way to express the changes relevant to us without duplicating the rest of the file. Consider the following script.

local super = this.knife.create;
this.knife.create = function()
{
  super();
  this.m.Description = "A short knife, totally made for combat.";
}

Something like this would allow us to change just the description while keeping the rest of the knife code the same, and we could put this script into its own file rather than overwriting the base knife file. The difficult part is getting this code loaded at the right time. By placing your file in a folder named !mod_mymod you can make a script load before most others, and by placing it in a folder named ~mod_mymod you can make it load after most others, but neither of those work in this case. That said, I did create a small mod called "modding script hooks[www.nexusmods.com]" that allows something like this:

::mods_hookNewObject("items/weapons/knife", function(obj)
{
  obj.m.Description = "A short knife, totally made for combat.";
});

On a side note, any sizeable mod will have to deal with official updates. You wouldn't want to redo your mod from scratch after each official update, nor would you want to continue to use outdated script files in your mod. The solution here is to set up version control. Extract the game data, decompile the scripts, and put it into a Mercurial or Git repository. Create a branch for the mod and edit your mod in the new branch. After each official update, switch back to the main branch, extract and decompile the game data again, and check in any changes. Then merge the changes into your mod branch. This way you keep your mod up to date with official changes without having to redo it from scratch. Also, the repository should of course just be on your local disk. Don't go putting all the game files up on GitHub. This also helps if you need to merge multiple mods together due to conflicts.

The mod kit

I packaged together a few tools into a very basic mod kit to get people started modifying the scripts for Battle Brothers. See the README.txt file within the package for details.

Download: Go here[www.adammil.net] for the latest version.

Changelog

• Version 1, January 20, 2019 - Initial release
  
• Version 2, January 21, 2019 - Attempted to fix a serious bug[github.com] in NutCracker. I'm not sure if the fix is correct. Hopefully the author will make an official fix.
  
• Version 3, January 22, 2019 - Attempted to fix another bug[github.com] in NutCracker.
  
• Version 4, January 23, 2019 - Attempted to fix another serious bug[github.com] in NutCracker.
  
• Version 5, January 26, 2019 - Added a program (bbrusher) that can unpack and repack the game's sprite sheets
  
• Version 6, January 27, 2019 - Fixed two[github.com] more[github.com] serious bugs in NutCracker
  
• Version 7, January 28, 2019 - Expanded the types of sprites that can be added with bbrusher. (Should be all types now.) Made Nutcracker's floating point output prettier. Added masscompile.bat tool.
  
• Version 8, February 3, 2019 - Fixed four[github.com] more[github.com] serious[github.com] bugs[github.com] in NutCracker
  
• Version 9, February 12, 2019 - Improved Nutcracker's handling of non-ANSI strings by using UTF-8

Postscript
So in the end, I got my updated character backgrounds spreadsheet[docs.google.com]. :-) I wrote a program to scan the script files and make a spreadsheet out of it. I guess that's mission accomplished!

[Adam]

@Exorciamus Hi. I looked at those files and I can say that the .fnt files are in an unknown format, but they look simple enough. They apparently contain information relating to the positions of the characters within the corresponding .png file.

I don't think you need to edit or understand the .fnt files, but just from a brief look I would say that the file name (e.g. cinzel_20) contains the basic grid size (i.e. 20x20) and the .fnt file probably describes deviations from the grid.

Anyway, I don't believe you need to edit the .fnt file because editing the .png file is enough if you just want to change the characters in it. cinzel_20.png apparently uses a 22x20 grid, reversed both horizontally and vertically for whatever reason. You can just try editing the file a see what happens.

All that said, it's quite possible that your edits will have no effect. There's a good chance that the game doesn't use those files at all, and instead uses the .ttf files for text. Still, you can try it.

[Exorciamus]

Hello! Is there any way to work with these files? I want to change some ACII characters to other characters I need for testing. But it seems that the .fnt files do not have a typical look for them, and are somehow encoded...

cinzel_20.fnt
cinzel_20.png
cinzel_bold_20.fnt
cinzel_bold_20.png
cinzel_bold_100.fnt
cinzel_bold_100.png

[pgames-food]

ok thank you adam

[Adam]

Well, the short answer is that to decrypt and decompile you do:
bbsq -d script.cnut
nutcracker script.cnut >script.nut

To compile and encrypt, you do:
sq -o script.cnut -c script.nut
bbsq -e script.cnut

Some care is required to keep track of whether the files are currently encrypted or not, since running bbsq -d on a file that's already decrypted, or bbsq -e on a file that's already encrypted, will result in them being double-decrypted or double-encrypted...

[pgames-food]

ah ok, yes its possible i wasnt following the syntax properly for the single file, will have another look, but so far so good with the batch tool :)

[Adam]

@pgames-food Thanks for the message. The masscompile and massdecompile scripts just call the other programs (bbsq, nutcracker, etc), so you might consider looking inside to see how they work.

[pgames-food]

hi adam, (while i dont know you from adam) i wanted to say thank you very much for the time youve spent in researching and making the modding tools :)

i was able to finally edit my local game, just so that items that are "worth xyz" are actually closer to that value when selling, instead of selling for like 10% :)
for example here:
https://steamcommunity.com/app/365360/discussions/3/2802882333588816330/

1 funny thing though, is that your nutcrackerexe file was not able to modify the .cnut i was trying to edit (world_assets.cnut) with the usual exe filename > output command,

BUT, simply using your masscompile/massdecompile batch files in a folder with just that file, actually did make it editable and back :)

thanks again

[Dealer Mangan]

@Adam Big thanks for the explanation!

[Adam]

@Ergo Breethyr Injuries are implemented as skills in the game, so to add an injury you add the appropriate skill. To add the brain damage injury, you could do something like:
[code]bro.getSkills().add(new("scripts/skills/injury_permanent/brain_damage_injury"));[/code]

I haven't tested that, but that's the idea. bro.addLightInjury() just reduces HP by a small amount, while bro.addInjury(...) takes an array of injury structures, selects one at random, and adds it. Const.Injury.Brawl is an array of typical injuries that might be obtained from brawling, in a form suitable to be passed to bro.addInjury(...). So bro.addInjury(Const.Injury.Brawl) adds a random brawling-type injury. You can see the definitions of these functions in scripts/entity/tactical/player.nut.

[Dealer Mangan]

also i'd be glad to learn what is the Const.Injury.Brawl, because just as it is i'm completely clueless what that thing stands for