Movie format and modification

[Lulech93]

Hi, all! Someone on Reddit pointed me to this group after I shared a bit of the work I've been doing on Nier: Automata's prerendered cutscenes :)

First things first: I've created a tool called NieRepack which can demux, convert, and repackage .usm videos with a number of customization options like support for various aspect ratios and compression to save disk space. You can check that out here: http://steamcommunity.com/sharedfiles/filedetails/?id=890543121

To go into more detail, Nier's .usm videos are, as with the rest of its data, a Criware format. Or rather, used to be. Criware's video encoder technology is actually in a bit of a confusing place, considering it was actually acquired by another company called Scaleform in 2009 which was then acquired as a whole by Autodesk a mere two years later. However, Autodesk hasn't really done much with the software, and in fact it's no longer even mentioned on any of their product pages save a few brief notes in their online documentation. The only feature they ever added was H.264 support, which itself comes with licensing fees thus sort of defeating the point of licensing a middleware video encoder at all. Aside from being an interesting bit of trivia, this means that Nier--which was built with Criware tools--is using an outdated video container and internal format besides.

Criware's .usm files use modified MPEG-1 streams and ADX audio. Nier's prerendered cutscenes do not store audio directly in the .usm container, however. For some reason, only one of the menu logos does, and it's a version of the Platinum logo you only rarely see in-game (file "PG5R.usm"). Since audio is not really a concern, then, it's actually quite easy to access the video stream of .usm files for playback, although certain changes to the format will confuse other encoders.

For example:

ffmpeg -f mpegvideo -i "input.usm" -q 0 -f mpeg1video "output.mpg"

With this method, FFMPEG can be used as a .usm 'extractor' by forcing it to treat the .usm as an MPEG video and then forcing the output to MPEG-1--the closest thing to the source format. However, while this works in most cases, the fact remains that .usm is modified from MPEG, and numerous imperfections remain. FFMPEG just doesn't know what to do with the internal .usm stream structure, resulting in videos that sometimes cut off too soon or have garbled images.

A more direct solution is to simply read the .usm data format, separate the video stream, and restructure the file to conform to MPEG standards again. This has been quite elegantly accomplished by HCS64 as part of his UTFTab suite[www.hcs64.com] of ripping tools (source here[github.com]). It's a simple matter of popping open a CLI and running the program with the target .usm file as the first argument.

usm_deinterleave.exe "input.usm"

With a simple, direct way to access the video portion of .usm files at their original quality, we can now do pretty much whatever we want to with them. Nier: Automata itself does not distinguish between resolutions, aspect ratios, bitrates, and so on, so as long as the videos are in the right container it will happily play them...stretched to fullscreen and locked at 30 FPS. And annoyingly enough, the original videos are stored at 1600x904 rather than 1080p. Considering this is also the resolution the PS4 version of the game runs at, it appears the devs rendered for that console first without considering the PS4 Pro and PC versions down the line. Apparently they didn't consider folks storing the game on hard drives instead of a Bluray disc either, because the videos are encoded at a whopping 40 Mb/s VBR, resulting in videos occupying a full 24GB of Nier: Automata's overall 50GB install size. Ouch.

Source resolution and bitrate issues aside, FFMPEG is great for handling all sorts of tweaks at this point. Change the resolution, aspect ratio, framerate, apply any FFMPEG filter you want to, or stick a jump-scare video in your friend's game for a good laugh. The only catch is that in order to package videos back up again, they have to be converted to .avi since that's the only input format Scaleform Video Encoder accepts.

And yes, you do still have to use the original application. Even though .usm is similar to MPEG there are no community-made tools to create files in that format, likely due to its proprietary nature prohibiting such tools from existing. Fortunately, an old version of Scaleform Video Encoder is pretty easy to find around the internet (Google is your friend), and the primary encoding application ("medianoche.exe") works in command-line and even comes with example batch files to show you how it's used. The best part: you can specify overall bitrate directly in Scaleform. In my experiments, I've found that 20 Mb/s VBR is virtually indistinguishable from the original videos and reduces filesize to just 13GB. Reduce it even farther to 10 Mb/s and quality is still perfectly acceptable while filesize drops to a mere 7GB instead.

So there you have it: the ins and outs of Criware/Scaleform/Autodesk's weird and outdated .usm format. Nier's movies are stored under ..\SteamLibrary\SteamApps\common\NierAutomata\data\movie and ..\movie_logo.

Have fun!

[The Engie]

Thanks for the tool and the documentation.