DQXIS-SDK 0.7.6.1 // rendering fix // dev console // first-person

[Havoc]

Now available on github: https://github.com/emoose/DQXIS-SDK

0.7.6 now supports the new 2021/05/28 Denuvo-removal update & the EGS/MS Store versions!

Should also support the JP Steam version as of 0.7.4!

0.7 adds TemporalAA support to USceneCaptureComponent2D, so character/monster viewers & fun-size forge can now look even better than ever! (just make sure to enable TAA in your game via INI tweaking first!)

DQXIS-SDK is a wrapper DLL that brings several enhancements to DQXIS, such as:

• Fixes render resolution of & allows Temporal AA to apply to character/monster viewers & fun-sized forge (along with anything else that uses USceneCaptureComponent2D), resolution of those should now match up with games. (comparison: https://slow.pics/c/NCyrOhgP, click image to compare)
  
• Adds FirstPersonCamera, EnterPartyChat and QuitGame input actions, that can be bound to any key/button via the Input.ini file.
  
• Unlocks the UE4 dev-console (with cheats for both 3D & 2D mode!) & allows loose, unpacked files to be loaded by the game.
  
• Fixes "Camera FirstPerson" camera height, and made it so NPCs no longer disappear when viewed up close in first-person.
  
• (optionally can make FirstPersonCamera always use the "Camera FirstPerson" camera instead, allowing the game to mostly be played in first-person!)
  
• Allows hiding the minimap when in first-person mode.
  
• Can prevent game from overriding/conflicting with any custom Input.ini bindings (the included Input.ini contains all the default binds used by the game, ready to be customized)
  
• Fixes some commonly-misconfigured cvars, such as r.JackLoadReduction.DisableMovementModeOptimization

The SDK also has a huge amount of the games classes/structs mapped out, giving a nice view into the games internals, including things like the structures of DataTables used by the game!

(For modders that want to edit the games data, such as monster stats, NPC details etc, I've also made DQXIAsset that allows deserializing the games DataTables to/from a human-editable CSV format: https://github.com/emoose/DQXIAsset - hope it comes in useful!)

Download/Install
You can download the plugin here: https://github.com/emoose/DQXIS-SDK/releases

To install just extract the xinput1_3.dll & DQXIS-SDK.ini file to the same folder as "DRAGON QUEST XI S.exe", and with that it's good to go.

If you're using the MS store version, or for some other reason can't copy xinput1_3.dll next to the game EXE, you can also use a DLL Injector tool (such as the one included with UUU: https://framedsc.github.io/GeneralGuides/universal_ue4_consoleunlocker.htm) to inject xinput1_3.dll at runtime (I recommend injecting before loading into a save though!)

Would appreciate any feedback, good or bad!

DevConsole + LooseFiles
DQXIS-SDK 0.4 now has DQXISHook merged in, allowing the dev-console to be unlocked, and letting the game load loose files without them needing to be packaged into .pak.

The guides section has some good info about playing with the dev console, I've also dumped a list of all the games cvars, the majority can be changed with the dev console fine: https://pastebin.com/kWxQVXfu

0.7.1 also enables calling into UTripleCheatManager so that 2D mode cheats can be used from the dev console - a list of them can be found at https://github.com/emoose/DQXIS-SDK/blob/master/DQXI-SDK/SDK/Triple_classes.h#L28 (note that not all cheats are functional however!)

Extra binds
Also adds extra custom bindable actions, FirstPersonCamera, EnterPartyChat and QuitGame, which can optionally be bound to any keys/buttons you like by editing the Input.ini file.

Since 0.5.3 the FirstPersonCamera can also have a "MovableCamera" mode enabled, letting the game play almost exactly like a first-person game, besides some tiny issues it works pretty well!

0.5 can also prevent the game from overriding any bound keys, though this requires a small setup step: after enabling "BindFromInputIniOnly" the first time you run the game you'll be prompted to install the games default bindings into your Input.ini, just select Yes to that, and now the game will only use bindings from your Input.ini file, with no risk of hardcoded game bindings conflicting/overriding your own!

Render fix
This fixes the low-res character viewer's resolution by changing the width/height of the USceneCaptureComponent2D texture used for it, by default this seems to always be rendered at 1280x720 (game requests it as 1080p, though due to a bug it seems to get scaled down to 720)
With the texture size changed to the actual game resolution it gives these viewers a lot more detail, and reduces aliasing a lot - on top of that, DQXIS-SDK 0.7 also allows TAA to be applied to these viewers, improving the quality even further (and even seems to help with performance slightly!)

Maybe other UE4 games that have similar low-res character viewers (eg. Bloodstained RotN) could be fixed with similar changes.

NPC pop-in fix
The previous version of DQXIS-SDK included code for fixing/removing NPC pop-in, but that's been removed since we found an INI setting that can do that instead (r.JackLoadReduction.HiddenThresholdDistance), the graphics mod thread & INI setting guides can show you how to change this setting if you like: https://steamcommunity.com/sharedfiles/filedetails/?id=2312807086

(I recommend looking into it, the default Switch-tier setting with NPCs disappearing just a few meters away is abysmal, only takes a minute to change the INI setting!)

SDK
The SDK lets you access all the games UObjects and modify the properties of them, or call functions exposed by them. (UnrealFinderTool managed to dump almost all the properties of each object, but I noticed there's some that aren't named and might need to be reversed manually, so don't be surprised if you notice something is missing)

The included code shows how to hook into a games function & use a structure from the SDK to modify some data related to it (in this case, the render resolution of the AJackCharacterCaptureCamera).
If you don't know what function to hook and just want to edit games objects instead, you can make a thread with CreateThread and then use UObject::FindObjects to search for different object types, the original release of DQXIS-SDK included code that would do this to edit the NPC distance values, you can find the 0.1a version with that code linked by Tim Allahn SnAckbarr below.
(editing stuff from a different thread brings some thread-safety issues with it though, so hooking functions is much more preferable, or some way of hooking the games main thread)

(Also despite the project filename this DQXI-SDK is only meant for working with DQXIS, I forgot to push a button while naming it :)

Todo
- Find way to hook into main game thread's loop, so we can change objects & run functions safely (not needed for rendering fix, but would be helpful for messing with the games objects)

Thanks
Huge thanks to idkicarus for providing me a copy of DQXIS!
Thanks to CorrM for creating UnrealFinderTool - 99% of the SDK work was made with that, with just a few tiny changes needed to adapt it for DQXIS.

[Tim Allahn SnAckbarr]

This is insane, nice find!! Perhaps my pastebin[pastebin.com] came in handy :D

I hope it's okay if I re-host it on a GDrive? I know the spam filter won't remove those links

Now i'm starting to wonder if some other graphic fixes might be possible in this way..

GDrive links:
DrawDistanceFix 0.1a -- https://drive.google.com/file/d/17SbZz7Lcb8w3yD8-Noaxfh6fniqBiaGq/view
RenderFix 0.2a -- https://drive.google.com/file/d/1PeD5zIkJ1XvH_xEDhnIib3qp85I3wosq/view

Install to <steamapps>\common\DRAGON QUEST XI S\Game\Binaries\Win64\
(where <steamapps> is typically C:\Program Files (x86)\Steam\steamapps\ depending on your installation )

--note: this will not work for the Gamepass / MS Store version as explained in this post.

[fastica]

OMG this is awesome!!!! I just tried it in Heliodor and it looks great without the distracting NPC popup!!! Thank you

[Tim Allahn SnAckbarr]

On the topic of the SDK, I had been researching a way to improve the overworld monster AI, using the UnrealFinderTool to dump object names, etc. It looks like your SDK includes the AIModule classes, functions, parameters and structs -- so maybe it's possible to change these properties through this method?

We were looking for a way to parse the DataTables to JSON, but even then I wasn't sure it was possible that way. It was almost looking like we'd need a custom solution for the DataTables, similar to Trials of Mana.

In any event, this is a huge development.

[Rayne_2007]

Works great, thank you very much for this and all your other work... I encountered zero NPC pop-in after installing this, which makes for a much more enjoyable experience.

After install, I did notice most NPCs will begin 'walking in place' while conversing with them, but I have several other mods installed as well [modified .ini, costume recolors and DQXISHook-0.6a].

Edit: Just discovered.... This has the side effect of locking you in conversation with a certain NPC in Act II at Insula Algarum, and possibly any NPC who has an interruption mid conversation [such as a short pause, followed by an exclamation mark appearing above their head before continuing to speak]. There is no way to advance or exit the conversation, without restarting the game.

[Havoc]

Originally posted by Rayne_2007:

After install, I did notice most NPCs will begin 'walking in place' while conversing with them, but I have several other mods installed as well [modified .ini, costume recolors and DQXISHook-0.6a].

Edit: Just discovered.... This has the side effect of locking you in conversation with a certain NPC in Act II at Insula Algarum, and possibly any NPC who has an interruption mid conversation [such as a short pause, followed by an exclamation mark appearing above their head before continuing to speak]. There is no way to advance or exit the conversation, without restarting the game.

Ah damn, thanks for letting me know, can confirm the walking-in-place bug happens with me too (just using this + INI mods).

I think the problem is something to do with some animations getting turned off for some reason, with the mod removed you can see the walking-in-place NPCs were trying to walk to turn toward you, no idea why it only affects some and not all though. I guess the problem with mid-convo interruptions is similar too.

Good news is that I think I've found the value that controls the pop-in distance, so instead of disabling this system entirely we can just increase the distance, seems to fix the walk-in-place NPCs at least, hopefully it'll help the other issue too.

I'll have an update posted soon in the next few minutes.

EDIT: Alright, I've found out there's actually a cvar that can control the distance for us:

r.JackLoadReduction.HiddenThresholdDistance

Normally it's set to 0, which applies the models default hidden-distance (normally 5000), 15000 is a good value if you want to get rid of all pop-in, 10000 could be a good choice if 15000 is too much for you, but you'll still get some (far away) NPCs popping in/out.

I'll update OP with instructions for changing this soon, hopefully others might find the SDK useful for changing other things.

[Havoc]

E: updated main post with render fix 0.2a, will now scale the character viewers resolution by r.ScreenPercentage value, so it should always be rendering at same res as the game itself now.

Also had a quick look at seeing what's left of first-person mode, seems there's 2 different first-person modes still in the game, both accessible with console commands:

- "Camera FirstPersonView" console command - seems to be the first-person mode that was in DQXI, LS zooms, RS rotates, B lets you return to normal view (or "Camera Reset" command)

- "Camera FirstPerson" console command - switches to a first-person view which is lower to the ground, but allows you to move around while using it. (IIRC there might be a cvar somewhere that controls camera height), pressing B/"Camera Reset" doesn't seem to have any effect at returning to normal view though unfortunately...

I wonder if changing DefaultInput.ini could restore the LS bind for it?

[Jacien]

Originally posted by Havoc:

E: updated main post with render fix 0.2a, will now scale the character viewers resolution by r.ScreenPercentage value, so it should always be rendering at same res as the game itself now.

Also had a quick look at seeing what's left of first-person mode, seems there's 2 different first-person modes still in the game, both accessible with console commands:

- "Camera FirstPersonView" console command - seems to be the first-person mode that was in DQXI, LS zooms, RS rotates, B lets you return to normal view (or "Camera Reset" command)

- "Camera FirstPerson" console command - switches to a first-person view which is lower to the ground, but allows you to move around while using it. (IIRC there might be a cvar somewhere that controls camera height), pressing B/"Camera Reset" doesn't seem to have any effect at returning to normal view though unfortunately...

I wonder if changing DefaultInput.ini could restore the LS bind for it?

If you can figure out how to make that work, let me know. I've tried of a long while to restore Switch's Party Chat shortcut (EnterNakamaKaiwa) but had no luck. Ini editing wasn't doing the trick.

And it's there commented out in the DefaultInput.ini, but I could never get it to work, by hook or making a new pak.

[Tim Allahn SnAckbarr]

Originally posted by Havoc:

Also had a quick look at seeing what's left of first-person mode, seems there's 2 different first-person modes still in the game, both accessible with console commands:

- "Camera FirstPersonView" console command - seems to be the first-person mode that was in DQXI, LS zooms, RS rotates, B lets you return to normal view (or "Camera Reset" command)

I wonder if changing DefaultInput.ini could restore the LS bind for it?

At one point, we found there was a line for it

; 一人称カメラ
+ActionMappings=(ActionName="FirstPersonCamera", Key=Gamepad_Special_Left)

But didn't have any luck getting it to work. Our guess at the time was that the ActionName tied to that mapping was removed.

In the Switch version, this was changed to a Party Chat shortcut

; 仲間会話　MARBLE では削除 : タスク #97909
; +ActionMappings=(ActionName="EnterNakamaKaiwa", Key=Gamepad_Special_Left)

Unfortunately, uncommenting this line also does nothing.

However, your SDK may be able to give us some insight (or maybe even more options) via some of the exposed methods.

Engine::InputSettings

void GetActionNames(TArray<struct FName>* ActionNames);
void AddActionMapping(const struct FInputActionKeyMapping& KeyMapping, bool bForceRebuildKeymaps);

[Havoc]

Ah dang, seems neither FirstPersonCamera & EnterNakamaKaiwa actions are bound to any code in here, guess they had to remove them for some reason.

In EnterNakamaKaiwa's case there is a AJackFieldPlayerController::NakamaKaiwa() function that opens the party chat, I'd guess that maybe EnterNakamaKaiwa was meant to run that.

Had a look at seeing if we could add custom actions ourselves, got a hook working which should be able to add new ones so an action can run our own DLL code, but so far I've only been able to get it working using action names that already have keys bound...

eg. if I name the action "MyTestAction" and try adding a +ActionMappings line for it inside Input.ini, nothing happens when I press the key, but if I name it "ToggleAutoRun" (which was already bound inside DefaultInput.ini) then pressing the button bound for that runs my custom action mapping fine.

E: Ah seems you can't use "+ActionMapping" inside input.ini, "ActionMapping" works instead, but might mean all the stuff from DefaultInput.ini has to be copied over, hmm.. (edit: nope)

Anyway custom mappings seem to work, made a FirstPersonCamera mapping that calls AJackFieldPlayerController::Camera("FirstPersonView") and bound it as N, seems to switch fine (same with EnterNakamaKaiwa)

There's a ~2 second delay that happens the first time a custom mapping is used though, might be related to how this SDK searches for UFunctions.. I'll look into it some more later.
E: changed it to search for UFunction during game load & cache the address instead, and now delay is gone :)
I'll post up a new plugin soon.

[Havoc]

0.3 release: hxxp:/www.mediafire.com/file/3dx8k8rhxyib4yc/DQXIS-SDK_RenderFix%252BExtraBinds_0.3.zip/file

This adds custom FirstPersonCamera & EnterNakamaKaiwa (party chat) actions, which can be bound to keys/controller inputs by editing Input.ini - the included Input.ini will bind first-person to N, and NakamaKaiwa to LS/M.

Source is included which shows how these actions are added, and how to call into the UFunctions that make them work.

I'll update OP in a little bit.

[Tim Allahn SnAckbarr]

Originally posted by Havoc:

0.3 release: hxxp:/www.mediafire.com/file/3dx8k8rhxyib4yc/DQXIS-SDK_RenderFix%252BExtraBinds_0.3.zip/file

This adds custom FirstPersonCamera & EnterNakamaKaiwa (party chat) actions, which can be bound to keys/controller inputs by editing Input.ini - the included Input.ini will bind first-person to N, and NakamaKaiwa to LS/M.

Source is included which shows how these actions are added, and how to call into the UFunctions that make them work.

I'll update OP in a little bit.

Very excited about this! Haven't had the chance to check either DefaultInput.ini atm, but I vaguely recall "MiddleMouseButton" being the default bind for "FirstPersonCamera" in OG? Also not sure if "Gamepad_Special_Right" is bound by default in DE, but that may be a suitable gamepad bind without interfering with party chat.

At some point I may post my own Input.ini when I get the chance to check.

Sounds like it may be possible to re-implement the ESC key (exit) bind in the same way, then?

I vaguely seem to recall skimming a file that referenced the alt+f4 bind (with yes/no/cancel dialogue), but for the life of me I can't remember where it was now... xD

In any event, much appreciated!

Edit: Still haven't found what I was looking for, but I did find this method defined in Engine.KismetSystemLibrary (not sure this is what i'm looking for either, truthfully 😅)

void STATIC_QuitGame(class UObject* WorldContextObject, class APlayerController* SpecificPlayer, TEnumAsByte<EQuitPreference> QuitPreference);

EQuitPreference struct

enum class EQuitPreference : uint8_t
{
	EQuitPreference__Quit          = 0,
	EQuitPreference__Background    = 1,
	EQuitPreference__EQuitPreference_MAX = 2
};

[Havoc]

I've pushed DQXIS-SDK to github: https://github.com/emoose/DQXIS-SDK will probably continue dev there and post major updates here.
Latest change there adds some checks to FirstPersonCamera/EnterNakamaKaiwa so they can only be activated when character has movement enabled.

@Tim Allahn SnAckbarr - unfortunately MiddleMouseButton is already bound to ToggleAutoRun, so it's a similar problem to binding FirstPersonCamera as LS, both actions will be used at once, making your character move away from the camera without it following :/

Haven't found a solution for unbinding yet, maybe replacing DefaultInput.ini could let us unbind the controller actions at least, but I think stuff like MiddleMouseButton & some other keyboard/mouse binds get bound by the game code itself, not sure what a good way to handle it might be (maybe patching out all the binds that aren't handled by the in-game bind editor and moving them to Input.ini instead would be good, so the user is in control of it all instead of the game EXE, hm...)

What did ESC key used to do? You can bind it to show the Items/Equipment/etc menu from the in-game bind editor at least, not sure if it could be made to work like alt+f4 though, unless original was able to?

BTW is anyone able to grab the Default INIs from the original and upload them somewhere? Would help a lot to compare things.

[Havoc]

I made a little progress with DataTables, UAssetParser needed some updating to allow getting the struct type from RowStruct property, instead of trying to read it from a PropertyTag, with that in place it can get the structure type of each row in the table fine.

Luckily all the row structures seem to be in the SDK I dumped, for example here's DT_MonsterFieldMovingBounds.uasset/uexp: https://i.imgur.com/EB9StbX.png
Needed a little work converting from C++ to C#, but that wasn't too hard for this struct at least, I'd imagine there's probably some structs that could be a headache though...

Unfortunately for us UAssetParser doesn't seem to have good support for working with structs (eg. values inside structs aren't printed in the JSON/YAML it creates, only DragonML/XML output contains the values, but even then some rows values aren't printed for some reason)

Seems like this is mostly for DataTables that use seperate UE4 properties/columns instead of raw structures, in a way DQ11 using raw structs might be good for us though, if we wanted to make our own parser it's a lot easier to just read in a block of data as a normal C/C++ struct instead of needing to parse all the UObject property types etc like UAssetParser does.

E: ah, seems it can work a little better with structs, here's some YAML output: https://pastebin.com/F21qx7az

Of course UAssetParser only allows deserializing objects though, so we can't convert that back into a UAsset file atm... still would probably make hex editing a little easier.

I wonder if adding the DataTable structs into a UE4.18 copy could let it work with these UAssets & convert between them though, maybe worth looking into some time.

[Tim Allahn SnAckbarr]

Originally posted by Havoc:

I made a little progress with DataTables, UAssetParser needed some updating to allow getting the struct type from RowStruct property, instead of trying to read it from a PropertyTag, with that in place it can get the type of each row in the table fine.

Luckily all the row structures seem to be in the SDK I dumped, for example here's DT_MonsterFieldMovingBounds.uasset/uexp: https://i.imgur.com/EB9StbX.png
Needed a little work converting from C++ to C#, but that wasn't too hard for this struct at least, I'd imagine there's probably some structs that could be a headache though...

Unfortunately for us UAssetParser doesn't seem to have good support for working with structs (eg. values inside structs aren't printed in the JSON/YAML it creates, only DragonML/XML output contains the values, but even then some rows values aren't printed for some reason)

Seems like this is mostly for DataTables that use seperate UE4 properties/columns instead of raw structures, in a way DQ11 using raw structs might be good for us though, if we wanted to make our own parser it's a lot easier to just read in a block of data as a normal C/C++ struct instead of needing to parse all the UObject property types etc like UAssetParser does.
Maybe worth looking into some more eventually.

Wow, nice!! May look at doing a hex edit using some of those float values now that I know what I was also looking for ;)

It's my hope that changing "ChaseRadius" will increase the range that monsters will chase you. I believe that the run speed values are actually stored in DT_MonsterPhysicalProperty ( my bad :X )

Originally posted by Havoc:

BTW is anyone able to grab the Default INIs from the original and upload them somewhere? Would help a lot to compare things.

Gladly!
OG: https://drive.google.com/file/d/1qhBwTivx73Ldr9zq_WqwEkDZhaL9Ln_A/view
DE: https://drive.google.com/file/d/1f9rjrFPMnj7nZTZ7th40uDpWcFLkpA4E/view

Originally posted by Havoc:

What did ESC key used to do? You can bind it to show the Items/Equipment/etc menu from the in-game bind editor at least, not sure if it could be made to work like alt+f4 though, unless original was able to?

The ESC key used to pull up the exit game prompt ("Are you sure you want to exit Dragon Quest XI?" yes/no). I believe this used to be a default behavior in older versions of UE4 Editor[forums.unrealengine.com], which carried over into OG. Whether this was intentionally removed or a side-effect of the newer engine, i'm not sure.

They also removed the "Exit Game" and "System Settings" options from the main menu (pretty sure this was a BP edit carried over from the Switch release), resulting in players being "unable to close the game" or not knowing how to access the settings. Thus they created a topic instructing players to use alt+f4.

[Havoc]

Originally posted by Tim Allahn SnAckbarr:

It's my hope that changing "ChaseRadius" will increase the range that monsters will chase you. I believe that the run speed values are actually stored in DT_MonsterPhysicalProperty ( my bad :X )

Hope it helps: https://paste.ee/p/wt57p

The enums like run speed/anything else starting with _ are single bytes btw, you can see the rough struct in the SDK as FJackDataTableMonsterPhysicalProperty (pretty much matches the struct in the uexp, except the UnknownData padding bytes aren't included there)