Tales of Zestiria - Anti-Aliasing Guide and comparison. (Mostly for Nvidia cards)

Being DX9 and supporting Arbitrary Resolutions gives users (Mostly Nvidia users) a wide variety of AA options available aside from the built in FXAA which doesn't do much on it's own.

This guide lists, describes and compares methods available to use and some basic recommendations for the best quality.

An introduction and short guide to Nvidia Inspector

It is highly recommended you use Nvidia Profile Inspector for Nvidia Users. As it allows greater control on a per profile basis than NVCP(Nvidia Control Panel).
It allows more options to be changed and is FAR more responsive than the sluggish NVCP. (Edit:2017, NVCP has been updated and isn't so sluggish anymore. But NPI is still preferred)

http://forums.guru3d.com/showthread.php?t=403676

Nvidia Inspector setup guide

1. Download the latest version here https://ci.appveyor.com/project/Orbmu2k/nvidiaprofileinspector/build/artifacts
   
   
2. Save it to a location that you can permanently use. Extract it.
   
   
3. When you open it for the first time you will be presented with
   Click the Tool icon next to the "Driver Version" Box to open up the driver profile settings.
   (As of 2017 the interface has changed slightly with more options. It will be similar)
   
   
4. You are now greeted with this screen and the default "Global Home" Profile
   http://u.cubeupload.com/MrBonk/Inspectorbasicexplan.png (You will want to create a desktop shortcut as noted. *Also note that Cubeupload's servers are currently down so you may not be able to see this picture* So to create a desktop short cut hit the icon at the top right next to the icon showing a Camera with a Green Arrow))
   
   These are the settings that globally apply to ANY given application. It is VERY IMPORTANT that you minimize what you force Globally to only basic things such as Texture Filtering settings and Performance Mode.
   
   
5. Here is my Global profile, I suggest you match it (Remember to hit "Apply Changes" afterward).
   
   
   It is worth noting however that whether you want to use "Adaptive" performance mode or "Prefer Max Performance" is up to you.
   Idling on air for me results in around a 10c difference with Adaptive. However with Adaptive enabled, in order to ensure it doesn't affect games you will either need to...
   
   
   • Set it to "Prefer Maximum Performance" each time globally you are going to play a game and reset the Display driver to make sure it takes effect. (Adaptive won't work for me unless I reset it first, same with PMP. Your results may vary)
     
     To reset the Display Driver, open "Device Manager" in Windows, right click your display adapter and hit "disable", then do it again to re-enable it.
     
     
   • Or Set every single game you play individually to "Prefer Maximum Performance" on their profiles.
   
   
6. Tales of Zestiria does not have a profile in the Nvidia Drivers by Default
   Following the notes on this picture
   Create a new profile named "Tales of Zestiria"
   
   
7. You are now ready to tweak Anti-Aliasing
   *If you matched my Global Profile settings, you will not need to change any Texture Filtering settings. As the new Profile will match the global setting by default of HQ 16xAF

Tales of Zestiria requires special Hexidecimal Flags in order to force Anti Aliasing (Different ones depending on the type).

These flags are pasted or typed into the "Antialiasing Compatibility" field at the top

A finished functioning ToZ profile with 8xSGSSAA enabled would look like this
You can additionally enable a negative LoD(Level of Detail) bias for textures if you'd prefer them to be sharper under "Texture Filtering - LoD Bias (DX)".
Or also enable an Automatic LoD adjustment listed as "Driver Controlled LoD Bias".
Though, be warned this can and will cause additional aliasing in some games.

For 8xSGSSAA -1.50 is the correct value and -1.00 for 4xSGSSAA.
Be aware that you do not need to set the "Negative lod bias" to "Clamp" in order for this to work anymore on cards Fermi and beyond.

ToZ supports MSAA,SGSSAA (Sparse Grid SuperSampling).

Here is an image explaining some of these modes (Taken from the second guide listed at the end )
These will all be detailed more thoroughly in the next section for what works with ToZ

To see references for more games see this guide here (That I am the maintainer of)
http://forums.guru3d.com/showthread.php?t=357956

Here is another guide as well maintained by a friend of mine. http://forums.guru3d.com/showpost.php?p=4908843&postcount=1

AA Flags and description for Tales of Zestiria

0x084012C5 - SGSSAA (4x,8x Only)

Do not enable the AA fix in this game. ("Antialiasing fix" in Nvidia Inspector)As it decreases the quality of the AA[screenshotcomparison.com]


*Do not enable the in game FXAA at the same time as it will cause a conflict with SGSSAA*

This flag gives the best results with the least UI blurring issues without affecting AA quality. (That's what the 5 on the end is for.)

The other flags/bits are necessary for; 1. AA to function, 2.For the Shadow Maps and Depth of Field to be AA'd.
(EDIT: Turns out, there are two separate DoF implementations. One that can be AA'd. The other runs at 1/4th resolution;0.5x0.5; fixed based on render resolution that can't be AA'd and is upsampled very poorly, creating a janky, pixellated and flickering effect. However using TZFix's "RemasterTextures=" setting can help a little bit with that by creating MipMaps. In the process increasing load times. SSD might be faster, and if you can Symlink the two big folders "PK0" and "PK1" to a RAMDisk, it might be faster as well. Untested mind you)
*However it is worth noting that for whatever reason Shadow Map Anti Aliasing does not work at resolutions above 2048x1152 for 4x or 8x. So keep that in mind*

DoF example comparison[screenshotcomparison.com] - The screenshots are upscaled to show the aliasing/undersampling artifacts in the shot without AA

0x08401245 - MSAA (No TrSSAA)

This is the optimal flag for MSAA.


*DO NOT enable the in game FXAA when using these modes. As it will cause the screen to flash like crazy during gameplay. This is a consequence of the right flag needed for MSAA to work.*


MSAA works about as well as you'd expect for geometry aliasing. It's not very costly either. Do note, that this won't do anything for various other types of aliasing.
It's still preferable for the most part to FXAA. And if it wasn't currently bugged, you could force Driver FXAA alongside it and mix that with downsampling too.

Other available AA methods.

Other methods available:

*Remember, downsampling + forced AA only works with custom resolutions and DSR!*

GeDoSaTo(Vendor Agnostic): Since this game is DX9, this means downsampling with GeDoSaTo is open to use if that is what you prefer. You can use this to also inject SMAA/FXAA and various shaders. (Using GeDoSaTo SMAA/FXAA on top of forced AA however isn't a good solution)

To Force AA and use GeDoSaTo at the same time requires the DisplayResolution and RenderResolution to be the same.(Also meaning that the resolution you want to use also has to be an available custom resolution you've created in NVCP)
With Nearest filtering enabled, it's essentially the same as using a custom resolution. Using Lanczos though to essentially pre-filter the image does actually improve IQ a noticeable amount

Nvidia Driver FXAA: The Driver FXAA is slightly better (IMO) than the built in FXAA. The downside is that it also affects the UI, but any issues from that will be mitigated from Oversampling (From Downsampling) if you choose to do so. Comparison[screenshotcomparison.com]

VSR and DSR(AMD<>NV): Because this is actually a pretty decent port, it supports arbitrary resolutions. Meaning the sky is the limit!
Specifically for Nvidia Users, you can use DSRTool as linked in the AA Document.[forums.guru3d.com] To get some insanely high resolutions with DSR that also work with ANY forced AA or the in game FXAA!

Custom Resolution Downsampling (Vendor Agnostic): I wouldn't really recommend this as it isn't the best quality. But it WILL result in the sharpest results at high ratios. At the cost of increasing the amount of aliasing. You can search on your own for how to do this if you want.

Picture comparisons.

Reference CPU/GPU in these screenshots is an i7 950 @4Ghz and a GTX 980 STRIX Overclocked.


No AA - Field[abload.de] | City[abload.de] | Cutscene[abload.de]

In game FXAA - Field[abload.de] | City[abload.de] | Cutscene[abload.de]

4xMSAA - Field [abload.de]| City [abload.de]| Cutscene[abload.de]


8xSGSSAA - Field [abload.de]| City [abload.de]| Cutscene[abload.de]

Recommendations

This is assuming you are playing at 30FPS
If you have a Mid to High-end or High-end card Is to use 8xSGSSAA at native if you have a monitor with a resolution under 2048x1152 if you would like Shadows to also be Anti-Aliased. Otherwise 2x2 DSR + 4xSGSSAA would provide slightly better results (And uses about the same or less as 8xSGSSAA at native), but of course this comes without Shadow Map Anti-Aliasing as well. So there are trade-offs.

If you have a Mid-end card is to use 8xSGSSAA at native (Even if over 1080p)if you can afford it (Uses up to 60% GPU or so on a GTX 980 at 1080p OverClocked for an example. So cards less powerful than this should be able to get away with using it.)

Or use 4xSGSSAA with 1.5x1.5(2.25x) DSR. Or as high of a ratio as you can get. (As 4xSGSSAA has a lot more performance room than 8x)


If you have a Low-end card try 4xSGSSAA at native and see if you can manage it.

If not, 4xMSAA with some DSR or Driver/In-game FXAA + 2x2 DSR (4.0x mode in NVCP) if you can manage it.
Here are a couple example shots of in game FXAA+3x3(9.0x- 4098x2304) DSR (Uses about 20-30% GPU)


For fine tuning the sharpness with DSR, just use the Smoothness slider. 10-15% For the sharpest without introducing more aliasing. 20-30% to get as close to the ground truth as possible. I wouldn't recommend anything other than this range. 0% is basically Point Sampling, it will be the sharpest but will also reduce the effect of oversampling and cause aliasing.

Here's a comparison of No AA vs 2x2 DSR +2xSGSSAA using Point Sampling with a different game[screenshotcomparison.com]


I highly recommend you use TZfix to fix the Post Processing scale. This will reduce a lot of flickering and crawling. Especially at high resolutions!

Another good option is with TZfix using the "RemasterTextures=true" setting on a fast SSD to and using DSRTool mentioned earlier in the guide with the built in FXAA. and a scale upwards of 3x3(9.00x) SSAA will look pretty good and will be less performance hungry. And will work better probably if you are aiming for 60FPS. This generates the MipMaps the game should've had to begin with and solves a lot of flickering/aliasing issues.


Edit:2017. TZFix has progressed quite nicely. Aside from the shadow map and dof AA, using TZFix with the Mipmaps you can get good results with just basic OGSSAA downsampling. Give that a shot too if you want a no fuss method.

Troubleshooting

I applied the settings matching your pictures but it still doesn't work.

Make sure you are hitting "Apply Changes" on the right upper corner.

Also, for whatever reason (Read the comments) several users have mentioned that GeForce Experience (GFE) installed with Nvidia Drivers by default seems to be conflicting with applying settings to the game.
Which is odd because I have spoken to others who have had no such issues forcing AA and using GFE simultaneously.

Uninstalling GFE seems to fix the issue for people who have had this problem.

[Sephy]

Can you post a picture of all the settings for 8xSGSSAA? I have no idea what I'm doing wrong since I have what I think is the correct settings but it still has so many jaggies on my screen :<.

(Copying your original profile, I changed [AA Comp], [AA - Setting: 8xQ], [AA-Trans: 8x SGSS]

[BONKERS]

I looked for a few HBAO+ flags without much luck for a stable one. Others have been trying too. If we find one, I will try to do the same :)

[Raikoh]

Thank you very much for the guide , it would be great if you can do one for ambient occlusion too.

[BONKERS]

Glad I could help and I agree that 8xSGSSAA really does look great here.
I hadn't actually tried 4+DSR, but assumed it would be best based on my past experience with tons of other games. (Which it usually is, but is not immune to exceptions)

[Antitype]

Great guide.

After testing all options, I settled with 8xSGSSAA. It's the best when it comes to reducing/eliminating crawling, which is sadly quite the eyesore in Zestiria, even at 4k downsampled. 4xSGSSAA + 4k is another great option for finer details (thought difference wasn't as big as I imagined it would be) at the cost of slightly more crawling and less smooth shadows than 8xSGSSAA.