Increase your virtual memory size(your page file).
It's by default in your OS partition. Instead letting the OS to manage it set it to twice your installed RAM for both min and maximum.
(to set pagefile size go to control panel, system, see processor and memory for W7 then advance setting, second tab, set it)
DO NOT set page file on mutliple partitions. This would be counterproductive. Set your page file on only a single partition, usually the C: partition where OS is installed.
And say goodbye to stutter.
If you still have stutter know it's now caused by your hardware. Reduce texture to normal and reduce drawing distance to 2,000. And or set trees to low.
To check/test load the default most demanding route which is the steam train routes in Trainz 12 and compare with previous experience.
Edit: Also don't use the custom profile in your NVIDIA graphics setting. Let everything set to application. NVIDIA detects Trainz 12 as Trainz Simulator 2004 and somehow if you mess with that custom profile the performance is going to suffer. I guess NVIDIA thinks you're playing Trainz 2004 and adjusts the setting to give you best performance for Trainz 2004.