Just loaded realMyst on my PC. First - simple specs:
Win 7 64-bit Intel Core i5 2500k AMD Radeon 6950 1 Gig Catalyst 11.12 drivers (yeah - I know they're old - it's a long story.)
I have realMyst set to 1920x1080 32 bit resolution and Texture Quality to High
I was having the crash when realMyst would try to play videos. I navigated to the C:\Program Files (x86)\Steam\SteamApps\common\Real Myst directory, and set RealMYST.exe to "Run this program in compatibility mode for:" Windows XP (Service Pack 3). That seemed to fix the issues - but I'm still testing it.
So far - so good. I'll post back if I need to tweak anything else.