The game is unable to perform important visibility optimizations when you go outside of playable areas. This causes everything to be rendered at once, which may be too much for the game or your hardware to handle (the map already pushes a bunch of limits as it is). Reducing the number of addons you have enabled and/or switching to the 64-bit branch of GMod may help. This might also help with the crash you're experiencing when walking into certain areas.

FYI, I'm deleting your posts in other threads where you described the same issue. Since you've already made this thread, let's keep the discussion here to prevent the discussion from getting fragmented.