It depends a lot on the map and the settings. He grabs a lot of resources and fiercely defends them. That could be fatal to you early game if there aren't that many resources.

His castle build order is far from optimised and he may struggle balancing building the castle with building an army. This is why I recommend pre-build castles.

Also as you can see in the screenshots, the castle is massive. So avoid small maps.

Is this lord only scaling and strong in later game or will he kick my ass early on also?

Im searching for a scaling lord :-)