I am playing as a free mercenary group and I was hired to rescue that guy. Upon releasing him from the jail to escape, he suddenly starts attacking me and of course I can't escape with him because is being aggressive with me. I was hired by an enemy faction of his sometime ago, but nobody of Fierdsvain is agressive towards me because I am now a free mercenary and it's been quite a time.

I honestly can't seem to find a way to rescue him because of this, maybe taking down the fortress? But honestly, this would be overkill just to release a random guy so I am guessing I will just fail the mission

Edit:
I just tried to rescue another 2 guys that aren't involved in a mission, it says they join me in prison break but he was moving away from me trying to attack me when I ask them to help me. If I tell them to stay still, they also keep moving trying to fight enemies and end up just KO'd.