Rescue from Fort Kastav

[Flyin' High]

so I finally got around to doing the empire civil war quests and this one is just strange to me

from what I can tell it's supposed to be like a stealth mission that turns violent after you free the prisoners but when I went to go do it I, didn't get a marker or anything for an Imperial camp, only marker was the fort itself, when I walked over to the fort I saw Hadvar fighting off like 40 stormcloaks in the courtyard with no other Imperials around
walked inside and the fort itself was completely empty (they were all outside I guess), freed the prisoners then Hadvar told me I did a good job and I finished the quest without ever seeing a single Imperial outside of Hadvar and the prisoners
so I ask, how is this quest actually supposed to go???

[DPMR]

https://en.uesp.net/wiki/Skyrim:Fort_Kastav

[Nabinkemper]

One issue is the Seducers camp that is located next to the Imperial spawn location outside of the fort. When you arrive, they will be in battle. The Seducers will kill all but Hadvar (essential). The player will have to kill all of the Seducer mercenaries before continuing with the quest.

Additionally, depending on character level, there may be frost trolls nearby to contend with.

My guess for the issue you had would be one of the above enemies ran toward the fort with Hadvar in pursuit and got within aggro range of the Stormcloaks.