Brawlers keep drawing their weapons and trying to kill me?

[Miss Vela]

I've brawled two people and both of them, after I get a few punches in, will draw their weapon and start trying to kill me and I have no idea why. First was the bard in Whiterun bar, where you brawl him to make him leave the veggie merchant alone, he drew his dagger so Lydia killed him. Second is Uthgerd the Unbroken, she drew her sword, so I reloaded and had Lydia wait outside, same thing. Dismissed Lydia, same thing.

Why are brawlers cheating and trying to kill me? I have several mods, but I don't think any of them would effect brawling.

[Nazenn]

Install the Brawl Bugs Patch from the nexus.
Any mod that adds in cloak spells as a way of giving you or NPCs around you effects, such as Enhanced Blood Textures or Wet and Cold (I think) or a dozen others will cause the NPCs to think you are cheating even if you arent actively using magic. Brawl Bugs Patch should fix that

[Miss Vela]

I do have Wet and Cold, Cloaks of Skyrim, Winter is Coming and Immersive Armors & Weapons. I guess I didn't realize mods that change the appearance of some npcs would cause brawlers to think I was cheating. I'll find the Brawl Bugs Patch, thank you.

[Hevy]

That happened to me as well . I just use Player.setav attackdamagemult to make my fists do more damage.