I defeated everyone using Sea raider chiefs and Forest bandits.

[njkkfish]

My first castle i took using less than a hundred sea raider chiefs and defended that castle against ridiculous odds. I added a heap of forest bandits and upgraded them, the combination of one of the best infantry units and the best ranged units meant that my army was nearly unbeatable.

At the end of the game they still made up the bulk of my army because i viewed the major factions as being trash in comparison.

[stecph]

Same,it's super fun! I am also using the mod that converts them into Huscarls,even more OP!

My troops are mixed anyways,but as you went 100% outlaws for a while,have you noticed Roughery skill improving? In theory it should if you use outlaws as your army.

[Zerosaiko]

As far as I can tell, other than selling prisoners you can get Roguery for:

Bribes (I think there's a % chance for some reason, which just feels like a pain),
Getting free from being captured
Being tortured for crimes
Sneaking into a town (the XP range is better if you don't get caught)
While raiding
Attacking villager parties or caravans
Force recruiting from villages

I haven't checked the ingame description for leveling Roguery in a while, but getting XP from recruiting bandits or thugs doesn't seem to be implemented, at least in 1.4.2. Don't have 1.4.3, so can't really check that.