Unsubscribing just sets a flag that steam doesn't take this mod into account when deciding which files to change with modded ones next time.

Steam only organizes (and deletes) files that were installed with the game, additional ones like mods or map scripts are totally ignored - ever.

To really uninstall a mod, you have to verify your game cache - that deletes all changed game files and reloades the default ones. At next game start needed files from still subscribed mods are extracted. Therefore if that is much data, it can lead to missing modded data when cancelling the intro (and extracting of mod data) too early.

For AoEs mod system uses additional files for several things (like mines, forests etc), and these files aren't organized by steam (as I mentioned above), even with verifying your game cache, sometimes there will remain modded data if you haven't deleted them manually.

So if you want to delete a sound mod, it can help to delete your game\sound folder before verifying.

In case of a graphic mod you should delete the game\data\slp folder.

And for steam sometimes gets confused about your mods status, it definatelly is a good descision to delete all unsubscribed mod files from your mods folder.