Pilgrimage won't get rid of my infamy

[Punished SKRUB ™]

I have visited all nine altars TWICE but they keep telling me to continue my penance. I thought that making the pilgrimage would get rid of my infamy but I still have 1 and visiting the nine altars done seem to work. Is there something I am missing or is this a glitch?

[Valden21]

It sounds like you're getting the altars and the wayshrines mixed up. Scattered through the wilderness are multiple wayshrines to different Divines, and visiting a wayshrine to each Divine is what resets your infamy. What you're looking for is a structure consisting of a small ring of pillars and a circular altar in the center. Each one of the wayshrines looks like that. The KoTN DLC should help with this, as it happens to be the very first quest of that DLC.