"Remove Party Member" does nothing.

[Spookworm]

I just started a new project, there are no plugins, and I am trying to start with a class selection to add and change characters depending on class.
The add party member after making the choice is working, because the new member shows up in the followers list. But I can't remove the main first actor to switch the player to the new character, because the remove party member function doesn't seem to have any effect in game.