When you remove Baeloth from your party he just disappears. No leaving conversation is initiated (to potentially change your mind) and the character is gone and unable to be talked to and asked to rejoin in future.
In addition, you are unable to save the game in the area that you remove Baeloth, it continually says 'you are unable to save at this time'. Even leaving the area and re-entering does not fix the situation.
Expected behaviour: Baeloth should initiate a leaving conversation on being removed, and should remain in the area in which he was dismissed (or move to a nearby inn). Removing Baeloth should not prevent saving the game in that area.