Feature to redistribute useless dual class proficiency slots

Hello, it always bugged me when i play a Kensai then dual to a mage, having only mage proficiencies to put pips into until i regain my Kensai levels back. So i would like the option of those points being redistributed to any Kensai (or whatever class u started with) available weapon, when i regain my old class back.
1