You can sell pretty much every gem you come across, there are some item upgrades that require some gemstones (off the top of my head: Diamond, Rogue Stone, King's Tear) but they become available much much later in the game.
For SoA, I think you only need to keep a Rogue Stone.
Even if you do sell them, you can always buy them back
For SoA, there's also one crafting which calls for a Diamond, another which calls for two Diamonds (and a Silver Necklace), and another which calls for a Beljuril.
For the avoidance of confusion, the Rogue Stone mentioned above by @DetectiveMittens isn't needed for crafting anything in SoA, but he's correct that there's a reason to keep one - there's a particular door which will open only if you have it in inventory.
In ToB, there are numerous jewels used in crafting - Rogue Stones, King's Tears, Star Sapphires, Emeralds and Diamonds are all involved, and in some cases more than one. However, ToB has its own supply of jewels to find, so you needn't worry about bringing them with you from SoA.
As for how to craft, in SoA ...
there's a dwarven smith called Cromwell in the Docks district who does most of your crafting, and a human smith called Maheer in Waukeen's Promenade who crafts a couple of items, and also if you do Neera's personal quest then there's a halfling tinker called Knocktor in her Hidden Refuge who will craft one quest-specific item for you ...
and in ToB ...
all your crafting is done by an impish smith called Cespenar in the Pocket Plane, except for one special set of equipment crafted by a gnomish researcher called Kerrick in Amkethran.
Edit: I'd omitted one special crafting, now added.
In BG2 there is a quest where you have to find some illithium metal for someone. This can be used for an upgrade although it's hard to figure that if you don't know.
Instead of completing the quest, you can bring it to Cromwell to upgrade a specific weapon. Cromwell usually tells you if you have a weapon that can be upgraded and what you are missing, but not in the case of this weapon.
Comments
For SoA, I think you only need to keep a Rogue Stone.
Even if you do sell them, you can always buy them back
For the avoidance of confusion, the Rogue Stone mentioned above by @DetectiveMittens isn't needed for crafting anything in SoA, but he's correct that there's a reason to keep one - there's a particular door which will open only if you have it in inventory.
In ToB, there are numerous jewels used in crafting - Rogue Stones, King's Tears, Star Sapphires, Emeralds and Diamonds are all involved, and in some cases more than one. However, ToB has its own supply of jewels to find, so you needn't worry about bringing them with you from SoA.
As for how to craft, in SoA ...
and in ToB ...
Edit: I'd omitted one special crafting, now added.
(is that cryptic enough without spoiling?)