Not sure about "at will", but IMHO there certainly ought to be some more convenient way of unslotting IOs than losing a full respec.
Maybe a "mini-respec" - a small, soloable, story arc that lets you visit a new contact and go straight to the 3rd respec screen...no chance to change powers or slot placement, but all enhancements pulled out and reslottable/keepable in the 10 enh tray. Remove the "sell" option to stop people using it at 27, 32, etc... to flog off all their SOs and use the inf to start over. 
Yeah, that's probably a better idea - works for me!
L50s: Tanks: Cryofission - Ice/EM - Dr Celsius - Fire/Ice - Saint George - SD/SS | Controllers: Psichosis - Ill/Kin - Major Chaos - Ill/Stm | Scrappers - Neutron Crusader - DM/SR
Currently Levelling: Angelic Blade - BS/WP Scrapper | Seeds of Destruction - Plant/Kin Controller
I find it extremely frustrating that you cant add and remove IO's without respeccing.
A respec is designed to allow you to re-choose powers and slots which is fine and suited to it's purpose it's also the only way to release IO's without having to drag them into the bin or lose them by putting another IO into the same slot.
Given the cost of IO's this is not a workable solution, so my suggestion is allow IO's to be added and removed at will.
L50s: Tanks: Cryofission - Ice/EM - Dr Celsius - Fire/Ice - Saint George - SD/SS | Controllers: Psichosis - Ill/Kin - Major Chaos - Ill/Stm | Scrappers - Neutron Crusader - DM/SR
Currently Levelling: Angelic Blade - BS/WP Scrapper | Seeds of Destruction - Plant/Kin Controller