When removing the modem drivers, it can help to boot up in safe mode and remove the device form the device manager, then shut down. Remove the card, boot up normally, shut down, put the modem back in, and finally turn it back on. Make sure you have the latest drivers for your modem when you do this. I realize these steps are many, but from experience, modems can be a bit of a bugger with the Win family.
Good luck.
Joshua