If I worried about anything, it would be the clients that will be connecting to my VPN. You absolutely do NOT want your employees using their own home computers to connect to your VPN, because you cannot control the security of their systems.
If an attacker can gain control of the client computer, then you may as well turn off your VPN because it provides absolutely no security. Once they get control of that system, then they can get both the private key and the passphrase.
We have a policy that no personal computers are used to access our network ever, even from home. That way we can control what applications are in use on every computer connected to our network. Plus, then you have fewer intellectual property issues, as you own the computer on which the work was performed.
pansophic