So far I have upgraded 7 of our backup servers from BrightStor v9 to BrightStor v11, here are my tips on a successful upgrade:
1. Uninstall v9: Uninstalling will get rid of the \DATABASE folder, but I have had mixed luck with the existing v9 databases when I upgrade to v11. Because of this, I found its simpler to let this folder get deleted and then let v11 create new blank databases for you. This will mean you would have to merge the tapes prior to arestore, but if you don't have to restore that often this isn't a big deal.
2. The \00000001.qsd folder: Prior to uninstalling v9, make a copy of this folder since this folder contains all your job information. After you uninstall v9 and install v11, stop the job engine, delete the new 00000001.qsd folder and then move the old one in. Start the job engine again and you will have a clean install of BrightStor v11 with good databases and all your old jobs.
Once I started doing these two things I have had no problems at all with the v11 installs. I just had way too many problems with the regular upgrades, so I uninstall v9 first now.
Hope this helps!