The best way to do this is as follows:
Install Backup Exec to the new server, including all agents, etc. Verify all the services start and you can use the software. If everything looks good, close Backup Exec, stop all the services (start with the Notification service) and then open Windows Explorer. Locate the Catalogs & Data folder where Backup Exec is installed. Rename these for now. CatalogsOLD and DataOLD is fine. Copy over the catalogs & data folder from the server you want to migrate from. Once everything is copied over, restart the services (start with Agent Browser). When you open Backup Exec now, you should have all your tapes listed in the restore selections & jobs & job history in the under Job Definitions & Job Monitor. Note: You will have to recreate media sets & drive pools if you had these unless you copy over the pvl.mdb file as well.
Always rename the files you intend on replacing in the event that the migration has problems. Call techsupport if you feel uncomfortable attempting this yourself.