I'm dealing with a similar problem, linking a BCM50 v1 and a BCM50 v2. I discovered yesterday that the router at Site B wasn't handling the H.323 packets correctly. A firmware update (a costly one, at that) now includes an H.323 proxy that SHOULD fix my problem.
If nothing on the Nortel side of things gives an indicator of the problem, you may want to check the logging on the router...
-mike