If you cout your pointers, make sure they're not char pointers unless you like seeing random strings of characters.
Also, there's no need to write a new swap function; the STL already has one. Just #include <algorithm>.
There's also an iter_swap. It swaps what each iterator points to (and pointers are considered iterators). I'm not sure if this is the behavior you're looking for, or if you just want swap.
Finally, in your initial post, you say yo were able to swap when you pass in by value... I'm not sure how you would have accomplished this. Is it possible you're getting the correct results with the code that's been posted, but are comparing them to the results of your "swap by value," which is actually incorrect?