I assume both the domain controller and terminal server are on the same domain, and they are connected to a fast network (at least 100Mbit).
Main problem is that the data has to shift across the network, which even at 100Mbit isn't as fast as a good hard drive. Also, load on the domain controller) and terminal server) will affect the speed of database access (as well as anything else it is used for concurrently).
John