Tell us a little about your setup. How was the system backed up? Compression/Multiplex/Multithread... Tape Technology (LTO/SDLT...). How many files are you trying to restore; a lot of small ones or just a few big ones? Network speed, 10/100/1000? Full Duplex?
A lot can effect restore processes. What I usually tell my customers is changes you do to increase the speed that the system backes up, there is a negitive effect to restoring the system. Not a one for one, but there is an effect.