When you restart the sshd server you just restart the listening service. Each scp session spawns a new sshd process. The 'stalled' process is unaffected by the restart.
Do a 'ps ax' or 'pstree -hupla' to list the running processes. 'top' will also show if you have a cpu-hog, or I/O-wait issue. 'lsof -i' will show which processes are listening, and which have active network connections.