Ok now it's clear.
Is this the only "command" that is slow to respond?
Everything else is good?
If this is the only one, do you have a failover unity server? If so are they on the same lan or are they spread across a wan link?
I would try rebooting your servers at some point, it might clear the issue.
4 seconds do sound long, have you verified that it is actually that long and that it is system wide and all the time?