at this point of thread, the question is not clear:
1) your server has only an IP-address and you want
that only spcified clients can access to it
2) your srv may have more than 1 address (more than 1
adapter or aliases on 1 card) and you want that any client
(or a subset of them) may have access only to 1 of server
ip address.
In case (1), for ftp, exists files that allow/deny client
to connect server (it does not exist for telnet)
In case (2) ip/filtering may be used (is not easy)
bye