Which platform? If it is Nokia IPSO, then there is a Voyager option to switch it off. http to the box and log in as admin.
If unix in general, then try man arp and looking for the option to remove an entry from the arp table. You would need to research how to make the change permanent - if required.