First off, make sure you are not being used as a relay. Secure your server.
The problem with spammers is they have there own mail servers that can say its from ANY domain. For example, if your email address is @1234.com I can set up my mail server to send mail from 1234.com and if the recipient isn't doing reverse lookups then they will accept the mail.
And there is nothing you can do about it really.
Have a look at the mail server forums on how to secure you server.
Iain