Is there any way I could check whether the server is vulnerable or not?

[jewell]

I have a server and I've just configured the SMTP server for it. Anyway, I'm a newbie system admin and I am not sure whether I've configured it properly or not. I mean I am afraid that some hackers/spam programs may be able to send emails via my mail server.
Is there any way I could check whether the server is vulnerable or not?
 

[sharif934]

You can always test a server for Open Relay via a command sequence like:
telnet domain.com 25
help senderdomain.com
mail from: [email protected]
rcpt to: [email protected]
data

Generally if a server does not allow Open Relay then "rcpt to" command will return an error.
Anyway, sometimes if you do the test from an intranet due to configuration settings that may not happen.

Having this in mind I would suggest that the best way is to use an online Open Relay checker like this one:
http://tools.geek-tools.org/en/open-relay/computerhope.com/1

For example computerhope.com has their mail server properly configured, they don't allow Open Relaying as you can see in the above example.

Hope this answers your question.