Good idea to replace a PSU with at least the same wattage as that removed; I would think that the Antec 350 ought to be adequate for your system, though.
Looks like system came with 128 MB RAM - have you added more?
I guess that I would: a) re-seat the RAM, b) disconnect everything except harddrive, monitor, mouse, keyboard. (Onboard video, right?) I'm still confused about the HD - is it good or not? If boot, add back peripherals one at a time.
If no boot, I'd call emachines and ask if they have a standard PSU or something proprietary - tell them what you've changed and see what they say. I'm still concerned that you've got that spontaneous start situation, and, it seems to me that that could be a sign of a non-standard PSU.
And...don't get mad
...are you sure that your monitor works?
Please try the above and post back.