Sorry for taking so long. After looking and asking around, I couldn't find a spare power supply to swap with mine for testing purposes.
What I will do is wait until I have the money to build myself a brand new computer and test with the newly bought PSU before building the thing.
In case the PSU turns out not being the problem, I'd like to know if there are other thing I should look for that could be causing these issues. This will save me a lot of troubleshooting time if I know in advance what to look for next.
If the motherboard or cpu turn out to be defective (which I doubt, as they seem to be working properly according to a few tests and visual inspection), well I guess it's gonna be trashed. It's pretty useless to look around for a P4 board nowdays. I'm pretty sure it's the PSU anyway, I noticed its fan never spinning whatsoever, although PSU temperatures are at a decent level, and the power button wont work powering off the machine after a crash, and will require a certain delay to power on.
