No. Do not replace the battery until you have made some tests
Some laptops, not all, will boot up normally without the battery in place. Consult your documentation for troubleshooting tips. Or go to the web site of the maker and see if that have tips about the Acer not starting up with power supply on the mains.
Acer Aspire laptop. No model number?