Well, you have the right slot for it, but you only have a 300W power supply, so you'll be somewhat limited in what you can upgrade to. I'm struggling to find a low powered card that is actually better than a 7660D onboard - the GT 620 is significantly weaker.
Edit: also, you are correct, you would only need to insert the new card and start the PC, then install the appropriate drivers. No other configuration should be necessary.