it appears that the registry has been corrupted (or, at least certain associations) wether this is a problem with malware, I'm not sure.
The "Scan" you saw within your browser was not, at that point, dangerous. The idea being that it will "scan" your PC (actually, it doesn't, but rather "pretends" to scan your PC, as it does not have access to your PC from the browser, and was likely just an animated picture). After the "scan" it would then tell you you need to download a program to fix it. However since you closed it it did not even get that far
. Additionally, since the "Scan" as well as the "popup" are merely images, pressing the "OK" button, specifically, is not possible, and as you've discovered the sneaky buggers simply hotlink the entire image (therefore the X button also takes you there).
I suspect that this new issue is merely a result of corruption of your registry data, and unrelated to your previous issues.
The caveat in this case is the program that might help fix it, System Restore, is- well, a program - and will likely give you the same error. quite the pickle indeed.
Can you run ANY programs? try using "Run as administrator", which executes another program with a parameter, and might succeed. Also, try booting into safe mode, which is achieved by pressing "F8" during startup immediately before the Windows logo appears. (Personally I just repeatedly press it after the initial boot-up screen).