There are certain Windows components which you can disable, but not permanently remove as they're built into Windows more or less. This includes Internet Explorer and Media Player. All removing them from the components list does is remove the shortcuts and file associations for those programs, but it doesn't actually remove the programs. They're still there and they can still actually be run by locating the executable on the hard drive or by typing the executable name (ex. iexplore.exe and wmplayer.exe) into the "Run..." dialog box.
There are ways to permanently remove them, but I don't recommend it as some programs automatically open those Windows programs instead of what you might have as your default player or browser.