This happened to me with IE on my desktop, had to kill IE with the Task Manager. I suspect the shortcut icon is hijacked, delete it & create a new one. Or are you referring to IE on the Start Screen of the tablet? You can pin & un-pin those icons. I rarely use the Start Screen on my tablet, since it takes you to the desktop for many apps, i.e. Firefox.
BTW, my tablet is an Asus Transformer T100.