My two cents:
FireFox has an addon called "FasterFox" which can speed up the application itself. DO NOT enable its caching utilities, its bad for servers.
Ultimately, you should judge a browser by what it has to offer you, and how easy it is for you to use. Speed shouldn't be youe first consideration.
I personally found that Internet Explorer 7 is faster than FireFox, but FireFox has features avaliable to me that are important: a good set of addons avaliable, spell check, better tabs, better quicklinks, advanced options that apply to me.