With Crysis' 32-bit executable I get about ~40fps on High settings. with the 64-bit executable I get a near constant 60fps.
If the software is written well to take advantage of 64-bit it will go faster. However, you can't run any 64-bit executable if you don't have a 64-bit OS. Also, since Windows x64 and it's many components are 64-bit executables and libraries they will largely run faster.
Windows x64 doesn't take any more memory then the 32-bit versions. As I've noted previously, the fact that handles take 64-bits rather the 32-bit is hardly even relevant. Many folks claim that the only reason to ever use a 64-bit OS is to access more the 3.5GB~ of RAM, but this is utter nonsense. you can access up to 32GB of RAM (or maybe it was 64, it was a lot, anyway) with Windows Server 2003, which is a 32-bit OS. But Windows Vista/7 x64 is both cheaper and still "in service" so to speak. Also, a 64-bit processor has the ability to handle 64-bit instructions, which have register sizes that are twice as large. This helps many games, which often use 64-bit data types (for some reason) which take 2-4 instructions to process on a 32-bit processor but only one on a 64-bit processor.
Also, many people would have you think that 32-bit programs run slower on a 64-bit Operating System. This is true for the Itanium processor, but with any consumer-oriented 64-bit system this is not true. the AMDX64 architecture in common use runs 32-bit code natively; the only "emulation layer" required is the WoW (Windows on Windows) set of libraries, which, like the 16-bit version of the same name that ran on 32-bit systems, is more designed so that you can do things like copy and paste and perform other operations between tasks; the actual instructions still run natively without any sort of "translation" by either the OS or the processor.
Either way, 2GB is pretty much the bottom rung for either the 32-bit or the x64 version of Vista or windows 7; Wether you stay with 32-bit or move to x64 is largely irrelevant because most of your speed is stifled by constant disk swapping. Therefore, as rthompson says, you would probably find a larger speed increase by adding more RAM.
On the other hand, upgrading to windows x64 shouldn't cost a dime. All windows licenses apply to both the x64 as well as the 32-bit version of the OS, from what I understand.