Ultimately it comes down to what you are doing, the Z620 is a good couple of generations newer than the Z600 so while it has less cores and probably less total CPU power, it'll be significantly more power efficient and therefore quieter. It'll also support newer standards such as USB3.0, SATA III and potentially newer PCI-E standards. The Z600 technically has more cores although with a pair of 130w TDP CPUs it's going to use pretty crazy amount of power compared to the Z620 with its single 95w TDP chip.
Another consideration would be the power supplies, both machines use proprietary units so upgrading it isn't really going to be an option for adding a GPU. You'll need to check your particular machines although from what I can see, the Z620 has a 800w PSU which would be plenty headroom for a decent GPU. The Z600 looks to only have a 650W unit which while it isn't terrible, you may risk pushing it a bit given the power required by the CPUs in that system.
The CPU in the Z620 is actually 6 core although if you check eBay you can easily pick up compatible 8 core Xeon E5 chips for well under $100 - There's currently a massive number of them available on the market due to regular refresh cycles in large enterprises and therefore the prices are ridiculously low!
The way I see it, unless you need the absolute most CPU power you can get, the Z620 would be the machine I would go for.
