The answer is simple.
Memory: 64-bit allows you to address 2^32 times more data in nominal memory
Memory use: New options available for 8 byte (64-bit) fundie data types. However, there is some confusion about what is going on, but it seems to me a char is still 8 bit, but ints and longs may move to 64 bit but haven't seen any answers anywhere. Data won't necessarily take up more space providing ther still exist int and float-types at 32 bits. Double useage is just the theoretical maximum. So theoretically, worst-case scenario you would only address 2^(32-1) = 2^31 more "real" memory provided memory modules become large enough. They already are in fact, as consumer 2 GB modules are available now. This lets you use more memory without trawling back and forth to the HDD which saves a lot more in performance than anything else that will be done.
Cache use: Yes, all addressing is 64-bit which means that "real" cache is cut in half. This reduces performance by about 3-10% depending upon your code.
Registers: 64-bit doubles the number of registers in your processor. Registers are what hold the information for manipulation, so you would be able to do more data manipulation at a time. The drawback is, the FPU's, ALU's and such aren't likewise doubled, but you can still get a 60% performance increase for your code.
Regular code: Effects from coding for 64-bit seem mostly negative. This is because some languages like Java have to generate extra code and procedures for dealing with 64-bit, the net result is an actual decrease in performance, up to 80% in the case of Java.
Summary of performance alterations:
HDD = 0% ~ lim(infinity)%
Cache = -3 ~ -10%
Registers = 0 ~ 60%
Code = 0 ~ -80%
Best case, you get infinite performance increase from not swapping from/to HDD, typical best case otherwise might be a 70% increase, but normal typical increases seem closer to 15-20%, and in the case of Java you could lose about 30% performance-wise.
Very wierd, but there are few-hard and fast rules unless you look at the fundamentals of what's going on in each case and construct the probable outcome from there.