32-bit versus 64-bit, which one should I choose?

A common question that pops up from time to time is whether someone who has a 64-bit CPU (such as an AMD64 or an Intel EM64T) should install a 64-bit or 32-bit version of the preferred OS. Since these processors are capable of running either one at full native speeds, what are the advantages and disadvantages to both? 64-bit just sounds cool and faster, but we all heard about headaches installing the OS, drivers and software. Are they still valid ...