64-bit does actually have a very small speed boost. Not usually worth it when dealing with hardware which may only have 32-bit drivers, as well as plugins that may not have 64-bit builds available.
But if your entire workflow is 64-bit, it can be a nice bump; especially to help out older hardware that might be a little borderline or prone to stutter, framedupe or skip, where 5-10% extra oomph can give you a bit more margin to handle load peaks smooth(ish)ly.