You are missing a very important point: Durability to power failures.
(Excuse me, but a majority of GNU/Linux users are not switched to a UPS
or something.) And that's where XFS totally fails[1][2].
Ext3 has the same problems when not properly configured:
Ext3 does not do checksumming when writing to the journal. If barrier=1 is not enabled as a mount option (in /etc/fstab), and if the hardware is doing out-of-order write caching, one runs the risk of severe filesystem corruption during a crash.
For the record I use ext3, I remember XFS as not being reliable enough (with power failures etc).