Yes reformat! You should always use a better file system than FAT unless you have to for compatibility reasons!
I'd suggest you format the drive using the same file system that you are rsyncing from otherwise you will potentially lose permissions and attributes that are set on your files and directories.
FAT (presumably FAT32) could be problematic to due maximum file sizes (4GB), maximum volume sizes (2Tb - one day anyway!), fragmentation etc.
However if none of this matters and you just want a simple, fast & maximum compatible file system then FAT will be OK. Speed wise, due to the lack of journalling, permissions etc, FAT could possibly out perform the alternatives - NTFS/Ext3/HFS...
