Speeding up local rsync transfers
Topic: Technology| 1 Comment »At long last, I am rebuilding my storage server. Apparently 82,000 pictures/movies/etc is just too much for my little 750 GB drive to handle.
So I turned to my Linux partition to get the job done. I’m building my storage server on Linux, and the data I had to transfer was on a windows NTFS drive, so I knew the transfer would be slow-er, thanks to NTFS-3G and FUSE but when I fired up rsync to (hopefully) replicate times, dates, and such on the EXT4 filesystem, I thought the transfer rate was a tad slow, ~15-19 MBps.
sudo rsync –verbose –progress –stats –compress –recursive –times –links –exclude System\ Volume\ Information –delete /tmp/evectra/ /tmp/storage/
So some googling later while the files were copying and I discover that on local transfers, –compress really slows down the transfer rate. So, armed with that information, I axe the option:
sudo rsync –verbose –progress –stats –recursive –times –links –exclude System\ Volume\ Information –delete /tmp/evectra/ /tmp/storage/
Apparently, in this case, –compress took 30% off my speeds. Yuck. Now it’s much faster, ~29MBps. Which isn’t great, but what can you do? Besides reboot into windows, remount the destination drive in another machine, samba mount the drive, and experiment with copying the files over Gigabit.
I’ve got to resist the urge to mess with things to get better speed. 