Skip to main content ↑ ↓

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: Keeping FLAC files on two drives in sync with rsync on linux (Read 1624 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

Keeping FLAC files on two drives in sync with rsync on linux

Hello, I have two external drives. One of the drives is named "drive2" and it contains a folder named "Music." This folder has the following structure:

Code: [Select]
drive2/
    /Music/
           Pink Floyd/
                      1982 - Album Name/
                                        01 - Track.flac
                                        02 - Track2.flac
                                        and so on..

So, because of the folder hierarchy I guess I need a recursive sync.

I need the entire Music folder from drive2 to be copied to drive1. I think I can use something like this:

Code: [Select]
rsync -av drive2/Music/ drive1/Music/

However, there are situations when I modify the metadata of certain songs. Those metadata modifications are small like changing the title of the album, and those modification don't necessarily change the size of the FLAC files, but their md5 fingerprint definitely changes because of altering the metadata. Right?

I noticed that when I use rsync -a, the rsync utility notices the metadata changes of FLAC files on drive2 and updates the files on drive1 as well to be in sync. Only the changed files gets transferred, which is exactly the behaviour I want. It seems to me that the -a (archive) flag implies the -u which tells to only update changed files.

However, I'm curious when rsync transfers the files that got their metadata updated, does the old files get overwritten completely on drive1? I mean they are replaced completely? Also, do you see anything wrong in the way I used the rsync command with the `-av` flag?

Re: Keeping FLAC files on two drives in sync with rsync on linux

Reply #1 –
Afaik old files will be completely overwritten. I don't see any wrong way of working, you can always us the
Code: [Select]
-n
option to do a 'dry run' btw.

On my system I use 
Code: [Select]
rsync -zavh --delete
to sync, which is basically the same but will also delete files on the target drive which are not on the source anymore.
 

Re: Keeping FLAC files on two drives in sync with rsync on linux

Reply #2 –
Quote from: zoliky on
does the old files get overwritten completely on drive1
What I understand is that RSYNC is very frugal on bandwidth.
It doesn't  look at "files" but at sectors on the HD being changed.
If I understand its working correctly it won't overwrite the entire file but only the sectors changed.

https://www.digitalocean.com/community/tutorials/how-to-use-rsync-to-sync-local-and-remote-directories
TheWellTemperedComputer.com

Re: Keeping FLAC files on two drives in sync with rsync on linux

Reply #3 –
AFAIK - and googling to refresh my knowledge, but don't trust it without checking for yourself - rsync distinguishes between the following two:

* "delta copy", where it overwrites merely altered chunks of a file;
* "delta transfer", where it only transfers what is needed for delta copy; but, using delta transfer it might still write a completely new file to drive, verify that is correct, and then delete the old. This for safety reasons in case something else is accessing the file.

Delta transfer is the --no-whole-file option. If you also want delta copy, you need to use --no-whole-file and --inplace, it seems.

Re: Keeping FLAC files on two drives in sync with rsync on linux

Reply #4 –
Quote from: zoliky on
However, I'm curious when rsync transfers the files that got their metadata updated, does the old files get overwritten completely on drive1? I mean they are replaced completely?

Yes.

Like Porcus said, there is a way to overwrite directly only the changes but it's not worth it because it requires reading all data on both drives anyway.
It's possible to make rsync transfer only the modified 8K worth of your 32M FLAC file and overwrite that in place (risky!) but before that happens you still had to read the whole 32M on both sides.
Read and write speeds are practically identical on most drives, so you're gaining nothing with delta transfers. They exist for saving network bandwidth at the expense of CPU and I/O load on the endpoints. If you're working with local drives, avoid checksum comparsion/delta copies like the plague.

If you expect to constantly tweak your tags, you should consider an external metadata solution like foo_external_tags or M-TAGS or one of the music database handling programs.

Or switch your filesystem to Btrfs and use snapshots and send/receive. Pros: does what you expect rsync to do, added bonus of version control. Cons: filesystem-specific, you must use Btrfs on all drives and can only read those drives on Btrfs capable systems