Sync large data

Zackri

New member
Joined
Jan 15, 2022
Messages
3
Reaction score
2
I have about 3 million files and folders sized (240G) on a local PC . Once the main job is synced only a thousand or so files change every day.

What I am trying to do is to sync the entire contents to a webhost via ftp and keep it updated as the files change on my local PC.

So what I need , is to transfer only the changes that happens on the files in the same time it changed on the destination .

I tried using GoodSync for this but it takes 15 hrs to analyses and by that time a number of files have changed.

Is there any software or string of batch files that would do this for me and utilize multiple connections for a ftp username?

- Greetings
Zack
 

Ian

Administrator
Joined
Jun 8, 2021
Messages
212
Reaction score
102
Is there a way you can use something else other than FTP? It's not really suited to syncing files like that unfortunately. From what I understand, it'll always be fairly slow via FTP as it's hard to check so many files to see if they have changed.

Depending on the destination system you could use something like rsync (which works fine in Windows 11 using WSL).
 

Zackri

New member
Joined
Jan 15, 2022
Messages
3
Reaction score
2
you are right, Onedrive would be a good choice, I am checking all ways now
rsync run on windows?
 

Ian

Administrator
Joined
Jun 8, 2021
Messages
212
Reaction score
102
rsync run on windows?

Yep, if you're familiar with Linux you can install the Windows Subsystem for Linux (native to Windows 11) and use terminal commands like rsync :)

There's a tutorial on it here:
 

Zackri

New member
Joined
Jan 15, 2022
Messages
3
Reaction score
2
I appreciate the helpful info which you provided and I will take it into consideration
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Top