Mirror to a slow destination over snapshot

[mapapo]

Hello and greetings from Austria,

I want to mirror the contents of a directory to my Amazon S3-Space which is mounted via Jungle-Disc as a drive Letter. The directory including all subdirectories has 90000 files, so the scanning for the changes takes awfully long.

Is there a way to:
1. Compare to a Snapshot of the Mounted Drive
2. Make alle the changes (Mirror including deletes and so on) on the Drive
3. Storing a new snapshot including all the changes made in 2

All this should be done via script and use just as less network-traffic as possible.

Many thanks

[Aaron]

Hello,

Firstly, scanning can take awhile, but how long is it taking exactly? What comparison criteria are you using? A timestamp/size comparison should not take too long, and if the whole solution is automated to run overnight it is built into the process. Using snapshots may help save some time if you run the comparison multiple times against a single snapshot, but as soon as you update the snapshot side, you will need to generate another snapshot. It also adds in a great deal of complexity to running the comparison.

To answer your questions:
1) Yes it is possible to make a virtual Snapshot. This would be a virtual list of the files on the JungleDrive, including timestamp, size, but no actual file contents.

2) You cannot commit changes locally, but you can use the Snapshot to run a timestamp/size, or CRC comparison if enabled. This allows you to select differences without connecting to the JungleDrive, then use the CopyTo command to copy those differences. This assumes the JungleDrive has not changed since the last snapshot was taken of it.

3) After successfully updating the JungleDrive, you could load it individually in a new tab and generate a new, up to date snapshot.

These commands can be done with script, but it would not be trivial to setup. I would strongly recommend using the graphical interface initially. It will provide the feedback and give previews of what is about to happen. Once you are comfortable with the steps necessary you could then Script a solution.

[chicagoskiguy]

Similar Question

We have a site with about 100k files. Users can both upload and then delete the files. We wish to schedule a nightly mirror of our live site to our remote server.

Can Beyond Compare be automated to handle the copying of any new/modified files as well as removing any files which users have deleted?

[Aaron]

Hello,

As long as the backup is one-directional, yes.

A Sync:Mirror command will make an exact copy of the Source to the Destination. This will copy any differences (older, newer, different files) that exist in the Source over to the Destination. If a file exists in the Source and not in the Destination, it will copy it. If it exists in the Destination and not the Source, it will delete it. As long as the Live site (source) has the only modifications, and the Remote server (destination) needs to be an exact copy of it, then the Sync:Mirror command could meet your needs.

I would strongly suggest testing the sync logic with already backed up, Test directories. Many sync actions such as copy or delete are not Undo-able. Once committed, the sync is not reversible. This is especially true with the Mirror command, since it is capable of deleting files.

[Aaron]

You can use our Folder Sync session type for a graphical preview of the sync. A Folder Compare session, using the Actions menu -> Synchronize tools could be used as well.

The automated part of the process would be to use Beyond Compare Script, which is called from the command line using: bcompare.exe @"C:\script.txt"
Scripting commands are documented in the Help file under the Scripting Reference section, and in Using Beyond Compare: Automating with Script section.
All scripting actions can be mirrored in the Folder Compare session's graphical interface, so I suggest testing with that first to get a preview of how the action will be performed. Script will not offer a preview before committing, and as I mentioned above is not Undo-able.