Large Folder Group, BC crashes

[Phototechman]

Hello Everyone,

I'm using the current version of BC and I'm having an issue with scanning large amounts of folders/files on a NAS. I have a 2tb NAS where I want to move all my images over to my new 16TB NAS.

I started the scan at 2pm friday to move all the files over to my new NAS. The scan crashed out at 6pm friday, the program was not responding.

I do have a TON of files and folders and images on the NAS, but windows can scan and give me the amount of files in about 15-20 min. Where BC was taking hours to scan. I tired to just sync and BC still was saying at the bottom 'scanning', this went on for about 4 hours then crashed BC again.

INFO about my systems

- all network is gigabit, all machines are gigabit.
- the pc i'm using is a i7, 16gigs of ram, 2x2tb in raid0, gigabit adapter
- one nas is 4x 1tb drives in raid 5
- other nas is 8x 2tb drives in raid 5
- current version of BC

So my questions are this.

1. is there any log files that BC creates, I didn't see anything useful in event viewer.

2. is there a way to just make the folder scan start, instead of just scanning all the files first then start the copy process?

3. Has anyone else had any issues with transferring a large group of files/folders. I have 2tb's of Tiff images, I'm guessing around 1-1.5 million files. (Yes, million.)

Any help would be great, I've been using the robocopy to complete this job but its wicked slow.

[Aaron]

Hello,

Unfortunately, BC3 may not be the right tool for this task for you. Beyond Compare 3 is designed to load and show all of your files graphically (with alignment, comparison, status color, logging, etc), and then have the user confirm what they would like to do. This is much more intensive than finding the Properties of a folder in Windows Explorer and counting the file sizes.

For a basic copy command, where you do not need to run a comparison, you can use a command line tool to throw all of your data over to your new 16tb NAS. You could use a program such as terracopy, xcopy, xxcopy, robocopy to perform the transfer over. These do not need to store and display all of the file information at once, so they can perform the copy without the full "loading" of the files.

Lastly, is Beyond Compare crashing and showing a dialog, or a Not Responding Window? If it is not responding, it may just be working in the background and too busy to respond to the Windows ping. If it is a crash dialog, is it our crash dialog (showing the option to Send Save or View the crash report, and to Continue, Restart, or Close the application)? If so, please send in the crash report and include in the text description info on the use case and how many files you were comparing, etc. The crash report includes your settings files, but does not include any information on what you were doing with the program at the time of the crash.

As for each of your questions:
1) logging is disabled by default, but can be enabled in the Tools menu -> Options, Folder Views, Log.
2) Scanning (pre-copy Scan done as you attempt to copy a file) will stop after 30 seconds if the folders are built. If it is not, it means the folders were still being built in the background before you attempted to perform the copy. Please wait until the folders are finished building entirely before attempting the copy command.
3) Our currently max user case tests are available here:

http://www.scootersoftware.com/support.php?zz=kb_maxfilev3

This was a local test, however. Copying over a NAS, for example, would require a download of the file, and then an upload to the other NAS device.

[Phototechman]

Thanks for the reply.

I want to use beyond compare because I can see how far along the process is, robocopy does not give any time frame of when it might complete.

I have 1.8 million images, but I still can not get a folder list. I click on the sync sides and it scans the folders for about 4 hours. Then goes to not responding, then closes its self with no other dialog boxes. I found a bug report it created in my appdata folder and attached it to this post..

I've been using beyond compare for 5 years now, this is the tool that I want to use.. I'm not using it to get a file count, I'm saying that this version 3 can't even scan the amount of folders I have to even start moving data..

What I'm trying to compare.
1.8million files on a NAS1 VS 0 files on NAS2
Trying to sync files over from NAS1 to NAS2.
I have 7 folders total, with about 600 subfolders in each one.

[Phototechman]

Here is the bug report, it didn't upload the first time

[Chris]

From the stack trace in the crash, it looks like BC is running out of memory. The only workaround is to break this up into smaller pieces.