Tweet
I read the forum topic "BC3 44% slower than BC2 on folder compares", but I'm starting a separate topic because (1) I'm not doing binary comparisons and (2) I'm using encryption.
I'm using TrueCrypt's "System-Drive Encryption" to encrypt my entire C: hard drive (including the Windows XP operating system). I know that encryption will affect the performance of BC's folder comparisons, but I don't know how much. After all, I'm not comparing contents, just the filename, size, & timestamp. I have only a handful of exclusion filters.
I know there are many factors that affect performance, so rather than list all of my software and hardware, I'll just say that my computer was above average when I bought it in 2003, and has a 2.8 Ghz CPU (the bottleneck during comparisons is the CPU, not the hard drive).
Before the test, I do a full refresh twice, so all the file metadata should be cached in memory.
I have 28,000 files in 3,600 folders (mirrored on the left and right sides). The folder comparison in BC takes 36 seconds.
I wrote a very naive, non-GUI C++ program that loads both the left and right folder trees into memory. For each tree, it recursively walks through each folder, and for each file, does fifty string comparisons of 100-byte strings to simulate comparing filenames, timestamps, exclude filters, etc. My test took 7 seconds total for all 28,000 files.
I know it's an extremely over-simplified test that ignores a lot of the complex processing and comparisons that BC has to do behind the scenes, not to mention that BC has to constantly repaint the screen, handle Windows events, and on and on. I know that it's not a fair comparison and please know that I think BC is an amazing piece of software and I use it several times every day.
Nonetheless, the difference between 7 seconds for my over-simplified test to 36 seconds for BC seems too big to ignore. I could more easily understand if BC took 2 times as long or even 3 times, but 5 times seems a bit unreasonable. Is there some kind of conflict between TrueCrypt's I/O drivers and the way that BC interacts with the file system (and that my little test somehow avoids)? Or is it unrelated to TrueCrypt and I'm seeing about the same performance as everyone else?
Note that of my 28,000 files, only about 200 of them change per day. So for 99% of my folders, all files in them should already be identical on the left and right sides. From my perspective, it seems that BC only has to verify this, add each file to the GUI list, and move to the next folder. However, it seems that BC might be over-complicating this and spending too much time with these already mirrored folders.
Another odd thing I noticed is how BC "jumps all over the place" in the GUI during folder comparisons. Of course I have no idea what BC is doing, but as a user who watches this happen several times per day on his screen, it doesn't *seem* very efficient. Instead, I would expect to see BC process the first folder at the top level, then all of its subfolders, then move to the second folder at the top level, and so on.
I would also prefer such consistent top-down visual behavior when doing the sync operation after the folder compare. Sometimes the BC window is "stuck" with unprocessed files in view at the top while BC is processing other files that are out of view. I would much rather see top-down processing so that I can always see the files that BC is currently syncing.
Here are the "Folder Handling" options I use:
[X] Automatically scan subfolders in background
...[ ] Automatically scan top-level orphan subfolders
[X] Expand subfolders when loading session
...[X] Only expand subfolders with differences
I can reduce the comparison time by five seconds if I disable the last two options, but that's still 31 seconds.
I realize that BC3 was just released and that the Scooter team probably hasn't had much (if any) time to do optimizations, but in the meantime I wanted to post my (unfair) test results.
I know that everyone has different hardware speeds, but I'm curious if BC's performance on my computer (36 seconds for 28,000 files) is approximately the same as other BC users, and also if anyone else is using TrueCrypt's system-wide encryption with BC.
I'm using TrueCrypt's "System-Drive Encryption" to encrypt my entire C: hard drive (including the Windows XP operating system). I know that encryption will affect the performance of BC's folder comparisons, but I don't know how much. After all, I'm not comparing contents, just the filename, size, & timestamp. I have only a handful of exclusion filters.
I know there are many factors that affect performance, so rather than list all of my software and hardware, I'll just say that my computer was above average when I bought it in 2003, and has a 2.8 Ghz CPU (the bottleneck during comparisons is the CPU, not the hard drive).
Before the test, I do a full refresh twice, so all the file metadata should be cached in memory.
I have 28,000 files in 3,600 folders (mirrored on the left and right sides). The folder comparison in BC takes 36 seconds.
I wrote a very naive, non-GUI C++ program that loads both the left and right folder trees into memory. For each tree, it recursively walks through each folder, and for each file, does fifty string comparisons of 100-byte strings to simulate comparing filenames, timestamps, exclude filters, etc. My test took 7 seconds total for all 28,000 files.
I know it's an extremely over-simplified test that ignores a lot of the complex processing and comparisons that BC has to do behind the scenes, not to mention that BC has to constantly repaint the screen, handle Windows events, and on and on. I know that it's not a fair comparison and please know that I think BC is an amazing piece of software and I use it several times every day.
Nonetheless, the difference between 7 seconds for my over-simplified test to 36 seconds for BC seems too big to ignore. I could more easily understand if BC took 2 times as long or even 3 times, but 5 times seems a bit unreasonable. Is there some kind of conflict between TrueCrypt's I/O drivers and the way that BC interacts with the file system (and that my little test somehow avoids)? Or is it unrelated to TrueCrypt and I'm seeing about the same performance as everyone else?
Note that of my 28,000 files, only about 200 of them change per day. So for 99% of my folders, all files in them should already be identical on the left and right sides. From my perspective, it seems that BC only has to verify this, add each file to the GUI list, and move to the next folder. However, it seems that BC might be over-complicating this and spending too much time with these already mirrored folders.
Another odd thing I noticed is how BC "jumps all over the place" in the GUI during folder comparisons. Of course I have no idea what BC is doing, but as a user who watches this happen several times per day on his screen, it doesn't *seem* very efficient. Instead, I would expect to see BC process the first folder at the top level, then all of its subfolders, then move to the second folder at the top level, and so on.
I would also prefer such consistent top-down visual behavior when doing the sync operation after the folder compare. Sometimes the BC window is "stuck" with unprocessed files in view at the top while BC is processing other files that are out of view. I would much rather see top-down processing so that I can always see the files that BC is currently syncing.
Here are the "Folder Handling" options I use:
[X] Automatically scan subfolders in background
...[ ] Automatically scan top-level orphan subfolders
[X] Expand subfolders when loading session
...[X] Only expand subfolders with differences
I can reduce the comparison time by five seconds if I disable the last two options, but that's still 31 seconds.
I realize that BC3 was just released and that the Scooter team probably hasn't had much (if any) time to do optimizations, but in the meantime I wanted to post my (unfair) test results.
I know that everyone has different hardware speeds, but I'm curious if BC's performance on my computer (36 seconds for 28,000 files) is approximately the same as other BC users, and also if anyone else is using TrueCrypt's system-wide encryption with BC.
Comment