Announcement

Collapse
No announcement yet.

Slow Performance with NAS Storage Unit

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Slow Performance with NAS Storage Unit

    I have no idea where the problem is, but I doubt if it is due to Beyond Compare. Still, I have to ask this question:

    I just built up a Buffalo TeraStation NAS, Model TS-TGL70C, with two 2TB Hitachi DeskStar (7200rpm, 32MB cache) hard drives configured as a RAID1 array. The Ethernet port on the Buffalo is rated as a 1000Mbps port.

    The firmware version is 1.12.

    I did a test run, transferring approximately 513GB of music files from a USB 2.0 drive on a Dell GX260 desktop through the Dell's 1000Mbps network port via a wired network with a single 1000Mbps switch also rated at 1000Mbps and then directly into the TeraStation's port.

    I used the latest version of Beyond Compare (Scooter Software) to sync the NAS with the USB.

    It took 25 hours to complete the file transfer! This is an average transfer rate of 45.6 Mbps, far short of what USB 2.0 and a 1000Mbps network and either hard drive setup should be capable of delivering.

    Where's the bottleneck?

  • #2
    My guess is that the usb interface for the external hard drives is limiting the transfer speed.

    What kind of transfer speed do you get if you copy a large number of files from an internal sata disk in the Dell to the NAS?

    You can also try the external USB to NAS copy with Windows Explorer to see if the speed issue is specific to Beyond Compare.
    Chris K Scooter Software

    Comment


    • #3
      NAS Storage slowness

      I use D-Link NAS storage, configured for Jbod on Gb ethernet.

      Running from WinXP laptop, connected through Gb USRobotics switch,
      it runs the way I expect it ... very fast.

      Switch to a Win7 X64 desktop machine attached to the same switch.
      transferring from very fast sata drives, I get performance similar to
      that reported by d2b ... glacially slow, measured in days, not minutes.

      BC 3.2.0 beta on all machines.

      Transfer from XP to Win7 .... very fast. All switch connections are Gb,
      no 10/100 connections.

      I suspect Win7 is at fault, and it is likely some tcp setting.
      Just haven't hooked up netmon to study it yet.

      Comment


      • #4
        It might help to try the same copy using Windows Explorer to see if the problem is BC specific or if it is a more general problem.
        Chris K Scooter Software

        Comment


        • #5
          I'll try Windows Copy next. Running sysinternals ProcessMonitor,
          BC3 is issuing hundreds of failing requests:

          CreateFile C:\Windows\CSC\v2.0.6\namespace\NAS "NAME NOT FOUND"

          The copy is from L: local esata disk to N: Network Mount \\NAS\Volume_1

          Call Stack:

          0 fltmgr.sys fltmgr.sys + 0x2027 0xfffff8800115a027 C:\Windows\system32\drivers\fltmgr.sys
          1 fltmgr.sys fltmgr.sys + 0x48ca 0xfffff8800115c8ca C:\Windows\system32\drivers\fltmgr.sys
          2 fltmgr.sys fltmgr.sys + 0x222a3 0xfffff8800117a2a3 C:\Windows\system32\drivers\fltmgr.sys
          3 ntoskrnl.exe ntoskrnl.exe + 0x374777 0xfffff80002dc0777 C:\Windows\system32\ntoskrnl.exe
          4 ntoskrnl.exe ntoskrnl.exe + 0x36a80f 0xfffff80002db680f C:\Windows\system32\ntoskrnl.exe
          5 ntoskrnl.exe ntoskrnl.exe + 0x36fb76 0xfffff80002dbbb76 C:\Windows\system32\ntoskrnl.exe
          6 ntoskrnl.exe ntoskrnl.exe + 0x376887 0xfffff80002dc2887 C:\Windows\system32\ntoskrnl.exe
          7 ntoskrnl.exe ntoskrnl.exe + 0x380498 0xfffff80002dcc498 C:\Windows\system32\ntoskrnl.exe
          8 ntoskrnl.exe ntoskrnl.exe + 0x6f853 0xfffff80002abb853 C:\Windows\system32\ntoskrnl.exe
          9 ntoskrnl.exe ntoskrnl.exe + 0x6bdf0 0xfffff80002ab7df0 C:\Windows\system32\ntoskrnl.exe
          10 csc.sys csc.sys + 0x1e85d 0xfffff88003ab385d C:\Windows\system32\drivers\csc.sys
          11 csc.sys csc.sys + 0x1e9c5 0xfffff88003ab39c5 C:\Windows\system32\drivers\csc.sys
          12 csc.sys csc.sys + 0x1e35b 0xfffff88003ab335b C:\Windows\system32\drivers\csc.sys
          13 ntoskrnl.exe ntoskrnl.exe + 0x68487 0xfffff80002ab4487 C:\Windows\system32\ntoskrnl.exe
          14 ntoskrnl.exe ntoskrnl.exe + 0x68441 0xfffff80002ab4441 C:\Windows\system32\ntoskrnl.exe
          15 ntoskrnl.exe ntoskrnl.exe + 0x7ff0a 0xfffff80002acbf0a C:\Windows\system32\ntoskrnl.exe
          16 csc.sys csc.sys + 0x1e459 0xfffff88003ab3459 C:\Windows\system32\drivers\csc.sys
          17 csc.sys csc.sys + 0x1ebbe 0xfffff88003ab3bbe C:\Windows\system32\drivers\csc.sys
          18 csc.sys csc.sys + 0x16cd1 0xfffff88003aabcd1 C:\Windows\system32\drivers\csc.sys
          19 csc.sys csc.sys + 0x16e84 0xfffff88003aabe84 C:\Windows\system32\drivers\csc.sys
          20 csc.sys csc.sys + 0x5dd5e 0xfffff88003af2d5e C:\Windows\system32\drivers\csc.sys
          21 csc.sys csc.sys + 0x17a36 0xfffff88003aaca36 C:\Windows\system32\drivers\csc.sys
          22 csc.sys csc.sys + 0x58967 0xfffff88003aed967 C:\Windows\system32\drivers\csc.sys
          23 csc.sys csc.sys + 0x12b4d 0xfffff88003aa7b4d C:\Windows\system32\drivers\csc.sys
          24 csc.sys csc.sys + 0x420b3 0xfffff88003ad70b3 C:\Windows\system32\drivers\csc.sys
          25 rdbss.sys rdbss.sys + 0x23a1c 0xfffff88003c23a1c C:\Windows\system32\DRIVERS\rdbss.sys
          26 rdbss.sys rdbss.sys + 0x22f67 0xfffff88003c22f67 C:\Windows\system32\DRIVERS\rdbss.sys
          27 rdbss.sys rdbss.sys + 0x2421f 0xfffff88003c2421f C:\Windows\system32\DRIVERS\rdbss.sys
          28 rdbss.sys rdbss.sys + 0x3aa0 0xfffff88003c03aa0 C:\Windows\system32\DRIVERS\rdbss.sys
          29 rdbss.sys rdbss.sys + 0x21614 0xfffff88003c21614 C:\Windows\system32\DRIVERS\rdbss.sys
          30 mrxsmb.sys mrxsmb.sys + 0x202c4 0xfffff880035672c4 C:\Windows\system32\DRIVERS\mrxsmb.sys
          31 dfsc.sys dfsc.sys + 0x9ce1 0xfffff88003b21ce1 C:\Windows\System32\Drivers\dfsc.sys
          32 dfsc.sys dfsc.sys + 0xa692 0xfffff88003b22692 C:\Windows\System32\Drivers\dfsc.sys
          33 dfsc.sys dfsc.sys + 0xdf6d 0xfffff88003b25f6d C:\Windows\System32\Drivers\dfsc.sys
          34 dfsc.sys dfsc.sys + 0x199a2 0xfffff88003b319a2 C:\Windows\System32\Drivers\dfsc.sys
          35 mup.sys mup.sys + 0x6c40 0xfffff8800196fc40 C:\Windows\System32\Drivers\mup.sys
          36 mup.sys mup.sys + 0x6157 0xfffff8800196f157 C:\Windows\System32\Drivers\mup.sys
          37 mup.sys mup.sys + 0x764b 0xfffff8800197064b C:\Windows\System32\Drivers\mup.sys
          38 fltmgr.sys fltmgr.sys + 0x323f 0xfffff8800115b23f C:\Windows\system32\drivers\fltmgr.sys
          39 fltmgr.sys fltmgr.sys + 0x222b9 0xfffff8800117a2b9 C:\Windows\system32\drivers\fltmgr.sys
          40 ntoskrnl.exe ntoskrnl.exe + 0x374777 0xfffff80002dc0777 C:\Windows\system32\ntoskrnl.exe
          41 ntoskrnl.exe ntoskrnl.exe + 0x36aa64 0xfffff80002db6a64 C:\Windows\system32\ntoskrnl.exe
          42 ntoskrnl.exe ntoskrnl.exe + 0x36fb76 0xfffff80002dbbb76 C:\Windows\system32\ntoskrnl.exe
          43 ntoskrnl.exe ntoskrnl.exe + 0x376887 0xfffff80002dc2887 C:\Windows\system32\ntoskrnl.exe
          44 ntoskrnl.exe ntoskrnl.exe + 0x319eab 0xfffff80002d65eab C:\Windows\system32\ntoskrnl.exe
          45 fltmgr.sys fltmgr.sys + 0x25dca 0xfffff8800117ddca C:\Windows\system32\drivers\fltmgr.sys
          46 fltmgr.sys fltmgr.sys + 0x2401e 0xfffff8800117c01e C:\Windows\system32\drivers\fltmgr.sys
          47 fltmgr.sys fltmgr.sys + 0x25361 0xfffff8800117d361 C:\Windows\system32\drivers\fltmgr.sys
          48 fltmgr.sys fltmgr.sys + 0x2513e 0xfffff8800117d13e C:\Windows\system32\drivers\fltmgr.sys
          49 fltmgr.sys fltmgr.sys + 0x10b2d 0xfffff88001168b2d C:\Windows\system32\drivers\fltmgr.sys
          50 fltmgr.sys fltmgr.sys + 0xabd6 0xfffff88001162bd6 C:\Windows\system32\drivers\fltmgr.sys
          51 fltmgr.sys fltmgr.sys + 0x11ad4 0xfffff88001169ad4 C:\Windows\system32\drivers\fltmgr.sys
          52 PROCMON20.SYS PROCMON20.SYS + 0x4a11 0xfffff88004817a11 C:\Windows\system32\Drivers\PROCMON20.SYS
          53 fltmgr.sys fltmgr.sys + 0x2027 0xfffff8800115a027 C:\Windows\system32\drivers\fltmgr.sys
          54 fltmgr.sys fltmgr.sys + 0x48ca 0xfffff8800115c8ca C:\Windows\system32\drivers\fltmgr.sys
          55 fltmgr.sys fltmgr.sys + 0x222a3 0xfffff8800117a2a3 C:\Windows\system32\drivers\fltmgr.sys
          56 ntoskrnl.exe ntoskrnl.exe + 0x374777 0xfffff80002dc0777 C:\Windows\system32\ntoskrnl.exe
          57 ntoskrnl.exe ntoskrnl.exe + 0x36aa64 0xfffff80002db6a64 C:\Windows\system32\ntoskrnl.exe
          58 ntoskrnl.exe ntoskrnl.exe + 0x36fb76 0xfffff80002dbbb76 C:\Windows\system32\ntoskrnl.exe
          59 ntoskrnl.exe ntoskrnl.exe + 0x376887 0xfffff80002dc2887 C:\Windows\system32\ntoskrnl.exe
          60 ntoskrnl.exe ntoskrnl.exe + 0x38f5b4 0xfffff80002ddb5b4 C:\Windows\system32\ntoskrnl.exe
          61 ntoskrnl.exe ntoskrnl.exe + 0x6f853 0xfffff80002abb853 C:\Windows\system32\ntoskrnl.exe
          62 ntdll.dll ntdll.dll + 0x5008a 0x76f4008a C:\Windows\SYSTEM32\ntdll.dll
          63 wow64.dll wow64.dll + 0x162d7 0x738862d7 C:\Windows\SYSTEM32\wow64.dll
          64 wow64.dll wow64.dll + 0xcf87 0x7387cf87 C:\Windows\SYSTEM32\wow64.dll
          65 wow64cpu.dll wow64cpu.dll + 0x276d 0x7380276d C:\Windows\SYSTEM32\wow64cpu.dll
          66 wow64.dll wow64.dll + 0xd07e 0x7387d07e C:\Windows\SYSTEM32\wow64.dll
          67 wow64.dll wow64.dll + 0xc549 0x7387c549 C:\Windows\SYSTEM32\wow64.dll
          68 ntdll.dll ntdll.dll + 0x7c367 0x76f6c367 C:\Windows\SYSTEM32\ntdll.dll
          69 ntdll.dll ntdll.dll + 0x32e2e 0x76f22e2e C:\Windows\SYSTEM32\ntdll.dll
          70 ntdll.dll ntdll.dll + 0x1fd06 0x770efd06 C:\Windows\SysWOW64\ntdll.dll
          71 KERNELBASE.dll KERNELBASE.dll + 0x19702 0x74da9702 C:\Windows\syswow64\KERNELBASE.dll
          72 BCompare.exe madTraceProcess + 0xd4980 0x54bad0 C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
          73 BCompare.exe madTraceProcess + 0x4e412c 0x95b27c C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
          74 BCompare.exe madTraceProcess + 0x4e6c4a 0x95dd9a C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
          75 BCompare.exe madTraceProcess + 0x4f4071 0x96b1c1 C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
          76 BCompare.exe BCompare.exe + 0x765c9 0x4765c9 C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
          77 BCompare.exe BCompare.exe + 0x2caef 0x42caef C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
          78 BCompare.exe BCompare.exe + 0x5c4a 0x405c4a C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
          79 BCompare.exe BCompare.exe + 0x764ab 0x4764ab C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
          80 BCompare.exe BCompare.exe + 0x76518 0x476518 C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
          81 kernel32.dll kernel32.dll + 0x13677 0x765a3677 C:\Windows\syswow64\kernel32.dll
          82 ntdll.dll ntdll.dll + 0x39d42 0x77109d42 C:\Windows\SysWOW64\ntdll.dll
          83 ntdll.dll ntdll.dll + 0x39d15 0x77109d15 C:\Windows\SysWOW64\ntdll.dll

          Comment


          • #6
            I changed the BC3 invocation to "Run as Administrator" on a very large .iso file.

            The copy is proceeding in 1MB chunks; the trace shows a single thread:

            R - read
            W - Write
            C - CreateFile NAS

            RWRWRWCC, RWRWRWRWRWRWC, RWRWRWCC, RWRWRWRWRWC, ...

            Comment


            • #7
              While I was writing this reply, the copy finished ... so it was running much quicker
              as administrator.

              Comment


              • #8
                Run -> Cmd -> xcopy /J xxx.iso N:
                /J Copies using unbuffered I/O. Recommended for very large files.

                xcopy is reading/writing 32K chunks.

                The trace pattern is something like:

                RRRWWWRW, RWRWRW, RRRRRWWRWRW ....

                xcopy activity starts with:
                Read Offset 129,536 length 7K
                Read Offset 25,952,256 lengtgh 32K
                Followed by:

                11 "Create File ..\NAS" requests (NOT FOUND)

                Those \NAS requests are not continuous as they are in BC3.

                xopy ran _MUCH_ faster than BC3 for this 559MB file.

                4:29:41.2492832 PM xcopy.exe 3208 ReadFile
                4:30:20.1974570 PM xcopy.exe 3208 CloseFile

                Perhaps the large blocks are causing packet re-assembly at the tcp layer,
                but there is also the nuisance of the Create File requests.

                xcopy is using
                ReadFile L:\xxx.iso (Local Disk)
                WriteFile \\NAS\Volume_1\xxx.iso

                Comment


                • #9
                  If xcopy has a "run unbuffered" command then it must include its own copy engine. BC relies on the Windows CopyFileEx routine. You might try TeraCopy and see if it is similarly speedy, since I know it has an optimized copy routine too.

                  Also, it looks like C:\Windows\CSC is "Client Side Cache" aka Offline Files. http://blogs.msdn.com/b/oldnewthing/.../27/54712.aspx
                  Zoë P Scooter Software

                  Comment


                  • #10
                    Figured it out, and now I'm back at _faster_than_xcopy speeds.

                    xcopy figured out it N: was a mounted filesystem with a UNC of \\NAS\Volume_1
                    and apparently switched the way it was doing writes.

                    When I ditched the mounted filesystem, and used the UNC in the BC3 folder,
                    the speed came back.

                    So perhaps BC3 can do this switch transparently the way xcopy did.

                    I'll read up on "Client Side Cache". The trace data doesn't make any sense;
                    why would Win7 check CSC the way it does? Why not every write ... or like xcopy,
                    figure it out once and "do the right thing".

                    Comment


                    • #11
                      "Offline Files" is a service running on Win7.

                      Since my machine is a desktop, I have no need for offline copies,
                      so I disabled the service:

                      Start -> Control Panel -> Administrative -> Services

                      Offline Files / Properties / Startup Type = Disabled

                      Comment


                      • #12
                        I've about given up on Win7 x64 ... even with all the changes,
                        it is painfully slow to synchronize disks.

                        My guess at the moment is that it is each new file that is queried
                        that is causing the slowdown.

                        To actually complete the synchronization, I switched to XP.
                        - mounted the network drive and the Win64 machine drive shared
                        - WinXP performs much faster, even though I have to drag the
                        files across the network through the XP box, doubling the network load.

                        Comment


                        • #13
                          Hello,

                          This may be driver related. Perhaps the Win7 drivers for your NAS device are not quite as good on Win7 as on XP? A few of our machines are Win7 in the office, and there are no big speed differences for remote comparisons in BC3.
                          Aaron P Scooter Software

                          Comment

                          Working...
                          X