Ventura public release appears to have a serious file copy issue that breaks BC

[mickjsanders]

After backing up my Monterey (twice), I installed Ventura public release today. After a few hours I noticed something odd and nasty - manually copying files via the Finder resulted in the files not actually copying! And replacing them with the same file resulted in them being deleted! Just sometimes.

Now a similar behaviour has stopped BC working. See the attached screenshots. I attempt to sync 2 identical SSDs and what happens is that the new/missing files (on the left) are copied across, but with the COPY TIME (not the actual file creation/mod time). Furthermore, when I repeatedly try to replace them with the "older" original, BC says it has copied them, but in fact it (or rather, the Ventura file system) hasn't copied the correct file time(s).

I'm not blaming BC; as far as I can see there is a monstrous bug in the file system in Ventura.

Can anyone please replicate? Thanks, M.

[M1 Max on MBP 16"]

[mickjsanders]

Further notes:

All external volumes are formatted as ExFAT.

(1) Under Ventura, BC definitely runs much, much slower (I'd estimate about 10 x slower).

(2) In the above scenario, if I choose to delete the "newer" files on the right hand (destination) disk, then re-start the sync, I end up in the same place (with destination files that are identical but with a time-stamp of whatever time they were actually copied (when they should of course have the same time-stamp as the source files).

The only solution I can find is to manually copy EVERY SINGLE FILE across, folder-by-folder, replacing the "newer" copies with the originals (So, no point in having a file compare/sync utility!)

UPDATE:

1 - my BC 4.4.3 configs used to work fine under Monterey (and its predecessors)
2 - BC 4.4.3 runs under Rosetta on Apple Silicon
3 - I just duplicated the compare/mirror config on "ChronoSync" (which is Apple Silicon-native) and the file system time-stamp problem does not exist
4 - Ergo, Apple has introduced some file system glitch in Rosetta on Ventura.

[Aaron]

Thanks for the report. I'm still working through the different scenarios, but I'm definitely seeing some poor behavior. I also see that a MacOS 13.1 beta is already available; we'll have to test 13.0 a bit more and get a test environment for 13.1 set up as well.

[mickjsanders]

Thanks for the update. FYI, using another compare/mirror app (ChronoSync) I have found that a similar issue occurs when copying from APFS to ExFAT.

In fact, if I just manually copy a moderately full folder of, say, 200MB from the APFS boot volume to a newly formatted ExFAT volume, it copies OK, *but* if you then delete the ExFAT folder and try to empty the trash macOS won't allow that (the errors seem rather random).

Please keep us posted. Thx, M.

[Aaron]

Alright, a quick update from my own testing:

exFAT devices definitely have a performance issue (10x sounds about right) and will always get the time of transfer as their timestamp, which as you note makes syncing particularly difficult. You could enable a content comparison, which would scan and find the content equal, but:
1) the performance issues make content scanning really slow
2) you also have to customize the sync to copy over any Older or Newer differences, since the destination side would always be Newer even if different, and the default Update only overwrites over Older files.

And, as you note, this behavior happens both with Finder and any other programs (including BC4).

I have not yet seen the most worrying thing you mentioned: copying caused the files to disappear. Though given this other behavior, I don't doubt it. I've also seen the OS and BC4 hang with a beachball; if I give it an extreme amount of time it does seem to recover, but it has made testing take longer than it normally would if I bump into it.

I do see there is already a 13.1 beta OS release available, and I'm going to upgrade to see if it helps with some of these exFAT issues. Unfortunately, while I was running each previous beta release, I was mostly interacting with the native APFS disks, and didn't notice the performance issues when working with a smaller exFAT thumbdrive.

[mickjsanders]

FWIW... I have a horrible suspicion that some of the exFAT issues, especially the nasty one of files not copying from APFS to exFAT (or disappearing) via Finder drag-and-drop, may only be present on M1 silicon with >= 64GB of RAM. In other words, only on M1 Max and M1 Ultra units that are loaded with RAM. I myself have a Max with 64GB.

I mention this because (a) no one else has yet been able to replicate that particular set of nasties, and (b) Apple has form here - when I got my M1 Max last November (concurrent with the release of Monterey 12.0.1) I found that installing any 3rd-party KEXTs broke Apple Pay, as well as Apple's own File System Replicator utility (which is used by 3 backup apps to create bootable M1/M2 images). I took a lot of heat because I couldn't convince Apple, or anyone else on the planet, that the problem existed. Eventually I wrote a complaint directly to Tim Cook, and was assigned an "Outreach" contact from Cupertino. Even then, it took 110 hours of my time, 50GB of videos and screenshots, and about 3 months, for Apple to fix it (around Monterey 12.4). In between I did find about 6 people worldwide on forums who had also experienced the issue; all of them had 64GB of RAM on M1s. The problem never occurred with smaller M1 systems, which is why almost no one could replicate it.

I fear that Apple are not regression testing on high-end systems (if they even regression test at all).

Without going right off-topic for Beyond Compare, I wonder if any support staff or users out there have an M1 Max/Ultra with >= 64GB of RAM. If so, could you please try lots of file copies from your boot volume to an external exFAT flash/drive? Also, I've found an occasional issue deleting the trash on exFAT volumes.

[Aaron]

Good suspicion. My own test machine is an older Intel macbook. We don't have any M1 machines with greater than 64gb of ram currently; we mostly top out around 32. We'll have to give that a try a bit to see what happens there.

[Chris]

https://threedots.ovh/blog/2022/06/q...macos-ventura/

The above link states that macOS Ventura 13 changed from kernel mode file system drivers to new user mode file system drivers for FAT, exFAT, and NTFS. I haven't tried it myself, but the link suggests you might be able to access a drive using the older kernel mode driver via the UNIX mount command in Terminal.

[Chris]

Apple's Mac Ventura release notes:
https://developer.apple.com/document...-release-notes

File System

New Features

• There’s a change to the implementation of the msdos and exfat file systems. Apps that check for those specific file system formats might not detect them. Please file feedback if this impacts your app. (90768681)

[lazy_atom]

Hi there. I just wanted to chime in with my own observations seeing as this is the only place I have found that has a similar story to mine.

I use Chronosync and just upgraded to Ventura before Christmas, 3 weeks ago. Over the weekend, having enjoyed the holiday break, I powered up my office environment and found 3 external drives were not mounting. I tried Disk Utility, Disk Warrior, etc, but no joy. Purchased Data Rescue, which actually could see content on the externals. I spent the weekend using it to transfer the files over to my Synology DS415+, which took 2 or three attempts due to the server randomly disconnecting from the system (an ongoing never-solved issue I have with my NAS) so I decided yesterday to amalgamate my recovered files into one big folder.

So I used Chronosync (which I love, btw, apologies) and at one stage had a 4TB folder of recovered files (which is about what I imagined the 3 externals combined would total). In my infinite wisdom, I decided that on the last Chronosync run, I would run the sync bi-directional, so I would end up with 2x4TB folders with identical content, only to discover this morning that both folders are now only 14kb each. It would seem that they replaced and wiped all the files in each folder.

The only common denominator I can see with the OP is that my iMac also has 64GB of RAM. I assume the Seagate drives in my NAS are formatted to Mac OS Extended, but I could be mistaken.

Anyway, apologies if this is completely off the rails. I have contacted eConn about this issue.

I am beginning to suspect that there was nothing wrong with my original drives, and it's a case of Ventura just simply not recognising them.

Either way, it's all just a horrible mess, and I have no idea how to regain my 20+ years iTunes library.

[Aaron]

Ah, that's awful!

Sorry that this has been the best place you've found to commiserate. As a sync utility ourselves, you can imagine my shudder at losing the data in both directions. I hope you can recover something, perhaps with a data recovery tool. Try not to leave the drive plugged in or act on it until you find something to scan it.

And we're glad to hear of other good sync programs. There are many tools out there and there are plenty of instances where one is better for a user for a specific workflow, and often BC can swing in and complement it by verifying the job and reconciling any remaining differences.