Improved JSON File Comparison?

[kevinmmd]

Unless I'm doing something wrong (i.e. don't know about some feature) it seems as if Beyond Compare doesn't do a great job of comparing two JSON files. JSON files, by their nature, don't care about sort order. Is it possible to either add a plugin for Beyond Compare or add the capability to ignore sort order differences? Alternatively, a feature to "sort" a JSON file, so that the fields are in the same sequence for easier comparison, would be fine too.

I'm posting to the forum in the hopes that there really is something like this already available and I just don't know about it?

Thanks,
Kevin.

[Zoltron]

I don't usually deal with json but on a rare occasion i came across a script called jq which allowed to convert a json file into a table (csv) which then i compared in BC3. instructing jq to convert the file was the real headache but if you only want to compare a dozen fields, it really does the trick: https://stedolan.github.io/jq/.

[Aaron]

Yep, in addition to Zoltron's general advice, we also have some pre-packaged File Formats for JSON with either Sorting or Tidied enabled:
http://www.scootersoftware.com/downl..._moreformatsv4

Note, if you edit and save, it will save the text in the new ordering, so you may want to enable Editing Disabled in the File Format's Conversion tab options to prevent that if needed.

[dmitry]

BC4 uses jq for preprocessing JSON files, for example when intelligently sorting content before comparing it. Would it be possible to extend this great idea and give us, users, an opportunity to provide a filter expression in jq language? Something like '[.[] | {message: .commit.message, name: .commit.committer.name}]' which jq would apply during preprocessing. It can be a setting in session config or something like that.

I am interested in comparing only relevant parts of large JSON files, not the whole files.

[Aaron]

Hello,

There would be two main approaches to this:

1) The current JSON conversions are called from .bat files. You can update/edit the bat file to limit the view (using your preprocessing logic) and at the end of the conversion pass back the plain .txt output file that BC4 should display. Please note that if you are cutting out large parts of the file, BC4 won't display any of that and is only showing the content of the .txt file as the rules-based scan results.
http://www.scootersoftware.com/suppo...rnalconversion

2) Update the JSON file format's grammar element list, and make only the text you want to compare (or text you do not want to compare) defined as a grammar. You can then update the Text Compare's Session Settings, Importance tab to mark an element as Important or Unimportant, and mark Everything Else as Important or Unimportant. This would keep the full content, but allow you to optionally ignore or filter out specific text.
http://www.scootersoftware.com/suppo..._unimportantv3

If I understand your above example, you are looking to mark specific "*.message" text combinations as important? I think grammar elements that match on them would be a good strategy.

[dmitry]

Thanks Aaron, excellent answer. I didn’t realise how powerful BC4 actually is: not only doing comparisons and merges, but able to serve as integrator of other software.

I experiment with both proposed solutions, starting with providing query string to jq inside of the batch file. I have created new File Format similar to existing JSON Sorted, provided my own filter string instead of existing "." and it works exactly as expected. As a bonus, I can go to Tools -> Edit File and apply my new format with proprietary filter to the loaded json file. Wonderful, I can now see only what I need to see out of thousands of lines!

The links you have provided are helpful, because there are not many detailed examples in the documentation. For example, it says: ‘The following variables will be replaced with the indicated information:

%s source file

%t target file

%o original file’

What is the difference between %s and %o? In what situations we should use original file parameter but not source file parameter? I didn’t find any examples where %o is used in File Formats.

[Aaron]

Correction:

%s is the source file when local, and creates a local temp file when necessary (such as dealing with a remote ftp location, clipboard, etc).

%o is always the original path, such as "ftp://" and relies on the conversion program to be able to handle any path we give it.

My previous reply assumed we always generated the %s as a temporary copy, but that isn't always the case.

[axiac]

As far as I understand, the solutions described here use `jq` to rewrite the input JSONs to a standard format but in the end, Beyond Compare still compares the JSON files line by line.
But the data encoded as JSON has a structure (nested objects and arrays).

Is there any plan to implement semantic comparison for JSON (and YAML)?

[Aaron]

A structured comparison view is something on our customer wishlist, which is where our developers look for ideas for future features and enhancements. For BC4, you would need to use JQ or a similar command line tool to sort and re-structure the data, similar to how XML and HTML Tidy performs, to open them in the Text Compare.