No announcement yet.

Ignore small decimal differences

  • Filter
  • Time
  • Show
Clear All
new posts

  • Ignore small decimal differences

    I compare two files that contain small difference in decimal part of numbers, due to rounding a number 109927.77777777 in one file is represented in another file as 109927.77777778 but i would like to ignore such small differences. I would appreciate if some of you could share an idea of how this can be acomplished.

    Volume = 113111.11111112 127422.22222222 113497.22222222 106088.37888889 109927.77777777 106419.00444444 106419.44444444

    Volume = 113111.11111112 127422.22222222 113497.22222222 106088.37888888 109927.77777778 106419.00444445 106419.44444445

  • #2

    If your data can be delimited and viewed in the Table Compare session type, each Numeric column can have an associated Numeric Tolerance to ignore a change like this.

    In the Text Compare session, the characters are just text (not a number), so ignoring it is a bit trickier. We'd have to create a grammar that matches on this spot of the line. I'd recommend defining a grammar for the Important part of the number:
    where "5" is the number of characters after the decimal that are important.

    Then in the Text Compare, Session Settings, Importance tab, you can check this new grammar's name, and uncheck Everything Else. This will make this part of the number important, and everything else on the line unimportant. If you need to define additional grammars (for "Volume =" or other text), these can also be Important.
    Aaron P Scooter Software


    • #3
      Thanks a lot for your help, Aaron!