3
$\begingroup$

I made the following observation following a suggestion from a friend. This question is about the Microsoft calculator program implemented on Windows. Since I don't have access to any other calculator at the moment, I cannot confirm whether this is a general feature.

If you enter an integer in the calculator program, say $N$, hit the minus button followed by the same integer again, you get zero as expected.

However, if you input $N^2$, then hit square root button, you get $N$, then subtract $N$, you get a very small number, but not zero.

This has been my observation for several integers that I have tried.

For example, with $N=5$, you get $1.232\times 10^{-24}$ in the final step.

Now, I understand computers use finite precision (and hence the square root function must be working under finite precision). However, I don't understand why this shows up in the final step only? Is it only because this is a small number in comparison to any integer, but when you get $0$, you can no longer make that approximation?

  • 0
    I suspect a very poor approximation to the square root is being used (but since the software is closed-source, your guess is as good as mine). Could you try in Excel?2010-11-26
  • 0
    @J.M.: Excel is the benchmark!2010-11-26
  • 0
    In case the gag was missed: Excel is also a floating-point headache. :P I really want to hear about more stories of stuff being ruined by sucky floating point arithmetic...2010-11-26
  • 0
    @J.M.: I never saved it, but I had an algebraic expression with plus, minus, mult and div in that was supposed to add up to 1 but in Excel (latest) it did not..2010-11-26
  • 0
    I agree. It is not hard to implement arithmetic using strings.2010-11-26
  • 0
    @AD: The division is a likely culprit. Adding and multiplying are no longer as troublesome as they were (especially now that hardware nowadays support things like FMA), but division remains to trap the unwary (anybody remember that famous division bug a while back?).2010-11-26

2 Answers 2

8

You'll find your answer and much more in the articles below:

What Every Computer Scientist Should Know About Floating-Point Arithmetic

Also see this article on Calculator Forensics, esp. "Hints and Tricks For Obtaining Guard Digits".

  • 0
    +1 for the beautiful article, which should be read after perusing the IEEE guidelines. Velvel Kahan's miscellaneous papers in his site, like [this one](http://www.cs.berkeley.edu/~wkahan/SQRTest.ps) are also an excellent read.2010-11-26
0

J.M., since when is 24 decimal digits accuracy considered a bad approximation?

Original poster, they could show sqrt(5)**2 as

5.000000000000000000000001232

but since the display probably has fewer display digits than this, they chop it to "5" as that is the closest representation in the allowed number of digits.

BTW, I just tried this in "calc" in windows 7 and the 5-sqrt(5)**2 is 1.7599e-37

If you want exact math, you will need to use a symbolic math package.

  • 0
    There is such a thing as a properly rounded square root (most certainly if the routine is to handle (near-)integer inputs). You might want to look at the link I gave in the comment to Bill's answer.2010-11-26
  • 0
    First, my apologies for posting what I intended to be a comment as an answer, which it isn't.2010-11-26