Floating Points - an explanation for non computer scientists

!!! Attention: this article (and in fact my whole blog) have moved to site:
http://thecsbox.com/2013/05/07/what-are-floating-point-numbers/ !!!

This article is not intended for very proficient programmers or computer scientists, but offers more of an introductory and high level explanation of floating point variables to the hobbyist programmer or high school student who likes to explore computers.

Let's look at Integers first

One of the first variable types you learnt were probably integer variables. With integer variables we can represent integers (i.e. 0, 1, 2, ..., -1, -2, ...). The numbers are represented accurately on the computer, meaning the exact mathematical value can be stored in the computer. However, note that a computer cannot represent the complete infinite set of integers (..., -2, -1, 0, 1, 2, ...) but only a finite subset of it, because a computer's memory is always finite. In C, the two limit values are stored in the constants INT_MIN and INT_MAX.

Getting Real

The limitations of integers are apparent when doing simple divisions like 1 / 3. For code like "int x = 1 / 3" the result would just be rounded down and we'd get 0 for x. This is why computer architects and mathematicians in the early days of computing thought of ways to represent real numbers in the computer. The mathematical set they were interested to do calculations with where the real numbers (containing rational numbers like 1/3 and irrational numbers like pi). The fundamental problem is that the real numbers are an infinite set, but the computer only has finite memory.

Infinity != Infinity ?

Note that with the set of integers, the problem of infinity was solved by specifying two limit values INT_MIN and INT_MAX, such that only the finite subset between those limits can be represented. Now, why can't we just take this approach over to the real numbers? The problem is that the real numbers present a "larger" infinite set than the integers. This was shown in Cantor's famous and lovely proof that the real numbers are an uncountable set whereas integers (and even rational numbers) are a countable set. Anyway, I'm deriving... (I just find these mathematical subtleties very lovely). If you haven't learnt about these concepts before, don't worry. The essential thing we're looking at here is how to represent real numbers in the best possible way on a computer.