Monday, 2 March 2009

floats and ints

you know how int's work, signed ints are not much harder to properly understand, but floats make some people whimper.

for starters, a float is not magic, it's just a number that indexes a non-smooth curve through the number space. The non-smooth curve has data points set according to the following rules:

if the sign bit is true, the number is negative.
you use the exponent to make a 2^x number which is then multiplies by the mantissa
you use the mantissa data to make a mantissa between 1.000001ish and 2.0

so, for a 1.0f, the data is (0)(-1)(1)
that is (no sign)( * 2^-1 == 0.5 )( * 1+1 = 2 )
sign bits can only be true or false. exponents can be anything between -127 and 127 (128 is reserved for infinities.) mantissa can be any value between 0.000001ish and 1.0 (using 23 bits of the float)

using this information, you can now see how easy it can be to double or halve a float (just +1 or -1 from the exponent), and you can see how making it negative is purely a 1 bit twiddle.

other fun facts about floats because of this:
  • After masking off the sign bit, floats can be compared for greater than and less than just like ints.
  • Because the mantissa is 23 bits, you can use the last few bits for other information without losing too much clarity (this is how we store whether to render a triangle or skip it in the PS2)
  • Almost any bit pattern can represent a float. Apart from when the exponent is 128.
hex and int versions of common floats:

0.5 : 0x3F00000
1 : 0x3F80000
2 : 0x40000000
-1 : 0xBF800000
0 : 0x00000000
256 : 0x43800000
-0 : 0x80000000

learn them and you can spot data going wrong in debuggers easier.

No comments: