Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/leegao/float-hacks

Floating Point Hacks
https://github.com/leegao/float-hacks

Last synced: 3 months ago
JSON representation

Floating Point Hacks

Awesome Lists containing this project

README 

        
# Floating Point Hacks



### Completely useless, but fun nevertheless.






*Equations for a "fast"  method.*



-------------------------------------



Table of Contents

=================



* [Floating Point Hacks](#floating-point-hacks)

  * [Usage](#usage)

     * [For Contributors](#for-contributors)

     * [Pow](#pow)

     * [Exp](#exp)

     * [Log](#log)

     * [Geometric Mean](#geometric-mean)

  * [Justification](#justification)

     * [Prelude](#prelude)

     * [Arbitrary Powers](#arbitrary-powers)

     * [Exp](#exp-1)

        * [Differentiating the l2f and f2l functions.](#differentiating-the-l2f-and-f2l-functions)

        * [A Tale of Two Functions](#a-tale-of-two-functions)

        * [Exp, redux.](#exp-redux)

     * [Log](#log-1)

     * [Geometric Mean](#geometric-mean-1)



-------------------------------------



This repository contains a set of procedures to compute numerical methods in the vein of the

[fast inverse root method](https://en.wikipedia.org/wiki/Fast_inverse_square_root). In particular,

we will generate code that



1. Computes rational powers () to an arbitrary precision.

2. Computes irrational powers () to within 10% relative error.

3. Computes  to within 10% relative error.

4. Computes  to within 10% relative error.

5. Computes the geometric mean  of an `std::array` quickly to within 10% error.



Additionally, we will do so using mostly just integer arithmetic.



## Usage



You can use everything in `floathacks` by including `hacks.h`



    #include 

    using namespace floathacks; // Comment this out if you don't want your top-level namespace to be polluted



#### For Contributors



This document is compiled from `READOTHER.md` by `readme2tex`. Make sure that you `pip install readme2tex`. You

can run



    python -m readme2tex --output README.md --branch svgs



to recompile these docs.



#### Pow



To generate an estimation for , where  is any floating point number, you can run



    float approximate_root = fpow::estimate(x);



Since estimates of `pow` can be refined into better iterates (as long as `c` is "rational enough"), you can also

compute a more exact result via



    float root = pow(x);



where `n` is the number of newton iterations to perform. The code generated by this template will unroll itself, so it's

relatively efficient.



However, the optimized code does not let you use it as a `constexpr` or where the exponent is not constant. In those cases,

you can use `consts::fpow(x, c)` and `consts::pow(x, c, iterations = 2)` instead:



    float root = consts::pow(x, 0.12345, n);



Note that the compiler isn't able to deduce the optimal constants in these cases, so you'll incur additional penalties

computing the constants of the method.



#### Exp



You can also compute an approximation of  with



    float guess = fexp(x);



Unfortunately, since there are no refinement methods available for exponentials, we can't do much

with this result if it's too coarse for your needs. In addition, due to overflow, this method breaks down

when `x` approaches 90.



#### Log



Similarly, you can also compute an approximation of  with



    float guess = flog(x);



Again, as is with the case of `fexp`, there are no refinement methods available for logarithms either.



All of the `f***` methods above have bounded relative errors of at most 10%. The refined `pow` method

can be made to give arbitrary precision by increasing the number of refinement iterations. Each refinement

iteration takes time proportional to the number of digits in the floating point representation of the exponent.

Note that since floats are finite, this is bounded above by 32 (and more tightly, 23).



#### Geometric Mean



You can compute the geometric mean () of a `std::array` with



    float guess = fgmean<3>({ 1, 2, 3 });



This can be refined, but you typically do not care about the absolute precision of a mean-like statistic.

To refine this, you can run Newton's method on . As far as I am aware, this is

also an original method.



## Justification



### Prelude



The key ingredient of these types of methods is the pair of transformations

 and 

.



*  takes a `IEEE 754` single precision floating point number and outputs its "machine" representation.

In essence, it acts like



        unsigned long f2l(float x) {

          union {float fl; unsigned long lg;} lens = { x };

          return lens.lg;

        }

*  takes an unsigned long representing a float and returns a `IEEE 754` single precision floating point number.

It acts like



        float l2f(unsigned long z) {

          union {float fl; unsigned long lg;} lens = { z };

          return lens.fl;

        }



So for example, the fast inverse root method:



      union {float fl; unsigned long lg;} lens = { x };

      lens.lg = 0x5f3759df - lens.lg / 2;

      float y = lens.fl;



can be equivalently expressed as






In a similar vein, a fast inverse cube-root method is presented at the start of this page.






We will justify this in the next section.



### Arbitrary Powers



We can approximate any  using just integer arithmetic on the machine representation of . To do

so, compute






where . In general, any value of `bias`, as long

as it is reasonably small, will work. At `bias = 0`, the method computes a value whose error is completely positive.

Therefore, by increasing the bias, we can shift some of the error down into the negative plane and

halve the error. 



As seen in the fast-inverse-root method, a bias of `-0x5c416` tend to work well for pretty much every case that I've

tried, as long as we tack on at least one Newton refinement stage at the end. It works well without refinement as well,

but an even bias of `-0x5c000` works even better.



Why does this work? See [these slides](http://www.bullshitmath.lol/FastRoot.slides.html) for the derivation. In

particular, the fast inverse square-root is a subclass of this method.



### Exp



We can approximate  up to  using a similar set of bit tricks. I'll first give its equation, and then

give its derivations. As far as I am aware, these are original. However, since there are no refinement methods

for the computation of , there is practically no reason to ever resort to this approximation unless you're okay

with 10% error.






Here,  is the [machine epsilon](https://en.wikipedia.org/wiki/Machine_epsilon) for single precision, and it

is computed by .



To give a derivation of this equation, we'll need to borrow a few mathematical tools from analysis. In particular, while

`l2f` and `f2l` have many discontinuities ( of them to be exact), it is mostly smooth. This

carries over to its "rate-of-change" as well, so we will just pretend that it has mostly smooth derivatives

everywhere.



#### Differentiating the `l2f` and `f2l` functions.

Consider the function






where the equality is a consequence of the chain-rule, assuming that `f2l` is differentiable at the particular value of

. Now, this raises an interesting question: What does it mean to take a derivative of ?



Well, it's not all that mysterious. The derivative of `f2l` is just the rate at which a number's IEEE 754 machine

representation changes as we make small perturbations to a number. Unfortunately, while it might be easy to compute

this derivative as a numerical approximation, we still don't have an approximate form for algebraic manipulation.



While  might be difficult to construct, we can fair much better with its sibling, .

Now, the derivative  is the rate that a float will change given that we make small perturbations

to its machine representation. However, since its machine representations are all bit-vectors, it doesn't make sense

to take a derivative here since we can't make these perturbations arbitrarily small. The smallest change we can make

is to either add or subtract one. However, if we just accept our fate, then we can define the "derivative" as the finite

difference






where






Here, equality holds when  is a perfect power of  (including fractions of the form ).



Therefore,






From here, we also have






#### A Tale of Two Functions



Given , antidifferentiating both sides gives






Similarly, since  satisfies , we have






This makes sense, since we'd like these two functions to be inverses of each other.



#### Exp, redux.



Consider






which suggests that .



Since we would like , we can impose the boundary condition






which gives . However, while this method gives bounded relative error, in its unbiased form

this is pretty off the mark for general purposes (it approximates some other ). Instead, we can add in an unbiased form:






where, empirically,  gives a good approximation. Notice that the  we've chosen is close to

, which is what we need to transform  to .

In particular, for all , the , , and  relative error is always below 10%.






### Log



In a similar spirit, we can use the approximation






to derive






Imposing a boundary condition at  gives , so we should expect






However, this actually computes some other logarithm , and we'll have to, again, unbias this term






where the  term came from the fact that the base computation approximates the 2-logarithm. Empirically, I've

found that a bias of  works well. In particular, for all , the ,

, and  relative error is always below 10%.






### Geometric Mean



There's a straightforward derivation of the geometric mean. Consider the approximations of  and ,

we can refine them as






Therefore, a bit of algebra will show that






which reduces to the equation for the geometric mean. 



Notice that we just add a series of integers, followed by

an integer divide, which is pretty efficient.



-------------------------------------



For more information on how the constant () is derived for

the cube-root, visit http://www.bullshitmath.lol/.



Equations rendered with [readme2tex](https://github.com/leegao/readme2tex).