[HN Gopher] Why does integer representation of a floating point ...
___________________________________________________________________
Why does integer representation of a floating point approximate its
logarithm?
Author : segfaultbuserr
Score : 98 points
Date : 2024-01-02 10:25 UTC (12 hours ago)
(HTM) web link (stackoverflow.com)
(TXT) w3m dump (stackoverflow.com)
| Anktionaer wrote:
| This can be useful:
| https://en.wikipedia.org/wiki/Fast_inverse_square_root
| https://www.youtube.com/watch?v=p8u_k2LIZyo
| SloopJon wrote:
| This may be the longest Stack Overflow question I've ever seen,
| with a code block, three graphs, and a list of links to related
| questions, comprising four and a half screenfuls on my display.
| (The accepted answer is five and a half screenfuls.)
| svat wrote:
| People who complain about their experience of posting questions
| on StackOverflow often seem to have started with the idea that
| it was a website for getting help, rather than what it actually
| is: a way to compete for the attention of a certain community
| of potential answerers. Like most games, this too has become
| harder over the years, in this case as the player needs to work
| harder to distinguish themselves from the mass of low-effort
| posters.
|
| (Not every question needs to be this long--this one's length
| seems to be because the asker is more a historian than a
| mathematician--but length and such sections--graphs, links etc
| --are one way it gets signaled that you really have put effort
| into the question.)
| d0mine wrote:
| There may be several ways to look at StackOverflow. For
| example, as someone who answered thousands of questions, I
| don't care about author's effort--I optimize for future
| readers from google when considering what question is worth
| the time to answer it.
| svat wrote:
| Yeah I agree; there are several strategies for the game as
| a question asker. For me when answering (only a few hundred
| answers across all Stack Exchange sites, so I'm already in
| the "tail" and probably not representative of the typical
| answerer), I mainly care about whether the question seems
| interesting and challenging to me (I'm especially likely to
| answer if I think no one else would answer). But the point
| remains that with several thousand questions being asked a
| day on Stack Overflow, most potential answerers are indeed
| likely considering "what question is worth the time to
| answer it" as you said, so a question needs _something_
| (effort, interest, broad utility... or just a stroke of
| good luck) to get "successful".
|
| So when asking (~80 questions asked so far across SE sites,
| only 8 on Stack Overflow), I try to put effort into it (and
| show it), but ultimately still just treat each question as
| a gamble and if it doesn't "work out" (question got
| few/zero/negative votes, or no useful answers) it's no big
| deal (just lost a turn, try again next time... at least I
| learned something about what sorts of questions work well
| on this particular site, and got an opportunity to state my
| question as clearly as I could, which is often the most
| useful part of the process). But I can see how people who
| approach asking questions on StackOverflow from a mindset
| of "I need help and hope to get it here" can easily get
| frustrated.
| mhh__ wrote:
| Yeah. Admittedly I haven't been burnt in a while but
| stackoverflow (stackexchange, really, i haven't truly learnt
| from stackoverflow in a while) is the highest signal to noise
| I know of online
| raphlinus wrote:
| In a recent talk[1], Bill Dally of Nvidia suggests using _actual_
| logarithms, as they 're more consistent in relative error than
| floats, which is especially relevant at low precision as
| increasingly used in AI. Multiplications then become simple
| additions, but then addition becomes more complicated, as in
| general it needs lookup tables. This is the same mechanism as a
| slide rule, and also reminds me of the way the Yamaha DX7 works.
| The lookup tables can be expensive, and the talk contains an idea
| how to batch them when adding up the results of many many
| multiplications (as is also often the case for AI workloads).
|
| [1]: https://www.youtube.com/watch?v=kLiwvnr4L80
| throwawaymaths wrote:
| I would guess that additions are pretty important in deep
| learning workloads. For backpropagation, accumulator on the big
| delta matrix is very important.
| FabHK wrote:
| Fun fact, related:
|
| Julia has a function to give you the next greater floating point
| number for any floating point number, nextfloat(x), and it is
| basically implemented by casting x to an integer, adding 1, and
| casting it back.
|
| https://github.com/JuliaLang/julia/blob/3120989f39bb7ef7863c...
| renonce wrote:
| Positive floating numbers can indeed be compared by their
| binary representation. The fact that incrementing the number by
| one gives you the next floating point number is a trivial
| corollary.
| mjburgess wrote:
| Roughly, the answer should be intuitable from what a float is,
| A float is ...-exp-int A log is exp-int -> int
| adastra22 wrote:
| What a wonderfully written question... with a patently obvious
| answer: IEEE-754 floating point formats store the exponent in the
| most significant bits (other than the sign bit, which is why the
| question is limited to positive values).
| monktastic1 wrote:
| Indeed! Given the effort and knowledge put into the question,
| the obvious answer seemed too... obvious for this poster to not
| already know it. I thought I must be missing something!
| naniwaduni wrote:
| A striking reminder that fluency is not the same thing as
| competence...
| sifar wrote:
| Perhaps because the poster seems to be historian and not a CS
| practioner.
|
| From the question
|
| "Why I am asking
|
| I am writing a history of the Fast Inverse Square Root,
| information about which is collected at 0x5f37642f.com. Part
| of investigating this as a historian means asking questions
| like this from the community. I do not have the expertise to
| distill, by myself, much of this because I am not a
| mathematician or a computer scientist."
| cryptonector wrote:
| The key that TFA probably missed is that the bit pattern
| following the exponent (which is by definition an
| approximation of the logarithm) is the "linear" bit of the
| "piecewise linear approximation", while the exponent is the
| "piecewise" bit. That's really all the answer needed to
| explain in the end.
|
| "Why is an IEEE 754 positive [normal] number interpreted as
| an integer a piecewise linear approximation of its
| logarithm?" -> "Because the bit pattern for it will have an
| exponent followed by a mantissa, where the exponent by
| definition is a very rough approximation of the logarithm,
| and where the mantissa can be interpreted as a linear
| interpolation between the exponent and the next exponent,
| thus the two pieces as the "piecewise" (exponent) and
| "linear" (mantissa) approximation of the number's logarithm."
|
| The only remaining bits of interest is the bias of the
| exponent and the assumed-1 leading digit of the mantissa
| being left out. The latter is a compression technique (why
| wastefully include a bit that's always set or always reset?)
| that happens to help here (avoiding having to shift the
| mantissa bits by one). The former is easy enough to deal
| with.
| stephencanon wrote:
| If njuffa and edp have weighed in on a floating-point question,
| it is canonically answered (if you're really lucky you might
| sometimes get one of the French to join in, but any combination
| of 2 suffices).
| tamarlikesdata wrote:
| In applications like graphics rendering or scientific
| computations, how does the choice of precision in floating-point
| representation (single vs double precision) affect the accuracy
| and performance of logarithmic calculations, especially when
| relying on this approximation method? Are there benchmarks or
| scenarios where the difference between these representations is
| particularly notable?
| amelius wrote:
| Let me just guess: floating point exponent is the same base as
| the logarithm base so you can just operate on the mantissa only
| and consider the exponent separately.
___________________________________________________________________
(page generated 2024-01-02 23:01 UTC)