[HN Gopher] Automatic Differentiation in 38 lines of Haskell
       ___________________________________________________________________
        
       Automatic Differentiation in 38 lines of Haskell
        
       Author : ttesmer
       Score  : 36 points
       Date   : 2022-09-17 18:05 UTC (4 hours ago)
        
 (HTM) web link (gist.github.com)
 (TXT) w3m dump (gist.github.com)
        
       | mountainriver wrote:
       | Being purely functional makes this quite easy, still a beauty to
       | see
        
         | sterlind wrote:
         | it's not just the functionalness. you couldn't do it this way
         | in Lisp/Scheme (I think?) because of the lack of multiple
         | dispatch.
         | 
         | If you did (f 'x) for instance, you'd end up with things like
         | (* 2 'x) which would blow up, since Lisp would try to compute
         | the answer instead giving you '(* 2 x) back.
        
           | cryptonector wrote:
           | The Common Lisp Object System has multiple dispatch.
        
       | green_on_black wrote:
       | Curious: I don't see the `^` op defined, or is it translated inti
       | `exp` im guessing?
        
         | ayjtyjtyjgfjc wrote:
         | It is one of three exponentiation operators in standard
         | Haskell, no translation involved.
         | https://stackoverflow.com/a/6400630/14768587
        
         | ttesmer wrote:
         | Since the type was made an instance of the Num typeclass, any
         | function that can be used with Num's, can now be used on the
         | type (Dual d). As per the Prelude[1], ^ is part of the Num
         | typeclass. Same thing for * for Floating[2]. The hyperbolic
         | tangent can also be used without being explicitly coded, as it
         | can be derived using cosh and sinh!
         | 
         | EDIT: As for the differentiation, it works for ^ since it is
         | just multiplication (https://hackage.haskell.org/package/base-4
         | .17.0.0/docs/src/G...) for which the derivative was defined
         | using the product rule.
         | 
         | [1]:
         | https://hackage.haskell.org/package/base-4.17.0.0/docs/Prelu...
         | [2]:
         | https://hackage.haskell.org/package/base-4.17.0.0/docs/Prelu...
        
         | tikhonj wrote:
         | The ^ operator is defined in Haskell's standard library for
         | raising values of any numeric type to non-negative, integral
         | powers. Conceptually a ^ n just expands to a * a * ... * a, but
         | the actual code is a bit more complex[1] for performance
         | reasons.
         | 
         | The neat thing with this approach is that ^ works for _any_
         | numeric type, including user-defined types like Dual in this
         | example. Since the Dual type can handle calculating derivatives
         | for *, it gets derivatives for ^ for free.
         | 
         | [1]:
         | https://hackage.haskell.org/package/base-4.17.0.0/docs/src/G...
        
       | sterlind wrote:
       | This is an interesting approach. Haskell is not a symbolic
       | language, but you take advantage of the abstractness of type
       | parameters in function definitions to thread your implementation
       | of "D x" through, and pattern match on that.
       | 
       | It's a neat design pattern. I bet it'd work in Julia too.
        
         | xiphias2 wrote:
         | Yes, but Julia has both forward and backward differention
         | implemented (backwards it's harder).
        
           | ttesmer wrote:
           | As I wrote in the Markdown file, there's also a usable
           | package for Haskell called `ad` on Hackage. It has both
           | forward and backward autodiff and prevents expression swell,
           | among other things. This gist is just for illustration
           | purposes.
        
       ___________________________________________________________________
       (page generated 2022-09-17 23:00 UTC)