[HN Gopher] Eigen: A C++ template library for linear algebra
___________________________________________________________________
Eigen: A C++ template library for linear algebra
Author : cpp_frog
Score : 144 points
Date : 2022-04-04 13:26 UTC (9 hours ago)
(HTM) web link (eigen.tuxfamily.org)
(TXT) w3m dump (eigen.tuxfamily.org)
| conformist wrote:
| Eigen is a great library.
|
| A similar one to consider that can at times be slightly easier to
| use coming from a python background is armadillo:
| http://arma.sourceforge.net/
| yarky wrote:
| I really like armadillo's syntax, it feels like your doing
| R/Matlab.
| structural wrote:
| That's not a positive for everyone, Matlab's API choices are
| a bit polarizing/confusing for people who don't use it.
| bayindirh wrote:
| I'm using Eigen extensively in a project of mine. It's
| extremely fast, versatile, easy to use and reliable. I'm a fan,
| one might say.
| hcrisp wrote:
| I wonder how Eigen compares to xtensor, which was inspired by
| Numpy and has support for views, slicing, and broadcasting?
|
| https://github.com/xtensor-stack/xtensor
| structural wrote:
| Eigen is similar to xtensor, but with 1) Eigen being
| significantly more mature, and used in production in
| simulation/research environments and 2) Eigen having not just
| basic operations on arrays, but also a full suite of linear
| algebra operations comparable to a full BLAS library,
| including a full test suite. xtensor wins on having better
| integration with Python and other languages out of the box.
| There's also some performance limitations due to being
| inspired by numpy, particularly with respect to memory
| management.
| jabl wrote:
| Slightly tooting my own horn here; In case anyone is interested
| in a (trivial) comparison between them I have a tiny example
| project implemented both with Eigen and Arma (And Fortran and
| Python/numpy and Julia, FWIW): https://gitlab.com/jabl/tb
| bobsomers wrote:
| Eigen is the standard choice (for good reason!) for many linear
| algebra projects, especially in robotics, but there is a big
| downside users should be aware of before they chose it.
|
| Eigen makes _extensive_ use of expression templates in C++ to
| collapse complex operation sequences into streamlined and minimal
| calculations. This is generally ok, until you need a debug build.
| I 've regularly seen debug builds of software using Eigen run
| 1000x to 10000x slower than the release build, which seriously
| complicates various debugging workflows. It also makes it a
| nightmare to run your test suite through valgrind, for example.
|
| I've seen several engineers attempt (and fail) to try
| creating/linking a release build of Eigen with a debug build of
| the rest of the program to try to regain most of that speed while
| still allowing a decent amount of debugability, but this is
| really hard due to all the aggressive inlining and heavy use of
| templates.
|
| In my experience, I would happily accept a 2x or more slowdown in
| linear algebra performance in release builds in exchange for
| significant boost in debug execution speed. If you're starting a
| greenfield project, you should consider how important decent
| debug performance is before choosing Eigen by default.
| hasmanean wrote:
| This is a problem with C++ in general.
|
| In debug mode it has no notion of performance whatsoever.
| ogogmad wrote:
| It supports operator overloading too, by the way. This makes it
| usable where many LAPACK wrappers like Numpy are not.
| phkahler wrote:
| My only complaint is that using OpenMP Eigen can be slower with
| SMT than without SMT. They even suggest telling to use half as
| many threads as you have "cores" when "cores" means twice as many
| due to SMT.
|
| Otherwise, we've seen a 8-10x performance increase in SolveSpace
| (CAD) in some situations after switching from home-grown matrix
| operations to Eigen.
| stingraycharles wrote:
| I think this is generally true if your workload is SIMD/AVX
| heavy: these types of "heavy" instructions cannot execute on a
| single core simultaneously.
| Symmetry wrote:
| If you have Eigen-like code that won't tend to have many cases
| where you're not having many branch mispredicts or loads the
| prefetcher can't figure out and you also have enough
| calculations that you can use the width of the core on a single
| thread then there really isn't any potential throughput gain
| with SMT but you still suffer from cache contention from having
| two threads. It's really not Eigen's fault, it's the nature of
| SMT that it doesn't help in all cases.
| jhartmann wrote:
| This has been true for years, Intel CPU's can't efficiently
| perform math operations when hyperthreading is involved.
| Generally there is only a single shared FPU/AVX/SSE unit doing
| the math over two hyperthreads. Since the Eigen implementation
| often can keep that unit 100% busy, it makes no sense to try
| and run two threads at full tilt through the units.
|
| I tested all this very heavily before Eigen had AVX-512
| support. In that environment there might be some differences
| and I would suggest you benchmark both configurations.
| Sesse__ wrote:
| > Generally there is only a single shared FPU/AVX/SSE unit
| doing the math over two hyperthreads.
|
| Hyperthreading does in general share units (both ALU and
| others); that's what hyperthreading is.
|
| Apart from that, it really depends on what operations you're
| doing; e.g., modern Intel CPUs have three ports that can
| issue a 256-bit FMA, each, every cycle.
| phkahler wrote:
| >> Hyperthreading does in general share units (both ALU and
| others); that's what hyperthreading is.
|
| Yes, I think the issue with Eigen is cache related. They
| apparently have optimizations that are aware of cache
| architecture and running 2 threads that share the same
| cache will screw that up, resulting in more misses. If this
| is the case, I'd prefer algorithms that are cache line size
| agnostic. It is still much faster than the simple hand-
| written code we had before!
| criddell wrote:
| The first thing I wondered was about the license. Fortunately, it
| mostly uses the MPL license:
|
| https://eigen.tuxfamily.org/index.php?title=Main_Page#Licens...
|
| > Note that currently, a few features rely on third-party code
| licensed under the LGPL: constrained_cg. Such features can be
| explicitly disabled by compiling with the EIGEN_MPL2_ONLY
| preprocessor symbol defined. Furthermore, Eigen provides
| interface classes for various third-party libraries (usually
| recognizable by the <Eigen/*Support> header name). Of course you
| have to mind the license of the so-included library when using
| them.
|
| > Virtually any software may use Eigen. For example, closed-
| source software may use Eigen without having to disclose its own
| source code. Many proprietary and closed-source software projects
| are using Eigen right now, as well as many BSD-licensed projects.
| digiou wrote:
| God bless Eigen, I am using it to implement the state matrices of
| a Kalman Filter and it's a joy to use its APIs.
| benibela wrote:
| I implemented the same thing for my master thesis project
| Rayhem wrote:
| Eigen has one of if not the best linear algebra APIs I've ever
| seen. In particular, vectors are column vectors by default and
| you never need to touch row vectors (if I can editorialize, row
| vectors shouldn't exist at all), and vectors are _not_ simply
| "n-by-1" matrices -- they're true vectors.
| mereel wrote:
| Could you elaborate a bit more about the difference between
| Nx1 matrices and "true vectors"?
| the_svd_doctor wrote:
| One has shape (N, 1) and one has shape (N). Always having a
| trailing 1 is a pain in many settings. Matlab has that
| "trailing 1" all the time and it's a PITA.
| vbarrielle wrote:
| But they are nx1 matrices in Eigen, you can call the rows()
| and cols() methods to check that. However, the nice part is
| that, since Eigen knows the number of columns at compile
| time, it enables vector methods such as size() in addition,
| which lets you handle them as true vector.
|
| But tge nice part is that these vectors can still interact
| well with parts of the code that expect a matrix.
| lacker wrote:
| Is there anything like Eigen for data that's kept on the GPU?
| I've been looking at porting some performance-critical scientific
| computing code from Python to C++ and numpy -> Eigen seems like
| an obvious migration path, but it's harder to figure out what to
| do with cupy matrix operations.
| ncmncm wrote:
| There is Kompute.
|
| It wraps Vulkan in a very thin layer, mostly to eliminate
| boilerplate.
| Communitivity wrote:
| Recently encountered Eigen. It seems awesome - easy to work with,
| with lots of options. mat.conjugate() * mat.transpose() seems to
| take a while for my stuff, but I think I just haven't found the
| right method (next is to look at mat.adjointInPlace()).
| waynecochran wrote:
| Probably won't ever catch Fortran, but using Eigen templates for
| reductions really opens the door for compile time optimizations;
| e.g. these are all reductions that do the same thing
| const float residual = (L.array() *
| P.array()).colwise().sum().square().mean(); const float
| residual = L.cwiseProduct(P).array().colwise().sum().array().squa
| re().mean(); const float residual = (L.transpose() *
| P).diagonal().array().square().mean();
|
| The compiler can optimize using static information, e.g. these
| would all be handled differently for the following types
| Eigen::Matrix<float,3,3> L(3,3), P(3,3);
| Eigen::Matrix<float,3,Eigen::Dynamic> L(3,K), P(3,K);
| Eigen::Matrix<float,Eigen::Dynamic,Eigen::Dynamic> L(N,K),
| P(N,K);
| Symmetry wrote:
| The compile time loop fusion is also particularly nice.
| waynecochran wrote:
| Thanks -- I just learned something new. Since matrices are
| primitive types in Fortran, I assume these kinds of
| optimizations are more abundant in Fortran. I wonder if
| adding matrices / vectors / tensors as primitive types has
| even been entertained by the various C++ committees.
| madduci wrote:
| A great library, a cornerstone. Many big libraries build on top
| of it (e.g. OpenCV, PointCloud Library)
| asimeqi wrote:
| Can you elaborate on how OpenCV is built on top of Eigen? From
| what I can google it seems that OpenCV can interoperate with
| Eigen but is not build on top of it.
| madduci wrote:
| There are functions related to camera calibration and
| adjustment of images whose internals are built using Eigen
| Vectors and Matrixes.
| markisus wrote:
| Eigen is really nice for getting code that looks like the
| underlying math and it also optimizes away unnecessary
| temporaries by using template expressions. However, when you mess
| up, you get a screen full of template errors that take some
| experience to understand and debug.
|
| It makes me wish for a language + compiler where linear algebraic
| objects are first-class values.
| deng wrote:
| > It makes me wish for a language + compiler where linear
| algebraic objects are first-class values.
|
| Like Fortran?
| adgjlsfhk1 wrote:
| Fortran gives you first class dense ND-array, but doesn't
| give you the flexibility to make other types of arrays
| (banded/blocked/etc) feel first class.
| tspooner wrote:
| While it's not exactly what you're asking for, I do find the
| LinearAlgebra (standard) library in Julia to be pretty
| fantastic:
| https://docs.julialang.org/en/v1/stdlib/LinearAlgebra/
| joelhaasnoot wrote:
| Ahh, the horror of getting this to compile last week in Python :(
| rough-sea wrote:
| This library is used heavily inside Tensorflow. It is "production
| ready".
| ska wrote:
| FWIW it's been used in production long before tensorflow
| existed.
| RavlaAlvar wrote:
| I can't stress enough how this library is a saving grace. It
| would be incredibly difficult to port my python prototype code
| with numpy to C++ production without using Eigen.
| tgot wrote:
| Not widely publicized, but the benchmarking code is in the
| source. At one point I was running it on my specific target
| machines to get performance estimates in support of porting some
| large-ish CPU stuff from Matlab into C++.
|
| The max performance was in Eigen-calling Intel MKL, but it was a
| big plus to not need MKL licenses on every development machine.
| Const-me wrote:
| Using it for years, and mostly happy with that library. The
| performance is awesome for very long vectors / large matrices.
|
| It's less than ideal for small things when the size is known at
| compile-time. If one knows SIMD intrinsics, in some of these
| cases the Eigen's implementation can be outperformed by a large
| factor like 2-4. Also it's very hard to mess with RAM layout of
| some things (like sparse matrices), just too many layers of
| abstraction and too much template metaprogramming.
|
| But still, out of the box the usability of Eigen is awesome. And
| until the code is written, debugged, integrated and benchmarked,
| it's generally impossible to tell whether a particular algorithm
| gonna be a performance bottleneck. That's why I'm mostly happy
| with the library.
| ensiferum wrote:
| Any idea about perf diff between this and GLM? In Computer
| graphics my use case is with fixed 4x4 matrices and vec4s.
| Thanks!
| Const-me wrote:
| I never really used GLM, but Eigen was substantially slower
| than DirectXMath https://github.com/microsoft/DirectXMath for
| these things. Despite the name, 99% of that library is OS
| agnostic, only a few small pieces (like perspective
| projection matrix formula) are specific to Direct3D. When
| enabled with corresponding macros, inline functions from that
| library normally compile into pretty efficient manually
| vectorized SSE, AVX or NEON code.
|
| The only major issue, DirectXMath doesn't support FP64
| precision.
| markisus wrote:
| Can confirm the poor performance on small matrices (less than
| 20x20). This is a problem particularly for robotics
| applications when your entities are positions, velocities, etc.
|
| https://stackoverflow.com/questions/58071344/is-eigen-slow-a...
| wbthomason wrote:
| Huh, interesting - this is news to me, as I use Eigen all the
| time/see it used all over for robotics. Is there a good
| replacement for robotics-specific operations/small matrices
| generally (I see some people mentioning DirectXMath?)? Or is
| the tradeoff just between spending the time and effort to
| write SIMD intrinsics yourself vs. lower performance but
| greater convenience with Eigen?
|
| One advantage of Eigen's approach that I haven't seen
| mentioned here is that its templated design makes it easy to
| substitute custom scalar types for operations, which helps
| enable straightforward automatic differentiation and other
| such tools (e.g. I'm currently using Eigen to make a tracing
| JIT for computations like FK, etc. over scenegraphs).
| jeffreygoesto wrote:
| Blaze [0] looks promising. There was a little discussion
| here [1] about it's comparison and performance, especially
| for small sizes.
|
| [0] https://bitbucket.org/blaze-lib/blaze/src/master/
|
| [1] https://bitbucket.org/blaze-lib/blaze/issues/266/blaze-
| vs-ei...
| wbthomason wrote:
| Thanks, this looks promising indeed! Especially because
| it similarly supports custom scalar types:
| https://bitbucket.org/blaze-
| lib/blaze/wiki/Vector%20and%20Ma...
| wbthomason wrote:
| One small downside of Blaze vs. Eigen for robotics is
| that Blaze seems to lack the highly convenient geometry
| types and operations that Eigen ships with. This wouldn't
| be difficult to build on top of Blaze (and could lead to
| some interesting performance comparisons), but does
| present some additional work necessary to use Blaze in
| many robotics applications.
| szundi wrote:
| An other answer just debates wether that was a benchmarking
| issue
| markisus wrote:
| Multiple other users confirmed the results and that one
| answer's author never followed up.
| Const-me wrote:
| > when your entities are positions, velocities, etc.
|
| For use cases where FP32 precision is enough, I usually use
| DirectXMath library https://github.com/Microsoft/DirectXMath
| for that. That thing is cross-platform in practice. Even when
| building things for ARM Linux, it's easy to copy-paste
| required pieces, NEON support is there.
|
| When I need FP64 precision on PCs, I usually proceed without
| libraries, using AVX intrinsics.
| justshowpost wrote:
| No estoy de acuerdo con tus politicas
| gnufx wrote:
| It may have improved recently -- I haven't measured -- but
| serial Eigen seems mostly a little less performant at plateau
| than optimized BLAS GEMM for reals, and about half as good for
| complex in results I've seen for v3.3. For
| multiplication/convolution of sufficiently small dimension
| matrices on x86 (aarch64 in development) you probably want
| libxsmm; it can be used header-only -- at least for C -- if
| that matters. I might guess Eigen does relatively better on L1
| and L2 than BLAS libraries.
| bayindirh wrote:
| In my case Eigen is handling some 3000x3000 and bigger
| matrices. For these scenarios, it's performance is about 98%
| of BLAS libraries, which is more than enough when combined
| with the ease and practicality of Eigen. It also handles RAM
| placement, so it doesn't get affected by memory
| fragmentation.
|
| Their current performance page is here [0].
|
| [0]: https://eigen.tuxfamily.org/index.php?title=Performance_
| moni...
| bee_rider wrote:
| That's interesting. I'd assume that a template library like
| Eigen would be most competitive for really small matrices and
| vectors of a known size -- it should have a fundamental
| advantage over something like MKL or BLIS, in that it can fully
| inline and unroll everything if it wants, right?
| Const-me wrote:
| My guess it's their deliberate design decision. It could be
| that most users of the library don't care about small things.
|
| About the template stuff, I think their main performance
| advantage over traditional BLAS is not even SIMD, it's lazy
| evaluation. Expressions like x=a*b+c never compute the
| complete a*b matrix or vector. The a*b expression returns a
| small placeholder object on the stack, of a scary type with
| couple lines of template arguments in the type name. This way
| the complete expression runs without making temporary
| matrices, instead it streams data from all 3 arguments and
| only writes to memory once. And if the `x` is of the correct
| size already, it doesn't call malloc/free.
| bee_rider wrote:
| Yeah, lazy evaluation/fusing operations is I think their
| best feature. Fixed interface BLAS has really good
| performance, but I mean, something as simple as DAXP _B_ Y
| is an extension. Selecting what functionality to super-
| optimize is hard for a fixed interface, and involves non-
| technical stuff like figuring out what subroutines people
| actually want.
|
| I'm sure someone has already looked at this, but I wonder
| if Eigen can just somehow nab kernels directly from BLIS,
| haha.
| liquidify wrote:
| This is called "expression templates"
| https://en.wikipedia.org/wiki/Expression_templates
| kolbe wrote:
| Thank you for this. I wrote a small linear algebra library
| using intrinsics in C#, and my code was beating Eigen at the
| n<50 level, but I figured it was just some error of mine, so I
| never had the balls to make the claim publicly.
| synergy20 wrote:
| Failed to see what chips are supported. Assuming x86 and ARM and
| their vector/SIMD are leveraged for performance, what about
| RISC-V's vector/SIMD, is there a plan to add those in the future?
| Bancakes wrote:
| BLAS vs Eigen
|
| Go
| bee_rider wrote:
| I believe (haven't looked at benchmarks lately) Eigen has
| trouble beating a good BLAS implementation like MKL or BLIS at
| doing BLAS stuff, but it is more expressive and has the ability
| to do lazy evaluation/fuse operations. Anyway since you can get
| it to call your favorite BLAS/LAPACK library, these are really
| complementary projects.
| ska wrote:
| > Go
|
| Category error.
|
| One is a standard the other a library. BLAS covers a subset of
| what Eigen does, and Eigen can use BLAS/LAPACK routines
| directly from other libraries (e.g. MKL) for those things.
| gnufx wrote:
| https://github.com/flame/blis/blob/master/docs/Performance.m...
| but ignore the SKX results with the old OpenBLAS there.
| onedognight wrote:
| Eigen calls BLAS when appropriate and with no overhead and the
| code looks close to the math. Eigen also supports small fixed
| size matrices where BLAS is not appropriate.
___________________________________________________________________
(page generated 2022-04-04 23:01 UTC)