[HN Gopher] EASTL: An Alternative C++ Standard Library From Elec...
___________________________________________________________________
EASTL: An Alternative C++ Standard Library From Electronic Arts
Author : optimalsolver
Score : 105 points
Date : 2021-08-04 13:21 UTC (9 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| Subsentient wrote:
| An innovative new STL with unlockable features, such as:
|
| std::vector: $5
|
| std::string: $15
|
| std::map: $30
|
| std::unordered_map: $100
| phone8675309 wrote:
| There's also a new version with renamed methods every year, and
| some of the methods move to other libraries in the middle of
| the project.
| jhgb wrote:
| Is that a pay-to-Win32 library?
| mike_hock wrote:
| std::cout << std::vector<char>{'h', 'e', 'l', 'l', 'o', ' ',
| 'w', 'o', 'r', 'l', 'd', '\0'}.data() << std::endl;
|
| I'm too cheap to pay triple the price for a little bit of
| convenience.
| marcodiego wrote:
| Actually some historic IBM computers were like that. You leased
| them and could pay more to enable some instructions on the
| processor.
| sigzero wrote:
| That's just funny.
| microtherion wrote:
| And occasional loot crates containing optimizations?
| teeray wrote:
| Those are per-month, I assume. Subscriptions are all the rage
| in software these days.
| paulddraper wrote:
| Correction: $5/month
| glouwbug wrote:
| Just get std::map. You can get std::vector<int> if you do
| std::map<int, int>
| MauranKilom wrote:
| Yikes for allocating all your elements separately, and with
| 400% - 700% memory overhead.
| danuker wrote:
| List size is O(n).
|
| https://eastl.docsforge.com/master/design/#listsize-is-on
|
| I am surprised compilers don't magically optimize the code and
| remove the size-remembering variable if it is not needed.
| jcelerier wrote:
| > I am surprised compilers don't magically optimize the code
| and remove the size-remembering variable if it is not needed.
|
| the problem is that technically anyone can come and do
| auto lib = dlopen("my_lib.so", 0); auto sym =
| dlsym(lib, "_ZNKSt7__cxx114listIiSaIiEE4sizeEv"); //
| std::list<int>::size() auto func =
| static_cast<void(*)(std::list<int>*)>(sym);
| std::list<int> some_local_list = ...;
| (*func)(some_local_list);
|
| and that is expected to not crash - template symbols are
| generally part of your shared library API (at least that's been
| the default, however bad it is, on Linux, for a very long
| time). If compilers were optimizing the layout of individual
| list instances, then the above wouldn't work anymore (unless
| the compiler would inline / create new symbols for each
| individual cases in your code which would make the object sizes
| go through the roof).
| Kranar wrote:
| There's no requirement or even expectation that your snippet
| of code in principle works (even after working out the issues
| in the snippet you posted, such as needing to use the .*
| operator, the fact that member functions are not the same
| size as void*, and other quirky details).
|
| The only way something somewhat similar to what you posted
| could possibly work would be to have some_local_list
| allocated/constructed dynamically from the same shared object
| that contains the the member function and to expose the
| member function using extern "C".
|
| For example something like: auto
| some_local_list = create_list(...);
| list_size(some_local_list);
|
| Where create_list is a function loaded from the same dlopen
| and returns a dynamically allocated and opaque handle to a
| list, and list_size is also a function loaded from dlopen
| that is exported using extern "C".
|
| With this approach it's certainly possible for a compiler to
| optimize out unused member variables. Any other approach is
| undefined behavior and may or may not work.
| jcelerier wrote:
| > There's no requirement or even expectation that your
| snippet of code in principle works
|
| "in principle" goes away as soon as we use dlopen, as it
| implies a lot of things on the way C++ will be supported on
| that given platform ; no one cares about C++ in a vacuum.
|
| In practice, different .dll / .so / .dylib communicate
| through C++ APIs all the time ; all relevant platforms have
| to support that at some level (which can sometimes cause
| _strong_ headaches, for sure:
| https://www.codesynthesis.com/~boris/blog/2010/01/18/dll-
| exp... ).
| Kranar wrote:
| When I say in principle, I don't mean according to a
| strict interpretation of the standard in a vacuum. I mean
| that even if I were to extend to you a great deal of
| liberties and operate at the level of what you're
| attempting to accomplish, your approach is fundamentally
| invalid and results in buggy code that will break and is
| entirely unneccessary.
|
| The way you accomplish dynamically loading member
| functions is by exporting a plain C function using extern
| "C" that takes an opaque handle to the object you wish to
| operate on, and whose implementation wraps the member
| function whose operation you wish to expose.
|
| Pointers to member functions are fundamentally not
| compatible with void* and hence may not reliably be
| returned using dlsym. Only once the member function is
| bound to an object (using the .* operator, ie.
| object.*member) is the resulting pointer compatible with
| a void* (in C++11 it's implementation defined). Until
| then, they not only have different sizes, their size may
| even be different within different translation units of
| the same application!
|
| In practice you are right that DLLs and shared objects
| communicate through C++ APIs all the time, and the
| reliable way that they do so is by using extern "C". The
| article you linked to is exactly the kind of pain,
| undefined behavior, and buggy problems you will encounter
| when you try to use any other mechanism than the plain
| and straight forward mechanism that exists precisely for
| the purpose of facilitating this kind of communication.
|
| The reason my point is worth making, as opposed to just
| being a pedantic technicality, is because this approach
| is precisely what allows compilers to make various
| optimizations that continue to work safely even in
| situations where objects, functions, and member functions
| are used across dynamic boundaries. If you don't follow
| this approach, then the compiler will make certain
| optimizations that will result in disastrous behavior.
|
| If you don't want to take my word for it, hopefully
| you'll take the advice of the ISO CPP [1]:
|
| "do not attempt to "cast" a pointer-to-member-function
| into a pointer-to-function; the result is undefined and
| probably disastrous. E.g., a pointer-to-member-function
| is not required to contain the machine address of the
| appropriate function."
|
| [1] https://isocpp.org/wiki/faq/pointers-to-members#addr-
| of-memf...
| tcbawo wrote:
| I have seen implementations of type erasure/delegates
| that store an instance pointer and pointer-to-member-
| function as an instance and pointer-to-member-function of
| a dummy type with the same calling convention. It's
| interesting how many different member function pointer
| sizes Windows has. Undefined behavior and compiler
| dependent, but consistent.
| josefx wrote:
| However list splice is O(1), since it doesn't have to count how
| many items it moves.
| gpderetta wrote:
| These sort of layout optimizations are extremely hard in C-like
| languages as the compiler needs to prove that the program can't
| tell the difference. At the very least you need whole program
| optimizations.
| T-zex wrote:
| Isn't space complexity for a list with cached count (size)
| property "c" still O(n + c) ~ O(n)?
| secondcoming wrote:
| Having compilers remove members would be a backwards
| compatibility shitshow.
| waynecochran wrote:
| Somewhere near the first paragraph there should be something that
| tells me what I get beyond STL and Boost. I still have no idea.
| throwaway0x wrote:
| Does it also include microtransactions?
| belazeebub wrote:
| It should be noted that Paul Pedriana, the original developer of
| EASTL and many other things, just recently passed away
| unexpectedly. He was an EA programming legend.
|
| Note of passing:
| https://www.facebook.com/groups/20296764839/posts/1015929527...
|
| EASTL paper: http://www.open-
| std.org/jtc1/sc22/wg21/docs/papers/2007/n227...
|
| Partial list of Paul's programming credits:
| https://www.mobygames.com/developer/sheet/view/developerId,2...
| adamrezich wrote:
| >Partial list of Paul's programming credits
|
| MS Hearts, wow!
| vlovich123 wrote:
| Shit. I just worked with Paul not too long ago at Oculus.
| That's a real loss.
| pjmlp wrote:
| This is pre-historic by now, in C++ history.
|
| Quite appropriate to this subject are the talks from Mathieu
| Ropert.
|
| Here is one of the latest ones,
|
| "This Videogame Developer Used the STL and You'll Never Guess
| What Happened", ACCU21
|
| https://www.youtube.com/watch?v=xoEUO9DezV8
| colinmhayes wrote:
| Is this really necessary? How many implementations of the
| standard library do we need? I get the feeling that engineers
| love reinventing the wheel so much that they forget to ask
| themselves what's wrong with the current wheel. My last company
| had its own implementation of the stl and it was a piece of crap,
| productivity would've gone way up if we just used the normal
| implementation and boost threading. Maybe I'm just projecting,
| but I get the feeling this company wasn't the only one that has
| made that mistake.
| JohnFen wrote:
| The STL is problematic and not a great solution for a lot of
| things, so replacing it with something else is very common
| practice.
| agent327 wrote:
| Could you show how it is problematic? And why replacing it
| with this one is better?
| tialaramex wrote:
| 1. The STL bakes in a bunch of assumptions about how the
| data structures you would want should work.
|
| These assumptions are often wrong. It's possible they were
| not wrong when the STL was first conceived, but we don't
| care about that because we're writing programs now, not
| then.
|
| For example the STL unordered_map thinks your hash map has
| buckets. After all, if you learned how to make such a data
| structure in a typical college course in like 1995 the hash
| map had buckets. So the _mandatory_ API for the STL 's
| unordered_map has buckets like in that college course.
|
| But today in many cases you don't want buckets, you've got
| a single unbucketed structure. How do these APIs work with
| your better structure? They don't. The STL is incompatible
| with your better structure.
|
| 2. The STL bakes in a bunch of assumptions about how C++
| works.
|
| Those assumptions were undoubtedly correct when the STL was
| first conceived, but since then there have been _major_
| changes to the language and all it can do is bolt on more
| and more, and more boilerplate to try to cope.
|
| Take emplace(). This _looks_ like it 's a better choice in
| a bunch of cases than say, insert() and then you dig into
| your STL implementation and you discover it had no choice
| but to construct your expensive object and then throw it
| away when it wasn't needed just as you might have with
| insert(). That's just how the class is defined, too bad.
|
| If EASTL is in fact just an STL then it might be no better
| than a modern STL you got with your compiler. But some
| people choose to have something better _instead_ of the
| STL. Abseil 's Swiss Tables for example offer a faster
| Unordered Map, it's just that it isn't, and can't be, a
| std::unordered_map
| not2b wrote:
| The STL interfaces work just fine with your alternative
| hash map, for example, those from Abseil. But to get the
| speed that they got, Abseil had to impose restrictions
| that make it less general. So yes, it can't be an
| std::unordered_map. That's OK. The algorithm APIs work
| just fine on any class that looks like a standard
| container.
| tialaramex wrote:
| This is all messier because C++ didn't have Concepts at
| the outset and when it got Concepts those had to have
| duck-typing instead of having the programmer write down
| which types implement a Concept.
|
| Both existing templates and any C++ Concept can
| _accidentally_ implicate something as a duck (or a
| container) when it actually isn 't. Meanwhile for an
| implementer, the only way to be sure you've written a
| working duck (or container) is to try it and see. You
| can't just assert "This is a duck" and have the compiler
| explain why it isn't AFAICT.
|
| Even if Abseil's SwissTables weren't containers from the
| point of view of STL algorithms, the only way for Abseil
| to stop STL algorithms assuming they _are_ anyway would
| be to purposefully sabotage the API and annoy users who
| don 't need any such guarantee. So that sucks pretty
| badly IMNSHO.
|
| When this is discussed there are usually two examples in
| play. One is ludicrous like Stroustrup's "CowboyWindow"
| from the 2nd edition of "The C++ Programming Language"
| which needs draw() for Window and draw() for Cowboy. This
| will be dismissed as a corner case that isn't going to
| have a real impact. It's hard to imagine some class that
| "accidentally" offers all the method signatures from your
| non-trivial concept when it's actually something quite
| different.
|
| The other is the "backward compatibility" example. You
| have a better_map your company used for decades, and now
| some asshole came along and said it isn't a container
| because you didn't write that down? Who does he think he
| is?
|
| But actually the problem you run into isn't CowboyWindow
| or better_map it's faster_map, which has exactly the same
| method signatures as better_map and so can be dropped in
| as far as the linker and compiler are concerned, but
| alas, for performance faster_map behaves a little
| differently and it must _not_ be used as an STL
| container. Oops.
| drno123 wrote:
| This is now a classic reading: (tldr: it is faster to call
| PHP process from C++ code and do regex in PHP than to use
| std::regex, and slow std::regex will never change because
| the Comittee doesn't want to break ABI in favor of speed):
|
| https://cor3ntin.github.io/posts/abi/
| sidlls wrote:
| The STL is meant to solve a fairly general class of
| problems. It isn't optimized for a number of use cases, and
| frankly some of the design and nomenclature is terrible.
| mempko wrote:
| What design nomenclature is terrible? Any examples?
| [deleted]
| maxlybbert wrote:
| EA and Bloomberg happened to make similar extensions to the
| standard library. Then, when the C++0x revision started, they
| both submitted proposals explaining their extensions including
| why they felt the extensions were necessary ( http://www.open-
| std.org/jtc1/sc22/wg21/docs/papers/2007/n227... ,
| http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2015/n446...
| ). The approach eventually made it into the standard (
| https://en.cppreference.com/w/cpp/memory/memory_resource ).
| typon wrote:
| I suggest watching this talk:
|
| https://www.youtube.com/watch?v=vElZc6zSIXM
|
| Or pretty much any other by Chandler to understand why the STL
| sucks and why you should write better datastructures for your
| own use cases.
| w_t_payne wrote:
| Every C++ shop that I've worked in has done the same thing.
| lalos wrote:
| I believe this is the library promoted by the exceptions-
| should-not-exist cpp crowd. So that explains the reinventing
| the wheel part.
| otabdeveloper4 wrote:
| It was probably made in the stone age sometime in the 1990's.
| cle wrote:
| Their docs clearly explain the difference and why this
| implementation exists, and reasons to use (and not use) EASTL:
| https://eastl.docsforge.com/master/faq/#info2-what-uses-are-...
| sho_hn wrote:
| Looks interesting, but the README is thin on espousing the
| benefits. Questions that come to mind:
|
| - What stands out about it?
|
| - Benchmarks?
|
| - What platforms are supported?
|
| - How is it differentiated today vs. 15 years ago? Qt wound up
| with its own 'standard library'-alike types mainly because the
| STL (and its implementations across platforms) were not really up
| to snuff when it started out. I suspect this may be similar.
|
| - Does anyone have direct experience to share? :-)
|
| Edit: Found some info at
| https://eastl.docsforge.com/master/faq/#info2-what-uses-are-...
| and benchmarks at https://eastl.docsforge.com/master/benchmarks/
|
| Especially interesting for me as Linux dev: "[...] EASTL is
| significantly more efficient than Dinkumware STL, and Microsoft
| Windows STL [...] EASTL is roughly equal in efficiency to STLPort
| and GCC 3.x+ STL, though EASTL has some optimizations that these
| do not."
| [deleted]
| zrail wrote:
| It's been a long time but from what I remember the selling
| point for EASTL is the ability to specify your own allocator
| instance for every container. EA writes portable code across
| lots of different architectures, many of which share a lot more
| with embedded systems than with PCs. They have constrained
| memory or special dedicated memory areas. Also, iirc EA uses
| tagging allocators extensively letting them trace against
| memory budgets down to the team level.
|
| Edit: there's an extensive FAQ in the docs folder:
| https://github.com/electronicarts/EASTL/blob/master/doc/FAQ....
| fnord123 wrote:
| Specifying your own allocator is like a main feature of bde
| from Bloomberg:
|
| https://github.com/bloomberg/bde
| jeffbee wrote:
| The FAQ seems to answer the question of why EASTL is better
| than some extremely old and bad standard libraries, not why
| it is better than modern ones.
| tyingq wrote:
| I don't know where they do their CI/CD now...they seemed to
| have moved off of travis. But, there's a snapshot of a
| benchmark from 2019 still here:
|
| https://github.com/electronicarts/EASTL/issues/267
| ThePadawan wrote:
| That was my impression of how EASTL came to be. Everybody
| used old and bad libraries, so they wrote ones that were
| better.
|
| Imagine they wrote their own Javascript engine before V8
| became really performant. That sort of thing.
| maxlybbert wrote:
| That is accurate. That general model eventually made it into
| the standard, because of EA and Bloomberg (
| https://en.cppreference.com/w/cpp/memory/memory_resource ).
| mseidl wrote:
| Don't forget about micro transactions and loot boxes.
| npalli wrote:
| They benchmark against Windows STL from Visual C++ 7.1 that
| came out in 2003!. Not sure how it compares to more modern
| versions much less something like abseil or Folly.
| jbluepolarbear wrote:
| I used EASTL on a few projects and it was 10x faster in debug
| mode. VS had a lot of checks in debug mode and it was almost
| impossible to play games in debug mode. I'm haven't used it
| since 2012.
| Negitivefrags wrote:
| I would highly recommend building with
| ITERATOR_DEBUG_LEVEL=0 in debug mode.
|
| It can be a bit annoying because all other libraries you
| link with must also be have this set the same.
| Const-me wrote:
| > Does anyone have direct experience to share?
|
| Used it a few times for some medium-complexity stuff, here's an
| open-source example: https://github.com/Const-
| me/vis_avs_dx/tree/master/avs_dx/Dx...
|
| Works OK, and indeed faster than VC++ (I was using VS2017 at
| that time), especially in debug builds.
|
| One interesting feature missing from the standard library is
| segmented_vector container.
| https://github.com/electronicarts/EASTL/blob/master/include/...
| wyldfire wrote:
| It would be interesting to see how well it performs on the libc++
| test suite. I know Microsoft recently explored running their C++
| lib against this test suite and I think libstdc++ has been run
| against it too.
| StephanTLavavej wrote:
| Well beyond exploring - microsoft/STL runs libc++'s test suite
| for every PR, skipping a set of known failures (for product
| bugs, test bugs, etc.), see
| https://github.com/microsoft/STL/blob/main/tests/libcxx/expe...
| . (I work on MSVC's STL.)
| jsnell wrote:
| Previous discussion (2016):
| https://news.ycombinator.com/item?id=11094436
| IvanAchlaqullah wrote:
| I wonder why EA need this when modern console (except Nintendo
| Switch) are all x86.
| maccard wrote:
| Ea were shipping games on PS3 and Xbox 360 until 2017. A modern
| game likely has code that datds back to the early days of that
| console, so it's a choice between maintaining eastl or porting
| all of their games to the standard library, along with the
| tooling theuve built around it (as someone else mentioned,
| tagged allocators are a big thing that aren't supported by the
| standard library.
|
| Also, just because something is x64, doesn't mean it's standard
| library is up to snuff, or that it has the same features. You
| might be using a 5 year old compiler with a standard library to
| match it.
| corysama wrote:
| 1. It was started back in the days of the PS2, I believe.
|
| 2. https://eastl.docsforge.com/master/faq/
| immmmmm wrote:
| somewhat unrelated but came across the Embedded TL recently,
| mostly for use on very small CPU, not exactly the same
| functionality but might be useful elsewhere as well
|
| https://www.etlcpp.com/
| darzu wrote:
| For Azure Sphere OS's user-land components we opted to use EASTL
| b/c of its small footprint. I don't recall any major issues
| arising in the two years I worked there.
___________________________________________________________________
(page generated 2021-08-04 23:00 UTC)