[HN Gopher] Reinvent the Wheel
___________________________________________________________________
Reinvent the Wheel
Author : zdw
Score : 555 points
Date : 2025-05-24 20:05 UTC (1 days ago)
(HTM) web link (endler.dev)
(TXT) w3m dump (endler.dev)
| moron4hire wrote:
| I've long said, I hate when people say, "don't reinvent the
| wheel." Even for the literal wheel, I know of at least 3 times it
| was usefully reinvented in just the last 20 years.
|
| People who say "don't reinvent the wheel" should come up with
| something new to say.
|
| But they can't, because they hate innovation so much.
| mcnamaratw wrote:
| I'm sorry but you're mistaken. The wheel was invented thousands
| of years ago.
| hoppp wrote:
| Wheels are constantly under development and are improving, but
| they are still derived from the previous concept, not a new
| invention
|
| I think reinventing the wheel would be like creating a wheel
| that does not spin or somethin... some youtubers have done it
| actually. Its fun to mess around with but hard to create
| something valuable.
| Ekaros wrote:
| Or they have noticed that some new wheels are not great. One
| example of wheel was this electric bicycle with massive hub. In
| essence almost as big constructions of bearings as the wheel
| itself... Horrible reliability and efficiency. Did look cool.
| hoppp wrote:
| You can reinvent the wheel for a specialized niche case but just
| dont expect many people to use it.
| mcnamaratw wrote:
| Yeah. Ok. You want the fine print? Reinvent the wheel ... for
| your own learning and fun. Or invent a different kind of wheel.
|
| Don't reinvent the wheel on a tight deadline while the rest of
| the team does useful work. Don't reinvent the wheel and then
| expect to be rewarded or respected as if you had really invented
| something.
| demirbey05 wrote:
| If you invent someting better, you can get reward, lots of
| examples out there.
| mcnamaratw wrote:
| Sure. But inventing something better is not reinventing the
| wheel. Those are two completely different activities.
| nssnsjsjsjs wrote:
| It doesn't have to be not better though. The wheel
| reinventor has hindsight.
| baxtr wrote:
| "Sometimes it's a good idea to reinvent the wheel" would have
| been a better title.
| nssnsjsjsjs wrote:
| Oh I totally assumed this applied to passion projects, making
| your own K8s on weekends kind of thing.
| wcfrobert wrote:
| Reinventing the wheel is the best way to learn. But imo that's
| really the only context where you should.
|
| I love my rabbit holes, but at work, it's often not viable to
| explore them given deadlines and other constraints. If you want
| your wheel to be used in production though, it better be a good
| wheel, better than the existing products.
| epolanski wrote:
| 99% of the people that reinvent wheels at work don't know how
| the wheel they don't like is even made and why it has the
| compromises it has.
| chad_c wrote:
| I'm reminded of Chesterton's Fence [1]. I just explained this
| to a coworker that was proud of indiscriminately eliminating
| 200+ positions.
|
| [1] https://www.lesswrong.com/w/chesterton-s-fence
| strken wrote:
| If your fencing was done by idiots, then at some point you
| have no alternative but to start tearing down fences and
| see what happens. Chesterton's Fence implies either a
| degree of design or a timeworn quality that most software
| products don't match.
|
| Chesterton himself was using it as a religious metaphor,
| and I think most of us agree that software engineers are
| not literal gods.
| chad_c wrote:
| It's not that you can't tear down the fences. It's that
| you studied why they were there in the first place.
| strken wrote:
| Sometimes the reason a fence exists is because the person
| who put it there was some combination of a) an idiot, b)
| extremely confused, or c) not qualified to construct
| fences.
|
| Also, in software, you may find the fence sometimes
| exists in higher dimensions than the usual three, can
| turn invisible for part of its length, and occasionally
| exists but is four inches tall and blocks only mice. And
| it may also be designed, at least in part, by a blind
| idiot god called ChatGPT on whose altar consciousness
| itself is sacrificed. At this point it's worth
| considering whether keeping the fence around is worse
| than letting the bull out.
| dclowd9901 wrote:
| It's funny. I had a similar analogy with some legacy
| systems recently. No one seemed to own or know where the
| data egressed to or whether it was even used anymore.
|
| But it was also low-value data and even in the worst case
| we surmised the most we'd do is anger someone in
| marketing briefly.
|
| I think so long as you can ascertain that the stakes are
| low, this is a good tactic.
| throwaway173738 wrote:
| The other 1% does it because the original wheel constrains
| them to an inferior approach that doesn't work for their
| employer.
| rTX5CMRXIfFG wrote:
| It's not really the best way to learn because it's the most
| expensive and time-consuming. What needs to be learned just
| needs to be well-documented and possible to tinker with, and
| clarity of communicating knowledge is a problem on its own, but
| you shouldn't have to build the whole thing from scratch.
| grg0 wrote:
| That sounds like learning something only very superficially.
| Rewriting from scratch is the only way to really learn any
| topic of non-trivial depth and complexity.
| rTX5CMRXIfFG wrote:
| It's not necessarily superficial learning. If you want to
| learn math, you'd be crazy to rewrite the rules of math
| just to learn it---you'd be better off with a good teacher
| and constant practice instead of building the theorems and
| formulas yourself, or you'll never finish learning math in
| your lifetime.
|
| Same can be said of learning a large inherited codebase. If
| you're assigned to a large piece of functionality and you
| need to understand it, your first instinct shouldn't be to
| rewrite the whole thing from scratch just to understand it;
| it should be to either read the code and documentation, or
| if those are impossible to comprehend, ask around what it's
| supposed to do, maybe even write tests to get a general
| sense of the behavior.
|
| I don't expect programmers to be cognitive psychologists
| here but the thing about learning is that you must do a
| "progressive overload" of complexity as you go, which is
| why building on top of what exists is the best way to go.
| You get a general idea first of what something is supposed
| to do, and then when you're good enough, that's when you
| can work backwards to building something from scratch where
| all the complexity at lower levels of abstraction wouldn't
| be too much for your brain to handle. Starting with a large
| amount of complexity will impair your learning and only
| discourage you.
| christophilus wrote:
| Different people learn differently. I learned computer
| science by first learning low-level languages, and then
| moving up the abstraction tree. Many go the opposite
| direction.
|
| Personally, I'm a fan of learning by doing. Reinventing
| the wheel works better for me than just about anything
| I've tried.
| bigstrat2003 wrote:
| > It's not really the best way to learn because it's the most
| expensive and time-consuming.
|
| The expense (time or otherwise) follows from how intimately
| you have to get to know the subject. Which is _precisely_ why
| it 's the best way to learn. It's not always viable, but when
| you can spare the expense nothing else compares.
| eviks wrote:
| No, the expense follows from how many wasted options you'll
| have to go thorough. The working intimate knowledge can be
| acquired directly without that waste.
| kupopuffs wrote:
| depends on what you're optimizing for
| dennis_jeeves2 wrote:
| >it's often not viable to explore them given deadlines and
| other constraints
|
| True for life in general. We have limited lifespans. Aging and
| death really are the great grand daddy of all problems.
| zero-sharp wrote:
| So you ignored the context surrounding this piece of advice?
|
| Yea, reinventing the wheel is a great way to learn. You're not
| going to hear an educator tell you to not reinvent the wheel.
| tehnub wrote:
| Yep, not reinventing the wheel is advice you should consider in
| a business context, not when you're programming for fun,
| although I'm sure there are hordes of people _giving_ this
| advice outside of the business context and perhaps that's what
| OP is responding to.
| pona-a wrote:
| There are many, many "educators" that will tell you to not dare
| think about a wheel more than it takes to mount a new one in.
| begueradj wrote:
| One of the most important reasons to reinvent the wheel which is
| is not mentioned by the author is to avoid adding complexity
| through unnecessary dependencies.
| underdeserver wrote:
| 100% this, and I'll add that libraries become popular because
| they solve an issue in many different scenarios.
|
| That menas that almost by definition, if a library is popular,
| it contains huge amounts of code that just isn't relevant to
| your use case.
|
| The tradeoff should be whether you can code your version
| quickly (assuming it's not a crypto library, never roll your
| own crypto), because if you can, you'll be more familiar with
| it and carry a smaller dependency.
| kortilla wrote:
| That's true for frameworks but not good libraries.
| layer8 wrote:
| Frameworks are clearly worse, that's true. But there are
| also kitchen-sink libraries that are too shallow in
| relation to their API surface, or libraries that perform
| certain background work or modify some external global
| state in a way that is unnecessary for your use case, or
| libraries that pull in transitive dependencies of that
| sort. You really want to minimize the code that executes in
| order to fulfill your use case, and also minimize the
| temptation to depend on additional, tangential functions of
| a library that you wouldn't have added for those functions
| alone.
| zzo38computer wrote:
| I also agree to avoid adding complexity through unnecessary
| dependencies.
|
| > if a library is popular, it contains huge amounts of code
| that just isn't relevant to your use case.
|
| It is true that many libraries do contain such code, whether
| or not they have dependencies. For example, SQLite does not
| have any dependencies but does have code that is not
| necessarily relevant to your use. However, some programs
| (including SQLite) have conditional compilation; that
| sometimes helps, but in many cases it is not suitable, since
| it is still the same program and conditional compilation does
| not change it into an entirely different one which is more
| suitable for your use.
|
| Also, I find often that programs include some features that I
| do not want and exclude many others, and existing programs
| may be difficult to change to do it. So that might be another
| reason to write my own, too.
| jfengel wrote:
| Unfortunately, if you depend on any libraries, there's a
| decent chance one of them depends on some support library.
| Possibly for just one function. And then your build tool
| downloads the entire Internet.
| thfuran wrote:
| That depends a lot on your language / build system. The
| easier it is to add a dependency, the more likely that is
| to be how they work, broadly speaking.
| rjsw wrote:
| A ruby application that I will soon need to use downloads
| 227 packages.
| jonny211 wrote:
| That's wild. Which gem is it? Something AWS?
| zoky wrote:
| OP said that was for an entire app, not dependencies for
| a single gem. And it's not really _that_ many. A bone-
| stock Rails app includes almost 120 gems out of the box.
| Add a few additional gems that each have their own
| dependencies and you can get up to over 200 total
| packages pretty quick.
| rjsw wrote:
| It is a command-line tool for document production.
| yetihehe wrote:
| > A bone-stock Rails app includes almost 120 gems out of
| the box.
|
| Maybe it shouldn't be necessary to have 120 different
| external libraries just to run bone-stock app? They
| should be placed in standard library.
| bitwize wrote:
| "Never roll your own crypto" usually means "never devise your
| own crypto algorithms". Implementing an established algorithm
| yourself is OK _provided_ you can prove your implementation
| works correctly. And... well, as Heartbleed showed, that 's
| hard even with established crypto _libraries_.
| jeff-davis wrote:
| Note that there are quite a few ways that crypto
| implementations can be insecure even if it's proven to be
| "correct" (in terms of inputs and outputs). For instance,
| it may leak information through timing, or by failing to
| clear sensitive memory due to a compiler optimization.
| speed_spread wrote:
| Getting the algorithm right is the easy part. It's the
| details of the implementation that kill you. Don't roll
| your own crypto.
| deadbabe wrote:
| Actually, now that we have decent LLMs, rolling your own
| crypto is probably feasible.
| immibis wrote:
| Any fool can write an encryption algorithm that he himself
| can't break. The NSA would greatly prefer that you did,
| too. Security is an arms race - you have to counter the
| latest generation of attackers.
|
| It's okay to write a compiler or a database if you only
| know the basic principles, but it's not okay to write an
| encryption algorithm, or even an implementation of one,
| using only basic principles, because someone who knows more
| than you will break it.
|
| For instance, were you aware that every time you write
| array[index], it leaks data to other threads in other
| processes on the same CPU, including JavaScript code
| running inside web browsers?
| deadbabe wrote:
| Yes of course, but do you know exactly who wrote your
| encryption libraries and what their qualifications are
| and who they work for or what their conflicts of interest
| might be?
|
| I really doubt people give it even a second thought.
| mrngm wrote:
| That holds not only for cryptography libraries, but
| generalizes to the entire computing stack. It's why, for
| example, coreboot exists, as well as various open source
| hardware projects. If it's fully open, you can inspect it
| yourself. Anywhere I see a branching statement within
| cryptography context, I'll know something's up.
|
| The problems introduced in xz are still fresh, but
| Dual_EC_DRBG[0] also comes to mind within the
| cryptography context.
|
| (Besides, getting cryptography right goes way beyond
| "just writing a library". As the parent commenter wrote,
| simple operations are the tip of the iceberg with regards
| to a correct implementation)
|
| [0] https://en.wikipedia.org/wiki/Dual_EC_DRBG
| deadbabe wrote:
| If you're not aggressively vetting the crypto libraries
| you're using, you're more or less exposing yourself to
| the same probability of risk as rolling your own crypto.
| immibis wrote:
| If you're picking some random blob of code from Github,
| then yes. If you're picking OpenSSL or Bouncy Castle,
| then no. Despite Heartbleed.
| mrngm wrote:
| I'll pick my battles, it's part risk appetite and part
| expected attacker model. It also depends on what I'm
| trying to protect.
|
| If it's in a standard library of an open source
| programming language, I'm less inclined to fully check
| the implementation.
| immibis wrote:
| When you use OpenSSL, you can trust that if you're
| screwed, so is the whole internet.
|
| That actually happened once.
| lobsterthief wrote:
| "Never roll your own crypto" is just this year's "never roll
| your own date library". There will always be something. Could
| I code this? Even if it's quick, there's ongoing maintenance
| cost and you lose out on the FOSS community identifying and
| fixing vulnerabilities as well as new features that you may
| need to use. Yes, the library might be large and contain
| things you don't need, but that's the tradeoff. You can
| mitigate this (depending on the platform and language)--for
| example, with ESM tree-shaking.
|
| I'd rather install date-fns or moment and let it decide what
| the fourth Sunday of a given month is in 2046, and also audit
| for the latest browser attack vectors.
| goodpoint wrote:
| > almost by definition, if a library is popular, it contains
| huge amounts of code
|
| popularity != bloat
| ale wrote:
| It almost always means bloat though, because any library
| that's not updated in the span of a year is considered
| "abandoned" and succumbs to feature creep.
| anyonecancode wrote:
| Less "reinventing" the wheel and more "uncovering the wheel."
| efavdb wrote:
| Another: getting good at invention / research is a skill that
| can be honed through practice - and you can practice on
| previously solved problems.
| skinowski wrote:
| +1, also sometimes to avoid complexity due to unnecessary
| abstractions/modularity, etc.
| sroussey wrote:
| Yeah, I built a library to run tasks based on a directed
| a-cyclical graph (DAG) and each task can optionally belong to a
| queue.
|
| So I had to write a simple queue, but since I wanted demos to
| work in the browser it has a IndexedDB backend, and I wanted
| demos it to work in an Electron app, so there is a SQLite
| backend, and I'll likely want a multi-user server based one so
| there is a Postgres backend.
|
| And I wanted to use it for rate limiting, etc, so limiters were
| needed.
|
| And then there is the graph stuff, and the task stuff.
|
| There are a lot of wheels to-create actually, if you don't want
| any dependencies.
|
| I do have a branch that uses TypeBox to make and validate the
| input and output json schemas for the tasks, so may not be
| dependency free for the core eventually.
| osigurdson wrote:
| Don't re-invent the wheel, use AirFlow :)
| Aeolos wrote:
| Not the grandparent, but Airflow is painfully slow and
| inefficient.
|
| Our reinvented wheel using posgresql, rabbitmq and EC2
| runners has ~10x better throughput and scales linearly with
| the number of pending tasks, whereas the airflow falls
| apart and fails to keep the runners fully occupied the
| moment you out any real load on it.
| sshine wrote:
| Reinventing the wheel is the real deal.
| ChrisMarshallNY wrote:
| That's the main reason that I tend to "Reinvent the wheel."
|
| Also, the dependencies often have a lot of extra "baggage," and
| I may only want a tiny bit of the functionality. Why should I
| use an 18-wheeler, when all I want to do, is drive to the
| corner store?
|
| Also, and this is really all on me (it tends to be a bit of a
| minority stance, but it's mine), I tend to distrust opaque
| code.
|
| If I do use a dependency, it's usually something that I could
| write, myself, if I wanted to devote the time, and something
| that I can audit, before integrating it.
|
| I won't use opaque executables, unless I pay for it. If it's
| no-money-cost, I expect to be able to see the source.
| fnord77 wrote:
| I was on a code review where someone imported an entire library
| just to use the library's Pair<> class, which Java does not
| have.
|
| But it is literally a one liner to declare a Pair<> type in
| java
|
| ``` record Pair<S, T>(S first, T second) {} ```
| scotty79 wrote:
| Custom solutions while initially potentially less complex
| gradually grow in complexity. There might be a time when you
| it's worth it to throw out your custom solution and replace it
| with more general dependency. There's still a benefit because
| dependency introduced at this stage is used way more
| thoughtfully because you know the problem it solves inside out.
|
| It might also change your psychological relationship with the
| dependecy. Instead of being disugsted by yet another external
| dependecy bringing poorly understood complexity into your
| project you are thankful that there exists a piece of code
| maintained and tested by someone else that does the thing you
| know you need done and lets you remove whole mess of complexity
| you yourself constructed.
| marginalia_nu wrote:
| An underrated middle ground, at least when it comes to open
| source, is vendoring the dependency, cutting out the stuff you
| don't need, and adapting the API so that it's not introducing
| more complexity than it has to.
|
| This is also generally helpful when you have performance
| requirements, as often 3rd party code even when optimized in
| general, isn't very well optimized for any particular use case.
| vendiddy wrote:
| I really like doing this.
|
| I find that in many cases you can cut out 80 percent of the
| code of the original library.
|
| Most of the deleted code is flexibility and features we don't
| need.
|
| It's surprising how small the essence of a solution can be.
| collyw wrote:
| I'll agree with this, though in a lot of cases reinventing the
| wheel is a bad idea. A previous coworker insisted om writing
| everything instead of using libraries so I had to maintains a
| crap undocumented buggy version of what was available in a
| library.
| turtleyacht wrote:
| > Reinvent for insight. Reuse for impact.
|
| When Primeagen was once interviewed, he built out a whole Java
| architecture; the interviewer asked him, "Have you heard of
| grep?" And that started a journey.
|
| If it were to happen to me, feels like a full circle to go from
| glue and dependencies to pointers and data structures. A welcome
| reverie.
| low_tech_punk wrote:
| I don't like vibe coding in general, but it is surprisingly a
| good tool to reinvent the wheel. Claude Code team allegedly built
| their own markdown rendering with Claude because the existing
| renderers don't meet their needs.
| whstl wrote:
| The fearlessness of AI is indeed refreshing.
| Blikkentrekker wrote:
| Ehh, I always took the "don't reinvent the wheel" advice in
| context of "wheels" being very simple things that everyone can
| create but no one wants to spend time on. It's typically not
| really a learning exercise to say implement quicksort or some
| hash table; it simply takes time.
|
| You will also rarely build a better implementation of these
| things than whatever is in the standard library or even some
| other library that already exists. If anything, it's better to,
| if one have a better idea, to contribute one's patches there.
| sfpotter wrote:
| It actually isn't that hard to come up with something better
| than what's in a standard library, but that requires
| understanding what better means for what you're working on.
| That's the hard part. In my experience, people who say "don't
| reinvent the wheel" are also the people who have the poorest
| understanding of requirements: what "better" means.
|
| A standard library data strucute or algorithm has to be
| something for everyone, so it can't be truly great at a
| specific thing. If you understand your specific use case
| extremely well it (and are competent...) it can be very easy to
| run circles around the standard library.
| graypegg wrote:
| I mean, the phrase isn't "Don't reinvent A wheel". I do feel like
| it's good advice as-is! To me, "reinvent" implies starting from
| nothing (inventing), again: If you can learn from why THE wheel
| is the way it is, you can make A better wheel. Yes it's good for
| learning, but it's also good advice for focusing on building
| things as a participant in our own shared history of invention,
| not trying to be the main character.
| Ekaros wrote:
| Go ahead build your own wheel. But often it is better just to use
| one already made.
|
| There is lot of complexity that mature wheels have taken into
| account or have had to solve and you are likely miss lot of it.
| Not that building your own does not help you to understand it.
|
| Still, I wouldn't replace wheels on my car with ones I made
| myself from scratch... Just like I wouldn't replace reasonable
| complex library.
| imiric wrote:
| Precisely.
|
| While sometimes reinventing the wheel is a useful exercise, as
| TFA lays out, this is often a symptom of a larger Not Invented
| Here mentality. This is generally a harmful tendency in some
| organizations that leads to derailing project deadlines, and
| misdirecting resources towards building and maintaining
| software that is not core to the company's mission.
|
| So in most cases the advice to not reinvent the wheel is more
| helpful. Deciding to ignore it, especially within a corporate
| environment, should be backed by very good reasons.
| basket_horse wrote:
| Sure, making a new wheel is fine if you're never actually going
| to use it. But if you're actually being serious, remember that
| you'll have to maintain it.
| jspdown wrote:
| There's no free meal and adding a dependency is far from being
| free. Each dependency you add needs to be carefully reviewed,
| each of its update as well. Though, apparently many people just
| YOLO this part.
| basket_horse wrote:
| I don't disagree, and it really depends on the complexity of
| what you are trying to do. If it's a simple util function, it
| makes total sense. But for complicated solutions where open
| source alternatives already exist, its a hard argument to
| spend your time reinventing it unless just for learning
| purposes.
| zzo38computer wrote:
| > But for complicated solutions where open source
| alternatives already exist, its a hard argument to spend
| your time reinventing it unless just for learning purposes.
|
| Even if the open source alternatives already exist, does
| not necessarily mean that they do what you want them to do.
| In some cases they can be fixed to do what you need (since
| it is open source, that is an advantage), but sometimes it
| cannot really be done without rewriting it and making a new
| one.
| jspdown wrote:
| Yes exactly, each situation is different.
|
| As a team you constantly need to assess whether a
| dependency should be brought in or of a re-implementation
| is not better.
|
| Sometimes, the re-implementation is so specific to you
| actual problem that maintenance becomes almost free. A
| generic solution always open more doors to problems and
| require more effort to maintain.
|
| For a rich client web application, you need a really good
| reason not to bring an external dependency such as React.
| whstl wrote:
| If "this will be properly maintained" was the bar for
| dependencies, hello world React projects would have a double-
| digits number of dependencies instead of 1484 [1].
|
| [1] https://news.ycombinator.com/item?id=39019001
| basket_horse wrote:
| case and point, "hello world React projects" are never
| actually used by anyone
| whstl wrote:
| 168,483 weekly downloads say otherwise.
|
| Plus, most professionally programmed React apps from the
| last 10 years were created like this. Vite only replaced
| CRA as the recommended tool quite recently.
| AlienRobot wrote:
| I think I got a deprecation warning for
| https://www.npmjs.com/package/boolean while installing React
| today.
|
| >3 million weekly downloads
|
| Dear God.
| megadragon9 wrote:
| Thanks for this inspiring essay, I couldn't agree more that
| "reinventing for insight" is one of the best ways to learn. I had
| a similar experience couple months ago when I built an entire
| PyTorch-style machine learning library [1] from scratch, using
| nothing but Python and NumPy. I started with a tiny autograd
| engine, then gradually created layer modules, optimizers, data
| loaders etc... I simply wanted to learn machine learning from
| first principles. Along the way I attempted to reproduce
| classical convnets [2] all the way to a toy GPT-2 [3] using the
| library I built. It definitely helped me understand how machine
| learning worked underneath the hood without all the fancy
| abstractions that PyTorch/TensorFlow provides. Kinda like
| reinventing the car using the wheel I reinvented :)
|
| [1] https://github.com/workofart/ml-by-hand
|
| [2] https://github.com/workofart/ml-by-
| hand/blob/main/examples/c...
|
| [3] https://github.com/workofart/ml-by-
| hand/blob/main/examples/g...
| richardjennings wrote:
| I very much agree with the Author. "Do not reinvent the wheel" is
| unfortunate in that it muddles several things into one piece of
| subjective advice.
|
| The advice as I would give it is:
|
| "Try to re-invent the things you are interested in".
|
| "Do not underestimate the value of the continued interaction
| between reality and established solutions."
| xipho wrote:
| Previously at HN https://news.ycombinator.com/item?id=43434730
| priorityfill wrote:
| I feel the article is missing the point behind the advice.
| Reinventing the wheel so you can learn ? Absolutely ! But be
| objective about what you built, and don't necessarily force your
| crappy square wheel onto your dev team when better wheels exist
| ("better" is subjective, it could be in terms of documentation,
| simplicity, features etc.). This is just like abstractions, they
| exist so as to make complex systems more manageable, but are not
| a substitute for not understanding what's behind them (even
| though in practice, most people don't bother to go deeper).
| whstl wrote:
| _> don 't necessarily force your crappy square wheel onto your
| dev team when better wheels exist_
|
| I would agree about this 15 or 20 years ago, where I saw some
| monstrosities by people who didn't knew that frameworks or ORMs
| existed, but the pendulum has swung too far in the other
| direction today.
|
| The "crappy square wheels" of 2025 are libraries that don't
| fully solve problems and are forced upon others by developers
| who are hostile to any approach that doesn't involve reaching
| out to a 20 github stars library from their package manager.
|
| And this has become extremely difficult to discuss because the
| discussion has also become binary with people treating any
| criticism as a sign of NIH.
| priorityfill wrote:
| I agree. The key term is "don't _necessarily_ force it on
| others " but it doesn't mean there aren't instances where it
| was the right thing to do. It's all about finding the right
| balance. A good rule of thumb could be to seek simple
| solutions and minimize accidental complexity (those crappy
| libraries you pointed out are exactly that). My broader point
| is that in order to assess whether your wheel is worth it,
| you have to be curious and learn about what other wheels are
| available, and how they work. If you just reinvent on your
| own, chances are you will have overlooked important factors
| that others have already considered. When people don't, the
| situation oscillate between two extremes.
| awinter-py wrote:
| engage rationally with build vs buy decisions
|
| accept that there are compatibility boundaries such that it is
| sometimes quicker to create a new X than locate it on the market,
| or that X is too expensive and it's time to pursue vertical
| integration
|
| but teams who can't do build vs buy properly are kind of doomed,
| sentenced to endless cycles of Not Invented Here syndrome which
| block other work.
|
| if you're in a meeting and someone says 'we can't launch our
| website until we develop a platform-native way to host websites'
| you're in the wrong part of the curve
| boricj wrote:
| I've reinvented my own wheel in a particular niche. I didn't set
| out to do that, but I rejected the existing state of the art as
| fundamentally misguided. Then, I attempted to divide-and-conquer
| my particular problem, something that is conventionally
| considered impossible.
|
| Against all odds, I not only succeeded (mostly thanks to
| ignorance and stubbornness), but my wheel turns out to be
| unbelievably good at what it does. Possibly even world-class.
| After further experimentation, it also enables feats that can
| only be described as pure heresy with troubling ease. Time passes
| and some people from that niche start picking up my wheel. They
| all hold it wrong at the beginning because it's so alien, but
| once they get the hang of it they never go back.
|
| I get bug reports and feature requests from all over the world
| for the oddest of use-cases and workflows. I have deep, in-depth
| technical discussions with brilliant people I would've never met
| otherwise. I've witnessed achievements done by others with my
| wheel beyond my wildest dreams. I discover things that keep me
| awake at night. I get kicks out of melting down the brains of my
| uninitiated coworkers and colleagues explaining what my wheel
| does and what I can do with it.
|
| Don't be afraid to reinvent the wheel. You never know what crazy,
| wild path it might roll you down to.
| nssnsjsjsjs wrote:
| I'll bite! What wheel is this?
| Archelaos wrote:
| Probably what he links to on his profile page.
| weaksauce wrote:
| got to imagine it is this https://github.com/boricj/ghidra-
| delinker-extension
| boricj wrote:
| This wheel: https://github.com/boricj/ghidra-delinker-
| extension
|
| It's a Ghidra extension that can export relocatable object
| files from any program selection. In other words, it reverses
| the work done by a linker.
|
| I originally built this as part of a video game decompilation
| project, having rejected the matching decompilation process
| used by the community at large. I still needed a way to
| divide and conquer the problem, which is how I got the funny
| idea of dividing programs. That allows a particular style of
| decompilation project I call _Ship of Theseus_ :
| reimplementing chunks of a program one piece at a time and
| letting the linker stitch everything back together at every
| step, until you've replaced all the original binary code with
| reimplemented source code.
|
| It's an exquisitely deep and complex topic, chock-full of ABI
| tidbits and toolchains shenanigans. There's next to no
| literature on this and it's antithetical to anything one
| might learn in CS 101. The technique itself is as powerful as
| it is esoteric, but I like to think that any reverse-engineer
| can leverage it with my tooling.
|
| In particular, resynthesizing relocations algorithmically is
| one of those problems subject to the Pareto principle, where
| getting 80% of them right is reasonably easy but whittling
| down the last 20% is punishingly hard. Since I refuse to
| manually annotate them, I've had to relentlessly improve my
| analyzers until they get every last corner case right. It's
| by far the most challenging and exacting software engineering
| problem I've ever tackled, one that suffers no hacks or
| shortcuts.
|
| Once I got it working, I then proceeded in the name of
| science to commit countless crimes against computer science
| with it (some of those achievements are documented on my
| blog). Cross-delinking in particular, that is delinking an
| artifact to a different platform that it originates from, is
| particularly mind-bending ; I've had some successes with it,
| but I sadly currently lack the tooling to bring this to its
| logical conclusion: _Mad Max_ , but with program bits instead
| of car parts.
|
| Ironically, most of my users are using it for matching
| decompilation projects: they delink object files from an
| artifact, then typically launch objdiff and try to create a
| source file that, when compiled, generates an object file
| that is equivalent to the one they ripped out of the
| artifact. I did not expect that to happen at all since I've
| built this tool to specifically not do this, but I guess when
| everything's a nail, people will manage to wield anything as
| a hammer.
| elorant wrote:
| In my almost 15 years in this community this is the first
| comment where I don't have a fucking clue of what is
| described.
| sureglymop wrote:
| When you write and compile code, there is a phase called
| "linking". For example, one could compile pieces of code
| and then link them together into one binary.
|
| A common challenge is decompiling and reverse engineering
| a compiled program, e.g. a game. The author realized that
| it's an interesting approach to do a sort of reverse-
| linking (or I guess unlinking) process of a program into
| pieces and then focusing on reverse engineering or
| reimplementing those pieces instead of the whole.
|
| When it comes to decompiling games, enthusiasts in that
| hobby want to be able to reproduce/obtain source code
| that compiles to exactly the same instructions as the
| original game. I think there is usually also a
| differentiation between instruction matching and byte for
| byte matching reproduction. But it definitely seems like
| a simpler and better approach to do this piece by piece
| and be able to test with each new piece of progress.
|
| That's my layman understanding of it having only dabbled
| in decompiling stuff.
| boricj wrote:
| This is pretty much spot on.
|
| The key insight of delinking is that object files are
| relocatable and the process of linking them together
| (laying their sections in memory, computing all the
| symbol addresses, applying all the relocations) removes
| that property. But by reversing this process (creating a
| symbol table, unapplying the relocations/resynthesizing
| relocation tables, slicing new sections), code and data
| can be made relocatable again.
|
| Since properly delinked object files are relocatable, the
| linker can process them and stitch everything back
| together seamlessly, even if the pieces are moved around
| or no longer fit where they used to be (this makes for a
| particularly powerful form of binary patching, as
| constraints coming from the original program's memory map
| no longer apply). Alternatively, they can be fed to
| anything that can process object files, like a
| disassembler for example.
|
| Of course, the real fun begins when you start reusing
| delinked object files to make new programs. I like to say
| that it breaks down the linear flow of toolchain from
| compilation to assembly to linking into a big ball of
| wibbly-wobbly, bitsy wimey stuff. Especially if you start
| cross-delinking to a different platform than the original
| pieces of the program came from.
| krackers wrote:
| I think this finally answers a question I've had since I
| started learning programming. All my life I've been told
| "you can't convert a shared library to a static library".
| When you try to probe for more details, all you get is
| some mumbling about how a shared library is already
| prelinked for mapping into memory so the relocation table
| is lost. When you press further about the precise
| information loss that prevents you from reconstructing
| it, you get the equivalent of "you just can't ok!".
|
| That was never satisfying to me because recompilation
| (decompiling to source code or IR that can be recompiled
| for another platform) is an academic field of study [1],
| so the problem of converting a shared library to a static
| one should be easier. After all, you have all the
| assembly and symbol information right there, it seemed
| like all you needed to do was massage it the right way.
|
| [1] https://rev.ng/
| mjanx123 wrote:
| You flipped the static and shared.
|
| Shared objects are compiled in a way that the exact same
| code works when loaded at any address, which typically
| requires base+offset addressing and is less performant,
| but the same physical pages holding the object can be
| mapped into different processes at different addresses,
| ie shared.
|
| Static object addresses the code and and data directly
| and does not waste a register and requires the addresses
| in its code to be modified depending on where it is
| loaded, and the code cannot be shared between different
| processes.
|
| You can't convert a static library to shared because you
| can't.
|
| You can convert a shared library to static but there is
| no point.
| krackers wrote:
| I'm not sure I follow why 'I flipped them'.
|
| My understanding a shared library is closer to an
| executable than it is to an object file, so it is in that
| sense that it is "prelinked and ready to map into memory"
| just as an executable is. (Once loaded dyld still has to
| do fixups in the GOT, but that's not touching the text
| segment.)
|
| I mostly agree with your definitions, but you _can_
| create a shared library out of a static library assuming
| the static library was compiled with fPIC, no? I mean
| I've never tried it but most linkers seem to allow it
| [1], and given that creating a full executable is similar
| to created a shared library, I don't see why it wouldn't
| work.
|
| And if the code wasn't compiled with fPIC, I still feel
| that it should be possible in theory since the whole
| point of object files is that they support relocation.
| Linking a non-PIC object to produce a shared object would
| just require patches within the text segment at runtime
| right? Modern linkers probably don't like doing this for
| security reasons. But otherwise seems isn't some
| fundamental limitation, just modern OSs and linkers not
| supporting an obscure use-case with security risk.
|
| When people say that you can't convert a shared library
| to a static one (or equivalently statically link against
| a shared library), they usually cite the lack of
| relocation table info as the reason. Like they have some
| hierarchy in mind that goes from source code -> object
| file -> executable/shared library, and so they say it's
| "obvious" that you can't go the other way. Which is
| mostly true in terms of the "golden path" and what you'd
| want to limit yourself to for production, but in a hacker
| sense you can obviously go the other way with assemblers
| and decompilers.
|
| So if someone claims that it's impossible to convert a
| shared library into a static library, there better be
| good justification for such a hard claim. And on the face
| of it, the claim doesn't seem watertight because the
| _stronger_ claim that you can't convert an executable
| back into recompilable code is falsified by the existence
| of decompilers that lower down to LLVM IR.
|
| [1] https://stackoverflow.com/questions/655163/convert-a-
| static-...
| michalsustr wrote:
| I think it's because of https://en.wikipedia.org/wiki/Add
| ress_space_layout_randomiza...
|
| My interpretation is it's basically a hack because C
| programmers can't make their libs reliable/memory safe,
| and it throws some logs under attacker's feet.
|
| There was this software called MagicErmine or statifier
| that could convert the two.
| krackers wrote:
| I don't see what ASLR has to do with the inability to
| convert a shared to static library though. In fact a
| shared library must be position independent so if
| anything it would make the job easier. For going from
| static (non-PIC) to shared, W^X is probably why modern
| dynamic linkers don't support patching text segment.
|
| Tools like Statifier to convert an executable + its
| dylibs into a statically linked executable are sort of an
| in-between hack that does the equivalent of prelinking.
| But that's not converting a shared library to a static
| library, because the result isn't an object file that can
| be linked.
|
| I guess I never stated the punchline explicitly, but
| boricj's tool seems to prove that there is in fact no
| such theoretical limitation in converting from shared ->
| static library. It's a bit hacky of course, but you can
| indeed reconstruct relocation tables with decompiler-like
| analysis to go back from executable -> object file for
| any function, and this also implies you can go from
| shared library -> static library.
| boricj wrote:
| > I guess I never stated the punchline explicitly, but
| boricj's tool seems to prove that there is in fact no
| such theoretical limitation in converting from shared ->
| static library. It's a bit hacky of course, but you can
| indeed reconstruct relocation tables with decompiler-like
| analysis to go back from executable -> object file for
| any function, and this also implies you can go from
| shared library -> static library.
|
| As far as the traditional linker goes, sure. This whole
| shtick relies on the fact that a platform has ABI
| conventions, so with a naive toolchain any translation
| unit (and function) compiled in isolation must be
| interoperable as-is, because the linker is completely
| oblivious to whatever's going on within an object file
| section (relocations excluded). Even then, platform-
| specific considerations might get in the way.
|
| For artifacts that went through inter-procedural and
| especially link-time optimization, this breaks down. The
| former will take advantage of the fact that anything goes
| as long as the visible interface of an object file
| section is ABI compliant, the latter effectively turns an
| entire program into one big translation unit. Among other
| things, functions may exhibit non-standard calling
| conventions, which wreaks havoc as soon as you start
| trying to meld delinked bits with freshly built ones,
| because the optimizer is unlikely to make the same
| choices in its custom calling conventions.
|
| I have a user who is decompiling a link-time optimized
| artifact built over 15 years ago. In order to graft newly
| compiled code, they binary patched the original compiler
| to basically add support for __usercall, in order to
| coerce the toolchain into making the same optimization
| decisions. If you're up against an artifact built with a
| non-traditional toolchain with optimizations on,
| delinking alone most likely won't cut it and additional
| magic will be required.
| krackers wrote:
| Thank you for chiming in! LTO wouldn't really matter for
| "delinking" exported symbols of shared libraries though,
| would it? The exported functions must necessarily follow
| platform ABI convention, and so long as those are copied
| over, things would seem to work fine.
|
| I guess the one catch with shared libraries is that if
| shared library A itself depends on another dylib B, then
| calls within A to functions of B would go via PLT
| indirection. So then creating an object file out of this
| would involve not just moving code and creating
| relocation entry but also possibly patching the assembly
| itself to remove the call indirection.
| boricj wrote:
| If you're delinking the entire LTO-optimized shared
| library as one big blob, then sure. LTO effectively turns
| a program/library into one huge translation unit, but if
| you're not cutting across it then the platform ABI
| mandates the observable boundary.
| atq2119 wrote:
| I don't think the answers from others in this thread are
| particularly satisfying either, at the time I'm writing
| this.
|
| Thinking about it a little bit, I'd say the major
| challenge for converting from shared to static library is
| that shared libraries have resolved relocations from text
| to other segments.
|
| In order to create a static library, you need to make it
| so that text and data can be moved relative to each other
| again.
|
| At a minimum, this requires disassembling the text
| segment to find PC-relative addresses and recreate
| relocations for them. That doesn't sound impossible, but
| there may be pitfalls I'm not thinking of right now that
| make it hard to find all necessary fix ups.
| immibis wrote:
| Static libraries can't be converted to shared because all
| shared library code on x86 needs to be compiled with the
| -fPIC (position-independent code) flag.
|
| Shared libraries could be converted to static. You would
| lose the ability of static libraries to only include part
| of the library.
| WalterBright wrote:
| Pretty awesome work!
| boricj wrote:
| This comment has made my day in ways words cannot
| express.
|
| But yeah, one recurring problem is that without
| literature on this esoteric and complex topic, any
| discussion remotely technical about it (especially
| without prior context) quickly bogs down, as it needs to
| be bootstrapped from first principles in order to make
| any sense.
|
| I've been meaning to write the Necronomicon of delinking
| to at least be able to point people to a document that
| contains everything they need to know in order to
| understand or at least decently grasp this. At the very
| least it should make for a good horror bed time story for
| linker developers.
| chinchilla2020 wrote:
| I've read some literature on this decompiling and reverse
| engineering work. It always blows my mind that there is a
| community of experts in it, toiling in science. Yet the
| academic and learning communities do not know anything
| about it. It's like an underground CS topic.
| spooneybarger wrote:
| This is insanely cool.
| abetusk wrote:
| Was this really a re-invention? Isn't this more akin to
| _invention_?
| boricj wrote:
| I did not actually invent delinking, there was some scant
| evidence of prior art online before I did my stuff. I did
| however come up with it independently from first
| principles.
|
| As for the reinvention part, I was referencing the wheel
| of video game decompilation specifically. By that point
| Super Mario 64 was already fully decompiled and several
| other projects were well underway. The established wheel
| was matching decompilation (either instruction or byte
| level) and the community was producing tooling for this
| sole purpose.
|
| In a sense, I did reinvent the wheel of video game
| decompilation in stark contrast to everything else that
| was out there at the time. It just turned out _horribly_
| right, because before I knew it I was making chimeras
| executables made out of bits of programs coming from
| different platforms.
| geoka9 wrote:
| > made out of bits of programs coming from different
| platforms
|
| By platforms you mean different binary formats, right?
| Not CPUs (just making sure I'm not going crazy here :))
| boricj wrote:
| Yup. I've shoved bits of a PlayStation game inside a
| Linux MIPS program and I've made a native port of a Linux
| x86 program to Windows, all without source code,
| decompiling anything or even fully reverse-engineering
| said bits. The toolchains happened to agree on calling
| conventions and structure padding/alignment, so it was
| mostly a matter of writing shims to glue everything
| together.
|
| I don't have the tooling to meld together incompatible
| ABIs. I tried once with two really similar ABIs (MIPS PIC
| and non-PIC) and while I've managed to work around
| incompatibilities with assembly trampolines, the
| debugging experience was so poor that I rage quit. Never
| tried with different CPUs.
|
| I'm crazy enough to think it ought to be possible
| (there's plenty of prior art for most of the needed
| pieces), but I'm already stuck inside a rabbit hole so
| deep I can't see the bottom... I'll let someone else
| investigate that.
| babuloseo wrote:
| Someone did something similar for the Davinci Resolve
| plugin, I think they have the crown for best VR addon atm.
| nyanpasu64 wrote:
| What's "wrong" with matching decompilation as an oracle for
| provably correct source code for a given binary?
| boricj wrote:
| I just don't want to deal with that. Identifying the
| right toolchain, finding the right options, rewriting
| code until the compiler is tricked into emitting the
| right sequence of instructions... I'm not saying that
| there's no value in a matching decompilation, but it's
| the kind of work I find thoroughly unappealing.
|
| Also, my decompilation project's goal was a remaster in
| the spirit of REDRIVER2, with bugfixes, new features,
| quality of life improvements and a PC port. The end
| result would not have been matching or even functionally
| equivalent anyways.
| ShroudedNight wrote:
| This is highly reminiscent of work I was exposed to in grad
| school on recombining library versions from different
| binary distributions in an attempt to diversify the attack
| surface of a system, but at a much more granular level:
|
| https://people.scs.carleton.ca/~soma/pubs/bfoster-
| gecco-2010...
| mjanx123 wrote:
| What you do is still a matching decompilation. The inserted
| step of delinking does not change that.
|
| It does have the potential to simplify and improve the
| decompilation over directly decompiling a linked binary, as
| it more naturally reverses the process.
| bigfatkitten wrote:
| Holy shit. This happens to be a thing I have an immediate
| use for. Thank you.
| dhash wrote:
| _fantastic_ -- I'm sure folks who have poked binaries or
| written patches have all had the desire for something like
| this. Glad to see that it exists!
|
| I'm definitely going to use this for solving some problems
| that i'm currently facing.
|
| For anyone curious about what hare-brained scheme that one
| could hatch with this, i'd like the ability to do something
| like a PGO'd shared library --- watch the process run and
| exit, tracing all dlopen's to create a new shared library
| with only the right functions from all the referenced
| libraries.
|
| Hopefully this works, and if not, i'll at least fail at
| something interesting :)
| immibis wrote:
| > Since I refuse to manually annotate them, I've had to
| relentlessly improve my analyzers until they get every last
| corner case right.
|
| I've fallen into this trap enough times to recognize it as
| a trap now. Good for you that it works for you, but in
| general, automation of loosely-defined problems just
| doesn't get 100% of cases right and once it works, you need
| to streamline the manual process instead.
| boricj wrote:
| The context of this is quite peculiar.
|
| Given the amount of information loss that occurs at the
| linking stage, delinking is an extremely intricate puzzle
| to solve. It's also very hard to be sure that you got it
| right, because there are many, _many_ ways it can go
| wrong in ways that are a nightmare to troubleshoot. The
| scale of the endeavor can be also staggering: one user
| has over 250 000 relocation spots in their artifact,
| pretty much all of them must be spot-on otherwise
| glitches and crashes will occur.
|
| The only way I've managed to pull it off is with an
| analyzer design that makes no assumptions, double-checks
| everything, logs anything that couldn't be successfully
| processed and heavily investing into regression testing.
| The manual process then consists of fixing inaccuracies
| within the Ghidra database, or (rarely nowadays) fix an
| unhandled corner case within the analyzer. Also, no
| manual annotations of relocations means they cannot go
| stale as the Ghidra database is modified.
|
| I also happened to attempt this first on possibly the
| worst architecture possible, MIPS. I won't go into the
| details, but split HI16/LO16 relocations, register
| dependency graphs and branch delay slots can produce some
| absolutely gnarly cases to untangle. Let's just say that
| my MIPS analyzer contains among other things one
| recursive function that takes 6 parameters, 4 of them
| shifts one place to the left at each iteration. I'm not
| exactly sure how it works.
| tough wrote:
| Thank you
| OtomotO wrote:
| These discussions (in the comments) tend to become this "to write
| everything yourself or not" discussions which are extreme in
| nature and very binary. While reality is way more nuanced.
|
| Why do you add a dependency? Because you need a certain
| functionality.
|
| The alternative to adding this dependency is to write the code
| yourself.
|
| That is totally feasible for a lot of dependencies.
|
| That is totally infeasible for a lot of dependencies.
|
| It's a trade off, as always. The fact is, that most of us, who do
| this for a living, need to ensure that our software runs and
| continues to run and may be adapted in a timely manner, when new
| requests come in from the client/owner.
|
| Using dependencies or not using depdencencies isn't gonna change
| that.
|
| Now, granted, some ecosystems are a bit extreme on the "let's add
| a dependency for this 1 line of code."
|
| On the other hand: should I really roll my own crypto? (Jk)
|
| Should I really write those 5000 lines of well tested code
| myself, just because I can? And because MAYBE it's not touched in
| a month from now?
|
| Every (later executed) line I add to the project, be it one
| written by myself or an external dependency, becomes part of the
| code. So I have to be able to maintain it. Might be easier if I
| write it myself. Might be way more difficult and time consuming
| if I write it myself...
|
| I have so many tasks to do and while I enjoy coding, I have to
| make my top priority a working system.
|
| So should I mindlessly add dependencies? Of course not!
|
| Should I just reinvent the whole world? Of course not! (Unless
| the client absolutely wants me to. And pays me to do it)
| incognito124 wrote:
| closely related: https://news.ycombinator.com/item?id=43434730
| random3 wrote:
| And the even more general advice: "First, break all the rules"
| (not (necessarily) the book).
| sfpotter wrote:
| So many bitter and jaded people in this thread.
| tomhow wrote:
| _Please don 't fulminate. Please don't sneer, including at the
| rest of the community._
|
| https://news.ycombinator.com/newsguidelines.html
| sfpotter wrote:
| Said with sadness, not sneeringly.
| tomhow wrote:
| Can you link to any of the comments that make you feel this
| way? I don't see it.
| semiinfinitely wrote:
| People advise not to "reinvent the wheel" because doing so imbues
| you with the ability to create new things yourself- increasing
| your agency and power thereby relatively decreasing theirs. Also
| it often comes at some expense to them if they have some claim to
| your time eg boss/coworker.
| isaacremuant wrote:
| I love how you selfishly think about your agency as if there
| was no team involved dealing with your "snowflake creation".
|
| When I give this advice it usually means I don't think the
| output is better than the existing thing and the dependency
| cost is better paid in the form of integration.I probably don't
| think you'll really maintain your creation or think about
| others using it when you do.
|
| As long as we are throwing shitty incentives around.
|
| But on a more neutral note, it's a tradeoff with many moving
| parts. Different choices for different scenarios.
| whstl wrote:
| If it were my team complaining about "wheel reinvention" or
| "NIH" I would be ok, but I never saw this kind of gatekeeping
| at work.
|
| It's always internet strangers who are too dogmatic even
| though there's zero context.
| hn_go_brrrrr wrote:
| I've encountered _plenty_ of stubborn dogmatism at work.
| People get an idea about "the right way" to do something,
| and refused to consider alternatives.
| EasyMark wrote:
| I've never encountered this person. Most people who say it,
| simply say it to save time and effort. I'm working on a product
| that allows my customer to do X job. I don't need to reinvent a
| json parsing library or sqlite-like database to get his site
| online and running his service, and he doesn't care that I
| didn't write those libraries. I don't feel guilty for skipping
| starting from zero. If I'm interested in a subject or algorithm
| then it makes complete sense to reinvent the wheel for my own
| personal advancement, knowledge, or amusement.
| mullingitover wrote:
| If you're working in a startup I hope you'll completely ignore
| this advice, unless the particular wheel you're reinventing is
| core to the product/service your startup makes. If it's not,
| you're likely just setting your runway on fire and crashing the
| plane before takeoff.
| epolanski wrote:
| You still want to build a startup with people that know how to
| build wheels, and thus have done it professionally, in oss or
| personal projects.
| bawis wrote:
| I guess it's more like an advice for personal wheels, not
| professional ones.
| lelanthran wrote:
| If you're in a startup and you aren't reinventing some
| particular wheel, where exactly is your value proposition?
| mullingitover wrote:
| That's why I added the clause:
|
| > unless the particular wheel you're reinventing is core to
| the product/service your startup makes
|
| If it's core to your product/service, absolutely reinvent the
| hell out of that wheel. That's your business. Put pneumatic
| spokes on it. Add a touchscreen. Embed a bespoke LLM. Go
| nuts.
|
| Just stay focused on your customers. Don't do a bunch of
| things that are frankly just resume builders for your senior
| engineers and don't add value for your customers.
| yxhuvud wrote:
| You are complaining about a symptom of not having enough
| experienced people onboard. Juniors will make a mess,
| regardless of if they try to use something existing or reinvent
| the wheel. Knowing when and what to reinvent is just a function
| of having worked at more places and for long enough to see what
| is good and bad with the existing solutions.
| collyw wrote:
| I have had to maintain rewritten libraries by seniors.
| Crappy, buggy and undocumented with no tests.
| vendiddy wrote:
| I don't find this to be absolutely true.
|
| At our company we reinvented a handful of wheels because the
| existing solutions were too complex.
|
| And complexity slows you down.
|
| It's not this clear cut.
| Waterluvian wrote:
| 90% of the time it's wrong to "reinvent the wheel." If you chose
| this option dogmatically, you'd be making the right choice 90% of
| the time. But this also represents a below average outcome as a
| one-sided die would roll this choice right 90% of the time, too.
|
| Your job as the decision-making engineer is to develop the
| expertise to be able to make the right choice more than 95% of
| the time.
| almosthere wrote:
| People that work at Radial, Michelin, Good Year, etc... probably
| are sick of hearing it.
| ppqqrr wrote:
| the obsessive narrative against redundancy has been the blight of
| our times. people are copies of each other, they eat the same
| food, work the same job, have the same needs. if you follow the
| Big Wheel propaganda to its logical conclusion, you'll have as
| your ideal a very specialized person who serves a very specific
| needs of some (probably) rich people, but knows or enjoys
| absolutely nothing else: never cooks, grows, loves anything,
| because those things are redundant.
| bigyabai wrote:
| "Don't reinvent the wheel" doesn't necessarily imply you
| shouldn't question anything, or even that you always have to
| take the path of least resistance. If your car tire pops, you
| _should_ consider all of your wheel options. You just shouldn
| 't reinvent the wheel; they give you a spare for a reason, and
| anything you can design or manufacture is going to be worse
| than the one from the factory.
| whstl wrote:
| The problem is that I work at a wheel design factory and get
| paid for it, unlike the hobbyists who make 95% of the wheels
| on Github.
|
| I'm all for reusing frameworks, standard libraries, crypto.
| Sure, those are things we don't want to recklessly reinvent.
|
| But there's a limit. There should be way more pushback
| against things like this:
| https://news.ycombinator.com/item?id=39019001
| kayodelycaon wrote:
| You need to know when not to reinvent the wheel.
|
| I've certainly done it at work because I didn't have time (or
| desire) to learn a library.
|
| But sometimes you have to understand the wheel to reinvent it.
|
| I'm not gonna let a someone roll their own user authentication
| and authorization system in a rails application.
|
| There's so many moving pieces you won't even think about and
| there's gems that solve this extremely well. Study the gem and
| you will learn a lot more. Then reinvent the wheel.
| dustingetz wrote:
| Time Is The Denominator,
|
| _especially_ in an industry context where you are being paid
| (there's that pesky denominator again). R &D, which is what this
| is, is a very expensive and risky endeavor. 99% of startup
| projects fail, for example. They would _not_ fail given infinite
| time, but you cant have that because time is not free:
|
| Interest Is The Price Of Time
| iamwil wrote:
| "Sometimes people reinvent the wheel not to get better wheels,
| but to get better inventors."
| gfalcao wrote:
| Thanks for sharing that article.
|
| A great friend of mine once told me the following quote from an
| unknown author: "Reinvent the wheel, not because we need more
| wheels but because we need more inventors." That quote has
| brought my mind and heart to some modicum of tranquility at
| various occasions when I wanted to learn some concept and
| resolved to write my own "toy version" of a software library,
| framework et cetera. Later on, when I learned about Feynman's
| quote "What I cannot create, I do not understand", amalgamated
| the sentiment that it is okay to build something in order to
| learn one concept. I have thus far learned that in every new
| journey to reinvent the wheel, so to speak, often led me to paths
| where my intuitions about the initial concept got stronger and
| beyond that, I learned several other concepts.
| artursapek wrote:
| This is so relevant to what I'm working on right now. Happy to
| see someone with this sentiment
| 65 wrote:
| When you rephrase it to "Make your own wheel" it sounds a lot
| more empowering.
|
| Many of my side projects are just my own version of things that
| are specifically tailored to my preferences. I have a notes web
| app that backs up to S3. I have my own music streaming service. I
| have my own music discovery algorithm.
|
| Sure these things exist, but because I made them, they do exactly
| what I want them to do. There is no one "wheel" - there are
| infinite permutations of what a "wheel" can be.
| brightball wrote:
| That's how you get better wheels
| cryptonector wrote:
| Yes, reinvent away, but _first_ learn about what came before.
| gblargg wrote:
| I think you need to invent it first, so you have no pre-
| existing ideas. Then you get the "doing it my way" out of your
| system and come to understand the challenges. Then when you see
| how it's done, you can see the wisdom of experience. Where it
| seems like it's poorly done, if you study you can see benefits
| to that.
|
| A huge benefit to not re-inventing is that standardization is a
| big value. Compare auto tires to phone batteries (back when
| they were replaceable). Auto tires are standardizes whereas
| every phone used its own slightly differently shaped battery.
| dclowd9901 wrote:
| > It usually comes from a good place, but is typically given by
| two groups of people:
|
| > those who tried to invent a wheel themselves and know how hard
| it is
|
| > those who never tried to invent a wheel and blindly follow the
| advice
|
| There's a third, and I think more common group: folks who know
| all that's involved with reinventing the wheel, and how to do it,
| and know the juice of doing it isn't worth the squeeze, and
| there's no value in doing it yourself, educational or otherwise.
| MetaMalone wrote:
| This group is much more tangible for me. Instead of trusting
| the status quo, this group fears it.
| 59nadir wrote:
| > There's a third, and I think more common group: folks who
| know all that's involved with reinventing the wheel, and how to
| do it, and know the juice of doing it isn't worth the squeeze,
| and there's no value in doing it yourself, educational or
| otherwise.
|
| I think this view of things is usually related to the fact that
| large parts of this group are simply too bad at reinventing
| wheels and in large part are gun-shy because of previously
| failed attempts at doing a good job.
|
| One of my former leads had worked for 7+ years in the same
| company by the time I showed up and he was absolutely useless
| as a programmer beyond just gluing together libraries because
| that's all he had ever done. He barely also knew the platform
| we ran on because he simply had developed no skill when it came
| to understanding things.
|
| For him and people like him reinventing wheels is simply not
| worth the time and since they're so bad at it and always fail
| they'll never actually learn anything. I heard from him last
| year (7 years after we both left the company we worked together
| at) and he still only glues libraries together, but he does it
| in Rust instead of Erlang nowadays.
| BenoitEssiambre wrote:
| I feel this article misses the main reason (unless it falls under
| "Build a better wheel") which is to build a wheel that is
| tailored and stays tailored to your purposes.
|
| How often do I see people metaphorically trying to use a car tire
| on bicycle with thee excuse of not re-inventing the wheel. There
| can be great benefits for the parts of your system to be tailor
| made to work together.
| drpixie wrote:
| Absolutely - there are so many really bad wheels out there:
|
| - wheels that force you to do things that way the maker wants,
| instead of the way that suits you;
|
| - wheels that are fine going downhill ... but nowhere else;
|
| - wheels require you to attach an ever more expensive widget;
|
| - wheels that promise but never quite deliver;
|
| - wheels that keep pushing you into services that you just know
| are going to be a pain.
|
| Often your own wheel is the best.
| caseyohara wrote:
| One that I run into most often:
|
| - A Formula 1 wheel when all you need is a bicycle wheel, but
| the person in charge of choosing wheels chooses it on the
| basis of "if we want to be as good as Formula 1, then we need
| to use the same wheels as Formula 1"
| skydhash wrote:
| It's worse when they bring parts of the car still attached
| to that wheel and wants you to fit all of it to the
| bicycle.
| KaiserPro wrote:
| Re-invent the wheel all you like, just don't try that shit in
| production, unless you want to cause a delay.
|
| Sometimes there are very good reasons to reinvent the wheel,
| where the buisness demands a faster x, or there are privacy
| concerns for y. But most of the time its just a bad idea to re-
| invent the wheel. Do that shit on your own time, and keep it out
| of our production codebase.
|
| I know there will be lots of people moaning about curiosity and
| the like. I somewhat agree. However imagine you are building a
| new table. Nothing interesting just metal frame with a composite
| wooden top. You are prototyping is, and suddenly, one of your
| engineers decides to reinvent screws. Not because there is a
| need, but because they are bored.
|
| Its not like you need hidden, or fancy screws/attachement
| devices, the engineer is just curious about trying out a lathe.
| Not that you have an "automatic" lathe, so it not like you can
| scale up production.
|
| Its fucking stupid right? yes.
|
| Sure, test out how to make your own fasteners. Even do it on
| company time. Just don't try and sneak it into production. Its a
| bad idea.
| pcthrowaway wrote:
| > Q: Will you mainline it?
|
| > A: While I am a strong believer in the future of the rotary
| dial as an input device, Linus Torvalds might disagree
| arionhardison wrote:
| I'm recreating the "Gov" - not really; just my idea of it,
| globally e.g.: ua.gov-ai.co / ua.ai-gov.co/ ng.gov-ai.co / ng.ai-
| gov.co - most progress made so far w/ CBER and DDP's
|
| * HHS -> FDA -> CBER
|
| It's important IMO (IMO only NOT AN EXPERT) because it helps you
| understand first principles better. As fundamentals change, its
| helps me to reevaluate these things even though I know nothing
| will ever come of them.
|
| I am 422 agencies in so far, hoping to finish in-time for
| Juneteenth. Cant post her because........... but yea.
|
| Re-invent the wheel!
| mediumsmart wrote:
| I created a bike with one html and one css wheel to cycle through
| the wheel reinvention factories in my area and I can confirm that
| rabbit holes are a thing.
| tough wrote:
| Im reinventing wheels daily
| hk1337 wrote:
| I wouldn't say it's harmful advice but it is often over-
| used/abused to continue working on a lopsided wheel because, "if
| it ain't broke..."
| fHr wrote:
| cries in working with osgi garbage in 2025, I don't want to
| reinvent the wheel on that one I just hope that shit dies
| Natsu wrote:
| > In Computer Science, for example, there are many concepts that
| are commonly assumed to be beyond the abilities of mere mortals:
| protocols, cryptography, and web servers come to mind.
|
| This baffles me, because a basic HTTP/1.0 server is about 20
| lines of printing certain human-readable text over a socket in
| Perl, and I'm not cheating by just importing a library that does
| all the things.
|
| I know because I have a www.pl script sitting around that I've
| frequently using for testing and such. I'm sure it violates
| various parts of the RFCs in different ways, but... sometimes
| that's what I'm testing anyway, and when I do want it to work, it
| simply prints file contents over a socket with a content-type
| header based on the file extension. It's even smart enough not to
| be vulnerable to directory traversal (i.e. it'll only serve files
| from the content directory), etc.
|
| Sure, that's in some sense cheating by leaving out a lot of hard
| parts from HTTP/1.1 or 2.0, but really, you're just reading a
| string coming over a network socket, responding with another
| string, and then closing the connection with HTTP/1.0. You're
| really just worried about printing out the right thing in
| response to input from the client.
|
| It's not magic and it's not even very complicated. At least
| crypto has a lot of formulas to keep track of, timing attacks to
| worry about, etc. You can have a simple web server that's too
| dumb to be vulnerable to most web attacks with a few lines of
| scripting, most of which involve formatting strings and printing
| them.
| immibis wrote:
| FWIW you're also allowed to close the connection in 1.1, but
| you have to send a header saying you're doing so. You also
| don't have to, as it's not that hard to allow pipelining.
| osigurdson wrote:
| What is the logical overlap between "re-invent" and "improve".
| When people say "don't re-invent the wheel" do they really mean
| "don't improve the wheel"? This seems unlikely to me. Of course,
| the "3000 BCE" retort is a good one and I hope to use it at least
| once in my lifetime.
| 90s_dev wrote:
| These points summarize the principles behind 90s.dev which I've
| been following for years and still stand by. Especially this:
| don't be afraid to try to innovate, to get a new answer to an old
| question, to start over from first principles, even if it fails.
| Because it might not.
| eviks wrote:
| None of these require reinvention
|
| > There are great reasons to reinvent the wheel:
|
| > Build a better wheel (for some definition of better)
|
| You can invent an improvement, no need to reinvent the existing
| wheel
|
| > Learn how wheels are made
|
| Read a book
|
| > Teach others about wheels
|
| Read a book and tell others about it
|
| > Learn about the inventors of wheels
|
| Read a history book
|
| > Be able to change wheels or fix them when they break
|
| Read an engineering manual
|
| etc.
| hahn-kev wrote:
| Reading is not the same as experience gained through doing
| eviks wrote:
| So how would that different experience in any way help you
| learn about the inventors of wheels?
| engeljohnb wrote:
| I would never listen to the advice of someone that said they
| read a book or a manual about [whatever project I'm working on]
| but never actually attempted one themselves.
| eviks wrote:
| That's fine, this is neither about your project nor giving
| advice to you
| PeterStuer wrote:
| We all work on mental models of the things we use, envision or
| encounter.
|
| I've often found that it was not until I tried to build that
| model for real that I had missed significant crucial parts, not
| just a matter of 'incompleteness' or a too high level
| abstraction, but that my model no matter how elegant was just
| wrong or unworkable.
| zmmmmm wrote:
| I think the article subtley misinterprets the whole idea of it
| and the problem of it.
|
| It's fine to invent your own wheel. But that doesn't mean you
| should put it in production or feel any entitlement for anybody
| else to use it there, just because you put personal effort into
| it. It's going to need to be at least as good, if not better, not
| just in whatever novel genius way you made it unique, but in all
| the boring ways - testing, documentation, supportability, etc.
| jonahx wrote:
| This is a good point, but there is a countervailing one:
| Sometimes the already invented, tested, and documented
| "standard" thing is much bigger than what you need. Perhaps you
| need one feature out of ten. It can make sense to re-invent the
| wheel to make exactly what you need, and nothing more, even at
| the expenses you've mentioned. Not always -- you need to weigh
| all the factors for your own situation -- but choosing "re-
| invention" is not _necessarily_ a merely-indulgent exercise of
| ego.
| augusto-moura wrote:
| Sometimes the size doesn't really matter, maybe if you are
| working on a constrained space (embedded software, let's
| say). But if you are doing Web or using any modern language,
| unused code is usually optimized out of the compiled bundle
| and you don't care about it. We care much more about tested
| and battle proven behavior
| tylervigen wrote:
| I learned a fun fact this year that completely changed how I
| think about wheels.
|
| Ancient wheels supported weight through compression: the cart
| pressed down, the wheel transferred that force straight into the
| ground. Simple and solid.
|
| Modern wheels? Totally different. Bicycle wheels hold weight
| through tension. If a bike is standing still, you could cut the
| bottom spokes and it would stay upright. Cut the top ones and it
| collapses--the rim is literally hanging from the hub.
|
| Even car tires follow this principle. It's not just a balloon of
| air under the wheel. The internal pressure forms a tensioned ring
| that holds the car up. The car is suspended from the tire, not
| resting on it.
|
| So while the article encourages reinventing the wheel
| metaphorically, I can't help but admire how we've already
| reinvented the wheel literally--by flipping the physics entirely.
|
| If you're visual like me, this video illustrates it perfectly:
| https://youtu.be/8I7QsgYm_NU?si=Vz1aqpHBAPjayqlG
| thunderbong wrote:
| That was very interesting. I had never thought about this
| earlier. I learned something new today! Thanks.
| astrobe_ wrote:
| Yes, as TFA alludes to, this expression is actually a bit wrong
| because the wheel has often been reinvented since ~4500 BCE.
| The latest "reinventions" are as recent as 19-20th century with
| the continuous track, monowheel, "hubless" wheel and mecanum
| wheel [1] [2].
|
| [1] https://en.wikipedia.org/wiki/Hubless_wheel
|
| [2] https://en.wikipedia.org/wiki/Mecanum_wheel
| moron4hire wrote:
| It seems to me that most people say, "don't reinvent..."
| because they believe there is an existing, ubiquitous,
| sufficient solution (well, that's why they think it. Why they
| say it out loud as a response to someone's project, I have
| different theories on that).
|
| But in ubiquitous solutions, we also find assumptions and
| trade-offs that require constraining the application to fit
| the solution. Thus it is there, in those assumptions and
| trade-offs, as these literal wheel examples demonstrate,
| where we find the ability to usefully reinvent.
|
| I have to laugh a little seeing your reference to Mecanum
| wheels and your post being dated 6 hours ago. 12 hours ago, I
| rewatched the Star Trek reboot film. In it, there is a scene
| where the characters are all loading into shuttles to head
| off to their assignments. In the background, we are supposed
| to be impressed with the technical advancement of the
| machinery in view, e.g. how something as familiar as a
| forklift has "futuristic" details like strange wheels. And
| those wheels on that forklift are Mecanum wheels. In-story,
| that makes them something like ~300 year old technology.
| agumonkey wrote:
| circular tensegrity
| fizlebit wrote:
| People say don't reinvent the wheel usually in a business context
| because writing from scratch is usually a lot more work than
| using existing technologies. Sure reusing technologies is also a
| lot more work than you would expect because most things suck (to
| different degrees), but so will your newly minted wheel. Only
| after a lot of hard lessons will it suck less, if at all.
|
| That said there are also contexts in which the existing system
| that was built sucks so bad that rewriting it usually a boon,
| even if the new wheel sucks, it sucks less from the start.
|
| You at a minimum should engage with the existing wheels and their
| users to find the ways in which they do and don't work.
|
| In your own time I think it is great to tinker, pull apart,
| assemble your own things. Every Jedi makes her own light saber
| right?
| Derbasti wrote:
| Especially at work, I find existing solutions often lacking. We
| tend to overestimate the complexity of reinventing many things,
| and underestimate the cost of ill-fitting abstractions.
|
| In particular, Google-scale frameworks are usually harmfully
| over-engineered for smaller companies. Their challenge is
| solving the Google scale, not the problem at hand. At the same
| time, their complexity implies that a small-scale solution will
| likewise require a hundred-programmer team to implement.
|
| But I find that far from the truth. In fact, many problems
| benefit from a purpose-built, small-scale solution. I've seen
| this work out many times, and result in much simpler, easier-
| to-debug code. Google-scale frameworks are a very bad proxy for
| estimating the complexity of a task.
| pzo wrote:
| This. Reinventing the wheel in business context IMOH should be
| only if:
|
| - you can afford it (money)
|
| - you have enough time or your customer can wait
|
| - provide significant benefit to you or your customers (have
| nothing better to do)
|
| - you don't have fierce competition in your domain
|
| - you are pretty much retired (or could) and treating it more
| like art/craft/passion project
|
| I compare it with building some custom made designer house and
| furniture. Many would want that, only some can afford it, even
| less will pay for it.
| lukaslalinsky wrote:
| When you have a niche problem, you can usually reduce it in scope
| to such level, that you can make significantly better solution
| than trying to adapt some existing technology. It's not going to
| be universal, but for your niche problem, it can be much more
| suited.
|
| Back in 2010, I started an open source project for identifying
| audio files based on fingerprints. I specifically designed
| everything, so that in the end I need to do super fast i32->i32
| lookups on scale. There were many inverted indexes, but they are
| usually for text search, which has a lot of overhead I don't
| need. I ended up writing a completely custom solution, and it
| allowed me to scale to millions and millions of fingerprints on
| such small servers that it was unimaginable to anyone who ever
| ran audio fingerprinting system before.
| tjpnz wrote:
| Was a post here recently about someone who did just this and was
| able to survive in a competitive niche. The key(?) is to look for
| those where a significant slice is owned by PE firms.
| sireat wrote:
| As you get older, you have to become much pickier which if any
| wheels to reinvent.
|
| As others said reinventing the wheel is fantastic for learning,
| but not so for production / real life stuff.
|
| So for example implementing your own transformers from scratch -
| great for learning but for 99% it would not meaningfully change
| how you use LLMs for mundane things.
|
| So only when the domain is really really meaningful to you, you
| could consider reinventing the wheel and even then do not expect
| any immediately payoffs.
|
| Those 4000 weeks pass by awfully quickly.
|
| You have to pick your battles / wheels.
| jaza wrote:
| Valid points in the article and in many of the comments here.
| BUT: taken to the extreme, "yes, reinvent the wheel!" quickly
| becomes NIH ("not invented here") syndrome, which I've seen one
| too many times in enterprise stacks, as I'm sure plenty of you
| folks have too. We don't need no stinkin' wheels from the
| OUTSIDE!
|
| I think every codebase needs a healthy and pragmatic balance of
| "this particular wheel we can and should build ourselves, and
| it'll work better (for us at least!) than any other wheels out
| there" vs "there is a standard wheel for this, 10 million other
| horse-and-carriages out there use the same wheel, it's not worth
| us using a non-standard wheel for this thing".
| sylware wrote:
| It is interesting to do it, if it significantly simplifies and
| goes better to the point.
|
| Often, it is to give a variant proposal to something, rarely a
| bad thing.
|
| For, instance in software, alternative implementations are
| somewhat critical. It is actually a necessity "in the current
| economy" in order to sanely "work".
| bullen wrote:
| When people say "don't reinvent the wheel" they really mean to
| say "don't copy the wheel".
|
| Reinvent means changing something, hopefully for the better:
|
| http://move.rupy.se/file/wheel.jpg
| kovac wrote:
| In the context of software, I wonder what would qualify as a
| wheel. I'd say implementations of standards like protocols, BLAS,
| etc probabaly are wheels and not trivial to reinvent. Would you
| consider something like Kubernetes, Docker, web browsers, web
| frameworks as wheels? I can't help but think the software quality
| would be much higher if the latter set wasn't considered as
| wheels and we tried to reinvent them often.
| roes9627 wrote:
| well i don't disagree
|
| so long as you have the time and your manager isn't rushing you
| to deploy that feature "ASAP" ;)
| SwtCyber wrote:
| Sometimes the value isn't in creating something new for the
| world, it's in creating something new for yourself... I like it
| tobr wrote:
| "Reinventing the wheel" is a misguided idea in the first place,
| because the invention of the wheel was really the invention of
| the road. So, it's a phrase that in more ways than one suggest
| that you have no idea what invention is about.
| didgetmaster wrote:
| Much of the bloated and inefficient code out there is the result
| of simply pulling something off the shelf and using it, just
| because it 'kind of' does what you need.
|
| Nevermind that a much simpler and efficient piece is possible, we
| have to ship ASAP so don't bother to spend any time on it.
|
| Include a whole set of dependencies just because we need a
| function or two in a huge library or framework. Worry about
| optimizations later (i.e. never).
| ernestohegi wrote:
| How do you explain this to product owners in economic terms?
___________________________________________________________________
(page generated 2025-05-25 23:01 UTC)