[HN Gopher] Micro-libraries should never be used
___________________________________________________________________
Micro-libraries should never be used
Author : nalgeon
Score : 103 points
Date : 2024-08-18 18:38 UTC (4 hours ago)
(HTM) web link (bvisness.me)
(TXT) w3m dump (bvisness.me)
| KaiserPro wrote:
| I suspect this might be seen as trolling, but why isn't there a
| standard lib of stuff like this?
|
| surely it can't be beyond the wit of programming kind to have a
| standard lib, or even layers of standard lib for Node?
|
| What is the argument for not having a standard lib, apart from
| download speed?
| qsort wrote:
| I'm not taking an absolute position either way -- the devil is
| in the details -- but here's my steelman for the opposing view:
|
| When you put something in the standard library, it's harder to
| take it out, meaning that you're committing development
| resources to support the implementation. Furthermore things
| change: protocols and formats rise and fall in popularity and
| programming style evolves as the language changes (e.g.
| callbacks vs. promises in JS). Therefore the stdlib becomes
| where libraries go to die, and you'll always have a set of
| third party libraries that are "pseudo-standard", like NumPy in
| Python.
|
| Having a minimal stdlib lets you "free-market" the decision,
| letting the community effects take care of what is considered
| standard in the ecosystem, and lets you optimize its minimal
| surface, like what happened with C.
| kwhitefoot wrote:
| Why isn't there some kind of 'compiler/linker/stripper' that
| would collect the functions actually used and compile them into
| an application specific library? Yes I know that dynamic
| dispatch makes that difficult but the programmer does surely
| know which functions he wants to call.
|
| I sometimes hanker for a return to Fortran IV where every
| routine was separately compiled and the linker only put into
| the object code those that were referred to by something else.
| mattlondon wrote:
| Do you mean tree-shaking? I think basically all JavaScript
| "compilers" do this these days so that the code you serve is
| only the code you actually use.
| AdrianB1 wrote:
| There are many ways to look at this. Maybe the standard lib of
| such functions should be implemented as native functions in the
| language (1). Or as a standard external function library (2).
| Or people should copy-paste the functions they need and keep in
| their organization's function library (3). I am sure there are
| a few more options.
|
| I moved to option 3: in all my apps I include a function
| library that I build over the years, so I don't start from
| scratch every time. I deeply hate ("hate speech" example here)
| dependencies to libraries from all over the Internet due to
| security reasons, but I copy-paste code when needed to my
| library after I read, understand and check the code that I
| copy. The biggest advantage is that some of this code is better
| than what I could invent from scratch in a busy day and I save
| the time of doing it right. The disadvantage is there is no way
| to reward these authors that contribute to humankind.
|
| PS. My function library has functions mostly written by me,
| over 80%, but it includes code written by others. In my case,
| every time I need a function I check my existing library first,
| then analyze whether to write or copy.
| richardwhiuk wrote:
| I assume you correctly obey the licensing when you copy and
| paste it in....
| AdrianB1 wrote:
| Yes. It is not worth not doing it.
| xmodem wrote:
| Every non-trivial Java project I've worked on ends up depending
| on Google's Guava and some combination of Apache Commons
| libraries.
| mkoubaa wrote:
| Why don't people just copypasta the library into their codebase
| with the license header intact and keep it out of npm
| tptacek wrote:
| I think the JS library ecosystem is a debacle, but there's really
| only one point in this post that grabbed me:
|
| "Would future updates be useful? No. The library is so simple
| that any change to the logic would be breaking, and it is already
| clear that there are no bugs."
|
| Maybe what you want is a library ecosystem where things can be
| marked "this will never change". Something crazy happens and you
| actually need to update "is-number"? Rename it.
|
| Of course, you can simulate that with a single large omnibus
| dependency that everyone can trust that pulls all these silly
| micro-libraries in verbatim.
| Lws803 wrote:
| I would argue that "They should either be copy-pasted into your
| codebase" would cause more code liabilities and maintenance
| required further down the line. I've personally seen codebases
| with a ton of custom code, copy-pasted code, inspired
| implementations before and it was horrible to get them up to
| speed with the latest functionality / best practices. I agree
| that having too many micro-libraries might not be beneficial
| though, but perhaps look for larger, more well-established
| libraries that encompasses those functionalities :)
| quonn wrote:
| What's an example for ,,latest functionality" or ,,best
| practices" that should or could possibly change for a function
| like leftPad and that would not automatically happen by virtue
| of being in the code base (such as formatting)?
| gjsman-1000 wrote:
| In my Laravel projects, there are a few packages of much more
| niche/hobbyist origin without corporate backing, some haven't
| been updated for a while, and others are perfectly fine and don't
| need much maintenance.
|
| Normally, packages are listed in my composer.json and stored in
| vendor/. For those packages, I created a separate folder called
| vendor_private/ which is part of my Git tree, put copies of these
| weird little packages in it, and set up my composer.json to
| consider that folder a repository.
|
| Works like a charm. My big important packages are still upstream.
| I can customize the little ones as needed to fit better, or have
| better code, and not worry about them going unmaintained. It's
| also way quicker than copying the files individually out of the
| package and into the right places (along with updating
| Namespaces, configuration, etc.) Once in a while, I'll go back
| and see if anything worthwhile has changed upstream - and so far,
| it never has.
| AndyKelley wrote:
| Missing benefit: Libraries are a way to distribute and share the
| costs of labor, resulting in a more efficient ecosystem.
|
| This doesn't apply to micro-libraries, but it looks like that
| cost/benefit list is intended to cover libraries in general.
| franciscop wrote:
| Seems a lot like the classic "I put only a couple of the strong
| advantages and enumerate everything I could think about as
| disadvantage". While I'm bias (I've done a bunch of these micro-
| libraries myself), there's more reasons I/OSS devs do them! To
| name other advantages (as a dev _consuming_ them):
|
| - Documentation: they are usually well documented, at least a lot
| better than your average internal piece of code.
|
| - Portability: you learn it once and can use it in many projects,
| a lot easier than potentially copy/pasting a bunch of files from
| project to project (I used to do that and ugh what a nightmare it
| became!).
|
| - Semi-standard: everyone in the team is on the same page about
| how something works. This works on top of the previous two TBF,
| but is distinct as well e.g. if you use Axios, 50% of front-end
| devs will already know how to use it (edit: removed express since
| it's arguably not micro though).
|
| - Plugins: now with a single "source" other parties or yourself
| can also write plugins that will work well together. You don't
| need to do it all yourself.
|
| - Bugs! When there are bugs, now you have two distinct "entities"
| that have strong motivation to fix the bugs: you+your company,
| and the dev/company supporting the project. Linus's eyeballs and
| all (yes, this has a negative side, but those are also covered in
| the cons in the article already!).
|
| - Bugs 2: when you happen upon a bug, a 3rd party might've
| already found a bug and fixed it or offered an alternative
| solution! In fact I just did that today [1]
|
| That said, I do have some projects where I explicitly recommend
| to copy/paste the code straight into your project, e.g.
| https://www.npmjs.com/package/nocolor (you can still install it
| though).
|
| [1] https://github.com/umami-
| software/node/issues/1#issuecomment...
| pton_xd wrote:
| Every team should eventually have some internal libraries of
| useful project-agnostic functionality. That addresses most of
| your points.
|
| Copy-paste the code into your internal library and maintain it
| yourself. Don't add a dependency on { "assert": "2.1.0" }. It
| probably doesn't do what you actually want, anyway.
|
| I think the more interesting point is that most projects don't
| know what they actually need and the code is disposable. In
| that scenario micro-libraries make some amount of sense. Just
| import random code and see how far you can get.
| userbinator wrote:
| _and because it updates fairly frequently_
|
| I fail to comprehend how a single-function-library called
| "isNumber" even needs updating, much less "fairly frequently".
|
| The debate around third-party code vs. self-developed is eternal.
| IMHO if you think you can do better than existing solutions for
| your use-case, then self-developed is the obvious choice. If you
| don't, then use third-party. This of course says a lot about
| those who need to rely on trivial libraries.
| guestbest wrote:
| I think the updates are more for bugfixes around edge cases
| than feature additions.
| foul wrote:
| >I fail to comprehend how a single-function-library called
| "isNumber" even needs updating, much less "fairly frequently".
|
| If someone uses isNumber as a fundamental building block and
| surrogate for Elm or Typescript (a transpiler intermediate that
| would treat number more soundly I hope), this poor soul whom I
| deeply pity will encounter a lot of strange edge-cases (like
| that one stated in the article: NaN is a number or not?) and if
| they fear the burden of forking the library they will try to
| inflict this burden upstream, enabling feature or conf bloat.
|
| I insinuate that installation of isNumber is, like most of
| these basic microlibs, a symptom of incompetence in usage of
| the language. A worn JS dev would try isNaN(parseInt(num+''))
| and sometime succeed.
| ristos wrote:
| Micro-libraries are really good actually, they're highly modular,
| self-contained code, often making it really easy to understand
| what's going on.
|
| Another advantage is that because they're so minimal and self-
| contained, they're often "completed", because they achieved what
| they set out to do. So there's no need to continually patch it
| for security updates, or at least you need to do it less often,
| and it's less likely that you'll be dealing with breaking
| changes.
|
| The UNIX philosophy is also build on the idea of small programs,
| just like micro-libraries, of doing one thing and one thing well,
| and composing those things to make larger things.
|
| I would argue the problem is how dependencies in general are
| added to projects, which the blog author pointed out with left-
| pad. Copy-paste works, but I would argue the best way is to fork
| the libraries and add submodules to your project. Then if you
| want to pull a new version of the library, you can update the
| fork and review the changes. It's an explicit approach to
| managing it that can prevent a lot of pitfalls like malicious
| actors, breaking changes leading to bugs, etc.
| mattlondon wrote:
| > Micro-libraries are really good actually, they're highly
| modular, self-contained code
|
| Well I think that is the point, they're not self-contained. You
| are adding mystery stuff and who knows how deep the chain of
| dependencies go. See the left-pad fiasco that broke so much
| stuff, because the chain of transitive dependencies ran deep
| and wide.
|
| NPM is a dumpster fire in this regard. I try to avoid it - is
| there a flag you can set to say "no downstream dependencies" or
| something when you add a dependency? At least that way you can
| be sure things really are self-contained.
| IgorPartola wrote:
| I remember adding a random date picker that pulled in a copy
| of React with it to a non-React project. NPM is a dumpster
| fire at a nuclear facility.
| ristos wrote:
| Yeah there's a way to do that, yarn and pnpm can flatten the
| dependency tree. You can add the fork directly too:
|
| yarn add <path/to/your/forked/micro-library.git>
|
| pnpm add <path/to/your/forked/micro-library.git>
| a_wild_dandan wrote:
| There is a "no downstream dependencies" option; it's called
| writing/auditing everything yourself. Everything else -- be
| it libraries, monolithic SaaS platforms, a coworker's PR,
| etc. -- is a trade off between your time and your trust. Past
| that, we're all just playing musical chairs with where to
| place that trust. There's no right answer.
| Toutouxc wrote:
| Do you know what else is all of that? Writing the five lines of
| code by hand. Or just letting a LLM generate it. This and
| everything else I want to reply has already been covered in the
| article.
| ristos wrote:
| Nothing wrong with that either, like I said copy paste works
| too. A lot of minimalistic programs will just copy in another
| project.
|
| Forking the code and using that is arguably nicer though IMO,
| makes it easier pull in new updates from the code, and to be
| able to track changes and bug fixes easier. I've tried both
| and find this approach nicer overall.
| tgv wrote:
| > I would argue the problem is how dependencies in general are
| added to projects
|
| But you need the functionality anyway, so there are two
| dependencies: on your own code, or on someone else's code. But
| you can't avoid a dependency, and it comes at a cost.
|
| If you don't know how to code the functionality, or it will
| take too much time, a library is an outcome. But if you need
| leftPad or isNumber as an external dependency, that's so far in
| the other direction, it's practically a sign of incompentence.
| reaperducer wrote:
| _The UNIX philosophy is also build on the idea of small
| programs, just like micro-libraries, of doing one thing and one
| thing well, and composing those things to make larger things._
|
| This year I started learning FORTH, and it's very much this
| philosophy. To build a building, you don't start with a three-
| story slab of marble. You start with a hundreds of perfect
| little bricks, and fit them together.
|
| If you come from a technical ecosystem outside the Unix
| paradigm, it can be hard to grasp.
| ivan_gammel wrote:
| > So there's no need to continually patch it for security
| updates, or at least you need to do it less often, and it's
| less likely that you'll be dealing with breaking changes.
|
| Regardless of how supposedly good or small is the library, the
| frequency at which you need to _check_ for updates is the same.
| It doesn't have anything to do with the perceived or original
| quality of the code. Every 3rd party library has at least the
| dependency on platform and platforms are big, they have
| vulnerabilities and introduce breaking changes. Then there's a
| question of trust and consistency of your delivery process. You
| won't adapt your routines based on specifics of every tiny
| piece of 3rd party code, so you probably check for updates
| regularly and for everything at once. Then their size is no
| longer an advantage.
|
| > Copy-paste works, but I would argue the best way is to fork
| the libraries and add submodules to your project. Then if you
| want to pull a new version of the library, you can update the
| fork and review the changes.
|
| This sounds "theoretical" and is not going to work at scale.
| You cannot seriously expect application level developers to
| understand low level details of every dependency they want to
| use. For a meaningful code review of merges they must be domain
| experts, otherwise effectiveness of such approach will be very
| low - they will inevitably have to trust the authors and just
| merge without going into details.
| ristos wrote:
| They don't need to understand the low level dependencies.
| People can create metapackages of a lot of a bunch of self-
| contained libraries that have been audited and forked, and
| devs can pull in the metapackages. The advantage is the
| modularity, which makes the code easier to audit and is more
| self-contained.
|
| When's the last time ls, cat, date, tar, etc needed to be
| updated on your linux system? probably almost never. And
| composing them together always works. This set of linux
| tools, call it sbase, ubase, plan9 tools, etc, is one version
| of a metapackage. How often does a very large package need to
| be updated for bug fixes, security patches, or new versions?
| layer8 wrote:
| What you call an audited metapackage is nothing other than
| a non-micro-library. The property that it has been
| audited/assembled/designed/tested in conjunction and would
| be updated as a whole is exactly the benefit that non-
| macro-libraries provide.
| ristos wrote:
| It's not the same as a non-micro-library, because a non-
| micro-library is a bunch of internal code that isn't as
| well-documented and isn't self-contained, and maybe come
| and go as the non-micro-library continues to churn. I
| can't easily change a large monolithic library or project
| do better optimize for my use case. I could do that much
| easier though if the large thing was composed of a bunch
| of small self-contained things.
| layer8 wrote:
| I was assuming that the constituents of the metapackage
| aren't completely independent. TFA is about micro-
| libraries the size of _is-number_ and _left-pad_. If you
| only consider completely independent libraries of that
| type, you won't get very far. A more realistic take would
| be hundreds of such micro-libraries that aren't self-
| contained, but instead build on each other to
| successively provide more powerful functions. And then
| the more transitive dependencies become more like
| "internal code" in regular libraries, and you'll get the
| same churn, and you'll be similarly unable to change
| things due to the interdependencies. I don't see how you
| can end up with a comparable functionality a regular
| library provides without the same potential of running
| into the issues you note.
| ristos wrote:
| What if you're using a larger library and you wanted to
| swap out a sorting algorithm for one that's optimized for
| your use case?
|
| I would say that the API boundary being more modular and
| explicit makes it possible to actually do those kinds of
| swaps if the larger library is composed of smaller
| modular code, in ways that you wouldn't be able to if
| it's buried in a bunch of internal code -- you would have
| to fork the library in that case.
| rrdharan wrote:
| > When's the last time ls, cat, date, tar, etc needed to be
| updated on your linux system? probably almost never.
|
| Bad example:
| http://www.slackware.com/security/viewer.php?l=slackware-
| sec...
|
| They find stuff like this fairly often in GNU coreutils
| even to this day.. it's the main reason there's a Rust
| coreutils effort.
| ristos wrote:
| It's probably still a good example. Looking up the CVEs
| for various search terms:
|
| coreutils: 17 results
|
| linux kernel: 6752 results
|
| x11: 184 results
|
| qt: 152 results
|
| gtk: 68 results
|
| docker: 340 results
|
| rust: 455 results
|
| python: 940 results
|
| node: 110 results
|
| javascript: 5657 results
|
| firefox: 3268 results
|
| chrome: 3763 results
|
| safari: 1465 results
|
| webkit: 1346 results
|
| The large monolithic codebases have a lot more CVEs. I'd
| also argue that patching a fix on code made up of small,
| modular parts is much easier to do, and much lower
| hanging fruit for any casual developer to submit a PR for
| a fix.
| ivan_gammel wrote:
| > How often does a very large package need to be updated
| for bug fixes, security patches, or new versions?
|
| I don't think you understand my comment, because you are
| asking the wrong question again. It is not how often you
| need to actually update one dependency, but how often you
| need to check for updates that matters. That has to be done
| frequently no matter what and must be automated. E.g.
| despite low number of CVEs in coreutils you have to check
| them very often, because the impact can be very high. Once
| you have this process in place, there's no advantage in
| using micro-libraries. I'd actually expect that in a micro-
| library environment most breaking changes happen when a
| library becomes unsupported and you need to choose an
| alternative, making the migration process more complicated
| than in case of a bigger framework which just changed the
| API.
| ristos wrote:
| Maybe I'm not understanding your argument. Are you saying
| that if all these larger programs wrote all those
| utilities from scratch, it makes it so that if someone
| messed something up, the rest of the large programs are
| unaffected?
|
| All of those larger programs can mess things up in
| different ways, opening up all sorts of attack vectors
| and bugs. And a smaller utility that's much more easily
| auditable and easier to contribute fixes to is arguably
| less likely to have attack vectors in the first place.
|
| I'm not sure you have to check large libraries less
| often. I would argue at least as much if not more often,
| because it's harder for people to audit larger codebases
| and to also contribute fixes to them. A significantly
| smaller number of devs can (or could, if they had
| bandwidth/time) understand a large self-contained
| codebase than a very small self-contained codebase.
|
| I think that if a larger library is made of many smaller,
| modular, self-contained parts, that are composed
| together, and you can swap out any smaller building block
| for something that fits your use case (inversion of
| control), then that's a really good way to write a large
| library. Sloppier code might find it's way in due to
| time/resource constraints though, or the authors might
| not notice that some of the code isn't entirely
| modular/composable/swappable.
| jaredsohn wrote:
| >I would argue the problem is how dependencies in general are
| added to projects
|
| I haven't done anything with this myself (just brainstormed a
| bit with chatgpt) but I wonder if the solution is
| https://docs.npmjs.com/cli/v10/commands/npm-ci
|
| Basically, enforce that all libraries have lock files and when
| you install a dependency use the exact versions it shipped
| with.
|
| Edit: Can someone clarify why this doesn't work? Wouldn't it
| make installing node packages work the same way as it does in
| python, ruby, and other languages?
| ristos wrote:
| I'm not sure why you're getting downvoted. The left-pad
| incident on npm primarily impacted projects that didn't have
| lockfiles or were not pinning exact versions of their
| dependencies. I knew a few functional programmers that would
| freeze the dependencies to an exact version before lockfiles
| came around, just to ensure it's reproducible and doesn't
| break in the future. Part of what was to blame was bad
| developer practice. I like npm ci.
| porcoda wrote:
| The UNIX philosophy is being a bit abused for this argument.
| Most systems that fall under the UNIX category are more or less
| like a large batteries-included standard library: lots of
| little composable units _that ship together_. UNIX in practice
| is not about getting a bare system and randomly downloading
| things from a bunch of disjointed places like tee and cat and
| head and so on, and then gluing them together and perpetually
| having to keep them updated independently.
| ristos wrote:
| They ship together because all of those small composable
| units, that were once developed by random people, were turned
| into a meta-package at some point. I agree with you that
| randomly downloading a bunch of disjointed things without
| auditing and forking it isn't good practice.
|
| I'm also not arguing against a large popular project with a
| lot of contributors if it's made up of a lot of small,
| modular, self-contained code that's composed together and
| customizable. All the smaller tools will probably work
| seamlessly together. I think UNIX still operates under this
| sort of model (the BSDs).
|
| There's a lot of code duplication and bad code out there, and
| way too much software that you can't really modify easily or
| customize very well for your use case because it becomes an
| afterthought. Even if you did learn a larger codebase, if
| it's not made up of smaller modular parts, then whatever you
| modify has a significantly higher chance of not working once
| the library gets updated, because it's not modular, and you
| updated internal code, and the library authors aren't going
| to worry about breaking changes for someone who's maintaining
| a fork of their library that changes internal code.
| porcoda wrote:
| > all of those small composable units, that were once
| developed by random people, were turned into a meta-package
| at some point
|
| No they weren't. Every UNIX I used in the 80s and 90s
| shipped with those little composable building blocks as
| part of the OS, and GNU bundled them in things like
| coreutils forever. It's not like there was some past time
| when there were independent little things like cat and wc
| and so on written by random people on the internet that
| somehow got bundled into a meta-package after they existed.
| That didn't happen.
| ristos wrote:
| They were developed by different authors...
| wizzwizz4 wrote:
| We should totally have a system like that, though. It'd be
| such a great learning environment.
| ronjakoi wrote:
| It's called Linux From Scratch.
| syncsynchalt wrote:
| > randomly downloading things from a bunch of disjointed
| places like tee and cat and head and so on, and then gluing
| them together and perpetually having to keep them updated
| independently.
|
| I have distressing news about my experience using Linux in
| the '90s
| foul wrote:
| Micro-libraries anywhere else are everything you said: building
| blocks that come after a little study of the language and its
| stdlib and will speed up development of non-trivial programs.
|
| In JS and NPM they are a plague, because they promise to be a
| substitute for competence in basic programming theory,
| competence in JS, gaps and bad APIs inside JS, and _de-facto_
| standards in the programming community like the oldest
| operating functions in libc.
|
| There are a lot of ways for padding a number in JS and a decent
| dev would keep an own utility library or hell a function to
| copy-paste for that. But no. npm users are taught to fire and
| forget, and update everything, no concept of vendoring (that
| would have made incidents like left-pad, faker and colors less
| maddening, while vendoring is even bolt in npm and it's very
| good!). They for years copy-pasted in the wrong window, really,
| they should copypaste blocks of code and not npm commands. And
| God helps you if you type out your npm commands because bad
| actors have bought the trend and made millions of libraries
| with a hundred different scams waiting for fat fingers.
|
| By understanding that JS in the backend is optimizing for
| reducing cost whatever the price, becoming Smalltalk for the
| browser and for PHP devs, you would expect some kind of
| standard for having a single way to do routine stuff. Instead
| in JS-world you get TypeScript, and in a future maybe WASM. JS
| is just doomed. Like, we are doomed if JS isn't, to be honest.
| orhmeh09 wrote:
| Could you link to somebody who is teaching npm users to "fire
| and forget?" Someone who is promising a substitute for
| competence in basic programming theory? Clearly you and I do
| not consume the same content.
| foul wrote:
| This is just a discourse based on "I need to churn out
| something, I need that fast and I didn't start in the web
| game when Backbone and E4X were solid corporate choices".
| If you are not in a hurry, work in a solid team and have a
| good attention span, a lot of clickbait idiocy around JS
| may not happen. It's just that the lone inexperienced guy
| is one of millions inexperienced guys who are taught the
| wrong ways everyday.
|
| I'm presenting you one of countless examples: a lot of
| coding bootcamps teach React, maybe with TS, maybe with JS.
|
| Enter react-create-app.
|
| https://github.com/facebook/create-react-app
|
| The docs are a link, while the commands you can copy and
| paste are laid out at 9th row in the README. That will
| become a habit for a junior.
| ivan_gammel wrote:
| The whole web stack must die and be replaced. JS, CSS, HTML,
| HTTP are huge cost center for global economy.
| edwinjm wrote:
| You can use Flutter now. Go use it. Stop whining.
| smitty1e wrote:
| The four abstraction layers reflect the reality of
| technological drift over time.
|
| Even stipulating a Wand of Internet Technology (WIT) that
| could produce the One True Stack (OTS), two things remains
| undone:
|
| - fixing all the old stuff (or does OTS emulate it all?)
|
| - precluding further drift (or does OTS end all that?)
| foul wrote:
| I think that it's ugly but okay-ish right now. What is very
| very bad is the tooling, and someone should remember people
| that Facebook and Google do things that serves Facebook and
| Google-scale needs (billions of users, thousands of devs
| working asynchronously, no time).
|
| What I end up thinking (maybe i'm wrong) is that node.js
| must be nuked out of backend and on frontend maybe some of
| the devs should use either a number of libraries under 15
| and write custom code for the rest, or use a language that
| transpiles to JS like TS, flutter, nim, Go or what have
| you.
|
| Maybe JS should be nuked out of tooling too, sometimes it's
| actively damaging and sometimes dead slow. Use something
| else if wrangling asset files are a problem.
|
| If you want a DX where backend is frontend, you must use
| the only three mantained languages that can do that without
| trying to actively damage you or users, which are a
| Smalltalk (like Pharo), a Lisp (like Clojure/Clojurescript)
| or Java.
| alerighi wrote:
| > The UNIX philosophy is also build on the idea of small
| programs, just like micro-libraries, of doing one thing and one
| thing well, and composing those things to make larger things.
|
| They have small programs, but that are not of different
| project. For example all the basic Linux utilities are
| developed and distributed as part of the GNU coreutils package.
|
| It's the same of having a modular library, with multiple
| functions in them, that you can choose from. In fact the
| problem is that these function like isNumber shouldn't even be
| libraries, but should be in the language standard library
| itself.
| bborud wrote:
| This has nothing in common with the UNIX approach. Awk, grep,
| sort, less and the like are perhaps small, but not that small
| and not that trivial.
| Barrin92 wrote:
| _" >The UNIX philosophy is also build on the idea of small
| programs, just like micro-libraries, of doing one thing and one
| thing well, and composing those things to make larger things."_
|
| The Unix philosophy is also built on willful neglect of systems
| thinking. The complexity of system isn't in the complexity of
| its parts but in the complexity of the interaction of its
| parts.
|
| Putting ten micro-libraries together, even if each is simple,
| doesn't mean you have a simple program, in fact it doesn't even
| mean you have a working program, because that depends entirely
| on how your libraries play together. When you implement the
| content of micro-libraries yourself you have to be at the very
| least conscious not just of what, but how your code works, and
| that's a good first defense against putting parts together that
| don't fit.
| ristos wrote:
| It's not a willful neglect of systems thinking. Functional
| programmers have been able to build very large programs made
| primarily of pure functions that are composed together. And
| it makes it much easier to debug as well, because everything
| is self-contained and you can easily decompose parts of the
| program. Same with the effectful code as well, leveraging
| things like algebraic effects.
| prng2021 wrote:
| Why even stop at micro-libraries? Instead of "return num - num
| === 0" why not create the concept of pico-libraries people can
| use like "return isNumberSubtractedFromItselfZero(num)" ? It's
| basically plain English right?
|
| You could say that if all the popular web frameworks in use
| today were rewritten to import and use hundreds of thousands of
| pico-libraries, their codebase would be, as you say, composed
| of many high modular, self contained pieces that are easy to
| understand.
|
| /s
| TacticalCoder wrote:
| Thought experiment: if a LLM can correctly produce the code for a
| micro-library like, say, leftpad... Should you call leftpad as a
| dependency or should you have the LLM generate that leftpad
| function for you?
|
| And if the LLM ain't good enough to write leftpad, how can I
| trust it to write anything at all?
| joshmarinacci wrote:
| TL;DR most micro libraries should be gists
| mirekrusin wrote:
| Personally I prefer sharing one level up from function to
| conceptual module, ie. instead of "left-pad" function, "string"
| module, ie. a bit like this [0] (`${authorshipDisclaimer}`).
|
| I'm also an advocate, against crowd, of qualified imports as they
| help with refactoring (renames are propagated, especially in
| monorepos), readability/reviews (functions are qualified, you
| know where they're coming from) and overall coding experience -
| qualified module name followed by dot gives good autocompletion,
| imports look neat in larger projects etc. The codebase written
| like this resembles extended standard library. It also helps with
| solving problems by encouraging first principle thinking, bottom
| up coding that produces auditable codebase with shallow external
| dependencies etc.
|
| [0] https://github.com/preludejs
| qwerty456127 wrote:
| > Micro-libraries should never be used. They should either be
| copy-pasted into your codebase, or not used at all.
|
| I would prefer them to be built straight in the languages.
| jdminhbg wrote:
| Yes, everyone seems to take the wrong lesson from left-pad. The
| reason left-pad happened on NPM isn't that there's something
| uniquely wrong with how NPM was built, but that JS has a
| uniquely barren standard library. People aren't writing their
| own left-pad functions in Java or Go or Python, it's just in
| the stdlib.
| xg15 wrote:
| Micro libraries are worse than no libraries at all - but I
| maintain they are still better than gargantuan "frameworks" or
| everything-but-the-kitching-sink "util"/"commons" packages, where
| you end up only using a tiny fraction of the functionality but
| have to deal with the maintenance cost and attack surface of the
| whole thing.
|
| If you're particularly unlucky, the unused functionality pulls in
| transitive dependencies of its own - and you end up with
| libraries in your dependency tree that your code is literally not
| using at all.
|
| If you're _even more_ unlucky, those "dead code" libraries will
| install their own event handlers or timers during load or will be
| picked up by some framework autodiscovery mechanism - and _will_
| actually execute some code at runtime, just not any code that
| provides anything useful to the project. I think an apt name for
| this would be "undead code". (The examples I have seem were from
| java frameworks like Spring and from webapps with too many
| autowired request filters, so I do hope that is no such an issue
| in JS yet)
| DonHopkins wrote:
| Sometimes importing zombie "undead code" libraries can be
| beneficial!
|
| I just refactored a bunch of python computer vision code that
| used detectron2 and yolo (both of which indirectly use OpenCV
| and PyTorch and lots of other stuff), and in the process of
| cleaning up unused code, I threw out the old imports of the
| yolo modules that we weren't using any more.
|
| The yololess refactored code, which really didn't have any
| changes that should measurably affect the speed, ran a
| mortifying 10% slower, and I could not for the life of me
| figure out why!
|
| Benchmarking and comparing each version showed that the
| yololess version was spending a huge amount of time with
| multiple threads fighting over locks, which the yoloful code
| wasn't doing.
|
| But I hadn't changed anything relating to threads or locks in
| the refactoring -- I had just rearranged a few of the deck
| chairs on the Titanic and removed the unused yolo import, which
| seemed like a perfectly safe innocuous thing to do.
|
| Finally after questioning all of my implicit assumptions and
| running some really fundamental sanity checks and reality
| tests, I discovered that the 10% slow-down in detectron2 was
| caused by NOT importing the yolo module that we were not
| actually using.
|
| So I went over the yolo code I was originally importing line by
| line, and finally ran across a helpfully commented top-level
| call to fix an obscure performance problem:
|
| https://github.com/ultralytics/yolov5/blob/master/utils/gene...
| cv2.setNumThreads(0) # prevent OpenCV from multithreading
| (incompatible with PyTorch DataLoader)
|
| Even though we weren't actually using yolo, just importing it,
| executing that one line of code fixed a terrible multithreading
| performance problem with OpenCV and PyTorch DataLoader fighting
| behind the scenes over locks, even if you never called yolo
| itself.
|
| So I copied that magical incantation into my own detectron2
| initialization function (not as top level code that got
| executed on import of course), wrote some triumphantly snarky
| comments to explain why I was doing that, and the performance
| problems went away!
|
| The regression wasn't yolo's or detectron2's fault per se, just
| an obscure invisible interaction of other modules they were
| both using, but yolo shouldn't have been doing anything
| globally systemic like that immediately when you import it
| without actually initializing it.
|
| But then I would have never discovered a simple way to speed up
| detectron2 by 10%!
|
| So if you're using detectron2 without also importing yolo, make
| sure you set the number of cv2 threads to zero or you'll be
| wasting a lot of money.
| conradludgate wrote:
| This is mortifying. This should not be acceptable implicit
| behaviour for imports to implicitly run code by simply
| existing
| DonHopkins wrote:
| I know, right??! I was alternating between hitting my head
| up against the wall in despair, and jumping for joy I
| accidentally found a way to speed up detectron2 by 10%.
| I'll take the win.
|
| YOLO: You Only Load Once
| qudat wrote:
| Partially disagree, JS has unique features that require small
| libraries: https://bower.sh/my-love-letter-to-front-end-web-
| development
|
| However, if I can inline a small function, I will, so in that
| sense I agree.
| PhilipRoman wrote:
| Micro libraries are fine (well... not really), the problem starts
| when each of those depends on 10 more "micro" libraries and so
| forth. The branching factor quickly leads to bloat. Libraries
| have a duty to minimize their footprint in ways that applications
| do not.
| rc_kas wrote:
| the entire nodejs ecosystem needs to die. You all are just
| keeping it alive.
| edwinjm wrote:
| You can start by ignoring it
| edfletcher_t137 wrote:
| Where do you draw the line: is 10 lines "micro"? 50? 100? It is
| never quantified within, yet the very click-bait-y title relies
| on the term. How many bugs can hide in 50 lines or 100? And you
| really want to copy-paste that code at a static point in time?!
| IshKebab wrote:
| I don't think they need to draw an exact line for us to know
| what they're talking about.
| edfletcher_t137 wrote:
| The fact that we're even debating it shows that they do. And
| I didn't ask for "an exact line". I asked for anything. What
| is your qualification for a "micro" library? I'm betting it's
| different from 9 other respondents. That's a problem.
|
| Furthermore, that's not even the main contention I was
| highlighting. Without a proper definition, the advice of
| "just copy/paste these" is dangerous. Someone will draw their
| line at something too large, copy/paste that in and inherit
| bugs/vulnerabilities they never fix. That's a big problem.
| unstable wrote:
| > You can write isNumber(foo) instead of typeof foo === "number".
|
| Indeed you can, but it depends what isNumber does. This is more
| like what it should do IMO:
|
| function isNumber( foo ) { return ( (typeof foo === "number") &&
| (foo == foo)) || ((typeof foo === 'object') && (foo instanceof
| Number) ); }
|
| And that is I think the value of micro libs, at least in JS, you
| don't want to think about all the edge cases when you only want
| to check if something is a Number.
| layer8 wrote:
| This is an argument for having a library that provides that
| function, but it is not an argument that it should be a micro-
| library.
| IshKebab wrote:
| Doesn't that exclude NaN (which you probably want despite the
| name)? I think this really highlights that you probably _do_
| want to think about those edge cases...
|
| In any case this is a bad example because Typescript exists.
| unstable wrote:
| Accepting NaN as a number can potentially crash your app,
| that's why I reject it in isNumber.
|
| I only tried to highlight some edge cases that I personally
| don't like to spend energy on, trying to get it right, when
| writing code. Btw, isNumber is a dynamic call in the example
| and unrelated to TypeScript. TypeScript doesn't exist at
| runtime.
| ristos wrote:
| Reminds me of the 2ality blog post on that:
|
| https://2ality.com/2017/08/type-right.html
| replete wrote:
| Not checking dependency updates need to die already. Not choosing
| better dependencies needs to die already.
| stevebmark wrote:
| This discussion is a complete waste of oxygen and bits. There is
| no point discussing this. This blog author should determine what
| the required minimum LOC is for any package in any ecosystem,
| then hit his computer with a hammer and go outside.
| johnnyanmac wrote:
| >This discussion is a complete waste of oxygen and bits. There
| is no point discussing this.
|
| Why?
| crabmusket wrote:
| While I mainly agree with the author's substantive point, though
| I find some of the ways it's presented in this post not entirely
| convincing or fair, I am interested that someone else has
| identified this:
|
| > I have talked a lot about the costs of libraries, and I do hope
| people are more cautious about them. But there's one factor I
| left out from my previous discussion. I think there's one more
| reason why people use libraries: _fear._
|
| > Programmers are afraid of causing bugs. Afraid of making
| mistakes. Afraid of missing edge cases. Afraid that they won't be
| able to understand how things work. In their fear they fall back
| on libraries. "Thank goodness someone else has solved the
| problem; surely I never would have been able to."
|
| I think this is true, but why does the JS ecosystem seem to have
| "more fear" than for example the Python ecosystem?
|
| I wrote about this a while ago. I think that actually JS does (or
| did) cause more fear in its developers than other programming
| languages. I described it as paranoia, a more insidious
| uncertainty.
|
| Quoting myself[1]:
|
| > There are probably many contributing factors that have shaped
| NPM into what it is today. However, I assert that the _underlying
| reason_ for the bizarre profusion of tiny, absurd-seeming one-
| liner packages on NPM is paranoia, caused by a _unique
| combination_ of factors.
|
| > Three factors have caused a widespread cultural paranoia among
| JavaScript developers. This has been inculcated over years. These
| factors are: JavaScript's weak dynamic type system; the diversity
| of runtimes JavaScript targets; and the physics of deploying
| software on the web.
|
| ...
|
| > Over the years there has been rapid evolution in both frontend
| frameworks and backend JavaScript, high turnover in bundlers and
| best-practises. This has metastasized into a culture of
| uncertainty, an air of paranoia, and an extreme profusion of
| small packages. Reinventing the wheel can sometimes be good - but
| would you really bother doing it if you had to learn all the
| arcane bullshit of browser evolution, IE8 compatibility,
| implementation bugs, etc. ad infinitum?
|
| > And it's not just that you don't understand how things work
| now, or how they used to work - but that they'll change in the
| future!
|
| [1] https://listed.to/@crabmusket/14061/javascript-s-
| ecosystem-i...
| DonHopkins wrote:
| >Micro-libraries should never be used
|
| Passive voice. WHO should never use micro-libraries?
| oftenwrong wrote:
| The primary cause of the left-pad incident was that left-pad was
| removed from the npm registry. Many libraries depended on left-
| pad. The same could have occurred with any popular library,
| whether micro or not.
|
| To reformulate the statement made in the intro of this post:
| "maybe it's not a great idea to outsource _any critical_
| functionality to random people on the internet."
|
| It has long been a standard, best practice in software
| engineering to ensure dependencies are stored in and made
| available from first-party sources. For example, this could mean
| maintaining an internal registry mirror that permanently stores
| any dependencies that are fetched. It could also be done by
| vendoring dependencies. The main point is to take proactive steps
| to ensure your dependencies will always be there when you need
| them, and to not blindly trust a third-party to always be there
| to give your dependencies to you.
| edwinjm wrote:
| If your only examples are leftPad and isNumber, I can't take this
| article seriously. There are so many really useful micro
| libraries.
| edwinjm wrote:
| So, everybody can contribute an npm package.
|
| The advantage: - everybody can contribute an npm package
|
| The disadvantage: - everybody can contribute an npm package
| Cheezmeister wrote:
| At no point does this article attempt to define the term "micro-
| library".
|
| Perhaps we should start there.
___________________________________________________________________
(page generated 2024-08-18 23:00 UTC)