[HN Gopher] Why I Prefer Makefiles over Package.json Scripts
___________________________________________________________________
Why I Prefer Makefiles over Package.json Scripts
Author : ducaale
Score : 205 points
Date : 2022-03-14 12:26 UTC (10 hours ago)
(HTM) web link (spin.atomicobject.com)
(TXT) w3m dump (spin.atomicobject.com)
| nicoburns wrote:
| I would like to call attention to `just`
| (https://github.com/casey/just) which is a modern take on "make
| if we didn't have to keep it backwards compatible", and more
| focussed on just task running rather than tracking dependencies.
| marcuskaz wrote:
| Yes! This is what a modern task runner should be. Includes
| ability to list recipes, and a consistent format between
| multiple platforms. Just two things Make doesn't have.
| michidk wrote:
| I was just going to say that. Just is awesome and I use it it
| most of my projects.
| enriquto wrote:
| There's no -j option nor mention of parallelism on the docs.
| Does it run independent targets in parallel? This is easily the
| _main_ feature of make.
|
| I don't understand how it can work in parallel if targets are
| not somehow associated to files. What's the advantage to a
| shell script with separate sub-commands, then?
| q3k wrote:
| 'just' apparently isn't a build system, just a 'task runner'.
|
| So I assume it doesn't need/want to implement parallelism.
| toast0 wrote:
| Parallelism is pretty useful for just a 'task runner' too.
|
| I've (ab)used Make for server pushes, and sometimes you
| want those to run one at a time, and sometimes you want -j
| 100 (sometimes -j 100 is even effective).
| jandrese wrote:
| The key aspect here is that since Make knows what the
| dependencies are it has enough information to
| automatically parallelize the task. A simple task runner
| would need to be specifically configured with the
| dependency graph, and if you have that you've just re-
| created Make.
| toast0 wrote:
| If I'm using the task runner for deployment, it doesn't
| need to have a sophisticated dependency graph, it "just"
| needs to be able to have a list of tasks that are
| declared as independent/parallelizable, and a way to
| specify how many to run at once.
|
| Sophisticated dependencies are nice to have of course,
| maybe make push-all transitively includes building the
| thing locally, but it's still useful to have just the
| push step (or the reload step, etc), have optional
| parallelism. If you start to build a tool that does do
| these things first, in order; then these things in any
| order; then these other things, that does look a lot like
| Make. but if you just handle ordered steps and
| independent steps, and let a person do the mixing, then
| it's not nearly as much work as Make.
| enriquto wrote:
| > Sophisticated dependencies are nice to have
|
| What are you talking about? Make does not deal with
| "sophisticated" dependencies. Make is insultingly simple,
| not sophisticated at all. It's just a damn directed tree.
| The simplest data structure in computer science after the
| array.
|
| There are also some fancier shortcuts for expressing
| rules in makefiles, but you can ignore them and use only
| the simple stuff. That's already a much more powerful
| tool than "just".
| dahfizz wrote:
| It sounds like `just` is not a replacement for make, then.
| It seems like it could be a nice wrapper on top, i.e. `just
| build` could invoke the make build system.
| pletnes wrote:
| The python library/CLI tool <<invoke>> is also nice, although
| requires python.
| jonhohle wrote:
| Tracking dependencies is what makes make make. Love it or hate
| it, Ant got this right (imho), and modern build systems almost
| universally get this wrong. Every language can run and compose
| tasks. Dependency tracking, parallelization, and language
| agnosticism are the killer features of make (again, imho).
| gnulinux wrote:
| I agree with you on all grounds (I personally use make daily)
| however note that `make` checks dependencies via last updated
| time flag on files. I think it would have been a better
| default to cache based on hash of file. This way if document
| was modified, by hash wasn't changed, make shouldn't rebuild
| tasks that depend on this file.
| zvr wrote:
| There are always different preferences on deciding when
| something needs to be run again. File modification time is
| a reasonable default that serves many (most?) cases pretty
| well.
|
| Yes, file hashes might be another way. On the other hand,
| if I modify a comment, the file hash will change, but re-
| compilation is not necessary. In this case, you might need
| yet another way...
|
| You can always write your Makefiles to compute and depend
| on hashes or whatever else you might consider appropriate
| for your exact use case.
| enriquto wrote:
| But computing the hashes of all the files would take the
| same time as rebuilding everything, or maybe even slower.
| And you'd need to re-compute all these hashes even for a
| no-op. It's absurd.
|
| The nice thing about dates is that they can be checked
| instantaneously, and independently of file size. If a file
| has changed, its date has, so it's all the information you
| need.
| anamexis wrote:
| > If a file has changed, its date has, so it's all the
| information you need.
|
| Unless the date hasn't changed -- there is no hard
| guarantee that mtime will be accurate.
| bityard wrote:
| If the file's contents have changed but the timestamp
| didn't, then something is broken and _that_ is the thing
| to fix, not rewriting the fundamental mechanism
| underneath the oldest relevant build tool.
| duped wrote:
| Computing the SHA sum of a file is plenty fast on modern
| hardware, nowhere close to the same time as a rebuild.
| slaymaker1907 wrote:
| Yep, I can take the sha256sum over my all my project's
| (vendored) dependencies in 10ms using the command:
|
| time find ./vendor -type f -print0 | sort -z | xargs -0
| sha256sum | sort -k2 | sha256sum
|
| Note if using this code for practical purposes, you
| should be sure your collation is set appropriately.
| That's taken with 1280 files that are in total 19MB.
|
| For raw throughput, you can use BLAKE2 for even more
| speed https://www.blake2.net/
| enriquto wrote:
| What % of that time is required for just computing the
| timestamps of these files? I bet it's a few orders of
| magnitude faster.
| gnulinux wrote:
| Rebuilding potentially has side-effects. You can have a
| build system that, if a file is changed, it makes HTTP
| calls, generate docker containers, store artifacts in S3
| etc. Given this, computing the hash would be negligable
| amount of work. Besides make does iterate over all
| dependencies every time to check last modified date
| filesystem keeps. So, at that point one can compute hash.
| IgorPartola wrote:
| You shouldn't have such a build system. Idempotence is a
| feature, not a bug.
| mmis1000 wrote:
| Side effects that target on non local dir seems shouldn't
| be included in the build step. It should be in a
| standalone script/ target/whatever. Or the build process
| will not be reproducible at all.
| kbd wrote:
| I haven't understood what the point is for a "task runner"
| over having a bin/ directory in the project with some shell
| scripts. If it's not going to do anything extra for you (like
| run commands according to a dependency tree like make), I
| don't see the point.
| FooBarWidget wrote:
| I've been using make for a long time but make really really
| sucks at parallelization and language agnosticism.
|
| I mean those things _work_ , but they're by no means nice.
| Parallelization works, but output of different tasks (and
| even the log that certain commands have been run) are
| interleaved, making it impossible to view progress. Language
| agnosticism works by operating on the level of shell
| commands, but that's really about it. If shell commands need
| to be customized on a per-system or per-OS level... well it's
| possible to hack around that with enough external scripts and
| even esoteric if statements inside the Makefile, but it's not
| nice.
|
| Documentation for make feels like it's from the 80s. Which it
| is.
|
| All these things (with the exception of having to rely on
| arcane sets of external scripts and if-statements-in-
| Makefile) are perfectly fixable. But they don't get fixed.
| oftenwrong wrote:
| `just` seems like a useful tool, but it only covers a secondary
| use case of `make`. `make` without a dependency graph is like a
| bicycle without wheels.
| masklinn wrote:
| All I can say is that I've only ever used make as a task
| runner myself, so just matches my need way better.
|
| IME make's model is often too simplistic to be of use, so it
| simply gets in the way of tools already doing things
| internally with proper insight, or requires force-cleaning
| everything because it's too dumb and just breaks the build
| when you e.g. switch branch.
|
| Just's author is also looking at integrating `redo` for
| building / dependency tracking, though the issue has no
| activity since it's been created.
| jandrese wrote:
| If you find yourself running "make clean" because it is
| "too dumb" you've built your Makefile wrong. Usually it
| means you've tried to manually re-create the inherent Make
| rules and have gotten it wrong somewhere. This is
| incredibly common, since the inherent rules are not always
| the same across various versions of Make and people don't
| always trust them.
|
| IMHO, this is the primary weakness of Make. The
| standardization efforts never really happened so you have
| various forks of 1980s code with annoying
| incompatibilities. BSD Make is missing some of the implicit
| build rules from GNU Make (only able to compile simple
| single source file applications without an explicit rule),
| and Windows Make tends to be extremely primitive. Worse,
| Make still in 2022 can't handle spaces in file names, like
| c:\Documents and Users\\.
|
| I would be so much happier if everybody could standardize
| on one flavor of Make. Probably GNU Make.
| teddyh wrote:
| > _I would be so much happier if everybody could
| standardize on one flavor of Make. Probably GNU Make._
|
| With included Guile scripting support!
|
| https://www.gnu.org/software/make/manual/html_node/Guile-
| Exa...
| slaymaker1907 wrote:
| Only if your version of Make is compiled with it. "GNU
| make may be built with support for GNU Guile..."
| https://www.gnu.org/software/make/manual/html_node/Guile-
| Int...
|
| I really wish it was more common because having it would
| be great.
| masklinn wrote:
| > If you find yourself running "make clean" because it is
| "too dumb" you've built your Makefile wrong.
|
| What's built wrong is make itself. Because the signal it
| uses for "this dependency has changed" is the mtime, any
| change which does not move the mtime forwards will be
| ignored. Switching between concurrent git branches, for
| instance.
|
| > Usually it means you've tried to manually re-create the
| inherent Make rules and have gotten it wrong somewhere.
|
| And pray tell how do you "manually re-create the implicit
| make rules" when such rules don't exist? GNU Make has
| implicit rules for C, C++, Pascal, Fortran, Raftor,
| Modula-2, and assembly. That is, frankly, not much. And
| mostly outdated.
| btbuildem wrote:
| Make has been around for decades, and with good reason. I've
| enjoyed watching the grunts and gulps and whatevers appear as the
| New Shiny Thing, become complicated and collapse under their own
| weight as people struggle with overly complex configs, as I
| quietly do the thing that has always worked and has not changed
| in forever.
| JohnHaugeland wrote:
| The second I see a node project built in make, I remove it
|
| That's a person who doesn't understand portability, tooling, or
| the need to stick to community norms. Everything about their
| library is going to be pure pain.
| tejohnso wrote:
| Pretty convincing writeup. I've only used makefiles when
| compiling C / C++ but lately I'm finding even those are moving to
| CMAKE with makefiles becoming less and less common.
| WesolyKubeczek wrote:
| Why not both?
|
| Use make for dependency graph, but also use "npm run" or "yarn
| run" to make sure you're running with the correct Node and
| correct paths to all your tools.
|
| I know you can specify those in your Makefile too, but the plus
| of running via npm/yarn is that they know where your binaries
| live, where your mode_nodules are, and that sort of thing.
| JamesSwift wrote:
| Yes, I usually treat Make as the user-friendly facade to the
| underlying framework tooling (which might be e.g. Rails,
| Docker, or npm). `make bootstrap` gets the project setup. `make
| up` starts the project.
|
| For npm projects these usually just wrap package.json stuff.
| But I don't need to remember what tech a project is in, or
| where to look for the commands, if Make is the entry point.
| andrew_ wrote:
| replace that with `pnpm` and you've got a winner. npm and yarn
| are old hat (unless you're working with react native, which is
| another discussion)
| karmicthreat wrote:
| Out in Grand Rapids, Mi where AO is located there seems to be a
| bit of a tradition of abusing make in a good way. It's where I
| picked up the habit as well. I think its just institutional
| knowledge/tradition that has creeped through the various firms
| here.
|
| I kind of wonder what other "traditions" get passed around on a
| regional level?
| dncornholio wrote:
| But you don't write scripts in package.json and it also failed to
| show an example how the javascript dependencies could be loaded
| with make.
|
| I think make works well in combination with package.json, but not
| as a replacement. You can also do the same stuff in an NPM
| command (which just loads node code) that you could do in a
| makefile script.
| andrew_ wrote:
| Not to mention that the top three package managers will load
| binary files from node_modules automagically by name only,
| without having to specify paths. That gets hairy in a monorepo,
| and package managers running package.json scripts manage them
| just fine.
| silverwind wrote:
| `npx` solves this elegantly as long as you `cd` to the
| project directory first.
| k__ wrote:
| A Makefile inside a JavaScript codebase is a good indicator that
| you will encounter other less idiomatic ways of doing things in
| that codebase.
| efortis wrote:
| It's helpful for mitigating an NPM arbitrary code execution
| vulnerability along with 'ignore-scripts=true'
|
| https://blog.uidrafter.com/getting-rid-of-npm-scripts
| sigzero wrote:
| Task is nice:
|
| https://taskfile.dev/#/
| Steltek wrote:
| This seems to be missing the obvious point: Make isn't anchored
| to any single language. If you will only ever use npm for your
| entire life, then you go ahead an live happily in package.json
| script land.
|
| But I've lost track of the number of languages and environments
| that I've worked in. Make ties it together by being the Good
| Enough anchor point that documents and launches all the other
| tools and compilers.
|
| - Compile this Rust app with Cargo then flash it to the esp32?
| Make.
|
| - Build this page with JS using Tool Of The Week and push it to a
| test container? Make.
|
| - Compile ancient C app and build a .deb? Make.
|
| And so on.
|
| Why not use shell scripts you may ask? Because shell scripts are
| way too free form and your tastes will change over time. Make
| forces just enough structure that you won't get carried away with
| yourself. Not for the basic task running stuff anyway.
| [deleted]
| dongping wrote:
| Bazel would be a better solution in terms of reproducibility
| and user-friendliness, though.
| nuccy wrote:
| I agree, but _make_ is usually more abundant on much more
| systems (which is useful when there is no root privileges
| available). Also people are used to run _make_ when they
| encounter a _Makefile_. I personally use _Makefile_ s even to
| create Docker images, I find it simpler to run _make images_
| or _make run_ , than remembering (or putting in a script and
| remember what parameters to use, as pointed in the comment
| above scripts are too free-style) how to do it manually.
|
| Though I also agree that _Makefile_ is not a silver bullet
| and some more complex /niche methods may be required in
| particular cases.
| physicsguy wrote:
| It's just a nightmare to manage unless you're a Google-sized
| company
| klodolph wrote:
| I've been working with Bazel by myself. My experience is
| that it is not a nightmare... it's just the documentation
| is missing some pretty crucial "how-to" guides, and there
| are a couple features that changed a lot prior to 1.0 (so
| using them is a bit difficult).
|
| Some stuff that is very easy with makefiles is a bit harder
| with Bazel, so I can't give it an unqualified
| recommendation. The payoff is that some things that are
| very, very hard with makefiles are much easier with Bazel
| (using multiple languages, cross-compiling, downloading
| dependencies automatically, distributed builds, etc).
| physicsguy wrote:
| My experience is mostly through consuming packages that
| other people have written, and the downloading existing
| packages is a big part of why it's problematic. If I
| already have PrettyCommonStandardLibraryX, I don't really
| want Bazel to download another copy of it for me just to
| incorporate your project. Other build systems are simple
| enough to change, or support this straightforwardly.
| Doing it with Bazel is quite painful IMO.
| slaymaker1907 wrote:
| The only bad part about Make is that it doesn't provide even
| the most basic utilities for doing cross platform work like
| removing a directory. It mostly works with Unix, but writing a
| Makefile that works on Windows and Unix is really hard.
|
| If Make had just a few more batteries included, it would be
| perfect. However, maybe adding those batteries would take away
| from its other qualities.
| Izkata wrote:
| Make is a file-based dependency tree that only defaults to a
| specific shell. If you really want to go crazy, you could
| actually change it to anything that can be invoked with
| "<executable> <file>", such as python, and use that language
| instead.
|
| As a really crude example: SHELL := python
| .ONESHELL: out.csv: import csv
| with open("$@", "w") as f: writer =
| csv.writer(f) writer.writerow(["header1",
| "header2"])
|
| and so: $ make import csv with
| open("out.csv", "w") as f: writer = csv.writer(f)
| writer.writerow(["header1", "header2"]) $ cat out.csv
| header1,header2
|
| I get that this doesn't exactly solve your "cross-platform
| work" gripe, but my point is that a lot of what people
| perceive as make providing isn't actually make in the first
| place.
| pqb wrote:
| Thank you for reminding me it! Awesome finding.
|
| Worth noting, each recipe can have its very own SHELL[0]
| (e.g., ruby recipe, that uses ruby command with -e flag):
| ruby: .SHELLFLAGS := -e ruby: SHELL := ruby
| ruby: greeting = "labas" puts "#{greeting},
| ruby!"
|
| [0]: http://agdr.org/2020/05/14/Polyglot-Makefiles.html
| andreineculau wrote:
| Nowadays there's Windows Subsystem for Linux. There's no
| excuse not to successfully run "Linux" scripts on Windows.
|
| I've been running very complex build systems via
| https://github.com/ysoftwareab/yplatform (disclaimer: author
| here) since 2016 on Linux, Mac and Windows without a problem.
| metaltyphoon wrote:
| That's why you always provide an nmake for the project :)
| kemenaran wrote:
| I like nothing more than dropping into a new project, and,
| rather than figuring out the commands to make it run, see that
| it has a Makefile with a handful of targets for common tasks.
|
| With some collegues we wrote an article about the benefits of
| this approach a few years ago:
| https://blog.capitaines.fr/2016/09/30/standardizing-interfac...
| pletnes wrote:
| I could not agree more. However most people are on windows
| for which obtaining gnu make is painful, or maybe even
| impossible - at least the path handling has many sharp edges.
| kemenaran wrote:
| Indeed - although I guess WSL made unix tools slightly
| easier to use nowadays.
|
| But even without executing the Makefile, simply reading it
| can tell new developers which language-specific command
| needs to be run to build the project (and then the command
| can be copy-pasted and run manually).
| pletnes wrote:
| WSL made running a linux VM less hassle.
| jokethrowaway wrote:
| I don't comprehend how people can develop on windows
| (without linux subsystem).
|
| I have to do that every once and then (to ship the
| occasional C++ or Rust build on Windows) and it's the stuff
| of nightmare. Stuff breaking randomly from one day to
| another, env variables to be set in weird ways, GUI
| installers, 8 different versions of mingw or similar.
| Recently I've seen that there are a few package managers
| now (I used chocolatey and at least 2 others just trying to
| get something to compile) but still, compiling something
| trivial is always an adventure.
|
| Mac OS X is kind of okay. Brew is barely decent and things
| mostly work (unless you discover you need to install 12GB
| of XCode for some dependency or your script is expecting
| coreutils instead of bsd).
|
| Every linux distro comes with a package manager and
| compiling is trivial
| wwalexander wrote:
| Homebrew is a subpar choice on macOS; MacPorts is faster,
| has more packages, and is implemented more correctly than
| Brew.
|
| Additionally, MacPorts was co-created by an engineer who
| also created the original FreeBSD Ports system, and thus
| hews much more closely to standard UNIX/BSD practices.
|
| I'm not sure how and when Homebrew became the standard,
| but it is definitively worse.
| jwdunne wrote:
| I came to realise this too.
|
| Using Homebrew and multiple users is excruciating and an
| eye opener on how system-level software should really be
| installed.
|
| Homebrew insists on avoiding root privileges whilst also
| installing packages system-wide. That works fine and is
| invisible with one user but falls down hard otherwise.
|
| Their documentation is incorrect too, saying that this is
| all fine because "we install in /usr/local/bin". It's not
| easy to change this.
|
| The solution was to embrace MacPorts which correctly
| requires root privileges to install system-wide packages.
|
| I haven't looked back since. I haven't missed brew or any
| software that's available on brew alone.
| Ygg2 wrote:
| > I don't comprehend how people can develop on windows
|
| They want to distribute their app on Windows?
| pletnes wrote:
| That's why, not how. And you can cross-compile from Linux
| for some stacks.
| kitkat_new wrote:
| For me compiling usually is something like `cargo build`,
| `ng build`.
|
| I remember having problems with libs that require
| installing & registering a library somewhere such that
| CMake can find it. However, I distanced myself a bit from
| C(++), so that doesn't really happen anymore :)
|
| I avoid mingw, don't use any package manager besides
| Windows Store (if you want to call that a package
| manager).
|
| Can't complain. Sometimes, there is stuff that simply
| doesn't support Windows -> WSL. When there is docker, it
| doesn't matter anyways...
|
| My strategy is don't fight Windows and you'll be happy
| jeeeb wrote:
| From your description it sounds like you might be going
| off the beaten track and hitting problems.
|
| When I was doing C++ on Windows getting a dev environment
| setup just meant installing Visual Studio with an
| appropriate Windows SDK version (or the Windows SDK +
| build tools for a build system).
|
| You can have multiple VS versions installed side-by-side.
| To get a terminal with environment variables set
| correctly you just need to use the shortcuts from your VS
| installation.
|
| For third party dependencies we checked the headers and
| (pre-built) binaries into the repository. I don't
| remember ever having more than a dozen or so in total. It
| was usually things like boost and zlib.
|
| Having done that you can just point CMake directly at the
| packages rather than worrying about FindPackage.
|
| Working in tools like Python and Node, personally I often
| miss the simplicity and stability of this approach.
| JohnHaugeland wrote:
| It depends a huge amount on the language and toolchain.
|
| MSVS is really quite nice.
| y4mi wrote:
| It's usually not by choice. Medium to large enterprises
| often demand this so they can manage the employee
| hardware.
| arwineap wrote:
| If you show up to the job with a windows box I expect you
| to know how to do the job in windows
|
| It's the same problem with docker-compose files; how do you
| expect the developers not running in windows to fix your
| windows problems?
| leephillips wrote:
| Are most people who would have a use for a tool like Make
| on Windows?
| pletnes wrote:
| Well, most developers I work with benefit from a task
| runner. Now, we use different runners for different
| repos. I'd prefer being able to go into a repo and do
| <<make test>> and have it work, regardless of language,
| framework, purpose of the repo.
| tom_ wrote:
| You can put a copy of make in the repo.
| matt123456789 wrote:
| For added stability, make sure it's statically compiled
| (which it might be already...)
| tom_ wrote:
| I've been using GNU Make 4.2.1 from here:
| https://github.com/mbuilov/gnumake-windows - depends only
| on kernel32.dll. I assume the latest 4.3 build there is
| the same.
| kernelsanderz wrote:
| I'm wondering if you could compile make to wasm and
| include it as a cross platform dev dependency?
| andreineculau wrote:
| I have never bumped into that article before, thanks for
| that.
|
| I did implement though that uniform interface in
| https://github.com/ysoftwareab/yplatform
| ptrvldz wrote:
| And it still holds true for development today! Really any
| organization that has multiple languages should consider
| making their devs lives easier with Make.
|
| And of course, it plays great with Docker & Compose, here's a
| write-up we did on using Make with container tooling:
| https://shipyard.build/blog/makefiles-for-modern-
| development...
| synergy20 wrote:
| I use makefile for all my builds.
|
| cmake can help portability across OSes but it has a layer of its
| own complexity, I mostly work on linux alone, makefile seems more
| than enough.
|
| google etc produces new tools to build its huge code base but I
| don't have that large scale source code, makefile so far worked
| well for my scale.
|
| unless you have specific needs I feel makefile will do just fine.
| it's simple, readable, manageable, and get the job done fast.
| gjvc wrote:
| I'm pleased to see this being espoused. One thing I always tell
| junior people is ""don't be tempted to think of make(1) as out-
| of-date [1], and to view packages available via npm etc as better
| replacements, as many tasks fit the model of make(1), if not the
| title "a program for directing recompilation" "" I am often
| viewed with suspicion until I show them a (much more compact)
| replacement for their custom python/js program expressed as a
| makefile without any of the process management requiring
| debugging. :-)
|
| An alternative approach for the "batches of files to process"
| situation is to generate the commands and feed them to GNU
| parallel for execution.
|
| [1] no pun intended
| nicoburns wrote:
| Agreed if you only have to support linux/unix, but the one good
| thing about the Node ecosystem is that they tend to work on
| windows too.
| verelo wrote:
| While i agree, I've spent literal days fighting with node
| modules in recent months. Sure make isn't perfect either but
| it's a lot less, confusing? Ie json isn't a language for
| humans, but often you really need to read it to correct some
| issue or bad merge from another contributor.
| nicoburns wrote:
| > Sure make isn't perfect either but it's a lot less,
| confusing?
|
| I think that's a matter of what you're used to. I don't use
| make all that often, so I find make files beyond a few
| lines quite a lot to get my head around, whereas I spend
| all day writing JavaScript so I can read/write JSON in my
| sleep.
| gjvc wrote:
| not the point, also "WSL"
| throw0101a wrote:
| http://gnuwin32.sourceforge.net/packages/make.htm
|
| https://community.chocolatey.org/packages/make
| Steltek wrote:
| Make has worked on Windows longer than Node.js has existed.
| nicoburns wrote:
| Make works fine, it's the commands you call from make that
| don't tend to as compatible. For example, try running rm
| -rf on windows. It might work if you have GNU tools
| installed, but it certainly won't work out of the box. On
| the other hand, the node.js `rimraf` package will do the
| same thing with no cross-platform compatibility issues.
| Steltek wrote:
| When using package.json, does npm intercept a script
| containing "rm -rf" and translate it to something
| different?
| nicoburns wrote:
| No, but it does put binaries from any installed node
| packages in your PATH (locally when running scripts).
| geewee wrote:
| No, that's why there's a bunch of packages such as
| rimraf[0] that implements that sort of functionality in a
| cross-platform way that most people use in their scripts
|
| [0]: https://www.npmjs.com/package/rimraf
| jchw wrote:
| The history of Make on Windows would Make one think twice
| about relying on it. What shell do you get when running
| Make on Windows?
| andrew_ wrote:
| Might it not be a disservice to juniors to point them away from
| deeply understanding the platform they're working with?
| gjvc wrote:
| Showing someone an alternative approach does not prevent
| someone "deeply understanding" their current method; it's not
| zero-sum.
| jjice wrote:
| Biggest pain of package.json to me is having to be crammed on
| one line. Slap in a conditional and now we have a messy thing
| to read, compared to its multiline counterpart.
| kall wrote:
| One thing I appreciate about the javascript ecosystem and
| "scripts" is the focus on keeping everything contained in the
| project. The only assumption about the system is that it has
| node, from there it's npm i, npm start. Scripts are a way to make
| sure people always run the project-local version of various
| development tools and not whatever they happen to have installed.
|
| When I see a makefile, I expect they will be making more
| assumptions about my system and expect a little more work.
|
| I do appreciate the "has worked fine for decades" aspect of
| makefiles, and I guess docker solves some of this, but I still
| prefer a fully self contained javascript project.
| des429 wrote:
| Yeah this seems to miss some of the biggest features of npm
| scripts. Access to the project's local packages is probably the
| biggest feature not mentioned IMO.
| throwawaycuriou wrote:
| npx to the rescue
| andreineculau wrote:
| When you say everything contained in the project, you forget
| that there's more to life than just node/javascript. Just two
| silly examples: you cannot run/deploy your API backend without
| docker/kubernetes/aws-cli/etc, nor your mobile app even if it's
| in JavaScript (react-native).
|
| The only way to keep to node/javascript is if you only develop
| libraries (a website that doesn't get deployed -> still on
| library level).
| kall wrote:
| That's true, kind of. If you have those kinds of tasks, I see
| the appeal of the makefile over various commands in
| package.json that won't run after a simple npm install
| anyway. In practice, you may be building your react native
| app with EAS Build and deploy your website/api to Vercel or
| to AWS with pulumi. That's all still npm dependencies.
| gedy wrote:
| API backends/middleware can be Node.js too
| yurishimo wrote:
| To add one tiny thing to this, it's absolutely imperative that
| your project be setup with safeguards for node version
| dependencies. I can't tell you how many projects I've joined
| that don't even have an nvmrc file, much less some guard to
| ensure a clean install and build.
| Mister_Snuggles wrote:
| I use make to run daily/hourly/etc batch processes in the ERP
| system at work. There's a Python shim to actually run processes
| in the ERP, plus a shell script to get things ready to run and
| send success/failure emails, and cron to kick batches off, but
| other than that it's pure make. In theory I should also be able
| to pass '-j 4' to make and have it run multiple processes at
| once, in the correct order with their dependencies properly
| satisfied, but I haven't actually tried that yet.
|
| It's clearly an abuse of a build tool, but it's also a testament
| to make's incredible flexibility.
|
| I've also heard of someone who replaced their Linux startup
| scripts (in the pre-systemd days) with make to speed up boot
| time. That's actually what inspired my batch schedule work.
| limonkufu wrote:
| We are using make targets to automate ci/cd, local dev and a lot
| of other things by adding a set of easy to extend, customise
| template makefiles as submodules.
|
| The main advantage for us is it's same everywhere and it's
| agnostic to underlying technologies. (we are only supporting unix
| based environments and have guidelines to setup gnumake in
| macos). By using this way, we ensure that the general SDLC is
| same across different repositories and underlying tooling can
| change anytime without inducing a lot of refactoring
|
| We are also using make targets to document themselves. How to
| use, what are other targets, variables you need to define etc.
| This makes it a powerful CLI app for us that can run and support
| many things.
|
| For anyone interested: https://gitlab.com/ska-telescope/sdi/ska-
| cicd-makefile and the similar parsing method for self documenting
| makefiles: https://gitlab.com/ska-telescope/sdi/ska-cicd-
| makefile/-/blo.... We didn't know about magicmake that's linked
| here that does the similar thing
| jsz0 wrote:
| Back when I was doing network engineering I used make to localize
| configs from boilerplate code templates. Not a common use for it
| or maybe even the best tool for the job but because I already
| knew Makefiles it was an easy solution for me to implement. Saved
| me endless hours of tedious error prone text editing. Just an
| example of why it's worthwhile to learn how to use these
| fundamental tools that have survived the test of time. Even if
| you don't have a specific use for it today it's a very useful
| tool to have in your toolbox.
| ThePhysicist wrote:
| I'm also a fan of make but whenever another frontend dev starts
| working with one of my projects the first thing they do is rip
| out the perfectly good Makefile and replace it with npm scripts
| (often introducing errors). I guess it's a generational thing.
| molszanski wrote:
| Here is Makefile "starter" I use:
| https://github.com/awinecki/magicfile
|
| People call this "self-documenting makefile".
|
| It migrated with me from company to company, from project to
| projects. Through node, php, aws, docker, server management, cert
| updates, file processing and many many more.
| rbongers wrote:
| The problem I have with Make is that it has a syntax that most
| web developers I have worked with would find archaic and
| unfamiliar.
|
| On some projects I've worked on, it's been used as basically a
| command alias system that only a few people can maintain. None of
| its caching or dependency chain capabilities were utilized.
|
| In these cases, shell scripts would have been a better option,
| and in some cases were later introduced with success.
| ravenstine wrote:
| I too have been using Makefiles in both my C++ and Deno projects
| and have been very happy with the unity it provides.
| silves89 wrote:
| I have very similar use-cases, and I also I found makefiles a bit
| limiting. I wrote lk[1] to make this sort of thing easier, and
| you can just write plain bash functions instead of makefiles.
|
| [1] https://github.com/jamescoleuk/lk
| Cthulhu_ wrote:
| I've only started using a Makefile ~two years ago when I started
| a big Go project at work, and I quite like it. There's a few
| caveats here and there - Makefile specific syntax, shell-script
| specific syntax, and whether I should put .PHONY in front of
| every command (I guess 'no unless you have a file with the exact
| same name'?), but it's quite compact and straightforward.
|
| The most complicated command I have is something that removes a
| folder or set of files, invokes the swagger generator with a list
| of options, copies it to a target folder, and passes it through a
| formatter/processor. But it's very straightforward, and it's
| "just" shell script, not shell script wrapped in a JSON document,
| or some Java tool invoked indirectly through an XML configuration
| file like back in the day with Maven.
| JamesSwift wrote:
| If using Make as a task runner and not build cache, I just put
| `.PHONY: %` at the top which means everything is marked
| `.PHONY`
| nablaone wrote:
| Thanks!
| spion wrote:
| Why I prefer package.json over makefiles:
|
| - turborepo (https://turborepo.org/) can describe dependency
| pipelines and provide automatic caching. Makefiles aren't
| designed for multi-input, multi-output scenarios - its really
| awkward:
| https://www.gnu.org/software/automake/manual/html_node/Multi...
| and https://stackoverflow.com/questions/39237306/makefile-
| compil...
|
| - turborepo can also run never-ending targets in parallel (e.g.
| API server and static file server in development mode). Not sure
| how well make supports that
|
| - turborepo can depend on env var values. Makefiles can to, but
| like everything with makefiles, its an awkward workaround:
| https://stackoverflow.com/questions/14840474/make-target-whi...
|
| - makefiles do not work on Windows (They are portable, just not
| to platforms most node devs care about)
|
| - Its unclear whether `make` will ever add features to remove the
| awkwardness for supporting the various scenarios above. It
| doesn't seem likely to happen.
| spion wrote:
| Also, it says something about the design of `scripts` that its
| so easy to build something like turborepo's dependency pipeline
| on top of it :)
| mark_l_watson wrote:
| I also use Makefiles, been using them for 40 years. I find that
| Makefile targets for misc. things like grabbing remote training
| data, running tests, building documentation, etc., etc. augments
| information in READ files, that is in addition to functionally
| saving time, serves as documentation when I haven't looked at a
| project in 6 months.
| deepsun wrote:
| What are the benefits of Makefiles compared to a bunch of shell
| scripts?
| elchief wrote:
| Resumability is the main one, in my mind
|
| Imagine a simple two-step process. The first takes a long time.
| The second one fails on occasion. You don't want to run the
| first one again. With Make, if the first one succeeds, it won't
| run again unless you `clean`
|
| Easy, but now imagine many, many steps
| deepsun wrote:
| Makefile just checks whether a file exists, nothing more.
|
| if [[ -f myfile ]]; then
| Izkata wrote:
| It also compares the last-modified timestamps of the source
| and target files so it only runs if the target is out of
| date. Plus because the dependencies are explicit, it's
| trivial to run in parallel the parts that can be.
| JamesSwift wrote:
| A single, system-and-language-agnostic entry point to the
| actions you want to run with very clear dependency rules built
| in.
| enriquto wrote:
| That you get parallelism and partial re-running for free.
| andrew_ wrote:
| parallelism is common in package managers these days
| https://pnpm.io/cli/run#--parallel
| monocasa wrote:
| No, make understands the directed acyclic graphs of
| dependencies, regardless of language, and will walk that
| graph in parallel. No forcing or ignoring sorting, and no
| enforced package level blocking on scripts.
| immibis wrote:
| It's a shame this doesn't work for Java in particular
| (not JS) as Java's coding conventions/requirements do not
| lend themselves at all to manually updating a list of
| file dependencies.
| mathgorges wrote:
| I've successfully used both together before :)
|
| The makefile lets me be very expressive about task dependencies
| and encode some developer conveniences. Ie make can know `make
| bootstrap` should invoke bootstap_windows.sh on Windows and
| bootstrap_osx.sh on Mac. Then the work happens inside the
| appropriate script.
|
| They're two tools that solve different problems but they work
| together quite well.
| andrew_ wrote:
| Stick around in any programming ecosystem long enough, and you
| live long enough to see strategy, taste, and "discovery" come
| full circle. We had make, then package.json "scripts," then
| grunt, then gulp, then the trend shifted back towards
| package.json "scripts." (Note: Using "scripts" to differentiate
| the package.json property versus the generic word use)
|
| I like that the author isn't being authoritative, but there could
| be some additional due diligence. I ran the make-is-faster loose
| benchmark via PNPM and runtime was 0m0.022s for make and 0m0.012s
| for PNPM. If I care about those 10ms, PNPM is my horse. Yarn is a
| glacier compared to PNPM.
|
| Another thing I would've liked to see comment on is the automatic
| pre and post paradigm that package.json "scripts" affords. The
| big three package managers all support pre and post, and it makes
| arranging "scripts" a breeze, it breaks down dependent steps into
| separate console output, and is generally easy to organize imho.
|
| All in all a nice write up for folks who might not really like
| package.json "scripts" to begin with, or for those who'd rather
| not gain more granular insight into how "scripts" works, but I
| don't see this being the nail in the coffin case against them.
| crate_barre wrote:
| What in god's name is pnpm?
| qayxc wrote:
| > What in god's name is pnpm?
|
| Take a look: https://pnpm.io
| andrew_ wrote:
| https://pnpm.io/ it ships with Node.js along side npm and
| yarn these days, has for about a year I believe.
| Guillaume86 wrote:
| What's your source on pnpm/yarn shipping with node? It
| doesn't in my experience.
| p8123 wrote:
| `corepack enable`
|
| https://nodejs.org/api/corepack.html
| mhio wrote:
| The caveat here is that the binaries aren't "shipped"
| with node.
|
| `corepack enable` adds shim scripts to node's $PATH to
| intercept calls to `npm` `yarn` or `pnpm` and downloads
| the binaries from a URL without much checking along the
| way. This functionality has had some vocal opposition.
| sigzero wrote:
| It's a replacement for NPM and YARN. Never used it myself.
|
| https://pnpm.io/
| FractalHQ wrote:
| I've used pnpm religiously for over a year and have seen it
| being adopted as a golden standard replacement for npm in my
| circles. Highly recommend it to anyone who works with
| JavaScript or Typescript!
| progx wrote:
| But why? Ok it is faster and save storage. Is that really a
| huge problem?
| [deleted]
| boondaburrah wrote:
| For a lot of the same reasons in this article I like to use
| tup[0] when I can. It doesn't integrate into anything which is
| both good and bad. I wish my IDE could check with tup to see what
| dependencies get pulled in by what files. However, it's nice that
| it doesn't care what language or ecosystem I'm using.
|
| Also, it's very strict about declaring dependencies properly, and
| will actually fail the build if you've set things up in a way
| that depends on something not tracked (by watching filesystem
| access as the compiler runs). That gives me warm fuzzies that my
| builds are reproducible.
|
| Also I can get a neat dependency graph as a PNG if I want.
|
| [0] https://gittup.org/tup/
| aitchnyu wrote:
| I was amazed to discover Taskfile. Didn't realise it was a copy
| of Make.
|
| https://github.com/adriancooney/Taskfile
| BeefWellington wrote:
| Over here in crazytown I'm using Makefiles to automate system
| deployments.
|
| I thought it was insane at first but actually I've come around to
| it; Make hits that sweet spot of being ubiquitous, simple, and
| flexible. There's a little bit of a learning curve but once
| you're over that initial hurdle it's pretty straightforward,
| especially for one-way operations, e.g. install without
| uninstall, build without clean, etc.
| nsonha wrote:
| At one point I tried to do the same thing but then I realized
| there isn't any make feature I need to use. The key thing about
| make is to avoid building already built artifacts, if you don't
| need that you end up with things like .PHONY. There is'nt
| anything in make that helps with composing or showing a list of
| scripts etc either. At the end I ended up with plain javascript
| scripts for js project and plain bash scripts for everything
| else.
| [deleted]
| davistreybig wrote:
| Earthly is an interesting open source project in this space that
| offers many of the benefits of a make file plus containerization
| plus some of the performance benefits of a Bazel/Pants.
|
| https://earthly.dev/
| benatkin wrote:
| Believe it or not, it isn't an open source project.
| https://github.com/earthly/earthly/blob/main/LICENSE
| adamgordonbell wrote:
| It's BSL which is source available with the one restriction
| being you can't build a commercial competitor. And the source
| becomes MPL2 after 3 years.
|
| https://earthly.dev/bslfaq
| gkbrk wrote:
| What you said does not contradict what the parent comment
| said. You're both in agreement that it is not open source.
| andreineculau wrote:
| https://github.com/ysoftwareab/yplatform that this a notch higher
| (disclaimer: author here)
|
| As others have mentioned on this thread, the problem is not just
| "use Makefiles instead of package.json scripts", but quickly
| ending up with duplicate Makefile content and then supporting
| more than just one language. It's inevitable.
|
| PS
|
| for those that don't know, npm's "scripts" functionality as we
| know it today was not "designed", but it was simply a byproduct.
|
| First introduced as an internal functionality for lifecycle
| support
| https://github.com/npm/npm/commit/c8e17cef720875c1f7cea1b49b...
| on 1 Mar 2010
|
| and later extend to run arbitrary targets
| https://github.com/npm/npm/commit/9b8c0cf87a9d4ef560e17e060f...
| on 13 Dec 2010
|
| This is the entire design backstory
| https://github.com/npm/npm/issues/298 .
|
| Needless to say that GNU Make's equivalent (or any other build
| system) is not to be found in npm's scripts.
| nwsm wrote:
| package.json scripts are definitely a bit out of place. I think
| their prevalence stems from popular npm projects using the
| pattern for easy cross-platform developer experience. I've seen
| some shell scripts in opensource node projects, but not much
| makefile, and I actually had an interviewer once joke that
| makefiles were old and out of style.
|
| But as I have worked with less Windows developers over time, my
| projects have relied more on shell scripts and makefiles. I think
| the containerization movement has started to shift software
| development towards Linux-based tooling as well. As a cross-
| platform effort this is a bit ironic, but having overlapping
| developer experience and CI/CD tooling is pretty convenient when
| you're on a Unix kernel or WSL2.
|
| However, much of the npm ecosystem and community are focused on
| frontend work that often does not involve infrastructure-related
| tooling. Without the absolute need for any OS-specific tools, and
| with the many composable crossplatform scripts in vogue already,
| package.json scripts make sense (until they get out of hand).
| mc4ndr3 wrote:
| (GNU) Make's advantages include ease of use, a modicum of
| portability, and support for a variety of different software
| development command line stacks.
|
| Unfortunately, the shell commands that are typically setup in a
| Makefile, are rather unportable. They waste a touch of time and
| space. They can break in surprising ways.
|
| For this reason, I try to write my build tasks in terms of the
| same programming language as the application language. For
| example, Grunt for JavaScript projects, Shake for Haskell
| projects, Gradle for Groovy projects, Rez for C/C++ projects,
| Dale for D projects, TinyRick for Rust projects, and Mage for Go
| projects. Leiningen for Clojure projects, SBT for Scala projects.
| Vast for shell projects.
|
| Make, like Python, suffers from the appearance of simplicity, at
| the cost of long term maintainability. Ideally both the build
| system and the main application are compiled. That doesn't have
| to be a cumbersome process, but rather a guide to identify
| potential bugs sooner during development.
|
| Make is my first choice for random projects, but only when better
| build systems are unavailable.
| SkyPuncher wrote:
| I don't have a problem with make files, but I do prefer to work
| directly with `package.json` when reasonable. Most notably, most
| of the IDE's work very well with `package.json`, but don't always
| handle make as cleanly.
| the-alchemist wrote:
| For the clojure crowd, there's bb tasks [0].
|
| * parallelism
|
| * hooks (before/after each task, etc.)
|
| * command-line arg parsing
|
| * --help equivalent
|
| * bash/zsh/fish autocompletion
|
| * you can also just spawn a shell with bb/shell
|
| * yes, dependencies
|
| * regular programming language (Clojure), with commonly used
| shell functions like glob()
|
| * import code from locations (don't need to copy/paste between
| Makefiles)
|
| * built-in JSON and CSV support, so you can use your app's JSON
| files right in your build file
|
| [0]: https://book.babashka.org/#tasks
| yboris wrote:
| Just wanted to share _nps_ as a nice way to manage scripts for
| your _package.json_
|
| https://www.npmjs.com/package/nps
|
| You can create a dedicated JS file (with comments and more!) that
| will house your scripts, which you can run by using "nps" instead
| of "npm" as a command.
| devn0ll wrote:
| May I also direct your attention to self-documenting Makefiles:
|
| https://www.thapaliya.com/en/writings/well-documented-makefi...
| molszanski wrote:
| Here is the one I use and recommend:
| https://github.com/awinecki/magicfile
| jitl wrote:
| I like Make, and use it in personal JavaScript ecosystem projects
| that do actually need to build things with interdependency (eg
| https://github.com/justjake/quickjs-emscripten/blob/master/M...).
| I've also seen Makefiles grown to be horrendous monstrosities
| masquerading as command-line tools; for about a year at Airbnb we
| used a 1000+ line Makefile as the main tool for fiddling with
| Kubernetes cuz one of our senior engineers didn't like Ruby, and
| I've seen another one get close to that level of cravenness.
|
| What I learned from supporting Make and shell among a few
| different audiences is that most developers have no interest in
| how to write or maintain shell-like tooling. They forget or mess
| up quoting rules constantly, and eschew learning things and good
| design in these tools to a much greater degree than in their
| "normal" work in Java/Python/Ruby/Typescript/Golang.
|
| For every POSIXly Correct HN Commenter (of which I count myself a
| member), there's 100 regular software engineers who won't read a
| `man` page on what $@ or $< mean in Make. I know that if I start
| writing a Makefile for $JOB, it's gonna be me and that one guy
| who uses tcsh who are gonna maintain it and answer questions.
|
| (Although for what it's worth we don't use package.json scripts
| either thank goodness. All our complicated build steps are
| typescript commands, and our glue is CI system YAML files.)
| unqueued wrote:
| I would try to use JavaScript tooling in JavaScript projects,
| especially if I share it with other developers.
|
| I would love to see better handling of filename transformation,
| file watching, and parallelization.
|
| A few times over the years I had given up and just used Make
| instead of Gulp or Grunt. Most recently, I wanted to watch if
| files had been deleted (if directory entries had been
| modified). It ended up being a few lines of Makefile.
|
| And once you understand the syntax, it is expressive and
| simple, and way more portable. There was some good discussion
| awhile ago on this[1]:
|
| [1]: https://news.ycombinator.com/item?id=7622296
| jgrahamc wrote:
| Reminds me of something I wrote years ago:
| https://blog.jgc.org/2010/11/things-make-got-right-and-how-t...
| jitl wrote:
| Has anyone Fixed Make in a way you find good and right since
| you wrote that?
|
| (My last few forays into the build systems domain have mostly
| turned up "Bazel but for X audience" so I can't recall a
| satisfying solution)
| mro_name wrote:
| one of the few annoyances is spaces in paths. Just avoid them
| anyway, right?
| beej71 wrote:
| This is one of the few fixes I'd really like to see. I'm from a
| Unix background so I don't tend to have spaces anywhere, but
| it's still a splinter that needs removing.
___________________________________________________________________
(page generated 2022-03-14 23:02 UTC)