[HN Gopher] Why Use Make (2013)
___________________________________________________________________
Why Use Make (2013)
Author : beardicus
Score : 65 points
Date : 2023-01-11 19:03 UTC (3 hours ago)
(HTM) web link (bost.ocks.org)
(TXT) w3m dump (bost.ocks.org)
| blacklight wrote:
| The Makefile syntax is beyond cryptic. And I'm speaking as
| someone who used the autoconf/automake chain for years to build
| software. The author only made a simple example that requires
| downloading, transforming and uploading a file. Try and do
| something a bit more complex, deal with m4 macros, maybe the
| autoconf syntax, and you'll feel like you're back in the 1970s.
|
| Don't get me wrong: make has been a loyal friend for most of my
| life. I've built a lot of software using make and the whole auto*
| build chain. And the author doesn't even mention it's biggest
| strength point: portability. Every single machine with a UNIX-
| like system has make. But I have to admit when some technology is
| ready for retirement. More modern alternatives exist today. Cmake
| is the most obvious, Bazel is also a promising one. Even simple
| shell scripts can do the job for easy things.
| convolvatron wrote:
| please don't perpetuate the idea that the auto tools are
| somehow an extension or part of make. autotools is a frankly
| insane model of nested scripts that attempts to be magic and
| fails. make is a really interesting declarative scripting
| framework that collapsed from a lack of minimal language
| features a long time ago.
|
| not the same.
| AshamedCaptain wrote:
| The make syntax is basically as simple as it gets; it's grammar
| is even shorter than the JSON spec itself... what you do with
| it is another story, but you don't have to use autotools.
|
| > Try and do something a bit more complex, deal with m4 macros,
| maybe the autoconf syntax, and you'll feel like you're back in
| the 1970s.
|
| I have been in the club from those who used the word
| "Autohell", and while I have used it I would most definitely
| _never_ write "GNU style" Autoconf script that would check even
| the system's max supported argv length; but frankly, just
| everything that has come since these "1970s" is just plain
| worse. These days I'm lucky if the build system doesn't have me
| learning languages that are _significantly_ more complex than
| m4 (e.g. bastardized copies of Tcl), or even truly general
| purpose languages like Python or JS. Have you tried debugging a
| failing "yarn build" for a random project recently ? It really
| makes me wish for the Autohell days.
| horsawlarway wrote:
| I think this is a cop out.
|
| Simple grammar does not mean easily understood syntax -
| particularly given that make is essentially dependent on the
| entire shell it's operating in (at a minimum, usually GNU
| tooling).
|
| Plus - At least in my experience it tends to deteriorate into
| many developers playing "code golf" with the commands: using
| obscure flags and inscrutable commands to save a few lines in
| the file.
|
| Worse, while splitting into several makefiles is possible - a
| lot of the tooling stops working particularly well (ex:
| autocomplete no longer works on tasks that are "included")
|
| Basically - I find that the only consistent way to use
| makefiles on a moderately sized team is a rather strict "one
| script per task" rule, with the script ideally written in the
| same language as the project. But at that point I might as
| well just use a native task runner in that language anyways.
| dwheeler wrote:
| I haven't had that problem, even with big Makefiles.
|
| I think the key is to acknowledge that it's a program. Just
| like any other program, instead of writing something that
| looks complicated, write it in a way that's easy to
| understand later. In particular, no code golf please.
|
| Also... just use GNU make, and use its extensions. The
| POSIX specification is so impoverished that you end up with
| complicated code (it has no functions for example). There
| are other tools, but I find make often does the job.
| ablob wrote:
| The syntax of lambda calculus is quite simple as well.
| However, I find it hard to say that any project of reasonable
| size would be anything but cryptic.
|
| A short grammar does not imply that the resulting language is
| comprehensible.
| anthk wrote:
| A fast guide to make: git clone
| git://bitreich.org/english_knight cd
| english_knight less english_knight
|
| Enjoy.
| jitl wrote:
| The best thing about make is that it's available _everywhere_ on
| POSIX systems. macOS comes with it pre-installed, and every Linux
| distro or *BSD does too, or offers a package for it that's often
| depended on by everything else devtools-wise. This means make
| skills are super transferable; possibly more so than bash /shell
| skills.
|
| Sure there are more user-friendly tools written in the
| programming language du jour, but those will always need some
| special snowflake setup process for all of your developers. If
| you're cross-platform, now you need encode that in a setup script
| before your project can build... Maybe you can offer a Makefile
| just to install `better-make`?
| blueblob wrote:
| I think Windows comes with nmake
| asveikau wrote:
| Nope, you have to install developer tools to get it. Usually
| that means visual studio. Historically they also ship
| compilers and command line dev tools as a separate package,
| I'm not sure if that's still a thing.
|
| Nmake is also not very compatible with gnu or bsd make.
| mkesper wrote:
| Beware, the macOS version is historic (as is macOS bash
| version).
| nine_k wrote:
| It's true, but the compatibility is limited.
|
| I had to resort to tricks to make Makefiles which work both on
| Linux and macOS. I had to mandate the use of GNU make on Macs
| in other cases.
|
| This is on top of having BSD vs GNU coreutils for things like
| grep, awk, etc, and the ancient bash on macOS.
|
| I really wish there was a self-contained tool that could work
| like make (building a dependency graph and only doing the
| needed things), with reasonable string / list / map processing
| built in. (In a limited way, GNU make is such a tool.)
| oehpr wrote:
| Maybe tup would interest you?
| https://gittup.org/tup/examples.html I've been considering it
| for the next time I do a project that needs this kind of
| ordered construction.
|
| tup comes packed with a lua parser that gets executed first,
| so if you need something fancy it can be expressed in a lua
| file with lua's tools
| https://gittup.org/tup/ex_lua_examples.html
| nine_k wrote:
| Tup is interesting, especially its dependency discovery
| mechanism.
| ddulaney wrote:
| Does ninja fit your needs? It's available on just about all
| of the Linux distros and it's extremely fast with very few
| bells and whistles. The language is (for better or worse)
| designed to be generated by a higher-level tool, so it strips
| out most of the complexity of GNU make, but it might go too
| far if you're looking to do list/map processing in it.
| hedora wrote:
| People always claim ninja is fast, but I can't figure out
| what they mean by that. A typical C++ project build uses
| 10-10,000 CPU core minutes, and make takes (maybe, in some
| pessimal situation) 100 milliseconds to schedule and
| coordinate the build invocations.
|
| Even if ninja is 100x faster, it really, really doesn't
| matter, at all.
| stabbles wrote:
| If you have a slow filesystem and you're compiling C,
| make has a "lot" of overhead due to implicit rules. There
| are many more rules than you write, unless you set stuff
| like .SUFFIXES:
|
| to nothing
| aseipp wrote:
| Ninja isn't necessarily faster in the slow path of
| "rebuilding the whole project" vs make, but it's often
| significantly faster in the fast path of "incremental
| rebuild given a small change in the input code" vs make.
| Which is what you're doing most of the time. You also do
| not have to abuse ninja for it to record certain changes;
| for example tracking CFLAGS as a dependency in Make can
| be awkward (e.g. write it to a file and all the
| associated overhead from the filesystem), but in Ninja
| it's "just" a variable binding, and the usage of
| variables in commands is tracked as a dependency, and so
| changing that variable and re-computing the needed set of
| commands to run is much, much faster. Those things add up
| in large builds.
|
| I have personally had Ninja turn multi-second long no-op
| rebuild times (e.g. run 'make' with no changes) into the
| 10s of milliseconds range. The no-op build is often the
| most extreme case but closest to the average case, which
| is "recompile after a small edit." The difference in
| interactivity is quite large in these scenarios.
|
| If your project builds in under like 5 minutes from
| scratch on a modern laptop it probably is not large
| enough to see _huge_ benefits (outside of pathological
| cases), but probably some benefit; but for larger
| projects the difference can be very pronounced, very
| quickly.
| drothlis wrote:
| They probably mean incremental builds, where the actual
| compilation doesn't overshadow the "coordination" work.
|
| In my (artificial) benchmark, make scaled poorly, taking
| 70 seconds to process 100k C files worth of dependencies,
| vs. ninja's 1.5 seconds: https://david.rothlis.net/ninja-
| benchmark/
|
| Most of make's time was spent processing the ".d" files
| containing header dependencies (Ninja has a special
| optimisation for these files, where it reads them the
| first time they're created, inserts the dependency
| information into a binary database, then deletes them so
| it doesn't have to parse them in future invocations).
|
| In real world projects, you often end up "abusing" make
| to add behaviour such as detecting if the compilation
| flags have changed, and this can make your makefiles
| slower; whereas ninja has those features built in.
| Apparently this made a big difference in build times for
| Chromium (where ninja was born). See this comment by the
| ninja's author:
| https://news.ycombinator.com/item?id=23182469
| nine_k wrote:
| If you build something much smaller, a difference betwen
| 5 sec and 0.5 sec is pretty noticeable for interactive
| work, even though 5 sec is not prohibitively long at all.
| bxparks wrote:
| The macOS version of GNU Make is stuck at 3.81, which I
| discovered does not print the information needed by vim's
| quickfix feature when traversing subdirectories using `make
| -C`. Installing the latest version of GNU Make (4.4) using
| `brew install make` fixed that problem.
| saurik wrote:
| The frozen macOS build also has some weird issue with my
| makefiles where it sometimes finishes the job but then sits
| there spinning at 100% CPU forever :(.
| rr888 wrote:
| Windows has always been a problem. Now lots of build tools work
| better cross platform than make.
| asveikau wrote:
| I use msys gnu make on windows for personal projects.
| jondeval wrote:
| An additional reason that drives me to use make on most new
| projects is the polyglot nature of many code repos. There are
| language/ecosystem-specific build tools: grunt, rake, etc. but
| often real world projects are a mix if different languages and to
| double down on just one language-specific tool feels
| unnecessarily constraining.
|
| Having a build tool like make that is more closely aligned with
| the system level feels more natural for orchestrating
| build/test/deploy tasks that by their very nature contain more
| cross-cutting concerns.
| Mister_Snuggles wrote:
| There are also some other 'interesting' ways to use Make.
|
| As a batch runner: https://news.ycombinator.com/item?id=32441602
|
| As an init system (which inspired my use of it as a batch
| runner):
| https://web.archive.org/web/20110606144530/http://www.ibm.co...
| dahfizz wrote:
| One big reason I rely on Make: its old. Make has been well
| maintained for nearly 50 years, and is deeply integrated into the
| programming ecosystem as a whole.
|
| I can start a new project that relies on Make, and be extremely
| confident that Make will continue to work and be maintained for
| the lifetime of my project. 20+ years from now, Make will still
| work. My Make knowledge will be relevant for my entire career.
|
| New and shiny replacements like Just are tempting, but you have
| to consider the real cost of:
|
| 1) Learning a new build system
|
| 2) Onboarding new devs with an unfamiliar build system
|
| 3) Dealing with the eventual deprecation of the new shiny, once
| something newer and shinier comes out. Rewrite your build
| scripts, GOTO 1.
| stabbles wrote:
| Do note that GNU make is still being developed. It even had
| breaking changes in the recent 4.4 release. Also GNU make 4.3
| has at least one bug (e.g. on Ubuntu 22.04) where
| tgt: gcc hello
|
| errors when you have a directory with the name gcc in your
| PATH. (This is a bug in gnulib, that's fixed, but gnulib is a
| rolling release typically vendored by every project on an
| arbitrary commit, sigh)
|
| Also system GNU make on Ubuntu 20.04 has a bug in ---output-
| sync. It doesn't sync.
|
| And finally GNU make on macOS is ancient by default. Like over
| a decade old. So what works on Linux may not always work on
| macOS.
| hedora wrote:
| I've yet to find a shiny replacement that is half as well
| thought out as make.
|
| I'll look at Just, but is there anything else actually in
| make's space (scripting language agnostic, target language
| agnostic, auto parallelized, declarative, ergonomic syntax that
| is not xml, json, etc).
|
| Many things claim to be a make replacement, but don't meet
| those basic requirements.
| imoverclocked wrote:
| There is nothing as general purpose as make because make
| already fills that space well. OTOH, there are special
| purpose replacements that do a much better job in their own
| domain.
|
| eg: I wouldn't want to maintain a _large_ Java project with
| make. Gradle has a lot of helpful built-ins for managing the
| complexity of subprojects /dependencies and also runs on the
| JVM which makes debugging for Java-programmers a little more
| in-reach than make.
|
| All of that being said, I do use make for some fun personal
| projects. Automatic failure (a more-powerful "set -e"
| equivalent in common shells) and dependency tracking mean I
| can write really powerful automation scripts which can easily
| parallelize for slow steps.
| zyedidia wrote:
| I've been working on a tool called Knit
| (https://github.com/zyedidia/knit) that I think is similar to
| what you are looking for. Essentially, a Knitfile is a Lua
| program with Make's declarative rule syntax baked in. Or in
| other words, it is like Make (with some additional changes
| inspired by Plan9 mk), but where Make's custom scripting
| language is replaced with Lua (but it still keeps the
| declarative rules language). It's still in progress (I'm
| currently using it in some projects, and then will likely
| make some more changes based on my experiences), but I hope
| to release a stable version in the next few months. If you or
| others also have feedback, please let me know!
| [deleted]
| jerf wrote:
| I'd love to look into a parallel universe where Make didn't make
| so many really basic mistakes by 2023 standards. For instance, it
| has a lot of the "was there an error? eh, just keep going"
| philosophy from early days. I'd like it to be an error if a make
| rule claims to make a certain dependency and it fails to do so.
| That one change in a single stroke would eliminate a lot of
| Make's hostility when trying to first understand it. There's a
| series of similar things that could be done.
|
| All of its replacements generally involve such a paradigm shift
| that it's no longer a comparable, to use a real estate term.
|
| I've got a whole list; $ being used for both make and shell
| replacements, leading to $$$$(var) abominations and the general
| lack of clarity as to which variables are for which things, many
| places where lists were clearly bodged in as after thoughts when
| they should be designed in from the beginning, tabs as meaningful
| whitespace is a classic but still a problem, .PHONY being a
| rather ugly hack when there should be a clear distinction between
| "a command I want to provide" versus "this is how to make a
| dependency", and while this may not be a user-visible behavior
| change, taking away all the C defaults so the strace is no longer
| a nightmare of trying every implicit C rule before actually
| trying the rule I want it to try. At least a _mode_ of invoking
| the shell that looks more like a programming language where I
| pass a clean array of strings rather than the shell language,
| already a nightmare of string interpolation on its own terms,
| buried in _another_ string interpolation language sitting on top
| of it. A thought-out solution to recursive versus included
| makefiles. And I never became a make master, but by my 21st
| century standards, trying to use make is just a fractal of bad
| 1970s ideas hitting me at every turn, so I 'm quite sure if I had
| to work with it a lot more I could go on for quite a while. As it
| is I think I've forgotten some.
|
| I think people, not entirely illegitimately, have trouble
| separating the essential concept of make, which I think was quite
| solid, from the accidental comedy of errors that we have as a
| received standard. So a lot of make replacements end up running
| from both of them, and often end up with a much more annoying
| essential concept at their core, such as a super-heavy-duty
| enumeration of "these are the exact things you may ever be
| interested in doing", which puts them behind the eight ball no
| matter how good their accidental choices may be.
| tannhaeuser wrote:
| > _For instance, it has a lot of the "was there an error? eh,
| just keep going" philosophy from early days._
|
| I don't know what you're talking about. make stops when a
| command it has launched terminates with a non-zero exit code,
| and if anything, "just keep going" is rather a thing in today's
| needlessly-async JavaScript tool chains.
| hedora wrote:
| Once you step off the beaten path, you find that errors from
| things like:
|
| false | true
|
| Get silently swallowed by bash (this is configurable, but the
| default ignores such errors). Also, the point about not
| noticing that a rule didn't create its target is a good one.
| (That behavior should be configurable; I don't think it is.)
|
| Anyway, with -j, make is as async as pretty much anything
| else out there.
| imoverclocked wrote:
| Some targets are virtual and don't create a file named as
| the target. Sometimes a target may or may not create a file
| based on internal logic.
|
| make itself is consistent in that it looks for errors from
| return status of the commands it spawns.
| Izkata wrote:
| > false | true
|
| That statement succeeds. Where's the error?
|
| To be more precise... "make" isn't "bash". There's no
| problem with "make" here and it has no way to see inside
| bash's internals. It's a bit like asking "make" to
| understand python, javascript, and java code and runtimes.
| peatfreak wrote:
| The article has the following near the end:
|
| "To see more real-world examples of makefiles, see my [World
| Atlas](https://github.com/mbostock/world-atlas) and [U.S.
| Atlas](https://github.com/topojson/us-atlas) projects, _which
| contain makefiles_ for generating TopoJSON from Natural Earth,
| the National Atlas, the Census Bureau, and other sources. "
|
| I checked those repositories because the descriptions of the
| makefiles sound interesting, but I couldn't find the makefiles.
| Am I looking wrong?
| Mister_Snuggles wrote:
| It looks like the author gave up on using Make with this
| commit: https://github.com/topojson/world-
| atlas/commit/8f92f6eb692d1...
| throwaquestion5 wrote:
| More like they moved from the need of using Make to create
| the topoJSON to use prebuild ones with Node.
|
| In the package.json of that commit:
|
| > - "description": "Roll your own TopoJSON from Natural
| Earth.",
|
| > + "description": "Pre-built TopoJSON from Natural Earth.",
| peatfreak wrote:
| It kind of makes the whole article irrelevant. Like a house
| of cards that is build on a foundation of Make, but when you
| get to the bottom there are no actual Makefiles there.
| throwaquestion5 wrote:
| It just means when the article was published, the need was
| real and make was useful. Context matters.
|
| Based on that commit they don't need to download the data
| to generate the .json file, so they don't, Make became
| irrelevant. If anything this shows that a tool can be
| really useful but you don't need to marry it. Don't use if
| you don't have to.
| barbazoo wrote:
| > Note: use tabs rather than spaces to indent the commands in
| your makefile. Otherwise Make will crash with a cryptic error
|
| Using this as an example, are there more modern equivalent tools
| that may be a bit more user friendly? I appreciate make and I get
| its age and the complexity and all, it's just sometimes I need
| something that's simple and explicit, without the historic
| baggage.
|
| Edit: It's not about spaces/tabs obviously, but about "Otherwise
| Make will crash with a cryptic error" which I used as an example
| Akronymus wrote:
| I'd say Shake fits your needs.
| https://hackage.haskell.org/package/shake
| dxhdr wrote:
| > Using this as an example, are there more modern equivalent
| tools that may be a bit more user friendly? I appreciate make
| and I get its age and the complexity and all, it's just
| sometimes I need something that's simple and explicit, without
| the historic baggage.
|
| Indenting with tabs too complicated? Let me introduce you to
| YAML.
| barbazoo wrote:
| > Indenting with tabs too complicated?
|
| I meant "Otherwise Make will crash with a cryptic error" and
| used that as a single example.
| hedora wrote:
| I've found the "make tabs are confusing" crowd heavily
| overlaps the "python is more readable than perl" crowd.
|
| I find the cognitive dissonance amusing.
| lmm wrote:
| Both views come from the same place: the idea that the
| semantic scoping should match the visual scoping, rather
| than counting tabs for indentation but not counting
| visually indistinguishable spaces (make) or ignoring your
| indentation entirely in favour of what the braces said
| (perl).
| imran-iq wrote:
| Since make 3.82 (which is about 12 years old now) supports
| `.RECIPEPREFIX` .RECIPEPREFIX = >
| all: > foo bar
| enriquto wrote:
| also, since forever, you can use a semicolon and put the rule
| on the same line: target : deps ; rule
|
| There are certainly a lot of legitimate criticisms of make,
| but the tabs issue is ridiculous (as are complains about
| .PHONY).
| hawski wrote:
| I don't understand why and don't know when spaces were started
| to be used for indentation. It never made sense for me. Your
| code editor surely supports setting tab width if you are not
| happy with the default.
|
| If the answer is vertical alignement I will weep and offer
| pity, but no sympathy. Even if that is the case you can still
| indent with tabs and aligns with spaces if you like to torture
| yourself, others and your RCS' history.
| lanstin wrote:
| I have preferred spaces to tabs since the mid nineties
| because everyone sees it the same. With tabs people use 2 4
| and 8, which can make my tendency to over indent code cause
| more trouble.
|
| I will say I have been doing go recently and have been forced
| into tabs and camelCase, and have found consistency really is
| better than The Right Way.
|
| For make, since using protobufs and code gen and being
| allergic to checking generated code in, have found make to be
| nicer than I remember it.
| chubot wrote:
| Another commenter and I both wrote some simple wrappers around
| Ninja in Ruby and Python -- I would recommend that approach
|
| https://news.ycombinator.com/item?id=32303692
|
| I started from ninja_syntax.py and then improved it based on
| what I needed. Ninja is fast and simple, but it's
| (intentionally) lower level than you might want.
|
| https://www.oilshell.org/blog/2022/10/garbage-collector.html...
|
| It does seem like there should be a "standard" thing that is
| maybe not so C++ specific (like CMake and Meson which both seem
| to use Ninja), i.e. for the use cases in the blog.
| drothlis wrote:
| I love Python + Ninja! I was going to link to my article on
| Ninja + OSTree, but I found in your blog you've already seen
| it. :-)
| beardicus wrote:
| I imagine many of Mike's points would be addressed just as well
| by Just or most any other task runner... but I thought his main
| point of "Makefile as documentation" was valuable.
|
| After reading this way back in 2015 I decided to give it a try
| for a not-code-related task: downloading a book from the internet
| archive, copying out all the images, and running some adjustments
| and conversions on them with ImageMagick:
|
| https://github.com/beardicus/bk-fig-phillips
|
| It was fairly ridiculous but works well and I learned a lot.
| debacle wrote:
| I haven't used make since college and will probably never use it
| again.
| asveikau wrote:
| Make is a great parable of our industry.
|
| * It's dead simple. It encodes dependency graphs and does stat()
| calls to check and compare timestamps. From that, you can have
| desirable features: minimal rebuilds of changed files, parallel
| build, etc ... Sure it's not perfect at this, timestamps can skew
| or whatever, dependencies can be improperly specified. But it
| does enormous heavy lifting with a very "dumb" implementation.
|
| * It's extremely influential. Even if you're not using makefiles,
| chances are some build tool is stat()ing files based on some
| representation of dependencies. In some form or another they
| probably got that expectation from make.
|
| * The original version was written by one guy over a weekend.
| Shows that our industry can have enormous, industry defining
| contributions from a small team.
___________________________________________________________________
(page generated 2023-01-11 23:00 UTC)