[HN Gopher] Show HN: Beast - The Build System
___________________________________________________________________
Show HN: Beast - The Build System
Beast is a build system built for speed and power - a tool for all
your build needs. GitHub: https://github.com/GauravDawra/Beast
Docs: https://gauravdawra.github.io/Beast-docs/ As a project grows
larger, it becomes difficult to keep track of all the build and
compilation procedures that need to be followed. So what should we
do??? Not to worry! Beast helps you build your projects with
minimal effort and high efficiency, bringing more power to you. In
addition, it is super easy to use and syntactically easy to
understand, making it suitable for both: beginners and highly
experienced programmers. With its new release: Nimble (v1.1.0),
Beast has become much faster and stronger than before. Its build
times are now overtaking or matching those of current community
standards!!! In contrast to other such build systems, Beast
focusses on both: ease of usability and speed!!!
Author : gaurav1804
Score : 23 points
Date : 2022-07-02 11:04 UTC (2 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| bjourne wrote:
| Awesome project! I'll use it for my next C++ project.
| evmar wrote:
| Congrats on making the release! I glanced through the docs and it
| seems Beast works via a graph over targets and mtimes?
|
| I recently wrote some retrospective notes about build system
| design in this area that seem like they might be useful for
| someone in exactly your place:
|
| https://neugierig.org/software/blog/2022/03/n2.html
| [deleted]
| [deleted]
| rkangel wrote:
| Increasingly, my view is that there is only room for two quite
| separate things:
|
| * Task runners (rake, simple Make rules, shell scripts) etc.
| Basically convenience scripts for devs. * Hermetic build systems
|
| Hermetic build systems (Bazel, Nix package manager) can be 100%
| confident that there aren't any secret implicit dependencies and
| this unlocks some really great stuff such as:
|
| * Complete confidence in incremental builds at all time. It isn't
| _possible_ for something secret to have changed * Excellent build
| caching, because you know all the inputs match the cache
|
| The classic example of this is C or C++. The build step is `gcc
| -c main.c -o main.o`, but that build command depends on a load of
| header files included by main.c. Both system ones and ones local
| to your project. If you change a header and rerun your build
| rule, it only knows about main.c, knows that it hasn't changed
| (by hash or timestamp) and doesn't do anything. You end up having
| to rebuild a load of stuff just in case. Your usual system (e.g.
| make) has no idea about this. Various solutions exist - Beast
| talks about the 'get gcc to tell you about the dependencies'
| approach but now you've got another file to generate and manage
| and have in your build system, and you still can't be _sure_ it
| 's right.
|
| There is a (good) argument to be made that these are problems
| that are only unmanageable in big software projects, and the rest
| of us should just take advantage of the simplicity of make (or
| newer better solutions like Beast). This is a similar argument to
| the one being made in another post about K8s and 'early
| optimisation'. But this is why every time someone posts a new
| build tool, I always hope it's going to be about 'power and
| reliability of Bazel' but as friendly to get going with a basic
| make rule.
| specialist wrote:
| > _Hermetic build systems_
|
| Wanting to know more, I found this:
|
| "Hermeticity: This page covers hermeticity, the benefits of
| using hermetic builds, and strategies for identifying non-
| hermetic behavior in your builds."
| https://docs.bazel.build/versions/main/hermeticity.html
|
| Sounds great.
|
| Ages ago, my teams had a policy of "one button build". Install
| VS C++ on a new box, open the project (from source repo), hit
| "Build". Tada.
|
| We could rebuild any revision on demand. Terrific for
| reproducing regressions and delta-debugging.
|
| In the Java world, with (misuse of) maven, gradle, jenkins,
| etc. attaining reproducible one button builds is quixotic.
|
| For hermetic builds, everything would be digitally signed
| (SHA256), right? There's a spec for signing Linux kernels,
| which I can't quickly refind. But the idea is to apply that
| strategy to everything, right?
|
| That sounds perfect.
| rkangel wrote:
| What you basically do is take all of the inputs (source
| files, compiler, exactly what command you're going to run)
| and then produce a hash of all of that together. Then the
| result of that becomes the input to the next thing etc.
|
| For caches you just say 'here's the SHA1 of the inputs and
| the cache server can just give you the output.
|
| Basel does this for each build command, e.g. for each object
| in a big C program. Nix does it for each package.
| gravypod wrote:
| > I always hope it's going to be about 'power and reliability
| of Bazel' but as friendly to get going with a basic make rule.
|
| Have you seen https://please.build? It is very close to what I
| think the answer to "What if Blaze was made today?" is. It is
| conceptually much simpler and doesn't suffer from some of the
| things that need to happen in Google-scale projects/teams. For
| example, they haven't done the same gross thing with
| proto_library that google did. You only define one and it only
| builds the bindings that are actually needed by the targets you
| are building.
| rkangel wrote:
| This looks really interesting!
|
| Doesn't support Windows which is a bit of a shame, but maybe
| manageable. Will have to check it out properly at work
| tomorrow.
| JonChesterfield wrote:
| If determined, the system headers can be elided. Bundle parts
| of musl with the application or work in -ffreestanding. Then
| it's self contained except for the compiler, which I believe
| some embedded systems commit to the repo along with the source.
| jitl wrote:
| Why would I use this instead of make? I read a bit of the guide
| to "beast files" here and haven't seen anything different from
| make other than some slight syntax differences.
| https://gauravdawra.github.io/Beast-docs/mainDocs/writingABe...
|
| You should make your value proposition clear in your project
| README, especially if your syntax and overall model appear to be
| quite similar to an existing tool. Saying you focus on "ease of
| usability and speed" doesn't tell me much.
| gaurav1804 wrote:
| Hi! I know I haven't supported my claims in the readme. But if
| you see the syntax for beast build files, you will see that
| indeed syntax in beast is much simpler and intuitive. For
| example, you have strings and integers for variables. You can
| dereference them easily and even create new variables of any
| type from other variables. You can declare variables inside a
| build rule. All in all, the syntax is inspired from python,
| since everyone loves it so much. This is not all, I'm currently
| benchmarking Beast. The new release: Nimble is turning out to
| be way faster than Make and even matching/overtaking Ninja
| (currently one of the the fastest build systems in the
| community) But you are right in saying that I should include
| this in the readme. I will surely post this after the
| benchmarks
| rkangel wrote:
| I haven't used Beast, but I have used make a lot. One answer
| might be "just in case someone wants to put a space in a
| filename, or in any parent folder of where my code is".
|
| Make does not cope with this at all. Yes I'm aware of all the
| workarounds and they all fall over at some point. Make works
| perfectly well on Windows for instance, except for the whole
| generation of Windows that had the users under "Documents and
| settings". Was almost always impossible to build code in the
| user home directory.
| kdumont wrote:
| Looks like a pretty neat extension of make syntax. Having
| better/simpler control of variables sounds great!
| nittanymount wrote:
| what languages it is for ? seems only for c/c++ ?
|
| or it can be for any language, just put the build command line in
| beast file? thanks.
| gaurav1804 wrote:
| Hi! It is not only for C/C++, but it can be used for any
| project in general. This is a low level build system which runs
| your commands in a shell depending on the rules you define in
| the beast.build file. Think of it as a faster and more user-
| friendly replacement to Make. It's 'pythonic' syntax makes it
| very easy to use and it's build speeds are matching/overtaking
| those of community standards!
|
| You are right in saying that since any command can be put in
| the beast file, it is suitable for building a project in any
| language.
|
| It is a general purpose build system, with focus on speed and
| usability!
| nittanymount wrote:
| thanks for reply,
|
| - examples for using it for android or iOS, or java project,
| and some numbers showing its better performance?
|
| - how to use it for an android project? gradle handles it all
| with target(sub targets tree structure of the build
| tasks...), if use Beast, will still be using gradle? or the
| build tools gradles is calling for different things (compile,
| packaging,signing...)
| gaurav1804 wrote:
| 1. Since Beast is new, there are not a lot of examples.
| Benchmarking is going on and it will take some time. In the
| mean time, you can take a look at how to write beast build
| files.
|
| 2. You can use it for any project you want if you know what
| are components of your project depend upon what other
| components. So as I see it, gradle is a build system for
| Java project in itself. Beast will not be using gradle if
| you are trying to compile a Java project. BUT.... you can
| for sure call `gradle` command from inside the beast.build
| file. Wherever build.gradle file is stored. So that in this
| case, beast provides you with an easier to use build
| system. Let us see an example:
|
| In your beast file, you can have something like:
| build hello: ! gradle -q hello
|
| and inside your gradle file: task hello {
| doLast { println 'tutorialspoint' }
| }
|
| Hope that helps
| prvnsmpth wrote:
| It doesn't make sense to have your build tool call
| another build tool to actually perform the build. In that
| case, why I would not just use Gradle directly?
| gaurav1804 wrote:
| There can be several reasons for that. Say your project
| doesn't only require gradle but other tools to be run
| too. Or say there are some post-scripts or pre-process-
| scripts that need to be run. They can be done easily
| like: build pre: ! echo 'pre
| script' build proj: ! gradle hello
|
| Plus, you might want to run some python scripts or run
| some other shell commands during or after you have built
| your gradle project.
| hiyer wrote:
| True. This seems more like a general-purpose task-runner
| like Task [1] than a self-contained build system.
|
| 1. https://taskfile.dev/#/
| pbackup12345 wrote:
| Hi there, it might be a good idea to clear this up early on
| the github page itself. This is a good idea, but I would not
| read on after the first page if I don't see that it checks my
| boxes for what I need. And as I can see it is still not
| really defining it.
| gaurav1804 wrote:
| Thanks! Noted... I will take care of it! Thanks again
| dale_glass wrote:
| The style looks extremely similar to make, in which case it needs
| a good explanation of what makes it better.
|
| It even seems to copy make's syntax, which IMO is a missed
| opportunity. One of the issues with Make is that the syntax is
| clever until you need to invoke multiple lines of shell script,
| like a loop. Then the tab-based cleverness becomes a pain.
| gaurav1804 wrote:
| Hi! Basically it is supposed to do what Make does... but in a
| faster and more user-friendly way. Note that Make doesn't have
| the best syntax. But Beast has sort of a 'python' syntax. It
| supports strings and integers. Variable modification
| assignment. It even supports scoping (within and outside beast
| build rules).
|
| Plus, it is an order of a magnitude faster than Make. All in
| all, I don't agree with the fact that it's syntax is similar to
| Make. Well for one thing it has legible special variable names
| like 'out' and 'dep'.
|
| So yes, while it is a low level build system, it is an
| improvement over Make in terms of syntax and speed.
| dale_glass wrote:
| How is it faster? And faster at what?
| gaurav1804 wrote:
| By 'faster' I mean better build times than Make for a
| project of particular size. Benchmarking is going on right
| now, and the difference in performance is clearly visible.
| dale_glass wrote:
| Okay, but what specifically is faster?
|
| Make is rarely a big bottleneck, except in very
| pathologically written projects, and when doing
| incremental builds. I think the main problem with big
| make setups is recursive make, which prevents it from
| having a global view on the project, and forces the main
| make to execute many sub-makes recursively, all of which
| has overhead.
| onion2k wrote:
| _As a project grows larger, it becomes difficult to keep track of
| all the build and compilation procedures that need to be
| followed._
|
| Anecdotally I've worked on projects that have thousands of
| scripts that could be built with a 5 line make file, and other
| projects that have a few tens of files that needed a 200 line
| Webpack config. I don't think build complexity is a function of
| project size.
| gaurav1804 wrote:
| Hi! The reason why you can compile such large amounts of files
| with only a few lines in a Makefile, is because most of these
| files might be carrying a pattern in their build rules (like
| for example most of them are to be compiled in an object file).
| But this is not always the case. Plus, note that I (and many
| others I have worked with) are of the opinion that Make may be
| popular but it is surely not user friendly. The syntax involved
| in writing a Makefile can get very messy very quickly with all
| the $<, $@ and what not... Beast's syntax is much more simpler
| and intuitive, so that everyone... (beginners or experienced)
| can use it
| mariuolo wrote:
| What are the advantages over cmake or meson?
| Rochus wrote:
| Looks conceptually quite similar to make, or did I miss
| something? Did you use other build systems like e.g. cmake, meson
| or gn? What makes beast more usable or faster than e.g. meson?
|
| What always amazes me: shouldn't the build system itself be as
| easy as possible to build (low requirements on the compiler,
| minimal dependencies, platform agnostic, etc.), e.g. just like
| "gcc -O2 buildsystem.c"? Also almost all of these systems seem to
| suffer from the same problems that were discussed (and solved) in
| the early years of software engineering, and e.g. hardly support
| modularization/encapsulation or static type checking. Cmake and
| meson are huge and complex, with a peculiar dynamic language
| each, not easy to install and use, and usually a factor ten
| bigger than what I want to build; also beast itself requires
| make, a recent gcc version and even flex and bison (so it doesn't
| e.g. run on windows, does it?).
| gaurav1804 wrote:
| Hi! You are right in saying that Beast is "conceptually"
| similar to Make (and Ninja). But it is much faster and much
| more user-friendly than Make.
|
| Second, you are listing tools like cmake and meson. Note that
| these are NOT build systems but they are META BUILD SYSTEMS.
| Meta build systems are tools that don't really build your
| project themselves... instead they rely on tools like Make,
| Ninja and Beast to build it for you. You must have seen that
| Cmake emits a Makefile (or Ninja file depending upon your
| choice). So comparison in speed to meson will not be
| appropriate. (Let us just call them high level build tools)
|
| About your third query... Well Beast is easy enough to compile.
| If you don't want to compile it, you can directly download the
| system binaries at
| https://github.com/GauravDawra/Beast/releases/tag/v1.1.0
|
| I'm currently using Make to compile Beast because I originally
| compiled it using Make. It will soon move on to CMake. Please
| note that it is totally OK to use flex and bison. You don't
| even need them since I have already provided the cpp/h files
| generated by them on github!
| Rochus wrote:
| > _these are NOT build systems but they are META BUILD
| SYSTEMS_
|
| That's technically true, but you could just integrate the
| Ninja source code into such a meta build system and run it
| e.g. with a different command line option and then at the
| latest the attribute "meta" would be no longer relevant.
| "meta" in this context doesn't actually mean "on a meta
| level", but just that the tool is a transpiler from a high-
| level to a low-level build description, instead of a full
| execution engine. make (in contrast to Ninja) has features
| for both levels. Personally I would prefer an integrated
| system, so bootstrapping would not involve two (artificially)
| separate parts. But as I understand your message is that
| Beast is rather a Ninja alternative than a Cmake alternative.
| That's ok, although we already have Ninja and it claims to be
| a very fast build tool. So in the question we then just
| replace Cmake/Meson by Ninja.
|
| > _since I have already provided the cpp /h files generated
| by them on github!_
|
| Ok, I see. So it then also runs on Windows? How do you build
| the tool on Windows?
|
| EDIT: actually if you're interested in GN, which has some
| very good concepts not seen elsewhere, I've written a tool to
| view and navigate GN source trees, see
| https://github.com/rochus-keller/GnTools. I usually use it
| with the Dart SDK source tree.
| JonChesterfield wrote:
| Agreed. Essentially the build system should do whatever
| crackpot things it likes to bootstrap with bespoke languages
| and so forth, but the output should be a single somewhat
| auditable C file that gets fed to a compiler, not a binary with
| exciting dependencies on wherever it was built.
|
| How sqlite constructs the single source/header pair looks
| ideal. Various efforts have bundled lua and some libraries into
| a single C file too.
|
| Check in that generated C file and you have trivial 'bootstrap'
| for everyone else. Ideally have some CI that checks the dev
| files still generate that exact C file on request.
| Rochus wrote:
| Lua is even simpler; there is an all.c which simply includes
| all other c files in a decent order, so you can just type
| "GCC all.c" resulting in a full Lua vm executable on all
| platforms. Other projects like e.g. Duktape offer amalgamated
| c files as you seem to suggest, but these are redundant.
| JonChesterfield wrote:
| This sounded wrong so I checked. You're correct. Current
| GitHub has a file onelua.c from three years ago.
|
| Turns out I forked the project four years ago. Slightly
| different implementation in that I expanded the #includes
| to get a single file I can copy around and hacked in a few
| libraries (libuv, lfs and penlight iirc). Not totally
| pleased to discover that was so long ago.
|
| Great to see that cc onelua.c gets the job done today.
| Rochus wrote:
| A file called all.c is also present in Lua 5.1 from 2006
| in the etc subdirectory.
| rjsw wrote:
| The ultimate build system should be written in a language with
| no implementations at all, failing this it should only be
| buildable using itself, examples of best practice being Bazel
| and Gradle /s.
| Rochus wrote:
| Well, you can expect that there is at least a compiler and
| runtime available compatible with the software you want to
| build; if your software is e.g. written in Java, then a build
| system also implemented in Java looks like a good fit. I
| usually develop in C++, so my preferred approach is to have a
| lean build system written in C89 which has only standard
| dependencies and compiles like "gcc buildsystem.c".
|
| I don't know how you would write and deploy a build system
| when there is no implementation for the language it uses;
| this looks like an unsolvable bootstrap problem to me; you
| would need a precompiled build system to build the build
| system, and that on all platforms you want to build; and
| anyway I don't think that a build system language should be
| Turing complete (or otherwise the language should at least
| support modularization and static typing).
|
| Bazel is certainly great for a lot of use cases, but it is
| one of those Google colossi where you have to hire a
| specialized team to manage and run your build. GN has a
| similar issue in that in practice there is an insurmountable
| dependency of a gazillion of Python scripts (depot tools and
| the like), even if GN and Ninja are written in C++ and only
| have standard dependencies.
___________________________________________________________________
(page generated 2022-07-04 23:02 UTC)