[HN Gopher] What's New in Bazel 6.0
___________________________________________________________________
What's New in Bazel 6.0
Author : siggi
Score : 54 points
Date : 2022-12-19 17:44 UTC (5 hours ago)
(HTM) web link (www.buildbuddy.io)
(TXT) w3m dump (www.buildbuddy.io)
| Leherenn wrote:
| We use Bazel at work for C++, and while defining targets and
| dependency is easy, I've really struggled with some stuff I would
| consider basic, like conditionally changing build flags,
| especially to external dependencies.
|
| Basically, I wanted to conditionally use sanitizers for a
| specific target and propagate the sanitizers compiler/linker
| flags to the whole dependency tree. For code you own, from what I
| could find, you need to duplicate each target and each dependency
| (and their own dependencies) for each possible sanitizer and then
| select the right one. And for external dependencies, you need to
| use "Aspects", if I understood correctly, to modify the build
| dependency graph and inject your flags. There's a question on
| stackoverflow about it, but the answer is very high level
| (presumably by a Bazel dev), and unfortunately I didn't have time
| to dig it further and just abandoned.
|
| I also had a lot of issues on an ARM Mac, including the linker
| issue that is apparently fixed (hooray) and recurrent crashes.
|
| Given we do not use any of best features of Bazel
| (remote/distributed compilation, multiple languages, and caching
| (because we basically spawn a new docker container for each
| build)), I think CMake would have made more sense.
| johntb86 wrote:
| I think you should be using user-defined transitions
| (https://bazel.build/extending/config#user-defined-
| transition... ) to use sanitizers. They essentially duplicate
| the build graph, and you can set build flags on the transition
| so that all dependencies will be built with those flags.
| nisa wrote:
| I'm still not sure if I should go the bazel route... recently
| found a long good explanation why anki switched away from bazel:
| https://github.com/ankitects/anki/commit/5e0a761b875fff4c9e4...
|
| I'd love to have cachable builds but I wonder how much effort it
| takes to maintain such a setup - especially in a small team very
| far away from silicon valley or google were nobody saw bazel
| before. Would be a perfect fit for my former company but even
| there I couldn't convince my colleagues - they build a monster
| using maven which would be more elegant in bazel but maven
| integration worked different than the maven algorithm at least 2
| years ago...so my PoC turned into reading source code of
| dependency resolution algorithm in the used scala tool
| coursier... in the end I had to give up after patching maven
| issues in several projects we build upon - something nobody is
| willing to do I guess - to be fair I tried to incorporate
| Alfresco ECM which is quite an enterprise Java monster on it's
| own.
|
| Also I couldn't find a good approach to use multiple git
| repositories almost all tutorials use a monorepo.
| ranting-moth wrote:
| In my experience you need a Bazel expert on the team.
|
| When everything works it's amazing. When it doesn't or
| additional work is required the hope your local expert is
| online and willing to sort it out.
| maccard wrote:
| This is true for any language ecosystem or build tool,
| whether it's bazel, maven, unrealbuildtool, Jenkins, cmake,
| gradle etc.
| oblio wrote:
| The thing is, if you hire 20 Java devs, at least one of
| them will already be an advanced Maven user. Same for
| Python and poetry/pipenv, Javascript/npm, etc.
|
| Bazel is kind of to the side everywhere.
| dbaupp wrote:
| We've had a good experience using Pants 2
| (https://www.pantsbuild.org/) and its dependency inference
| (https://blog.pantsbuild.org/why-dependency-inference/) for
| Python. Although we are in a monorepo, rather than a "multi-
| repo".
| jitl wrote:
| It's cool that Pants has enough community to evolve beyond
| being a Twitter-led project. It seems Twitter itself migrated
| from Pants to Bazel, and no longer works on Pants:
| https://groups.google.com/g/pants-
| devel/c/PHVIbVDLhx8/m/LpSK...
|
| I find it hard to buy into one of these "mega build system"
| ecosystems because there are so many competing tools that
| descend from Blaze: Bazel, Pants, Buck, Buck2, etc. The
| migration path to these systems takes substantial effort, so
| I wouldn't want to bet on the wrong one and have to do it all
| again a few years later. I'm content to wait patiently for
| Bazel or Buck2 to clearly win before I invest.
| dbaupp wrote:
| I have the sense that migrating to _any_ hermetic,
| cacheable build system will be more effort than migrating
| from one to another: once a project is "safe" to have
| everything run in sandboxes, the mechanisms for setting up
| those sandboxes is less important. For example, once one
| system has accurate dependency information, that can be
| used as a base for setting up the second system (and
| presumably this info can be exported for machine-assisted
| migration).
|
| Of course, the details of each system differ enough that I
| imagine it's not _trivial_.
| jen20 wrote:
| Do you know what the current maintenance story for Pants is
| following Twitter's move to Bazel?
| dbaupp wrote:
| It seems to be quite active, with regular new releases. I'm
| lead to believe that https://www.toolchain.com/ is the main
| sponsor, but there are maintainers from other companies
| too. I do not know the details, though.
| aseipp wrote:
| Was your project built in multiple languages and tied together
| using Maven? Or was it just JDK languages/Java?
|
| Tools like Bazel or Nix or <whatever you enjoy> are incredible
| for many reasons but they are, to a first approximation, very,
| very high-effort and complex tools that require care. It's like
| a racing car. You need an engineer on hand to keep the car
| running, and someone to drive it too. Maybe you're both of
| those people, but only a driver isn't enough.
|
| They also suffer from another problem which is that most people
| don't care as much as you or I do. :) So to make it a slam
| dunk, you have to make a pretty clear case that it's not "just"
| 2x better, but actually 10x better. And it's actually even
| harder than that: it can't just be 10x better in practice, it
| has to _appear_ 10x better _at a glance_. Like, at the window
| shopping phase, it needs to catch their eye. That 's difficult
| to nail and basically impossible to do perfectly.
|
| Personally? I'm all-in on "whole sale" build tools like this
| that hit many nails with one hammer. They solve a tremendous
| amount of issues. But they aren't always good and I can really
| see why they can fail to gain traction for many teams: they
| simply aren't necessary for them, even if they're really a lot
| better in many ways. I get it.
| nisa wrote:
| > Was your project built in multiple languages and tied
| together using Maven? Or was it just JDK languages/Java?
|
| Exactly that - for one it was a plugin for Alfresco ECM that
| disobeyed every Alfresco SDK rule :), then a custom Angular
| frontend that needs to be build according to the plugin -
| furthermore the business was of that kind that every customer
| had a specialized build with customization - Bazel would have
| solved that in a very neat way - including overriding a few
| source files for a specific customer.
|
| We tried to build then docker images with maven which was
| just a shitshow considering our problems - however I'm not
| there anymore but I'd like to have the exact setup that you
| describe - the big hammer to don't have to deal with all the
| stupid things - especially as my background is sysadmin and
| every problem I don't have to solve anymore is good for me :)
| Groxx wrote:
| Just speaking to multiple git repositories: could you use
| submodules for a conglomerated-monorepo?
|
| Otherwise: you can pull in repos as bazel-file-containing
| additions. I forget the details, but I believe it's something
| like https://bazel.build/extending/repo . The troubleshooting-
| while-developing experience is not great though imo, I'd try
| submodules first.
| nisa wrote:
| Thanks - repository_rule looks good. So it's either git
| submodules to maintain updates or checksums / commits in
| repository_rule - I probably need to just play around with
| both approaches and choose the one that annoys me the least
| :)
| actionfromafar wrote:
| I've been meaning to try out XMake [0]. It has caching and
| distributed build and seems much easier to understand than
| Bazel.
|
| 0: https://xmake.io/#/getting_started
| [deleted]
| eklitzke wrote:
| I'm not going to try to convince you to use Bazel. I think it's
| great, but ultimately choosing one build system or another is a
| big decision that involves weighing a lot of different factors,
| including what your coworkers are amenable to and have
| experience with. However I don't think Bazel is that difficult
| to get started with.
|
| The Bazel docs are admittedly a bit challenging, but the
| easiest way to learn things when you're first getting started
| is to just look at other projects using Bazel and see how they
| do things.
|
| Using multiple git repositories is easy, you just use the
| git_repository() rule which has been in Bazel since pretty much
| forever. Nowadays you can also use bzlmod (introduced in bazel
| 5.x) which is a slightly different approach to this problem.
| There are a bunch of rules in
| https://github.com/bazelbuild/bazel-central-registry which you
| can look at to see how to write your own bzlmod registries.
| nisa wrote:
| bzlmod looks amazing! Thank you! I don't need to be convinced
| but I need to sell it to my team and I must be able to solve
| our problems in it - imho getting it working solves 90% of
| annoying problems in this whole ci/cd/container mess and
| that's why I'd like to push for it.
| hobofan wrote:
| The BCR also has a web UI at https://registry.bazel.build
| jeffreygoesto wrote:
| Maybe check out justbuild [0], the main author used to be a
| bazel dev and wanted to combine it's strengths with the
| possibility of multi repos and more possibilities to interact
| with the dependency tree. Quite interesting and very responsive
| team.
|
| [0] https://github.com/just-buildsystem/justbuild
| hobofan wrote:
| Aaarg, I hate to always be the person that points out "naming
| issues", but there is already a pretty popular command runner
| with the name just: https://github.com/casey/just
| tinglymintyfrsh wrote:
| 0. Does it do creating static libraries yet?
|
| 1. Does it do linking to a mix of static and dynamic libraries
| yet?
|
| 2. Anyone try buck2, sort of bzl-like?
| lalaland1125 wrote:
| 0 and 1 have been supported for a while. See the "linkstatic"
| feature.
| en4bz wrote:
| No, they are extremely broken. You can only choose:
|
| - Link everything static.
|
| - Link everything dynamic.
|
| - Link user libs as static and system libs as dynamic.
|
| There is no easy way to link a single user lib static/dynamic
| without resorting to hacks/workarounds like re-importing the
| shared library or defining weird intermediate targets. It's
| completely broken.
| lalaland1125 wrote:
| You can use the "linkstatic" feature on the cc_library
| level. Then that library will be linked statically while
| other cc_library's will be dynamically linked.
| en4bz wrote:
| The logic is backwards though. I may have multiple
| consumers of a library some of which may want static some
| of which may want dynamic. You need to create/import new
| targets to do this even though the original target
| creates both static and dynamic libs by default.
| emidln wrote:
| I don't know what you're talking about. This is trivial to
| accomplish using `linkstatic` as documented on `cc_library`
| and `cc_binary`. I uploaded a demo to github here: https://
| github.com/emidln/bazel_static_dynamic_c_demo/blob/m...
|
| Try it out like this: # you'll need a c
| compiler installed, xcode is fine on macos git
| clone https://github.com/emidln/bazel_static_dynamic_c_demo
| cd bazel_static_dynamic_c_demo bazel run //:foo
| ldd bazel-bin/foo # otool -L bazel-bin/foo if you're on
| MacOS
| simplotek wrote:
| From the announcement:
|
| > One of Bazel's most powerful features is its ability to use
| remote caching and remote execution.
|
| What's the value proposition of Bazel when build systems like
| cmake support this out of the box with third-party tools like
| distcc and ccache?
| en4bz wrote:
| None, bazel's caching implementation is broken because they
| don't even know or specify what constituents a build hash/key.
| See this issue from 2018 that's still open [1].
|
| [1] https://github.com/bazelbuild/bazel/issues/4558
| lopkeny12ko wrote:
| Probably because building targets with tools outside of the
| workspace is an antipattern, as it violates hermiticity
| principles. In fact, Bazel generally makes it quite hard to
| do this, so anyone who ends up in this scenario must have
| jumped through many hoops to get there.
|
| I agree that the linked issue is legitimate, but I'd argue
| that this isn't a problem Bazel itself needs to solve--you
| should fix your build to be fully hermetic.
| en4bz wrote:
| Non-Hermetic is the default for C/C++. And if you plan on
| using system provided libraries to support multi OSes then
| you can't use it.
| lopkeny12ko wrote:
| That is precisely the point--using system-provided
| libraries in your Bazel project is an antipattern that
| should be avoided.
| pests wrote:
| It's not out of the box if you need third party tools is it?
| lalaland1125 wrote:
| The main change here is that Bazel now has a package repository
| that people can pull/push external dependencies to:
| https://registry.bazel.build/
|
| This is a huge change as external dependencies used to be one of
| the big pain points with the Bazel pipeline.
| no_wizard wrote:
| NX is a very bazel like for frontend focused repositories.
|
| I think what I struggle with the most is the opinions of the
| rules / plugins. Either I use it as a core task runner (easy, but
| lots of config the team has to maintain setting up script entries
| in packages etc) or I have to write custom plugins for anyting
| not on the "happy path".
|
| I think thats my general frustration with these tools, they
| aren't terribly well built for SDK type consumption, they don't
| expose a core set of APIs directly, instead relying on a plugin
| system that can sometimes be very opaque
___________________________________________________________________
(page generated 2022-12-19 23:01 UTC)