[HN Gopher] Bringing supply chain security features to the Go co...
___________________________________________________________________
Bringing supply chain security features to the Go community
Author : lmas
Score : 166 points
Date : 2021-07-23 08:25 UTC (14 hours ago)
(HTM) web link (github.blog)
(TXT) w3m dump (github.blog)
| debarshri wrote:
| I got few notifications yesterday for our golang projects. I was
| pleased to see less noisy alerts as compared to React based UI we
| have. It is terrible to say the least for npm projects.
|
| Would love to know if dependabot creates noisy alerts for other
| languages too.
| playcache wrote:
| meh. well that disappointing, its basically dependabot / graph
| and advisories.
| doctor_eval wrote:
| Well maybe it's meh if you already had it, but as a Go shop
| this is the first time we've been able to access these features
| so now I'm in party mode!
| lmas wrote:
| Yeah same for me. Seen it being used for npm and friends and
| been wanting some for Go too, for a long time now! I love
| graphs and dependency monitoring!
| rob74 wrote:
| Well, yeah, the title says they are "bringing supply chain
| security features to Go" - so Go gets the same features other
| languages already have, no new exciting stuff to be expected
| here...
| baby wrote:
| Had the same issue for Diem a while ago (I don't work there
| anymore) and we ended up looking into software to manage
| dependencies in Rust. Check out https://github.com/diem/whackadep
| kardianos wrote:
| Are the github's authors aware of Go's modules Vulnerability db?
|
| https://golang.org/design/draft-vulndb
| greysteil wrote:
| We are, and we're talking to them. In an ideal world we would
| have loved to have merged the databases.
|
| I expect the two databases will move in sync. The GitHub
| advisory database is licensed as CC BY 4.0 (i.e., attribution
| only), so we actively encourage others, including the Go
| Vulnerability Database, to pull from it.
| ikiris wrote:
| https://xkcd.com/927/
|
| Can you.... not?
| jd3 wrote:
| I was also wondering about this -- I feel like it was just a
| couple of weeks ago that I read about the Go team's new unified
| vulnerability schema/Open Source Vulnerabilities (OSV) database
|
| Definitely a step in the right direction on both ends, though!
|
| https://osv.dev/list
|
| https://security.googleblog.com/2021/06/announcing-unified-v...
|
| https://opensource.googleblog.com/2021/02/launching-osv-bett...
| leo_bloom wrote:
| GitHub's dependabot is causing a ton of "spam" in our frontend
| (Angular) repositories, as it seemingly opens 1-5 PRs per day to
| bump random dependencies. I really hope this does not become
| common practice for our Go repositories.
|
| Is there an option to tell dependabot "make one PR per week at
| most, please, and bundle your changes"?
| Cthulhu_ wrote:
| Part of that is the Node / JS ecosystem that churns out
| libraries for every tiny thing. The Go ecosystem is a lot more
| streamlined; adding dependencies is frowned upon, building
| small libraries is frowned upon, etc.
|
| That said, if it's just version bumps then they should be
| bundled. But what about security issues? Those need to be fixed
| and updated asap.
| rudian wrote:
| Disable it. If you don't want to, limit it to weekly or
| monthly. I agree that Dependabot is super noisy especially by
| default.
| whymarrh wrote:
| Yup, there's quite a few different configuration options
| available: https://docs.github.com/en/code-security/supply-
| chain-securi...
| gherkinnn wrote:
| Here's some information on these "vulnerabilities":
| https://overreacted.io/npm-audit-broken-by-design/
|
| As far as I'm concerned, there's no need for Dependabot to
| create PRs. The notifications in the security tab are enough.
| Mark the unnecessary ones as benign.
| dane-pgp wrote:
| And here's the HN discussion on that thought-provoking post:
|
| https://news.ycombinator.com/item?id=27761334
| staticassertion wrote:
| The PRs are really helpful when you do actually want to
| update.
| gherkinnn wrote:
| You can have Dependabot create a PR by hitting a button
| somewhere in the vulnerability details.
| X6S1x6Okd1st wrote:
| Yup, my PR review requested page is near useless because of all
| of the dependabot PRs.
| starefossen wrote:
| The amount of pull requests from dependency updates can become
| more manageable if you enable GitHub's Auto Merge feature[1]
|
| [1] https://docs.github.com/en/github/collaborating-with-pull-
| re...
| ilammy wrote:
| Exactly what you want to have: get a bot merge fixes for
| security issues prepared by another bot without any human
| oversight. Then let Copilot add features to your software
| automatically as well.
| dmitriid wrote:
| You can't enable granular auto merges: I'm fine with auto
| merging minor/security update PRs from dependabot. Not
| anything else. You can't set up that.
| koblas wrote:
| The "biggest" problem in the JavaScript ecosystem (NPM) is that
| dependadabot doesn't know how to discriminate between
| dependancies and devDepandancies.
|
| I don't really care if jest includes a package that has a regex
| issue. It's not production code. I do care if babel introduces
| a backdoor, but somehow they're treated with equal importance.
| leftshift wrote:
| It depends, attacks via build pipelines can be devastating.
| tailspin2019 wrote:
| Yep.
|
| Unfortunately it's not clear cut that we don't have to
| worry about devDependencies vulnerabilities.
| greysteil wrote:
| PM for security products at GitHub here.
|
| Totally agree. The good news is that I think GitHub is in a
| position to fix this - expect progress in the next 12 months.
|
| One change that's needed here is data for each vulnerability
| on whether it's ever relevant in development. The advisory
| database that powers GitHub's security alerts (and npm audit,
| and NuGet audit) now has a dedicated curation team and is
| ready to curate more ecosystems and more information.
|
| That data then needs to be hooked up with GitHub's security
| alerting logic. That shouldn't be too difficult - we already
| detect whether a dependency is used in development or
| production, and the team here is growing.
|
| Finally, for this to work we'd need a new UI concept for
| vulnerabilities you aren't affected by. We're already working
| on functionality with a very similar requirement (one that
| tells you whether you're using the vulnerable function within
| a dependency).
|
| I can't make promises, but I can say that GitHub has an
| increasing amount of energy. Expect progress :-)
| staticassertion wrote:
| It seems to primarily be a frontend thing. I get tons and tons
| of them, it's really annoying.
| hiq wrote:
| I also get a lot of spam from dependabot because of
| prereleases, I wish they would fix this bug:
| https://github.com/dependabot/dependabot-core/issues/2547
|
| As others have pointed out, you can opt for daily / weekly or
| monthly updates, I'll stick to monthly until they fix this bug.
| greysteil wrote:
| Hmmm, it's been a long time since I worked on Dependabot
| Core, but I think I can fix that one - we just need to
| special case `native-mt` as a version type (as opposed to
| applying standard Maven version comparison rules on it). I'll
| try and get a PR in today and will tag you.
|
| Update: https://github.com/dependabot/dependabot-
| core/pull/4077
| greysteil wrote:
| PM for security products at GitHub here (and one of the
| original authors of Dependabot).
|
| Sorry to hear that. I wouldn't expect us to be telling you
| about 1-5 security issues a day - do you maybe have (non-
| security) version updates enabled? If so and they feel like
| spam to you I'd recommend turning them off. (I wish I had a
| better suggestion, but until Dependabot supports grouped
| updates it sounds like it just isn't right for you.)
|
| Dependabot doesn't support grouped updates yet but we hear the
| feedback and the team wants to work on them. Most of the
| investment in Dependabot recently gone towards improving our
| infrastructure and improving the experience for security
| updates. The team is still relatively small (it's 7 people),
| and supporting a service like this at GitHub scale is hard, but
| we're keen to keep improving.
| plasma wrote:
| HN is great, can I chime in with some feedback too, for NuGet
| I'm seeing Dependabot open separate PRs for the same
| dependency that's out of date in the one repo that has
| multiple projects that relate to each other (they reference
| one another in csproj files).
|
| They thus also need to be updated together, but I wish one PR
| was opened to update the dependencies in all projects at
| once, instead of multiple PRs I to merge.
|
| PS: I think you should highlight dependabot updates on the
| Security tab in GitHub repo, I thought it was on before (but
| was actually just the security notices) because dependabot
| itself is hidden away in Insights -> Dependency graph ->
| Dependabot which was a bit surprising.
| greysteil wrote:
| Feature requests always welcome! I'll pass it on to the
| Dependabot team.
| jacobparker wrote:
| For NuGet, consider using centralized package references:
| https://github.com/NuGet/Home/wiki/Centrally-managing-
| NuGet-...
|
| You get one file defining the set of all packages used in
| your repo (or some subset of your repo, etc.) and
| Dependabot will update this file directly. Individual
| projects can choose to use a package but won't specify the
| version.
|
| It requires that your projects are all in sync with package
| versions but (1) that sounds like what you want (2) it's
| usually the best thing.
| jacobevelyn wrote:
| Thanks for all the hard work, greysteil! Long-time Dependabot
| user here and a big fan (currently in the throes of updating
| many repos to GitHub-native Dependabot).
|
| I've reported all this in the past, but since we're giving
| feedback in this thread, some big pain points for us are:
|
| 1. The lack of a way to view all Dependabot security
| alerts/security PRs (across repos) in one place. With Old
| Dependabot, we could do a GitHub Issues search like `is:open
| is:pr label:security org:<our-org>` and see them all, and
| actually built quite a lot of automation around PRs with the
| `security` label. But New Dependabot has no way to configure
| security PRs to get the `security` label, so it's easy to
| miss vulnerabilities when you have two dozen repos, and our
| automation no longer works. :/
|
| 2. Dependabot reporting `No security update is needed as
| <dependency> is no longer vulnerable` when in fact there are
| multiple versions of the dependency, including vulnerable
| ones, in our yarn.lock file. (Webpack has a _lot_ of
| transitive dependencies!)
|
| 3. Dependabot can't update a vulnerable version of, say,
| rails, because rails requires the same version of
| activerecord, even though these are essentially the same
| project. (I get the sense this is being worked on as "grouped
| dependency updates"?)
|
| P.S. Just wanted to clarify: feedback is love. I wouldn't be
| writing this if we didn't find Dependabot valuable (Old
| Dependabot was actually the reason we moved to GitHub years
| ago!).
| greysteil wrote:
| Thanks for the feedback Jacob, and for all the support over
| the years. I'm going to pass those three on to the
| Dependabot team, who are best placed to think through
| solutions.
|
| (FWIW, I'm sure we can think of something to fix (1) which
| balances security and usability - it just needs some
| thinking. (2) sounds like a bug to me - I'm sure the team
| will want to look at that. And (3) is a tricky case where
| Dependabot's Ruby logic needs to be able to unlock a
| dependency's "parent" or peers in order to make security
| fix (e.g., rails in the case where the vulnerability is in
| activerecord). Entirely possible but hard!)
| zdw wrote:
| It would be wonderful if there were bulk editing of
| Dependabot settings across multiple repos. The weakest part
| of GitHub is the lack of bulk administrative edits (repo
| settings, not code changes) across multiple repos.
|
| This affects group (team) permissions as well - having to add
| a new team to multiple repos with specific permissions is a
| manual slog and hard to audit. Even the expanded permission
| categories are frequently inadequate - there's no CI focused
| "Let these users write to the repo and change hooks"
| permission level - all CI systems that self-manage hooks need
| full Admin rights.
|
| The other code hosting I actively use is Gerrit, and the
| ability to hierarchically control what groups have specific
| permissions on multiple repos is far simpler and easier to
| use, from an admin perspective.
| xorcist wrote:
| Puppet and Ansible are great for this type of things. If
| you want a new repo for your group, you make a pull request
| for our management repo.
|
| There are a bunch of access rights, hooks and tokens that
| need to be set up in a certain way depending on the type of
| repo, and Ansible regularly makes sure no one deviates from
| the intended configuration.
|
| Github (or Bitbucket or Gitlab) isn't different from any
| other software in this regard.
| H12 wrote:
| I'd love it if there was more org-level functionality in
| general, for both content and configuration.
| xevrem wrote:
| This is why we use GitLab over GitHub.
| mikewhy wrote:
| Does GitHub have "system level" webhooks like gitlab?
| That's what I've done previously, so when anyone creates a
| repo, it's configured as the org expects.
|
| This page[1] seems to mention organization level hooks that
| receive repository created events, if making your team edit
| a terraform/puppet/ansible/chef repo to simply create a
| repo seems overkill to you
|
| [1]: https://docs.github.com/en/developers/webhooks-and-
| events/we...
| veverkap wrote:
| At a previous employer, we used Terraform to manage GitHub
| teams/repos/permissions.
|
| It worked ... okay. Might be worth a try for you?
| newman314 wrote:
| Can you expand a bit more on how you achieved this? It
| sounds interesting.
| onei wrote:
| There's a GitHub provider for Terraform [1]. I tried it
| out on a GitHub Enterprise instance last year and found
| it a little awkward because they didn't have support for
| the latest GH APIs and the version before was missing a
| bunch of features. When I looked recently, they seemed to
| have got past that.
|
| The principle is broadly the same as other Terraform
| workflows: import existing config and update it when you
| need to. You can manage repo permissions, people and
| teams, all centrally and in code. Super useful for
| tracking changes (the GH audit logs are a bit lacking in
| my experience) and identifying discrepancies between the
| expected and actual configurations.
|
| [1] https://registry.terraform.io/providers/integrations/
| github/...
| d4mi3n wrote:
| We did something similar to this at an org I worked at.
| We'd pull users from specific groups from our identity
| provider (Okta) and use that to provision corporate
| accounts to a Github Enterprise instance.
|
| One big downside to this approach is Github teams often
| don't line up 1-to-1 with an organization's reporting
| structure, so you don't really ever get away from needing
| to manually manage user-team memberships, but it does
| remove the need to onboard/offboard users manually.
| Jenk wrote:
| > PM for security products
|
| Please, please, _please_, add organisation scope to github
| action tokens (GITHUB_TOKEN in actions). It is painful that
| we have to create a PAT to access packages from within an
| organisation.
|
| https://github.community/t/github-token-cannot-access-
| privat...
| greysteil wrote:
| I'll make sure the Actions PMs see this. I _think_ I heard
| from a colleague that there's progress here.
| wolverine876 wrote:
| > The team is still relatively small (it's 7 people), and
| supporting a service like this at GitHub scale is hard, but
| we're keen to keep improving.
|
| While I empathize with your team regarding their apparent
| workload, that's only because they (probably) don't control
| hiring. GitHub isn't a struggling startup; it's not a non-
| profit; it's a mature, sizeable organization whose owner has
| over a trillion dollars in assets. If the team is too small
| to do a job, it's not because you lack personnel, it's
| GitHub's choice not to do it. You're _GitHub_ , owned by
| _Microsoft_ ; GitHub doesn't care about these issues enough
| to staff them.
|
| EDIT: Removed something provocative.
| samatman wrote:
| The Mythical Man-Month is probably older than you.
| wolverine876 wrote:
| Yes, I thought of that, and it's a relevant point IMHO.
| But while hiring 10,000 people off the street won't
| improve things, GitHub hardly has its hands tied; they
| know how to get things done if they want to. We're
| talking about adding some basic features. Somewhere in
| GitHub and Microsoft, there probably are some additional
| highly productive developers who could be assigned to the
| project.
| burnished wrote:
| That book is an incredible read for anyone, even just the
| first couple chapters, but the central thesis is "you
| probably cannot pay any amount of money to make an over-
| due software development (or many other kinds of
| development) project be an on-time project", and not
| "there is no point in growing a team to solve bigger
| problems faster". You can't hire another orchestra to
| help play the same song faster, but you can hire another
| orchestra for more music over a longer time.
| dmitriid wrote:
| A must: allow Dependabot to merge PRs automatically. The
| reason to refuse this is frankly ridiculous [1]. For JS
| projects Dependabot is little less than a spammer, and the
| ability to automatically merge minor/patch versions is a
| must. Having to rely on third-party actions to do this is
| probably a larger security risk than letting people decide
| what PRs they want to automerge.
|
| [1] https://github.com/dependabot/feedback/issues/954#issueco
| mme... and the original issue:
| https://github.com/dependabot/dependabot-core/issues/1973
|
| And why it doesn't fly:
| https://github.com/dependabot/dependabot-
| core/issues/1973#is...
| newman314 wrote:
| While we are on the feedback train, please consider making
| the Docker dependabot action more intelligent.
|
| Right now, I have a monorepo with multiple Dockerfiles in
| subdirectories.
|
| I end up having to add an entire docker block scoped to each
| directory that has a Dockerfile which is painful. Ideally, I
| would have a way to specify using */Dockerfile as a way to
| scan for Docker dependencies.
|
| Also, multi-stage Dockerfile configs seem to confuse
| Dependabot where it'll submit a pull request to update the
| "AS build" but not the image to be used for packaging...
|
| I really like the promise/premise of Dependabot, just want it
| to be easier to use (and edit across multiple repos).
| skybrian wrote:
| This seems like good motivation to prune your dependencies if
| they're getting out of hand. Of course, easier said than done,
| but at an ecosystem level maybe it will help?
| mukesh610 wrote:
| Well angular repositories tend to have tons of dependencies.
| tailspin2019 wrote:
| In the case of NPM, the only way to not have your
| dependencies be "out of hand", is to not use NPM.
|
| /s (but also, for realsies)
| hakre wrote:
| it was easy: block the user.
|
| then it got harder: create one branch that blocked the
| dependabot branch.
|
| then it got harder again: create 00 00 00 - FF FF FF branches
| (let it run with parallelism and over night).
|
| github support is not helpful on this.
| skipants wrote:
| A setting I turned on to make it more manageable is to ignore
| patch updates and have dependabot only make PRs for major &
| minor version bumps.
| boarnoah wrote:
| I usually ignore those and bundle a bunch in a single PR.
|
| The worst I found was another team which merges in depandabot
| PR's one by one. Their git history isn't pleasant to look
| through :(
| karlding wrote:
| The git log command has a --perl-regexp flag [0], which
| allows you to filter commits that match a Perl-style regular
| expression. I guess in that situation you could specify a
| regular expression that excludes dependabot in --author.
|
| If typing --perl-regexp gets old, you can always alias it or
| add a repo/global config option via grep.patternType [1]. Or
| you can create an alias for adding the less-powerful
| --invert-grep [2].
|
| [0] https://git-scm.com/docs/git-
| log/2.32.0#Documentation/git-lo...
|
| [1] https://git-scm.com/docs/git-
| config/2.32.0#Documentation/git...
|
| [2] https://git-scm.com/docs/git-
| log/2.32.0#Documentation/git-lo...
___________________________________________________________________
(page generated 2021-07-23 23:00 UTC)