[HN Gopher] Why did Meta open-source Llama 2?
___________________________________________________________________
Why did Meta open-source Llama 2?
Author : rckrd
Score : 60 points
Date : 2023-07-21 18:55 UTC (4 hours ago)
(HTM) web link (matt-rickard.com)
(TXT) w3m dump (matt-rickard.com)
| lincon127 wrote:
| Did it? I don't think they did
| [deleted]
| innagadadavida wrote:
| All the responses I'm reading so far are rather shallow and fails
| to consider the overall landscape and how it will evolve and who
| the big losers will be. The way I see it, that the current
| players include Google (which has a lot to lose), OpenAI (unclear
| business model) and upcoming startups (can disrupt
| Google/OpenAI). Meta releasing these models will impact Google
| and OpenAI the most by helping upcoming startups to inflict fatal
| blows or slowly chip away at their business models by means of a
| race to the bottom. The main issue preventing Google or OpenAI to
| succeed is that the regulatory landscape will pose a huge risk
| and Meta knows that. Startups are not hampered by this as they
| are small fry and before anyone can notice, they can/will land a
| blow on Google/OpenAI. To all those people complaining on this
| not being open source - Zuck is playing chess, while you play a
| much simpler game. Advancing SOTA and a bit of Open source is a
| side benefit.
| seba_dos1 wrote:
| I don't care whether it's open source or not. I do care about
| it being called open source when it's not.
| jraph wrote:
| > I do care about it being called open source when it's not.
|
| Yep. We don't need additional confusion on this term.
| cloudking wrote:
| They are taking a similar AI strategy to Google's mobile strategy
| with Android.
| Tepix wrote:
| So, rckrd wrote a (rather short) article about the license of
| Llama 2 but got it all wrong: Even though the press calls it
| open-source, it's not.
|
| Open Source has a very clear definition. Llama 2 fails in
| multiple regards.
|
| First, the license by itself is not an open source license. It
| has important restrictions that make it non-open source.
|
| Second, the distribution. You have to apply for the download with
| a web form and you are not allowed to redistribute the model.
|
| Third, the source code, i.e. the data used to train the model.
| Meta is not telling us about it. One of the core features of open
| source is that you can recreate the binary (in this case, the
| model weights) by yourself. You can't do that here.
| taneq wrote:
| Are we gonna have to do the FOSS vs. 'source available' thing
| again? Also, while I think your 'source=training set+model,
| binary=weights' analysis is correct, I'm not sure we have 100%
| consensus on this yet?
| charcircuit wrote:
| You can redistribute the model.
|
| A model is not a binary. Training code is not source code for
| the model. The training code is closer to an editor. You can
| release open source software created by a closed source IDE.
| seba_dos1 wrote:
| They didn't. It's not open source.
| mrtranscendence wrote:
| Definitions are determined by actual usage. If people call
| Llama 2's license open source then ipso facto it's open source.
| seba_dos1 wrote:
| There's about 25 years of actual usage that clearly states
| otherwise. We can give your idea some weight after 25 years
| of people consistently calling Llama 2 open source,
| effectively diluting the term to mean nothing in the process.
| Meanwhile it's just as if I told you that the sky is green.
| stale2002 wrote:
| We don't have to play dumb. Everyone in this thread knows
| exactly what the article means, when it says that Llama was
| open sourced.
|
| This obviously means that the weights were released, and
| everyone knows this, regardless of any pedantic definition of
| what "open source" means.
| pengaru wrote:
| > We don't have to play dumb. Everyone in this thread knows
| exactly what the article means, when it says that Llama was
| open sourced. > > This obviously means that the weights were
| released, and everyone knows this, regardless of any pedantic
| definition of what "open source" means.
|
| It's no surprise you're disinterested in the meaning of "open
| source" given how much you misuse "everyone".
| stale2002 wrote:
| And it is no surprise that someone who cares only about
| pedantry will fail to address the argument, and instead go
| off on an irrelevant tangent.
|
| But feel free to reread the original comment and actually
| talk about the substance of the claim, instead of doing the
| normal "Well actually" hacker news comment.
| seba_dos1 wrote:
| Calling Llama 2 "open source" is obviously a misleading
| marketing tactic to capitalize on positive connotations of
| the term, while in reality it's nowhere even close to being
| open source. This has nothing to do with any kind of
| pedantry, "open source" is a specific term with specific
| meaning, and "releasing the weights" is completely orthogonal
| to it being open source or not.
| andy99 wrote:
| A challenge here is that it is pedantic, even if it's true.
| I've been very vocal about how the LLaMAs as well as other
| models like those released under RAIL licenses are not open
| source. But the truth is very few people care and attitudes
| like the one above are common. What needs to happen is for
| a strong, and I'd suggest copyleft, alternative to emerge
| that's good enough to make people have to care more about
| license terms. That plus continued advocacy in the face of
| people dismissing pedantic distinctions as irrelevant.
| serf wrote:
| >A challenge here is that it is pedantic, even if it's
| true.
|
| it's not pedantry unless it's a minor detail, and given
| that the entire license behind llama has next to nothing
| to do with the open source model or existing open source
| licensing, this isn't a minor detail.
|
| the court-room doesn't care if 'few people care'.
|
| it's obvious to anyone that has been to a CS convention
| in the past 20 years that the whole open-source thing
| from large corporations is used to leverage good-will to
| recruit useful labor and to collect community 'altruism-
| points' so that they can make shadier decisions later on
| and hopefully trap a captive audience, not to follow the
| ethical ideals of open-source but to enhance their
| bottom-line.
|
| 'Trojan-horse open-source'.
| nomel wrote:
| > and I'd suggest copyleft, alternative to emerge that's
| good enough to make people have to care more about
| license terms
|
| I keep seeing this, but I can't understand how it would
| work.
|
| Bored nerds will volunteer their time to open source,
| with little return. That's why I did it.
|
| Generally, nobody will volunteer their _money_ to open
| source. This is unfortunate, and a huge problem.
|
| Training neural nets requires the _money_. Where does it
| come from?
| gtirloni wrote:
| https://en.wikipedia.org/wiki/List_of_highest-
| funded_crowdfu...
| andy99 wrote:
| I'm not saying it's a solved problem though there are
| already initiatives like AI Horde that are doing
| volunteer distributed inference (it it looks like now
| fine-tuning) on GPUs https://stablehorde.net/
|
| Plus it can be a strategic choice for institutions and
| companies, just like Linux is for example.
| [deleted]
| _emacsomancer_ wrote:
| Given legal implications, 'pedantic' is a rather inaccurate
| descriptor.
| topynate wrote:
| They can keep pretending it's an open source licence, and I'll
| keep pretending to respect their copyright in algorithmically
| generated weights.
| throwaway29812 wrote:
| can you expand on that?
| mindcrime wrote:
| It's closer to "shared source" or "source available". The
| license is not compliant with the OSD[1] which is the de-
| facto (but not de jure, for you pedants out there) definition
| of "Open Source" in common usage.
|
| [1]: https://opensource.org/osd/
| jraph wrote:
| > but not de jure, for you pedants out there
|
| This means that the term "open source" is not legally
| defined (and not recognized by lawyer / judges) [and the
| OSD], right?
| mindcrime wrote:
| Yes, "de jure" more or less translates to "by law", and
| "de facto" is something like "by practice" or "by
| convention". So the Open Source Initiative folks have no
| legal basis to enforce use of their definition, but in
| practice it's so widely used and acknowledged that for
| all practical purposes it _is_ the definition.
| _Parfait_ wrote:
| Basically under the actual terms of what open source is there
| can be no limits to particular users. Meta, in order to
| protect their competitive market share has said explicitly in
| Paragraph Two of their licensing terms
|
| --- " If, on the Llama 2 version release date, the monthly
| active users of the products or services made available by or
| for Licensee, or Licensee's affiliates, is greater than 700
| million monthly active users in the preceding calendar month,
| you must request a license from Meta, which Meta may grant to
| you in its sole discretion. --
|
| Doing so, technically, takes the license out of the category
| of "Open Source."
|
| Sources: Open Source Initiate Argument
| https://blog.opensource.org/metas-llama-2-license-is-not-
| ope...
|
| Llama Terms https://ai.meta.com/resources/models-and-
| libraries/llama-dow...
| wolpoli wrote:
| I wonder who they are targeting with this restriction on
| 700 million MAU.
| onedognight wrote:
| It's open but not open source. Getting Llama is like getting
| a binary executable. You don't get the source (training data
| / model / code), so you can't make changes and recompile, but
| you can use it and fine tune.
| jraph wrote:
| What does open mean? I'm not sure we really have a widely
| shared, common definition of this, contrary to open source.
|
| > You don't get the source
|
| You do IIUC. But your can't use it for all purposes (like
| serving 1B users), which is what makes it non open source.
| seba_dos1 wrote:
| Its license is quite obviously not matching the definition of
| Open Source:
|
| > v. You will not use the Llama Materials or any output or
| results of the Llama Materials to improve any other large
| language model (excluding Llama 2 or derivative works
| thereof).
|
| > 2. Additional Commercial Terms. If, on the Llama 2 version
| release date, the monthly active users of the products or
| services made available by or for Licensee, or Licensee's
| affiliates, is greater than 700 million monthly active users
| in the preceding calendar month, you must request a license
| from Meta, which Meta may grant to you in its sole
| discretion, and you are not authorized to exercise any of the
| rights under this Agreement unless or until Meta otherwise
| expressly grants you such rights.
|
| > Prohibited Uses: <a whole page full of text>
| andy99 wrote:
| The license has a bunch of terms and restrictions that make
| it incompatible with accepted definitions of open source
| software.
|
| LLaMA2 isn't "Open Source" - and why it doesn't matter
| https://www.alessiofanelli.com/blog/llama2-isnt-open-source
|
| Meta can call Llama open source as much as it likes, but that
| doesn't mean it is https://www.theregister.com/2023/07/21/lla
| ma_is_not_open_sou...
|
| Software licenses masquerading as open source (I wrote that
| one) http://marble.onl/posts/software-licenses-masquerading-
| as-op...
|
| There have been lots of other posts on here about this too.
| frankreyes wrote:
| Divide et impera
| phkahler wrote:
| To increase adoption. They are also working with Qualcom [1] to
| bring it on-device. Not sure if they're licensing it, but when
| you tweak hardware for something specific, you kinda want people
| to use it.
|
| [1] https://www.qualcomm.com/news/releases/2023/07/qualcomm-
| work...
| dwrodri wrote:
| Meta made Llama 2 source-and-weights available because they
| agreed with the observations in the leaked Google memo[1]. Meta
| got a huge amount of infra/research/experimentation work done on
| top of LLaMa. Pre-training wasn't cheap, but they got datapoints
| no one else in big tech had, abd that is very valuable,
| especially when building a bridge into a new frontier like LLM-
| driven products.
| m3kw9 wrote:
| This analogous to freeium model for apps?
| andy99 wrote:
| Llama had already become a defacto standard for LLMs, between all
| the fine-tunes and llama.cpp. Giving it a wider license really
| cements it as a standard while making everybody use it on Meta's
| terms. It's an "open source" strategy where all the benefits
| accrue back to Meta instead of the community but that's
| permissive enough to placate most people.
|
| Personally I feel like it's a dangerous precedent because it
| shifts open source from a community concept to something that a
| company lets you have with a bunch of conditions.
| brucethemoose2 wrote:
| Stable Diffusion is a good example of this progression.
|
| There are new models/papers/backends coming out all the time
| (including SDXL, which is similar to the original SD), but the
| community is so entrenched in Stable Diffusion 1.5 and the old
| Stability AI PyTorch implementation that moving to anything
| else hasn't really happened yet.
|
| That being said, I think Llama won't stick around as the "de
| facto" standard for over a year unless Meta keeps releasing
| better foundational models.
| pavlov wrote:
| Why wouldn't they? Meta seems to have the money, the
| hardware, the talent, and the executive motivation to keep
| producing better models.
| ldjkfkdsjnv wrote:
| Any open source is good open source, meta has contributed react
| and pytorch to the world. They should be applauded for their
| contributions to software. They have done far more than any
| other big tech company.
| tonmoy wrote:
| I was with you until the last sentence. Google has given us
| Android, Kubernetes, Golang, Tensorflow (not to mention
| Angular and Chromium)
| mrtranscendence wrote:
| > Kubernetes
|
| Which they never apologized for!
| sanxiyn wrote:
| Apple did LLVM and WebKit, which seem far more important than
| React and PyTorch.
| remcob wrote:
| WebKit was more or less 'stolen' from KDE's KHTML and KJS.
| They forked it, developed it in secret, and did not publish
| anything until it diverged so far from source that it could
| not be merged back in. While technically legal, it
| definitely goes against the spirit of open source. LLVM is
| a better story: also not invented at Apple, but they hired
| the core developers to continue working on it.
| palata wrote:
| > it shifts open source from a community concept to something
| that a company lets you have with a bunch of conditions.
|
| Because you think that Protobuf, Android, Chromium, and all the
| other open source projects by Big Tech are there for the
| community? It's about control.
|
| Not saying it's fundamentally bad, but you can't deny that the
| fact that Android exists means that it is very difficult to get
| adoption for an alternative "open" mobile OS, and Google
| benefits greatly from it. Same for Chromium, and basically
| everything.
|
| When Big Tech open sources something, it's a strategic
| decision. Not philanthropy.
| psd1 wrote:
| Protobuf seems like the odd one out. I understood that
| googlers created it for their own benefit and tossed it over
| the wall because "why not". Am I mistaken?
| palata wrote:
| I don't know about the actual decision, but I can see the
| result: Protobuf is ubiquitous. Now take it like this: if
| Protobuf was not open source, wouldn't there be an
| alternative? I would say probably, since there are already
| alternatives (just not as popular as Protobuf).
|
| So from Google's perspective, _not_ open sourcing Protobuf
| does not really give them an edge: alternatives exist.
| However, by open sourcing it, they end up with a ton of
| devs who learn Protobuf outside of Google. That makes it
| easier for Google technology to be adopted if they release
| something new, and when they hire a new engineer, it 's a
| win if he already knows Protobuf.
|
| Same applies to gRPC and others.
| littlestymaar wrote:
| > Personally I feel like it's a dangerous precedent because it
| shifts open source from a community concept to something that a
| company lets you have with a bunch of conditions.
|
| It's not really a unique precedent, Chromium and Android fit
| the same model, but it's still very much better than closed-
| source (electron and custom Android ROMs are something you
| couldn't have otherwise).
___________________________________________________________________
(page generated 2023-07-21 23:01 UTC)