[HN Gopher] Sharing new research, models, and datasets from Meta...
___________________________________________________________________
Sharing new research, models, and datasets from Meta FAIR
Author : TheAceOfHearts
Score : 173 points
Date : 2024-06-18 17:01 UTC (5 hours ago)
(HTM) web link (ai.meta.com)
(TXT) w3m dump (ai.meta.com)
| pbronez wrote:
| Interest that they're releasing a deepfake detector. I expect
| that to get integrated into image generation training pipelines.
| ronsor wrote:
| Throw "deepfake loss" onto the pile of loss functions.
| sghiassy wrote:
| Is it just me or does Meta seem to be doing better than most
| companies right now open sourcing their AI research?
| IncreasePosts wrote:
| Yes, I'd also like to know why Meta has chosen this path,
| whereas many of the other big players haven't. Usually they all
| settle upon the same viewpoint.
| hlfshell wrote:
| It started by accident, with the original llama weights being
| leaked by two separate employees. They've since embraced
| opening the weights, which I'm all for.
|
| As for why? I have a theory: Meta is not in a position to
| capitalize upon the model itself. Yes, they can use it
| internally, and maybe their competitors can copy it to - but
| there are no real competitors to Facebook or Instagram that
| can benefit from it enough to make it a differentiating
| facet.
|
| Thus, releasing stuff for open source does two things:
|
| 1) Make them more attractive to research talent (Apple
| famously recently started to publish research because their
| traditional secrecy was causing issues with hiring top
| talent) and...
|
| 2) Continues to undermine the ability to make $$$ off of
| model alone, driving it towards being a commodity rather than
| the long term profit engine for other companies.
| sdenton4 wrote:
| In short: "Commoditize the complement."
| onurcel wrote:
| > It started by accident, with the original llama weights
| being leaked by two separate employees.
|
| This is not true. Meta Fair has been built on openness from
| day 1. We published many papers and open-source d many
| repositories to reproduce the work
| ipsum2 wrote:
| Wrong. FAIR has been open sourcing ML models and source
| code for the last 10+ years. It did not start with llama.
| Also, llama was not leaked by employees, but people in the
| broader community, who the weights were shared with.
|
| For example:
|
| Faster R-CNN - state of the art image segmentation,
| released in 2017.
|
| FastText - text embedding models, 2016.
|
| FAISS - vector DB, 2018.
|
| https://github.com/orgs/facebookresearch/repositories has
| over 1,000 repos.
| hlfshell wrote:
| I was speaking about LLM weights specifically and llama,
| not all models and work at FAIR.
|
| Per your links, it's clear that FAIR does have a good
| history of open source work.
| freehorse wrote:
| > weights being leaked
|
| You can hardly call that "leak" when they basically were
| sending the weights to thousands of people who applied for
| access. It is not that they kept them secret.
| righthand wrote:
| Same reason they open sourced reactjs or encourage jestjs
| usage. If they give it away they can benefit by being
| embedded in the stack. Then all the scale issues they can
| beat implementers on because they have the money to run it.
| To keep ahead of people implementing their tech they have
| their own data trove to train on. You only get a small piece
| of it. It's all to position themselves as a sensible
| solution.
| downWidOutaFite wrote:
| My guess is that when Llama leaked on 4chan and it blew up in
| the community it somewhat forced their hand to go with an
| open strategy. But they also have a history with pytorch of
| reaping the benefits of an open strategy. The benefits are
| well explained in Google's leaked "We Have No Moat" document.
| bg24 wrote:
| 1/ Build a community (think Linux for OS, Android for mobile,
| React for frontend) and figure out monetization later. What
| is clear to most people is that something fundamental is
| changing in how we build and consume applications.
|
| 2/ Prevent OpenAI from cornering the future $$$ market.
| Unfortunately, Google search is hit as well, but it is more
| due to the generational shift.
|
| 3/ Attract the best AI researchers. A product is a good as
| its core set of people (often just a few).
| timy2shoes wrote:
| They've chosen the path of commoditizing their complement. To
| ensure that ML capabilities are not a differentiating factor
| in the market, make ML capabilities a commodity available to
| everyone at the marginal cost.
| nicce wrote:
| There isn't conflict in business model. And if people can
| help to improve their models, they can extract better value
| by themselves.
|
| Also, maybe they need to improve their brand. Hoarding data
| for over a decade, maybe bringing something back now.
| gillesjacobs wrote:
| Facebook/Meta has been doing open research in ML and NLP for
| far longer than the current LLM era: Convolutional Neural
| Networks for Sentence Classification (2014), FastText (2016),
| PyTorch (2016), fairseq (2017), LASER (2018), RoBERTa (2019),
| XLM-R (2019), and BART (2019),
|
| They were always present at ACL with decent open research as
| far as I have been studying/working in NLP (2014).
|
| It's part of a strategy to attract top talent in the field.
| If you want top researchers you have to let them publish,
| which in turn hones a reputation of solid research,
| attracting more talent.
| behnamoh wrote:
| > It's part of a strategy to attract top talent in the
| field. If you want top researchers you have to let them
| publish, which in turn hones a reputation of solid
| research, attracting more talent.
|
| This. Although, it turns out, if you pay them well enough
| (like OpenAI), they'll forego publishing. If you can make
| enough to retire by 40, why work at places that pay less?
|
| ofc, not all researchers think that way, and there are
| those who are in it for the science, not just money.
| lllaaaammmaaaaa wrote:
| All their users are writing "content" for free and
| communicate with each other. AI generated "content" does not
| really fit into this.
|
| They do not want their users to go to ClosedAI or similar and
| communicate with an Artificial Stupidity instead talking to
| each other on Facebook.
|
| So it is in their interest to undermine the market for
| Artificial Stupidities by releasing the models for free.
| dvngnt_ wrote:
| they've been on this wave for years.
|
| thanks to meta i have been creating instrumentals of some of my
| favorite songs with vocals
| https://github.com/facebookresearch/demucs
| hiddencost wrote:
| They are unable to compete with the top players on quality, so
| they instead compete by winning the largest user base with open
| models.
| 12345hn6789 wrote:
| Note that due to the licenses no company will touch any of
| these with a 10 ft pole. Great for individuals looking to
| experiment though
| occamrazor wrote:
| How so? Maybe in the past, but nothing announced today is open.
| Filligree wrote:
| Pity they aren't including image generation. Multimodal
| generation with reference inputs is my #1 ask for things like
| novel illustrations.
| advael wrote:
| It seems like their image-tokenization model might be useful
| for this, but also have you looked into stuff like ControlNet?
| mi_lk wrote:
| Is Llama3 400B still going to be released?
| ai_what wrote:
| Yes, Yann LeCun posted about it on twitter ~2 weeks ago.
|
| Edit, it was a month ago:
| https://twitter.com/ylecun/status/1793181068943639014
| ein0p wrote:
| Under a "do not even look at it" license I'm sure. /s
| RobotToaster wrote:
| > non-commercial/research-only license.
| zxcb1 wrote:
| Last time they did this, I was not granted access
| ai_what wrote:
| Try a different email address, that's what fixed it for me.
| zxcb1 wrote:
| Sure, that may work; but what does it mean?
| ai_what wrote:
| I can't speculate that for you. Try reaching out to them if
| you must know.
| zxcb1 wrote:
| When we say open X, we expect something like the Via
| Negativa principle; instead of inviting members into a
| closed community, exclude those that violate the code of
| the open community.
| ffsm8 wrote:
| This is false. Any "open" event (or anything, really) can
| be closed for select individuals.
|
| The reason for this exclusion just can't be a banned
| criteria such as race.
|
| (I.e. kicking someone out after they've stirred up
| controversy)
|
| I'm not saying that this happened to you, I'm just
| addressing the point you made, that anything "open" can't
| or shouldn't be exclusionary.
| zelphirkalt wrote:
| Considering, that this is all based on data and personal
| information of people, the people in general should have the
| right to dictate Facebook what they can and cannot do with that
| data and what happens with the products made using the data. It
| is laughable, that they act here as if they are doing something
| friendly or fair. Same is true for other companies like Microsoft
| and "Open"AI.
| tech_ken wrote:
| What's the argument that the early-fusion setup scales/maintains
| easier than late-fusion? Does this come at a tradeoff or is it
| just uniformly better?
___________________________________________________________________
(page generated 2024-06-18 23:00 UTC)