[HN Gopher] Meta open-sources multisensory AI model that combine...
___________________________________________________________________
Meta open-sources multisensory AI model that combines six types of
data
Author : rntn
Score : 126 points
Date : 2023-05-09 15:45 UTC (7 hours ago)
(HTM) web link (www.theverge.com)
(TXT) w3m dump (www.theverge.com)
| fnordpiglet wrote:
| Primary source HN discussion :
|
| https://news.ycombinator.com/item?id=35875700
| bhoops wrote:
| It seems to me that we put this inside a boston dynamics robot
| and we have Terminator
| WinLychee wrote:
| In the quest to placate shareholders, one of the failing FAANGs
| will make a panicked announcement they have partnered with
| Boston Dynamics, to ensure AI is democratized and equitable for
| all (technically true when we're all on the run).
| renewiltord wrote:
| Well, it's unlikely to be Google. They've sold them off once
| already.
| gremlinsinc wrote:
| nothing says equality when terminators are annihilating the
| human race, non-discriminatory.
| transformi wrote:
| Any thoughts about possible applications of this model? Why do
| they made this source available?
| [deleted]
| seanlinehan wrote:
| Link to the ImageBind site: https://imagebind.metademolab.com/
| mkaic wrote:
| Additionally, here's a link the blog post about it, which I
| feel might actually be a better place for this post to link to
| instead of Verge's coverage of it:
| https://ai.facebook.com/blog/imagebind-six-modalities-bindin...
| [deleted]
| revskill wrote:
| They should rename Meta to MeetAI now.
| sroussey wrote:
| MetAI
| nprateem wrote:
| Ha ha they're trying to revive the metaverse with the one killer
| app that'll save it: immersive AI porn. Probably a good time to
| buy some meta shares in case they "pull it off"
| gloryjulio wrote:
| It's a bit late for that. They already almost tripled from the
| low point
| omeze wrote:
| I see this sentiment of "the metaverse is dead" very often on
| HN but VR usage is at an all time high and meta has the leading
| hardware. Genuinely wonder: Why do you think the metaverse is
| dead, and by metaverse do you mean "horizon worlds", VR,
| "metaverse apps" or something else?
| throwuwu wrote:
| I dunno, everyone pivoted to AI because it's clearly a massive
| force multiplier but they're all bringing those gains back to
| whatever they were working on previously. I've already seen
| some great uses of AI generated voices, scripts, textures, and
| sky spheres in VR. AI is also a huge resource for learning how
| to do things you've never attempted before. Shit's going to be
| popping off all over the place this year and I don't think
| it'll stop.
| [deleted]
| digitcatphd wrote:
| It seems like this is a substantial step in Yann LeCun's approach
| to AGI https://www.shaped.ai/blog/yann-lecun-a-path-towards-
| autonom...
| flangola7 wrote:
| Maybe, I take anything LeCun says with many grains of salt.
| Much like Elon Musk, any time he says something that appears to
| make sense I find myself asking "what don't I know enough
| about, to know why he's wrong?"
| seydor wrote:
| > text, audio, visual, movement, thermal, and depth.
|
| > It's only a research project
|
| Nice try , zuckercop. Facebook policing and terminating real
| people is truly their natural evolution.
| soulofmischief wrote:
| Audio2image looks insane. Can't wait to see how this tech
| develops over time.
| thrtythreeforty wrote:
| Once again, Facebook tries to have their cake and eat it too.
| This model is not open source, despite the claim.
|
| https://github.com/facebookresearch/ImageBind/blob/main/LICE...
|
| The source is available to view and use under a _non-commercial_
| license. It 's extremely frustrating that major AI players are
| trying to shoehorn a redefinition of a term that's well-
| understood, just to further their reputation.
| godelski wrote:
| There's a lot of people arguing about OSS vs FOSS.
| opensource.org has a specific definition but this is not
| universally accepted (there's a lot on this in the wiki page
| btw). Stallman even wrote about this[0].
|
| If you're passionately arguing and downvoting others that are
| claiming that ImageBind is in fact OSS then you're also
| actively demonstrating naivety. Definitions are not always
| universal and this is an easy to verify example. Instead of
| just posting another link to opensource.org read their about
| section[1] and consider why that leads to non-universality. And
| maybe also consider the words themselves: is the source
| available? Even if the only correct definition were from
| opensource.org, I'd expect everyone here to have some
| understanding why others might be confused since the words
| imply something different. There's some evidence to the other
| side's view, so it isn't obscene or idiotic. You don't have to
| accept the other view, just consider its existence.
|
| Take a step back and consider that your view isn't the only
| possible one. Maybe we can have more constructive discussions
| if we do so. We don't need these holy wars, they don't
| accomplish anything.
|
| [0] https://www.gnu.org/philosophy/open-source-misses-the-
| point....
|
| [1] https://opensource.org/about/
| getcrunk wrote:
| It's not about what views are possible. It's about honesty
| and humility which are both things the modern world doesn't
| seem to care about. The more narrow definition of oss
| (rms/gnu) is un ambiguous as is the meaning of source
| available. People choosing to use oss with the ambiguous
| meaning are liable for their ambiguity, and therefore worthy
| of op's criticism.
| mindcrime wrote:
| Welp, I guess it can't hurt to ask:
|
| https://github.com/facebookresearch/ImageBind/pull/6
| lanza wrote:
| You don't understand what opensource means...
| squokko wrote:
| "Open source" is not "free software."
|
| This is absolutely open source and will enable a lot of
| academic, governmental, and nonprofit uses. Just because it
| doesn't allow someone else to make money from it doesn't make
| it not open source.
| thrtythreeforty wrote:
| https://opensource.org/osd/, specifically:
|
| > 6. No Discrimination Against Fields of Endeavor
|
| > The license must not restrict anyone from making use of the
| program in a specific field of endeavor. For example, it may
| not restrict the program from being used in a business, or
| from being used for genetic research.
|
| It's true that people are starting to make a distinction
| between "free software" and "open source," but generally the
| distinction is smaller than what you're saying.
|
| The broader class of programs whose code is available for
| viewing and (potential) use or modification subject to
| restrictions, has been called "source available," and I think
| that's a great description of what it is and sets
| expectations well. You may or may not have a license to use
| it, but you can see the source code.
|
| For example, the Business Source License is generally not
| considered open-source. It places restrictions on what you
| can do.
|
| I kind of hate being pedantic about this, but I also dislike
| the deliberate misuse of the term, especially by a company
| that is obviously quite familiar with publishing large
| amounts of high-quality software that's genuinely open
| source.
| mindcrime wrote:
| _Just because it doesn 't allow someone else to make money
| from it doesn't make it not open source._
|
| Actually it does, per the OSD[1].
|
| [1]: https://opensource.org/osd/
| jacooper wrote:
| This is source available, not open source.
| nextaccountic wrote:
| Indeed
|
| https://en.wikipedia.org/wiki/Source-available_software
|
| > Source-available software is software released through a
| source code distribution model that includes arrangements
| where the source can be viewed, and in some cases modified,
| but without necessarily meeting the criteria to be called
| open-source.
| gumballindie wrote:
| > Those opposed to open-sourcing, like OpenAI, say the practice
| is harmful to creators because rivals can copy their work
|
| OpenAI complaining about people copying their work is hilariously
| hypocritical.
| ChatGTP wrote:
| Totally, I think open ai believes they will build an AGI and
| rule the world with importance before everyone realised they
| just steal and compress everyone's stuff and sell it via hosted
| database.
| swyx wrote:
| > Those opposed to open-sourcing, like OpenAI,
|
| so we're just saying the quiet part out loud now huh
| gumballindie wrote:
| Which is the quiet part?
| FeepingCreature wrote:
| Did they actually say this?
| ajayyy wrote:
| https://cdn.openai.com/papers/gpt-4.pdf
|
| > Given both the competitive landscape and the safety
| implications of large-scale models like GPT-4, this report
| contains no further details about the architecture (including
| model size), hardware, training compute, dataset
| construction, training method, or similar.
|
| They did say "competitive landscape"
| yreg wrote:
| If that's it, then it's quite different to Verge's misquote
| imo.
| derefr wrote:
| Are multi-format embeddings -- like this one, or simpler ones
| like text-to-image -- inherently bijective? I.e. could a model
| like this be configured to use any of its trained sense data
| types as either an input or output? Or is there some bias in the
| way the training data is ingested that makes the model better at
| transforming in certain directions?
| jacobr1 wrote:
| I haven't yet read this paper, but other multimodal models have
| different layers for mapping the input/output of each modality
| to the common embedding space. So, to simplify, there is some
| high-dimensional coordinate that the audio sample for the
| morphemes for "frog," the text word "frog" and maybe the same
| in multiple languages like, "rana" - they all basically have
| the same point in the abstract feature space. The additional
| layers of the model then provide "translation"
| tkellogg wrote:
| Like laser eyes?
| alhirzel wrote:
| Has anyone found info on what the hardware requirements are to
| run this?
| capableweb wrote:
| For training, it probably looks very different, but the usage
| example in https://github.com/facebookresearch/ImageBind#usage
| seems to use just one cuda device, so you might be able to run
| it on a consumer GPU
___________________________________________________________________
(page generated 2023-05-09 23:02 UTC)